71ee0163de qt: 30.0rc3 translations update (Hennadii Stepanov)
Pull request description:
This PR updates the Polish (pl) translation and addresses [this](https://github.com/bitcoin/bitcoin/pull/33275#issuecomment-3329617679) comment.
Updates for other languages have been skipped, as I believe the review effort would not be worthwhile at this stage of the release process.
ACKs for top commit:
maflcko:
Though, this lgtm ACK 71ee0163de
janb84:
ACK 71ee0163de
Tree-SHA512: e9ee8146542d0928f5ddaac5b2cb0c0032cd7d236fc19b38fd4ebe91e13eccdfa7d25ff279376c3f27f0bc461729b2c0574fedbb670a2555bc7a534e5ec0cd76
QT translations are optional, but the script would error when
'translations_dir' falls back to its default value NULL.
This PR fixes it by moving the set-up of QT translations under
the check for 'translations_dir' presence.
Github-Pull: #33482
Rebased-From: 7b5261f7ef
Modern Ubuntu isn't shipping with this library installed by default.
Staticly link it to remove the need for end-users to install it.
Closes#33432.
Github-Pull: #33434
Rebased-From: eca50854e1
Not enforcing TRUC topology on reorg was the intended
behavior, but the appropriate bypass argument was not
checked.
This mistake means we could potentially invalidate a long
chain of perfectly incentive-compatible transactions that
were made historically, including subsequent non-TRUC
transactions, all of which may have been very high feerate.
Lastly, it wastes CPU cycles doing topology checks since
this behavior cannot actually enforce the topology in
general for the reorg setting.
Github-Pull: #33504
Rebased-From: 26e71c237d
Using bypass_limits=true is essentially fuzzing part of a
reorg only, and results in TRUC invariants unable to be
checked. Remove most instances of bypassing limits, leaving
one harness able to do so.
Github-Pull: #33504
Rebased-From: bbe8e9063c
A target field was added to the getblock and getblockheader RPC calls in bitcoin#31583, but it mistakingly always used the tip value.
Because regtest does not have difficulty adjustment, a test is added for mainnet instead.
Github-Pull: #33446
Rebased-From: bf7996cbc3
The next commit requires an additional mainnet block which changes the difficulty.
Also fix a few minor mistakes in the test (suite):
- rename the create_coinbase retarger_period argument to halving_period. Before bitcoin#31583 this was hardcoded for regtest where these values are the same.
- drop unused fees argument from mine helper
Finally the CPU miner instructions for generating the alternative mainnet chain are expanded.
Github-Pull: #33446
Rebased-From: 4c3c1f42cf
7ebdfa2173 ci: link against -lstdc++ in native fuzz with msan job (fanquake)
ecbcef33d6 doc: remove unrelated `bitcoin-wallet` binary from `libbitcoin_ipc` description (Sebastian Falbesoner)
8c973d6614 ci: disable cirrus cache in 32bit arm job (will)
2378bbf356 ci: refactor docker action to return provider str (will)
acf7d53ace test: split out `system_ram_tests` to signal when total ram cannot be determined (Lőrinc)
ce56548c63 system: improve handling around GetTotalRAM() (Vasil Dimov)
5226a92f28 coins: warn on oversized -dbcache (Lőrinc)
49d4ebcbfe system: add helper for fetching total system memory (Lőrinc)
0a80b1ae62 doc: update manual pages for v30.0rc2 (fanquake)
b8fb918969 build: bump version to v30.0rc2 (fanquake)
792a75ac86 build(windows): Remove lingering registry entries and shortcuts upon install (Hodlinator)
1bc3be1962 p2p: Increase tx relay rate (Anthony Towns)
4b02bc1a72 test: Avoid interface_ipc.py Duplicate ID errors (Ryan Ofsky)
Pull request description:
Backports:
* #28592
* #33302
* #33333
* #33420
* #33422
* #33425
* #33435
* #33459
Finalise `v30.0rc2`
ACKs for top commit:
willcl-ark:
ACK 7ebdfa2173
hebasto:
ACK 7ebdfa2173, I applied all backports locally without conflicts and obtained a zero diff with this PR branch.
Tree-SHA512: 73d641a5d783511a959e63f240453bb020705cb620b85a5a0968b32b937ac28816ef142f78bdf41976ed1c2bee431def945c5c37da33621031e3198cfdae51f3
`bitcoin-wallet` as-is is merely an offline wallet inspection tool
(introduced more than 9 years ago in PR #13926) that doesn't have any
relation with IPC/multiprocess, so remove it from the list of binaries
that use `libbitcoin_ipc`.
Github-Pull: #33459
Rebased-From: fbde8d9a81
Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>
Add an optional matrix field allowing opt-out of configuring cirrus
GHA cache when not using cirrus runners.
This is not needed for the cirruslabs/[save|restore]-cache actions, as
they automatically fallback based on runner type.
Github-Pull: #33302
Rebased-From: 00c253d494
when `GetTotalRAM` returns an `std::nullopt` now we're getting:
```
The following tests did not run:
106 - system_ram_tests (Skipped)
```
GitHub-Pull: #33435
Rebased-From: 56791b5829
Oversized allocations can cause out-of-memory errors or [heavy swapping](https://github.com/getumbrel/umbrel-os/issues/64#issuecomment-663637321), [grinding the system to a halt](https://x.com/murchandamus/status/1964432335849607224).
`LogOversizedDbCache()` now emits a startup warning if the configured `-dbcache` exceeds a cap derived from system RAM, using the same parsing/clamping as cache sizing via CalculateDbCacheBytes(). This isn't meant as a recommended setting, rather a likely upper limit.
Note that we're not modifying the set value, just issuing a warning.
Also note that the 75% calculation is rounded for the last two numbers since we have to divide first before multiplying, otherwise we wouldn't stay inside size_t on 32-bit systems - and this was simpler than casting back and forth.
We could have chosen the remaining free memory for the warning (e.g. warn if free memory is less than 1 GiB), but this is just a heuristic, we assumed that on systems with a lot of memory, other processes are also running, while memory constrained ones run only Core.
If total RAM < 2 GiB, cap is `DEFAULT_DB_CACHE` (`450 MiB`), otherwise it's 75% of total RAM.
The threshold is chosen to be close to values commonly used in [raspiblitz](https://github.com/raspiblitz/raspiblitz/blob/dev/home.admin/_provision.setup.sh#L98-L115) for common setups:
| Total RAM | `dbcache` (MiB) | raspiblitz % | proposed cap (MiB) |
|----------:|----------------:|-------------:|-------------------:|
| 1 GiB | 512 | 50.0% | 450* |
| 2 GiB | 1536 | 75.0% | 1536 |
| 4 GiB | 2560 | 62.5% | 3072 |
| 8 GiB | 4096 | 50.0% | 6144 |
| 16 GiB | 4096 | 25.0% | 12288 |
| 32 GiB | 4096 | 12.5% | 24576 |
[Umbrel issues](https://github.com/getumbrel/umbrel-os/issues/64#issuecomment-663816367) also mention 75% being the upper limit.
Starting `bitcoind` on an 8 GiB rpi4b with a dbcache of 7 GiB:
> ./build/bin/bitcoind -dbcache=7000
warns now as follows:
```
2025-09-07T17:24:29Z [warning] A 7000 MiB dbcache may be too large for a system memory of only 7800 MiB.
2025-09-07T17:24:29Z Cache configuration:
2025-09-07T17:24:29Z * Using 2.0 MiB for block index database
2025-09-07T17:24:29Z * Using 8.0 MiB for chain state database
2025-09-07T17:24:29Z * Using 6990.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
```
Besides the [godbolt](https://godbolt.org/z/EPsaE3xTj) reproducers for the new total memory method, we also tested the warnings manually on:
- [x] Apple M4 Max, macOS 15.6.1
- [x] Intel Core i9-9900K, Ubuntu 24.04.2 LTS
- [x] Raspberry Pi 4 Model B, Armbian Linux 6.12.22-current-bcm2711
- [x] Intel Xeon x64, Windows 11 Home Version 24H2, OS Build 26100.4351
Co-authored-by: stickies-v <stickies-v@protonmail.com>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: w0xlt <woltx@protonmail.com>
Github-Pull: #33333
Rebased-From: 168360f4ae
Added a minimal system helper to query total physical RAM on [Linux/macOS/Windows](https://stackoverflow.com/a/2513561) (on other platforms we just return an empty optional).
The added test checks if the value is roughly correct by checking if the CI platforms are returning any value and if the value is at least 1 GiB and not more than 10 TiB.
The max value is only validated on 64 bits, since it's not unreasonable for 32 bits to have max memory, but on 64 bits it's likely an error.
https://learn.microsoft.com/en-us/windows/win32/api/sysinfoapi/ns-sysinfoapi-memorystatusex
> ullTotalPhys The amount of actual physical memory, in bytes.
https://man7.org/linux/man-pages/man3/sysconf.3.html:
> _SC_PHYS_PAGES The number of pages of physical memory. Note that it is possible for the product of this value and the value of _SC_PAGESIZE to overflow.
> _SC_PAGESIZE Size of a page in bytes. Must not be less than 1.
See https://godbolt.org/z/ec81Tjvrj for further details
Github-Pull: #33333
Rebased-From: 6c720459be
Prior releases installed using these paths. Especially annoying was that the lingering registry entry for the uninstaller would show up as "Bitcoin Core (64-bit)" besides the current "Bitcoin Core" entry in the list of installed programs, and whichever was uninstalled last would fail to work as they would default to the same install directory.
Github-Pull: #33422
Rebased-From: 79752b9c0b
In the presence of smaller transactions on the network, blocks can sustain a
higher relay rate than 7tx/second. In this event, the per-peer inventory queues
can grow too large.
This commit bumps the rate up to 14 tx/s (for inbound peers), increasing the
safety margin by a factor of 2.
Outbound peers continue to receive relayed transactions at 2.5x the rate of
inbound peers, for a rate of 35tx/second.
Co-Authored-By: Suhas Daftuar <sdaftuar@gmail.com>
Github-Pull: #28592
Rebased-From: b81f37031c
This change should fix issue https://github.com/bitcoin/bitcoin/issues/33417
reported by zaidmstrr. It's possible to reproduce the `mp/proxy.capnp:0:
failed: Duplicate ID @0xcc316e3f71a040fb` error by installing libmultiprocess
system-wide, or to one of the locations listed in the python test's `imports`
list before the local libmultiprocess subtree, and then running the test.
Github-Pull: #33420
Rebased-From: e9c52272eb
33a0d4bb5b qt: 30.0rc2 translations update (Hennadii Stepanov)
Pull request description:
This PR updates Spanish (es) and Czech (cs) translations and addresses the following comments:
- https://github.com/bitcoin/bitcoin/pull/33275#issuecomment-3315273628
- https://github.com/bitcoin/bitcoin/pull/33275#issuecomment-3316206549
Updates for other languages were skipped, as I believe the review effort would not be worthwhile at this stage of the release process.
ACKs for top commit:
fanquake:
ACK 33a0d4bb5b.
Tree-SHA512: 94c1c1fb4a0079f3e733c573ba1fddd149307ec39220e811d33f5bbfd929a634b24ef9adbe9e789bd0127539ce5e134dde3a241db0e233d53446abf96a4d49b6
c9f751090c cmake: Install `bitcoin` manpage (Hennadii Stepanov)
2327b2b0db net: Do not apply whitelist permission to onion inbounds (Martin Zumsande)
26208b3a0c test: Add submitblock test in interface_ipc (TheCharlatan)
3ae592537d test: Prevent disk space warning during node_init_tests (Ryan Ofsky)
5dbb1bae38 ci: Enable CI_LIMIT_STACK_SIZE=1 in i686_no_ipc task (MarcoFalke)
c7faf72ac6 test: Fix CLI_MAX_ARG_SIZE issues (MarcoFalke)
0a2afbeb77 cmake: Fix regression in `secp256k1.cmake` (Hennadii Stepanov)
75026cddea wallet: Add m_cached_from_me to cache "from me" status (Ava Chow)
bbb4e118f3 test: Add a test for anchor outputs in the wallet (Ava Chow)
b85dc7ed3a wallet: Throw an error in sendall if the tx size cannot be calculated (Ava Chow)
d2be9a22d8 wallet: Determine IsFromMe by checking for TXOs of inputs (Ava Chow)
ad6c13e041 test: Test wallet 'from me' status change (Ava Chow)
35038b03c9 trace: Workaround GCC bug compiling with old systemtap (Luke Dashjr)
f7eded1dca ci: always use tag for LLVM checkout (fanquake)
6b19ede1a5 gui: Avoid pathological QT text/markdown behavior... (David Gumberg)
Pull request description:
Backports:
* #33243
* #33268
* #33310
* #33364
* #33379
* #33380
* #33391
* #33407
* https://github.com/bitcoin-core/gui/pull/886
ACKs for top commit:
darosior:
utACK c9f751090c
hebasto:
ACK c9f751090c, I applied all backports locally without conflicts and obtained a zero diff with this PR branch.
Tree-SHA512: 257cc5bd0423fbf2aff62c72957faea3de8731353d809b11e18d0e5cad174c7023dca9dedd0c73e07497eb804b7c48355a055b4461db260e2f0a5712d2514ff6
Tor inbound connections do not reveal the peer's actual network address.
Therefore do not apply whitelist permissions to them.
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Github-Pull: #33395
Rebased-From: f563ce9081
mzumsande pointed out https://github.com/bitcoin/bitcoin/pull/32345#issuecomment-3286964369 that this test was causing a warning:
Warning: Disk space for "/tmp/test_common bitcoin/node_init_tests/init_test/bf78678cb7723a3e84b5/blocks" may not accommodate the block files. Approximately 810 GB of data will be stored in this directory.
Fix by setting regtest instead of mainnet network before running the test.
Github-Pull: #33391
Rebased-From: bdf01c6f61
m_cached_from_me is used to track whether a transaction is "from me", i.e. has
any inputs which belong to the wallet. This is held in memory only in
the same way that a transaction's balances are.
Github-Pull: #33268
Rebased-From: 113a422822
Instead of checking whether the total amount of inputs known by the
wallet is greater than 0, we should be checking for whether the input is
known by the wallet. This enables us to determine whether a transaction
spends an of output with an amount of 0, which is necessary for marking
0-value dust outputs as spent.
Github-Pull: #33268
Rebased-From: 39a7dbdd27
If something is imported into the wallet, it can change the 'from me'
status of a transaction. This status is only visible through
gettransaction's "fee" field which is only shown for transactions that
are 'from me'.
Github-Pull: #33268
Rebased-From: e76c2f7a41
Rather than trying to match the apt installed clang version, which is
prone to intermittent issues. i.e #33345.
Github-Pull: #33364
Rebased-From: b736052e39
d00b82fc96 doc: update manual pages for v30.0rc1 (fanquake)
25f699daa5 contrib: add bitcoin binary to gen-manpages (fanquake)
8578991348 doc: generate example bitcoin.conf (fanquake)
9b75222b5e doc: point to v30.0 release notes draft (fanquake)
e69aba63cd build: bump version to v30.0rc1 (fanquake)
Pull request description:
* Bumps version to `v30.0rc1`.
* Generates example bitcoin.conf.
* Generates the manpages (includes backport of f5887a8de4 from #33348).
* Points release-notes.md to the devwiki (https://github.com/bitcoin-core/bitcoin-devwiki/wiki/v30.0-Release-Notes-Draft).
ACKs for top commit:
hebasto:
ACK d00b82fc96. On Ubuntu 25.04, I've got the same generated files.
janb84:
ACK d00b82fc96
stickies-v:
ACK d00b82fc96 - getting identical manpages and bitcoin.conf output on macos 15.6. Other changes LGTM too.
Tree-SHA512: 7c1cf6442f2380c90d6395d07f75297718bc323a740209efaf2020d7c94598a28c73ab5a638e1fd4ddf2b38cc6aaebe046ea968688f695abf8735b0d9315cd68
9f744fffc3 build: bump CLIENT_VERSION_MAJOR to 30 (fanquake)
Pull request description:
Last step before branch off.
ACKs for top commit:
hebasto:
ACK 9f744fffc3.
Tree-SHA512: f8ddbaa56213707c4d1719a6ade89103bcc1142d71f47cc527a20669995c1598ddbd61a88487841aa794340219e956deed403d8a7c229fc8b526e67e07dd7d69
fa8f081af3 ci: Checkout latest merged pulls (MarcoFalke)
Pull request description:
Currently, the `actions/checkout@v5` checks out pull requests merged against master, which is what we want.
However, it checks out ancient/stale merge commits on a re-run. This is documented (https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs):
> Re-run workflows [...] will also use the same GITHUB_SHA (commit SHA) and GITHUB_REF (git ref) of the original event that triggered the workflow run.
For example:
* https://github.com/bitcoin/bitcoin/actions/runs/17458152407/job/49579638898?pr=29641#step:9:914 compiles with IPC=ON, even though latest master is at ed2ff3c63d
* https://github.com/bitcoin/bitcoin/pull/32989#issuecomment-3133536724 (example explained in comment)
This is problematic, because:
* Unrelated CI failures and intermittent issues, which are fixed or worked around in latest master can not be cleaned by re-running the task. The author has to actively go out and (force-)push the branch, invalidating review.
* It is odd to have a recent CI run, but it uses code and config from the past.
* Detecting silent merge conflicts by re-running the CI task is impossible.
Fix all issues by checking out the latest merged state of the pull request. The behavior is unchanged for non-pull-request actions. This patch changes the "re-run" default behaviour. Forcing it to use the new state instead of running the old state again.
ACKs for top commit:
janb84:
re ACK fa8f081af3
hebasto:
ACK fa8f081af3.
Tree-SHA512: c22c6f837402f61ec46be46817473e1946424b5312e36ed0e246cadb1ca89c04163bb471f71c309765a3d327f198a83cd83679d231f03828a99a97562a622fdd
5eeb2facbb ci: reduce runner sizes on various jobs (will)
Pull request description:
These jobs can likely use reduced runner sizes to avoid wasting our CPU quota, as much of the long-running part of the job is single-threaded.
This will also give us more (job) parallelisem from the same number of CPU that we are using.
Suggested in: https://github.com/bitcoin/bitcoin/pull/32989#discussion_r2321775620
ACKs for top commit:
kevkevinpal:
ACK [5eeb2fa](5eeb2facbb)
m3dwards:
ACK 5eeb2facbb
janb84:
ACK 5eeb2facbb
Tree-SHA512: 6fb0352bc40623dd63b9bd6169d753d1ec9667c272445fda7a2db8bbedfa35350a51d08c1adf3fa5e070e84855c3f491668726d3c7ded07a39f2f9c63edacefc
790b440197 Fix benchmark CSV output (Hennadii Stepanov)
Pull request description:
The `SHA256AutoDetect` return output is used, among other use cases, to name benchmarks. Using a comma breaks the `bench_bitcoin` CSV output.
This PR replaces the comma with a semicolon, which fixes https://github.com/bitcoin/bitcoin/issues/33331.
ACKs for top commit:
Raimo33:
Code Review ACK 790b440197
l0rinc:
Code review ACK 790b440197
janb84:
code review ACK 790b440197
Tree-SHA512: 096bfa29a0639a4d97d510a3e2a15f071f384148c3035e4d0fc525794682e499c45a0d0c95728d5c78010098393b2c486a7fa9c21c1e2fbb600dea7c5638a55f
3cceda9f48 guix: strip binaries in libexec (fanquake)
Pull request description:
#31679 moved some internal binaries to `libexec/`, but the Guix build wasn't updated to stip these binaries of their debug symbols.
ACKs for top commit:
achow101:
ACK 3cceda9f48
ryanofsky:
Code review ACK 3cceda9f48. Good catch and thanks for the fix
hebasto:
ACK 3cceda9f48, I've checked Guix build outputs.
Tree-SHA512: 96ad57c2d3670a9ae8c58cdb428918d1dc0fa90014bb7c6fb7a7a68b3ece3fbea9c4fac90a626a005a0edb3cca8b6a33adc9a037fe6b915319387588ffe09e7b
8b62647680 test: send duplicate blocktxn message in p2p_compactblocks.py (Eugene Siegel)
5e585a0fc4 net: check for empty header before calling FillBlock (Eugene Siegel)
Pull request description:
This avoids an Assume crash if multiple blocktxn messages are received. The first call to `FillBlock` would make the header empty via `SetNull` and the call right before the second `FillBlock` would crash [here](689a321976/src/net_processing.cpp (L3333)) since `LookupBlockIndex` won't find anything. Fix that by checking for an empty header before the Assume.
ACKs for top commit:
instagibbs:
reACK 8b62647680
fjahr:
tACK 8b62647680
achow101:
ACK 8b62647680
mzumsande:
Code Review ACK 8b62647680
Tree-SHA512: d43a6f652161d4f7e6137f207a3e95259fc51509279d20347b1698c91179c39c8fcb75d2668b13a6b220f478a03578573208a415804be1d8843acb057fa1a73a
c767974811 clang-tidy: Fix critical warnings (Fabian Jahr)
54dc34ec22 index: Remove unused coinstatsindex recovery code (Fabian Jahr)
37c4fba1f4 index: Check BIP30 blocks when rewinding Coinstatsindex (Fabian Jahr)
51df9de8e5 doc: Add release note for 30469 (Fabian Jahr)
bb8d673183 test: Add coinstatsindex compatibility test (Fabian Jahr)
b2e8b64ddc index, refactor: Append blocks to coinstatsindex without db read (Fabian Jahr)
431a076ae6 index: Fix coinstatsindex overflow issue (Fabian Jahr)
84e813a02b index, refactor: DRY coinbase check (Fabian Jahr)
fab842b324 index, refactor: Rename ReverseBlock to RevertBlock (Fabian Jahr)
Pull request description:
Closes https://github.com/bitcoin/bitcoin/issues/26362
This continues the work that was started with #26426. It fixes the overflow issue by switching the tracked values that are in danger of overflowing from `CAmount` to `arith_uint256`.
The current approach opts for a simple solution to ensure compatibility with datadirs including the previous version of the index: The new version of the index goes into a separate location in the datadir (`index/coinstatsindex/` rather than `index/coinstats/` before, the new naming is more consistent with the naming of the other indexes). There is no explicit concept of versioning of the index which earlier versions of this PR had. Having the two different versions of the index in separate places allows for downgrading of the node without having to rebuild the index. However, there will be a warning printed in the logs if the new code (v30) detects the old index still being present. A future version could delete a left-over legacy index automatically.
The PR also includes several minor improvements but most notably it lets new entries be calculated and stored without needing to read any DB records.
ACKs for top commit:
achow101:
ACK c767974811
TheCharlatan:
ACK c767974811
mzumsande:
Tested / Code Review ACK c767974811
Tree-SHA512: 3fa4a19dd1a01c1b01390247bc9daa6871eece7c1899eac976e0cc21ede09c79c65f758d14daafc46a43c4ddd7055c85fb28ff03029132d48936b248639c6ab9
Previously in debug builds, this would cause an Assume crash if
FillBlock had been called previously. This could happen when multiple
blocktxn messages were received.
Co-Authored-By: Greg Sanders <gsanders87@gmail.com>
188de70c86 net: Add interrupt to pcp retry loop (TheCharlatan)
Pull request description:
Without this interrupt bitcoind takes a long time to exit if requested to do so after a failed pcp lookup on startup.
ACKs for top commit:
achow101:
ACK 188de70c86
fjahr:
utACK 188de70c86
hodlinator:
utACK 188de70c86
Tree-SHA512: 426dabd10ac0ef5de246c83d281ba70957e4032d251054aa6028b4d7ce4e35cd35ac70e67dc07bd418673bcdd2f4457b76f174ac5e7d0dd3caa05de5da952dac
589b65f06c clang-tidy: Disable `UndefinedBinaryOperatorResult` check in `src/ipc` (Hennadii Stepanov)
Pull request description:
The warnings are false positive and have been fixed upstream. See: https://github.com/capnproto/capnproto/pull/2334.
This PR:
1. Disables the `UndefinedBinaryOperatorResult` clang-tidy check for source files generated by the `mpgen` tool.
2. Is an alternative to the draft https://github.com/bitcoin/bitcoin/pull/33281.
3. Fixes https://github.com/bitcoin/bitcoin/issues/33256.
ACKs for top commit:
Sjors:
ACK 589b65f06c
fjahr:
ACK 589b65f06c
achow101:
ACK 589b65f06c
ryanofsky:
Code review ACK 589b65f06c. Thanks for the fix!
Tree-SHA512: 6d376a82641a5b85d4dd1fa164fdcbd8e15f1262e7d4f582f4d9959031d35852e28ff1b8268336e39ba6779fdd10ecdb986af42407d0545f4217f41d64556272
The `SHA256AutoDetect` return output is used, among other use cases, to
name benchmarks. Using a comma breaks the CSV output.
This change replaces the comma with a semicolon, which fixes the issue.
a2a35b58cb doc: fix LIBRARY_PATH comment (fanquake)
Pull request description:
Now that we build capnp, qt isn't the only native package.
ACKs for top commit:
l0rinc:
code review ACK a2a35b58cb
willcl-ark:
ACK a2a35b58cb
Tree-SHA512: 7b4a54b53b4c673e31dea011e569c6ef9e7238a56a7f09c7f5a7bc521fb0ffdda3794f2b65764908005333a30f7c8c7b4d6b45d985bc92a70dc2daf8c3d1ff3d
The std::move in coinstatsindex was not necessary since it was passed as a const reference argument.
The other change in the utxo supply fuzz test changes a line that seems to have triggered a false alarm.
The coinstatsindex currently looks for block data at a hash key if the prev block in CustomAppend is different than expected. This is not needed since base index should always prevent us ending up in this scenario since it should rewind the index before calling CustomAppend in this case. But even if we run into this and our belt-and-suspenders code is getting hit, the index could not recover properly from the hash key index data so it can be removed without any real impact.
This is practically irrelevant due to the unlikeliness of a re-org
reaching so deep that it would drop the BIP30 blocks from the chain
(91842 and 91880). However this serves as documentation and ensures that
the functions RevertBlock and CustomAppend are consistent.
The index originally stored cumulative values in a CAmount type but this allowed for
potential overflow issues which were observed on Signet. Fix this by
storing the values that are in danger of overflowing in a arith_uint256.
Also turns an unnecessary copy into a reference in RevertBlock and
CustomAppend and gets
rid of the explicit total unspendable tracking which can be calculated
by adding the four categories of unspendables together.
a341e11ac9 ci: test IPC on additional hosts (Sjors Provoost)
6aee573bfc ci: enable IPC tests in CI (Pieter Wuille)
8d2ee88fa2 tests: add functional tests for IPC interface (Pieter Wuille)
3cc9a06c8d test: Add TestNode ipcbind option (Ryan Ofsky)
3cceb60a71 test: Provide path to `bitcoin` binary (Ryan Ofsky)
8c7f005629 test: add is_ipc_compiled() and skip_if_no_ipc() functions (Ryan Ofsky)
Pull request description:
This adds support to the functional test framework to run the multiprocess `bitcoin-node` binary, and then tests it in a new `interface_ipc.py` functional test through the `pycapnp` module.
ACKs for top commit:
Sjors:
ACK a341e11ac9
ryanofsky:
Code review ACK a341e11ac9. Changes since last review: rebasing, switching to miniwallet and expanding wallet test, improving pycapnp install steps in instructions and CI.
TheCharlatan:
ACK a341e11ac9
Tree-SHA512: 98330283cf0d66d5537eec1219345b8aec5740dbc2171bd8b70680d7a282e0962fcdf0588a75518110761e8bc95af57d6097c93937cac710805b0df10837093c
Install pycapnp on all (active) CI hosts which have IPC enabled and
run the functional tests.
Except for previous_releases, which uses an older version of pip
that doesn't support --break-system-packages.
With this change, tests can specify `self.extra_init = [{ipcbind: True}]` to
start a node listening on an IPC socket, instead of needing to choose which
node binary to invoke and what `self.extra_args=[["-ipcbind=..."]]` value to
pass to it.
The eliminates boilerplate code #30437 (interface_ipc_mining.py), #32297
(interface_ipc_cli.py), and #33201 (interface_ipc.py) previously needed in
their test setup.
Set new `BitcoinTestFramework.binary_paths.bitcoin_bin` property with path to
the `bitcoin` wrapper binary. This allows new tests for `bitcoin-mine` in
#30437 and `bitcoin-cli` in #32297 to find the `bitcoin` binary and call
`bitcoin -m` to start nodes with IPC support. This way the new tests can run
whenever the ENABLE_IPC build option is enabled, instead of only running when
the `BITCOIN_CMD` environment variable is set to `bitcoin -m`
61ec8866c6 [doc] archive v29.1 release notes (glozow)
Pull request description:
Copied from https://github.com/bitcoin/bitcoin/blob/v29.1/doc/release-notes.md
This is needed for announcement links and so we can see historical release notes in master.
ACKs for top commit:
l0rinc:
review ACK 61ec8866c6
Tree-SHA512: da9692c8cd8de54e848caab19da41975e8e75049b4fd3e1c6475ee86bf9947132597ceb4bf2e217710a73178b54c05b8f27668c67da202ba5fb1799b582fb15d
c9d5f211c1 depends: strip when installing qt (fanquake)
Pull request description:
Otherwise we end up with ~1.5GB of binaries (Linux) when `DEBUG=1`. This isn't great generally, but is worse in the CI, where disk may be limited (#33293).
ACKs for top commit:
TheCharlatan:
ACK c9d5f211c1
hebasto:
ACK c9d5f211c1.
Tree-SHA512: bf83e0d8c41c64aaa6d841e24c4f25bbe33034ae54a32f34ca14aca59eaa1a004809d48acf171414ed43b99f7d3d1f4b973aee0b272475bd7cc2ca708718b8da
4f1a4cbccd net: Quiet down logging when router doesn't support natpmp/pcp (laanwj)
Pull request description:
When the router doesn't support natpmp and PCP, one'd normally expect the UDP packet to be ignored, and hit a time out. This logs a message that is already in the debug category. However, there's also the case in which sending an UDP packet causes a ICMP response (type 3, code 3 "port unreachable"). This is returned to user space as "connection refused" (despite UDP having no concept of connections).
Move the warnings from `Send` and `Recv` to debug level too, to reduce log spam in that case.
Closes#33301.
ACKs for top commit:
willcl-ark:
utACK 4f1a4cbccd
sipa:
utACK 4f1a4cbccd
davidgumberg:
Tested ACK 4f1a4cbccd
achow101:
ACK 4f1a4cbccd
darosior:
utACK 4f1a4cbccd
mzumsande:
utACK 4f1a4cbccd
Tree-SHA512: 2c99a5679720482ece47af33616b6b207509fb58ba1962a1c2d30f8d0e68554f8f5ef25224313d93f4c5a1cc702183fcf8e6119abc411209c9884119ef680aad
The warnings are false positive and have been fixed upstream.
See: https://github.com/capnproto/capnproto/pull/2334.
This change disables the `UndefinedBinaryOperatorResult` clang-tidy
check for source files generated by the `mpgen` tool.
When the router doesn't support natpmp and PCP, one'd normally expect
the UDP packet to be ignored, and hit a time out. This logs a warning
that is already in the debug category. However, there's also the case in
which sending an UDP packet causes a ICMP response. This is returned to
user space as "connection refused" (despite UDP having no concept of
connections).
Move the warnings from `Send` and `Recv` to debug level too, to reduce
log spam in that case.
Closes#33301.
fae610d858 ci: Remove redundant RUN_UNIT_TESTS_SEQUENTIAL (MarcoFalke)
Pull request description:
`RUN_UNIT_TESTS_SEQUENTIAL` is useful to detect cases where global state is left dirty in the test process and leads to subsequent unit test cases failing. However, one CI task is sufficient to catch this.
As there already is one, add docs there and remove this env var (and extra logic).
ACKs for top commit:
fanquake:
ACK fae610d858
Tree-SHA512: b7ace1257d039f144cb0acb08d5d19d641028464517e6a2468e248ed79b2511512dc904867dacd66157b7483ec8041c95cce00f8ce3c89f3a2c3bb47939d7ff9
88db09bafe net: handle multi-part netlink responses (willcl-ark)
42e99ad773 net: skip non-route netlink responses (willcl-ark)
57ce645f05 net: filter for default routes in netlink responses (willcl-ark)
Pull request description:
...for default route in pcp pinholing.
Currently we only make a single recv call, which trucates results from large routing tables, or in the case the kernel may split the message into multiple responses (which may happen with `NLM_F_DUMP`).
We also do not filter on the default route. For IPv6, this led to selecting the first route with an `RTA_GATEWAY` attribute, often a non-default route instead of the actual default. This caused PCP port mapping failures because the wrong gateway was used.
Fix both issues by adding multi-part handling of responses and filter for the default route.
Limit responses to ~ 1MB to prevent any router-based DoS.
ACKs for top commit:
achow101:
ACK 88db09bafe
davidgumberg:
Code Review re-ACK 88db09b
Sjors:
re-utACK 88db09bafe
Tree-SHA512: ea5948edebfad5896a487a61737aa5af99f529fad3cf3da68dced456266948238a7143383847e79a7bb90134e023eb173c25116d8eb80ff57fa4c4a0377ca1ed
af4156ab75 build: set ENABLE_IPC to OFF when fuzzing (fanquake)
Pull request description:
A `BUILD_FOR_FUZZING` build will currently failure to configure, with missing `capnp`.
ACKs for top commit:
Crypt-iQ:
tACK af4156ab75
marcofleon:
ACK af4156ab75
dergoegge:
utACK af4156ab75
janb84:
ACK af4156ab75
Tree-SHA512: e3c5238cb5823116a958502eab84ee72a94cac0853fc3908ef97b6b6dc037db27806be0726f321d70ab706c37924dec526b46a3a46ea3f3f3684ce48da46a803
Handle multi-part netlink responses to prevent truncated results from
large routing tables.
Previously, we only made a single recv call, which led to incomplete
results when the kernel split the message into multiple responses (which
happens frequently with NLM_F_DUMP).
Also guard against a potential hanging issue where the code would
indefinitely wait for NLMSG_DONE for non-multi-part responses by
detecting the NLM_F_MULTI flag and only continue waiting when necessary.
7270839af4 doc: truc packages allow sub min feerate transactions (Pol Espinasa)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/32067
Some policy documentation is outdated since TRUC. This PR aims to update the documentation to the actual policy state.
ACKs for top commit:
w0xlt:
reACK 7270839af4
glozow:
ACK 7270839af4
Tree-SHA512: 1272e7acc76c76d7e145cdd07827ece31253dba4b99b9a22fc986fcd538830e46392fda877736cb496f3e53a0abcb9d8403d439bb1da63b88da7f8b6f17b6c8b
3c5da69a23 ci: remove un-needed lint_run*.sh files (willcl-ark)
2aa288efdd ci: fix annoying docker warning (will)
dd1c5903e8 ci: add ccache hit-rate warning when < 75% (will)
f427284483 doc: Detail configuration of hosted CI runners (will)
3f339e99e0 ci: dynamically match makejobs with cores (will)
4393ffdd83 ci: remove .cirrus.yml (will)
bc41848d00 ci: port lint (will)
d290a8e6ea ci: port msan-depends (will)
9bbae61e3b ci: port tsan-depends (will)
bf7d536452 ci: port tidy (will)
549074bc64 ci: port centos-depends-gui (will)
58e38c3a04 ci: port previous-releases-depends-debug (will)
341196d75c ci: port fuzzer-address-undefined-integer-nodepends (will)
f2068f26c1 ci: port no-IPC-i686-DEBUG (will)
2a00b12d73 ci: port nowallet-libbitcoinkernel (will)
9c2514de53 ci: port mac-cross-gui-notests (will)
2c990d84a3 ci: force reinstall of kernel headers in asan (will)
884251441b ci: update asan-lsan-ubsan (will)
f253031cb8 ci: port arm 32-bit job (will)
04e7bfbceb ci: update windows-cross job (will)
cc1735d777 ci: add job to determine runner type (will)
020069e6b7 ci: add Cirrus cache host (will)
9c2b96e0d0 ci: have base install run in right dir (will)
18f6be09d0 ci: use docker build cache arg directly (will)
94a0932547 ci: use buildx in ci (will)
fdf64e5532 ci: add configure-docker action (will)
33ba073df7 ci: add REPO_USE_CIRRUS_RUNNERS (will)
b232b0fa5e ci: add caching actions (will)
b8fcc9fcbc ci: add configure environment action (will)
Pull request description:
This changeset migrates all current self-hosted CI jobs over to hosted [Cirrus Runners](https://cirrus-runners.app/).
These runners cost a flat rate of $150/month, and we qualify for an open source discount of 50%. Therefore they are $75/month/runner.
One "runner" should more accurately be thought of in terms of the number of vCPU you are purchasing: https://cirrus-runners.app/pricing/ or in terms of "concurrency", where 1 runners gets you 1.0 concurrency.
e.g. a Linux x86 Runner gets you 16 vCPU (1.0 concurrency) and 64GB RAM to be provisioned as you choose, amongst one or more jobs.
Cirrus Runners currently only support Linux (x86 and Arm64) and MacOS (Arm64).
This changeset does **not** move the existing Github Actions native MacOS runners away from being run on Github's infrastructure. This could be a follow up optimisation.
Runs from this changeset using Cirrus Runners can be found at: https://github.com/testing-cirrus-runners/bitcoin2/actions which shows an uncached run on master ([CI#1](https://github.com/testing-cirrus-runners/bitcoin2/actions/runs/16298637161)), an outside pull request ([CI#3](https://github.com/testing-cirrus-runners/bitcoin2/actions/runs/16303305483?pr=1)) and an updated push to master ([CI#4](https://github.com/testing-cirrus-runners/bitcoin2/actions/runs/16304182527)).
These workflows were run on 10 runners, and we would recommend purchasing a similar number for our CI in this repo to achieve the speed and concurrency we expect.
We include some optional performance commits, but these could be split out and made into followups or dropped entirely.
## Benefits
### Maintenance
As we are not self-hosting, nobody needs to maintain servers, disks etc.
### Bus factor
Currently we have a very small number of people with the know-how working on server setup and maintenance. This setup fixes that so that "anyone" familiar with GitHub-style CI systems can work on it.
### Scaling
These do _not_ "auto-scale"/have "unlimited concurrency" like some solutions, but if we want more workers/cpu to increase parallism or increase the runner size of certain jobs for a speed-up we can simply buy more concurrency using the web interface.
### Speed
Runtimes aproximate current runtimes pretty well, with some jobs being faster.
Caching improvements on pull request (re-runs) are left as future optimisations from the current changeset (see below).
### GitHub workflow syntax
With a migration to the more-commonly-used GitHub workflow syntax, migration to other providers in the future is often as simple as a one-line change (and installing a new GitHub app to the repo).
If we decide to self-host again, then we can also self-host GitHub runners (using https://github.com/actions/runner) and maintain new GH-style CI syntax.
### Reporting
GitHub workflows provide nicer built-in reporting directly on the "Checks" page of a pr. This includes more-detailed action reporting, and a host of pretty nice integrated features, such as [Workflow Commands](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/workflow-commands-for-github-actions) for creating annotations that can print messages during runs. See for example at the bottom of this window where we report `ccache` hitrate, if it was below 90%: https://github.com/testing-cirrus-runners/bitcoin/actions/runs/16163449125?pr=1
These could be added conditionally into our CI scripts to report interesting or other information.
## Costs
### Financial
Relative to competitors Cirrus runners are cheap for the hosted CI-world. However these are likely more expensive than our current setup, or a well-configured (new) self-hosted setup.
If we started with 10 runners to be shared amongst all migrated jobs, this would total $750/mo = $9000/yr.
Note that we are not trying to comptete here on cost directly.
### Dependencies
We would be dependent on Cirrus infra.
## Forks
- Forks should be able to run CI without paid Cirrus runners. This behaviour is achieved through a rather verbose `runs-on:` directive.
- This directive hardcodes the main repo (unfortunately you cannot use the `env` github context in this field in particular, for some reason).
- This directive also allows for a fork to patch the `runs-on:` field in the ci.yml file if they want to use Cirrus Runners too.
- The workflow otherwise will fallback to the GitHub free runners on forks.
- This cirrus cache action transparently falls back to github actions cache when not running on cirrus, so forks will get some free github caching (10GB per repo).
All jobs work on forks, but will run (slowly) on GitHub native free hosted runners, instead of Cirrus runners. They will also suffer from poor cache hit-rates, but there's nothing that can be done about that, and the situtation is an improvement on today.
## Migration process
The main org should also, in addition to pulling code changes:
1. Permit the actions `docker/setup-buildx-action@v3` and `docker/login-action@v3` to be run in this repo.
## Caching
For the number of CI jobs we have, cache usage on GitHub would be an issue as GH only provides 10GB of cache space, **per repo**. However cirrus provides [10 GB per runner](https://cirrus-runners.app/setup/#speeding-up-the-cache), which scales better with the number of runners.
The `cirruslabs/action/[restore|save]` action we use here redirects this to Cirrus' own cache and is both faster and larger.
In the case that user is running CI on a fork, the cirrus cache falls back transparently to GitHub default cache without error.
### ccache, depends-sources, built-depends
- Cached as blobs via `cirruslabs/actions/cache` action.
- Current implementation:
- On `push`: restores and saves caches.
- On `pull_request`: restores but does **not** save caches.
This means a new pull request should hit a _pretty relevant_ cache.
Old pull requests **which are not being rebased on master** may suffer from lower cache hit-rate.
If we save caches on all pull request runs we run the risk of evicting recent (and more relevant) cache blobs.
It may be possible in a future optimisation to widen this to save on pull request runs too, but it will also depend on how many runners we provision and what cache churn rates are like in the main repo.
### Docker build layer caching
- Cached using the `gha` cache backend
- These cache blobs compete for space with `ccache`, `depends-sources` and `depends-built` caches
- `gha` cache allows `--cache-from` to be used from pull requests, which does not work using a registry cache type (technically we could use a public read-only token to get this working, but that feels wrong)
This backend does network i/o and so are marginally slower than our current disk i/o cache.
## But what about... `x`?
We have tested many other providers, including [Runs-on](https://runs-on.com/), [Buildjet](https://buildjet.com/), [WarpBuild](https://www.warpbuild.com/), and GitHub hosted runners (and investigated even more). But they all fall short in one-way or another.
- Runs-On and Buildjet (and others) require installing GH apps with much too-liberal permissions (e.g. `Administration: Read|Write`) for our use-case.
- GitHub hosted runners suffer from all of high costs, lower speed, small cache, and the requirement for a GitHub Teams subscription.
- WarpBuild seems to be simply too expensive.
## TODO:
To complete migration from self-hosted to hosted for this repo, the backport branches `27.x`, `28.x` and `29.x` would also need their CI ported, but these are left for followups to this change (and pending review/changes here first).
-----
Work and experimentation undertaken with m3dwards
ACKs for top commit:
maflcko:
re-ACK 3c5da69a23 🏗
m3dwards:
ACK 3c5da69a23
achow101:
ACK 3c5da69a23
janb84:
re ACK 3c5da69a23
Tree-SHA512: 9f7f2dddf1a5eebc56b4101663283d4219d189cda6054dba760f1288bed9e6ed3f2fa029a5caedc76c31b1271ea0a0cb0967a796086360d8f5be8277379b6397
2885bd0e1c doc: unify `datacarriersize` warning with release notes (Lőrinc)
Pull request description:
Follow-up to https://github.com/bitcoin/bitcoin/pull/32406
---
The [release notes](a189d63618/doc/release-notes-32406.md (L1)) claim
> [...] marked as deprecated and are expected to be removed in a future release
but the [warning itself](2885bd0e1c/src/init.cpp (L907)) claims
> [...] marked as deprecated. They **will** be removed in a future version.
To be less aggressive (since some have objected against this version online) - and to unify the deprecation warning with the release notes - I have changed the warning to communicate our expectation in a friendlier way.
ACKs for top commit:
cedwies:
ACK 2885bd0
ryanofsky:
Code review ACK 2885bd0e1c. I don't think it is good for the release notes and the runtime warning message to say two different things. I'd also be happy if release notes were updated to match the runtime warning, instead of vice versa. Whatever is more accurate is better.
ajtowns:
ACK 2885bd0e1c
kevkevinpal:
ACK [2885bd0](2885bd0e1c)
achow101:
ACK 2885bd0e1c
janb84:
ACK 2885bd0e1c
Zero-1729:
crACK 2885bd0e1c
jonatack:
ACK 2885bd0e1c
hodlinator:
ACK 2885bd0e1c
w0xlt:
ACK 2885bd0e1c
optout21:
ACK 2885bd0e1c
Tree-SHA512: a9d2a64ab96b3dd7f3a1a29622930054fd5c56e573bc96330f4ef3327dc024b21b3fbc8a698d17aea7c76f57f0c2ccd6403b2df344ae2f69c645ceb8b6fa54a5
ci/lint_run.sh: Only used in .cirrus.yml. Refer to test/lint/README.md on how to run locally.
ci/lint_run_all.sh: Only used in .cirrus.yml for stale re-runs of old pull request tasks.
Docker currently warns that we are missing a default value.
Set this to scratch which will error if an appropriate image tag is not
passed in to silence the warning.
Previously jobs were running on a large multi-core server where 10 jobs
as default made sense (or may even have been on the low side).
Using hosted runners with fixed (and lower) numbers of vCPUs we should
adapt compilation to match the number of cpus we have dynamically.
This is cross-platform compatible with macos and linux only.
When using hosted runners in combination with cached docker images,
there is the possibility that the host runner image is updated,
rendering the linux-headers package (stored in the cached docker image)
incompatible.
Fix this by doing a re-install of the headers package in
03_test_script.sh.
If the underlying runner kernel has not changed thie has no effect, but
prevents the job from failing if it has.
To remove multiple occurances of the respository name, against which we
compare `${{ github.repository }}` to check if we should use Cirrus
Runners, introduce a helper job which can check a single environment
variable and output this as an input to subsequent jobs.
Forks can maintain a trivial patch of their repo name against the
`REPO_USE_CIRRUS_RUNNERS` variable in ci.yml if they have Cirrus Runners
of their own, which will then enable cache actions and docker build
cache to use Cirrus Cache.
It's not possible to use `${{ env.USE_CIRRUS_RUNNERS }}` in the
`runs-on:` directive as the context is not supported by GitHub.
If it was, this job would no longer be necessary.
Whilst the action cirruslabs/actions/cache will automatically set this
host, the docker `gha` build cache backend will not be aware of it.
Set the value here, which will later be used in the docker build args to
enable docker build cache on the cirrus cache.
This sets the build dir at build time so that Apple SDK gets installed
in the correct/expected location for the runtime to find it.
Co-authored-by: Max Edwards <youwontforgetthis@gmail.com>
Reverts: e87429a2d0
This was added in PR #31545 with the intention that self-hosted runners
might use it to save build cache.
As we are not using hosted runners with a registry build cache, the bulk
of this commit can be reverted, simply using the value of
$DOCKER_BUILD_CACHE_ARG in the script.
link: https://github.com/bitcoin/bitcoin/pull/31545
Using buildx is required to properly load the correct driver, for use
with registry caching. Neither build, nor BUILDKIT=1 currently do this
properly.
Use of `docker buildx build` is compatible with podman.
Another action to reduce boilerplate in the main ci.yml file.
This action will set up a docker builder compatible with caching build
layers to a container registry using the `gha` build driver.
It will then configure the docker build cache args.
If set, Cirrus runners will be used on pushes to, and pull requests
against, this repository.
Forks can set this if they have their own cirrus runners.
Add "Restore" and "Save" caching actions.
These actions reduce boilerplate in the main ci.yml configuration file.
These actions are implemented so that caches will be saved on `push`
only.
When a pull request is opened it will cache hit on the caches from the
lastest push, or in the case of depends will hit on any matching depends
hash, falling back to partial matches.
Depends caches are hashed using
`$(git ls-tree HEAD depends "ci/test/$FILE_ENV" | sha256sum | cut -d' ' -f1)`
and this hash is passed in as an input to the actions. This means we
direct cache hit in cases where depends would not be re-built, otherwise
falling back to a partial match.
Previous releases cache is hashed similarly to depends, but using the
test/get_previous_releases.py file.
The cirruslabs cache action will fallback transparently to GitHub's
cache in the case that the job is not being run on a Cirrus Runner,
making these compatible with running on forks (on free GH hardware).
b7b249d3ad Revert "[refactor] rewrite vTxHashes as a vector of CTransactionRef" (Anthony Towns)
b9300d8d0a Revert "refactor: Simplify `extra_txn` to be a vec of CTransactionRef instead of a vec of pair<Wtxid, CTransactionRef>" (Anthony Towns)
df5a50e5de bench/blockencodings: add compact block reconstruction benchmark (Anthony Towns)
Pull request description:
Reconstructing compact blocks is on the hot path for block relay, so revert changes from #28391 and #29752 that made it slower. Also add a benchmark to validate reconstruction performance, and a comment giving some background as to the approach.
ACKs for top commit:
achow101:
ACK b7b249d3ad
polespinasa:
lgtm code review and tested ACK b7b249d3ad
cedwies:
code-review ACK b7b249d
davidgumberg:
crACK b7b249d3ad
instagibbs:
ACK b7b249d3ad
Tree-SHA512: dc266e7ac08281a5899fb1d8d0ad43eb4085f8ec42606833832800a568f4a43e3931f942d4dc53cf680af620b7e893e80c9fe9220f83894b4609184b1b3b3b42
493ba0f688 threading: reduce the scope of lock in getblocktemplate (kevkevinpal)
Pull request description:
This change was motivated by https://github.com/bitcoin/bitcoin/pull/32592#discussion_r2294770722
It does exactly what is said in the comment. Reducing the scope of the lock by a bit before it is needed
ACKs for top commit:
stickies-v:
re-ACK 493ba0f688
maflcko:
lgtm ACK 493ba0f688
Tree-SHA512: aa3a21ef3da6be6c0af78aa2dda61ee21c3f6d4d9c897413dba9e7d7d2a91e9e069bbc6b6684b45aadaa28d8603dd310f2c2d2e58c31bb4d864204e468fefaf1
509ffea40a ci: return to using dash in CentOS job (fanquake)
Pull request description:
`dash` is available again: https://bugzilla.redhat.com/show_bug.cgi?id=2335416.
ACKs for top commit:
maflcko:
lgtm ACK 509ffea40a
davidgumberg:
ACK 509ffea40a
janb84:
crACK 509ffea40a
Tree-SHA512: c57194b6158f6453cadb2487be232af5e37aa2234852f04a76fc80909fbfa48c7f8dd30e7be41be67dedb7ec4886930e165fdbaf746d358bb94c6ccc49d6bde6
46ca7712cb threading: remove unused template instantiations (Cory Fields)
b537a6a6db threading: remove obsolete critsect macros (Cory Fields)
0d0e0a39b4 threading: use a reverse lock rather than manual critsect macros (Cory Fields)
3ddd554d31 tests: Add Assertions in reverse_lock tests to exercise thread-safety annotations (Cory Fields)
c88b1cbf57 tests: get rid of remaining manual critsect usage (Cory Fields)
Pull request description:
Now that #32467 is merged, the only remaining usage of our old `CRITICAL_SECTION` macros (other than tests) is in `getblocktemplate()` and it can safely be replaced with a `REVERSE_LOCK`.
This PR makes that replacement, replaces the old `CRITICAL_SECTION` macro usage in tests, then deletes the macros themselves.
~While testing this a few weeks ago, I noticed that `REVERSE_LOCK` does not currently work properly with our thread-safety annotations as after the `REVERSE_LOCK` is acquired, clang still believes that the mutex is locked. #32465 fixes this problem. Without that fix, this PR would potentially allow a false-negative if code were added in the future to this chunk of `getblocktemplate` which required `cs_main` to be locked.~
~I added a test for the reverse lock here in the form of a compiler warning in `reverselock_tests.cpp` to simulate that possibility. This PR will therefore cause a new warning (and should fail a warnings-as-errors ci check) until #32465 is merged and this is rebased on top of it.~
Edit: Rebased on top of #32465, so this should now pass tests.
ACKs for top commit:
maflcko:
review ACK 46ca7712cb📌
fjahr:
Code review ACK 46ca7712cb
TheCharlatan:
ACK 46ca7712cb
furszy:
ACK 46ca7712cb
Tree-SHA512: 5e423c8539ed5ddd784f5c3657bbd63be509d54942c25149f04e3764bcdf897bebf655553338d5af7b8c4f546fc1d4dd4176c2bce6f4683e76ae4bb91ba2ec80
a602f6fb7b test: index with an unclean restart after a reorg (Martin Zumsande)
01b95ac6f4 index: don't commit state in BaseIndex::Rewind (Martin Zumsande)
Pull request description:
The committed state of an index should never be ahead of the flushed chainstate.
Otherwise, in the case of an unclean shutdown, the blocks necessary to revert
from the prematurely committed state are not be available, which would corrupt the coinstatsindex in particular.
Instead, the index state will be committed with the next ChainStateFlushed notification.
Fixes#33208
ACKs for top commit:
achow101:
ACK a602f6fb7b
stickies-v:
re-ACK a602f6fb7b
Tree-SHA512: 2559ea3fe066caf746a54ad7daac5031332f3976848e937c3dc8b35fa2ce925674115d8742458bf3703b3916f04f851c26523b6b94aeb1da651ba5a1b167a419
1c3db0ed8e doc: use new block_to_connect parameter name (stickies-v)
Pull request description:
The parameter name was previously changed from `pblock` to `block_to_connect` in 9ba1fff29e, without updating the documentation.
Addresses https://github.com/bitcoin/bitcoin/pull/33078#discussion_r2279914775.
ACKs for top commit:
purpleKarrot:
ACK 1c3db0ed8e
janb84:
ACK 1c3db0ed8e
musaHaruna:
ACK [1c3db0e](1c3db0ed8e)
Tree-SHA512: 8b12243f1d9e5586e487dd705dc5b40ff12025bb5539eb4195f7fde4df38a9fe8eb0a9570a72f9463a2420f7307358409804fcb23bb73e32ff691ac4ef5bc35a
de65c86572 doc: capnproto instruction for Alpine and Arch (Sjors Provoost)
49d1a1a363 doc: add capnproto-devel to Fedora build instruction (Sjors Provoost)
eab5518913 doc: mark bitcoin-{node,gui} installed in files.md (Sjors Provoost)
2a815d126b doc: link to capnp version bump PR (Sjors Provoost)
Pull request description:
- have `dependencies.md` link to the PR that updated the capnp version: https://github.com/bitcoin/bitcoin/pull/31802#discussion_r2290393840
ACKs for top commit:
maflcko:
lgtm ACK de65c86572
janb84:
re ACK de65c86572
Tree-SHA512: 842d7a89ef18a8c597ca05720c41a72e67500bc93430cf2c3b074cb2f4b936f1df58b5b1e99010e1ea5c1f8a9f8875fb9c20398f915feeacecee9b2fed3cb03c
966666de9a doc: Remove wrong and redundant doxygen tag (MarcoFalke)
Pull request description:
`param@[in]` is not a valid doxygen tag. Also, no other function in this file uses the annotations, and they are redundant with the line above, so just remove them in `feerate` to fix all issues.
In other places, fix them.
ACKs for top commit:
cedwies:
ACK 966666d
janb84:
ACK 966666de9a
pablomartin4btc:
ACK 966666de9a
w0xlt:
ACK 966666de9a
Tree-SHA512: fcb6aa75c0f03b36f3caad023854ba276e0335cf47908a77006e182633b6a68f7b7d3115ef9fb97d143ca23730def05550f970265bb1fde97594ba68e724bde9
7392b8b084 miner: clamp options instead of asserting (Pieter Wuille)
Pull request description:
The `BlockAssembler::ClampOptions` function currently doesn't actually clamp most of the provided settings, but asserts that some are in range. This made sense while it was a purely internal interface.
However, with the mining IPC interface exposed in #30510, these options are now externally accessible, and it is not entirely intuitive how to set them. In particular, calling `Mining::createNewBlock` with a default-constructed `BlockCreateOptions` will right now instantly crash the bitcoin node.
This isn't a security issue, as the IPC interface is considered trusted, but it is highly unexpected I think, and rather unergonomical to have the node crash while developing against the interface.
An alternative would be exposing a way for the interface to return a failure, but I think in this case, just correcting to reasonable values is acceptable.
ACKs for top commit:
Sjors:
ACK 7392b8b084
achow101:
ACK 7392b8b084
stickies-v:
ACK 7392b8b084
ryanofsky:
Code review ACK 7392b8b084. I think ideally this would throw an exception and return a clear error to the caller, or maybe log as stickies suggested, but clamping is much better than crashing.
Tree-SHA512: 7a1e05b68edbf57beb682ee63e27666f42af6a2b70a81874d368a2cb10d107a589e0a388658c1039330b8cc9f6048479870095a9d552ca387a250ac118c1abf2
The committed state of an index should never
be ahead of the flushed chainstate. Otherwise, in the case
of an unclean shutdown, the blocks necessary to revert
from the prematurely committed state would not be
available, which would corrupt the coinstatsindex in particular.
Instead, the index state will be committed with the next
ChainStateFlushed notification.
be776a1443 wallet: Remove isminetype (Ava Chow)
009a69a616 wallet: Remove ISMINE_USED (Ava Chow)
6a7aa01574 wallet: Remove COutput::spendable and AvailableCoinsListUnspent (Ava Chow)
620abe985e interfaces, gui: Remove is_mine output parameter from getAddress (Ava Chow)
Pull request description:
The remaining isminetypes are `ISMINE_SPENDABLE` and `ISMINE_USED`.
`ISMINE_USED` is only used as a filter for caching balances and is never actually returned from `IsMine`. Since we do still want this behavior, This PR changes the caching to utilize bools and explicit members variables to account for the avoid_reuse case. This allows us to remove `ISMINE_USED`.
`ISMINE_SPENDABLE` and `ISMINE_NO` are the only things that are returned by `IsMine`. This is a bool, so it can be replaced as such.
After removing `ISMINE_USED` and `ISMINE_SPENDABLE`, we are able to remove isminetypes altogether.
ACKs for top commit:
murchandamus:
ACK be776a1443
fjahr:
reACK be776a1443
davidgumberg:
crACK be776a1443
enirox001:
re-ACK be776a1
jlest01:
reACK be776a1443
Tree-SHA512: 689759f6a6ba20a1ae988b0c3abacb15424844f29a1ec2fcb2d1ca9d87b44ae68313e8f61d6fd310281b681144f0ade67e90fcfab807e982b52ed99441d9c987
ce7d94a492 doc: add release note (Sjors Provoost)
71f29d4fa9 doc: update build and dependencies docs for IPC (Sjors Provoost)
3cbf747c32 cmake: set ENABLE_IPC by default (Sjors Provoost)
32a90e1b90 ci: use bitcoin-node for one depends job (Sjors Provoost)
b333cc14d5 ci: build one depends job without multiprocess (Sjors Provoost)
16bce9ac4c build: depends makes libmultiprocess by default (Sjors Provoost)
Pull request description:
Have depends make libmultiprocess by default. This PR causes the following behavior changes:
1. **bitcoin-node and bitcoin-gui binaries are included in releases**, due to `ENABLE_IPC` option being switched on by default in depends builds
2. `ENABLE_IPC` is also switched on by default in non-depends builds (instructions updated, #33190 does this as a standalone PR)
3. Various changes to CI: switching on `ENABLE_IPC` on in most configurations and using `bitcoin-node` binary (`bitcoin -m`) for functional tests in two of them.
4. The `bitcoin-node` and `bitcoin-gui` are added to `Maintenance.cmake` (since they're now in the release)
This PR doesn't need to do all of these things at once. However it's is simpler, avoids code churn (especially in CI), and probably less confusing to make all these changes in the same PR.
Windows is not supported yet, so `ENABLE_IPC` is off by default for it. It can be enabled after https://github.com/bitcoin/bitcoin/pull/32387.
The initial main use case for IPC is to enable experimental support for the Mining IPC interface. A working example of a Stratum v2 Template Provider client using this interface can be found here: https://github.com/Sjors/bitcoin/pull/48.
See #31756 for discussion of when this should happen. Supersedes #30975.
## Wait what, why?
The [Stratum v2 spec](https://stratumprotocol.org/specification) has been around for a few years now, mostly stable but with [ongoing activity](https://github.com/stratum-mining/sv2-spec/commits/main/) to clarify and fix more subtle issues encountered by implementers. Most of the implementation is built in Rust in a project called the Stratum Reference Implementation ([SRI](https://github.com/stratum-mining/stratum)).
[Braiins](https://demand.work) added Stratum v2 support to both their (custom) firmware and pool several years ago, though they have fallen behind on recent spec changes (update: it seems they've fixed that). Apparently [new hardware is underway](https://github.com/bitcoin/bitcoin/pull/31802#issuecomment-3189623427) that supports Stratum v2 without the need for custom firmware.
[DMND pool](https://www.dmnd.work) is Stratum v2 native from the start and employs several of the SRI developers (they haven't fully launched though). The industry is rather secretive, but apparently [there is more underway](https://github.com/bitcoin/bitcoin/pull/31802#issuecomment-3190601926).
What does Bitcoin Core have to do with this? Well, in Stratum v2 jargon we are the Template Provider.
Or at least, the Template Provider role needs us to make block templates. Initially back in 2023 the plan was to have Bitcoin Core implement this role entirely, see #23049. It would speak the sv2 encrypted message protocol. In fact the spec was designed around this assumption, making sure to only use cryptographic primitives already in our codebase.
I took over that effort in late 2023, but during 2024 it became quite clear there was [strong resistance](https://github.com/bitcoin/bitcoin/pull/29432#pullrequestreview-2132699185) to the idea of including all this new code, opening another network ports, etc.
At the same time there was the long running multiprocess / IPC project #10102, and the idea was born to apply that here: instead of including Stratum v2 specific stuff, we offer a general Mining interface via an IPC connection that can e.g. push out fresh block templates as fees rise above a threshold (something not possible and/or very inefficient with `getblocktemplate`). A client sidecar application then sits between the Stratum v2 world and our node.
Currently there's only one such sidecar application, maintained by me, and reusing the same codebase from the integrated approach. An attempt has been made to connect to our interface from Rust https://github.com/bitcoin-core/libmultiprocess/issues/174, which would pave the way for SRI include the Template Provider role. Plebhash below indicates he's also working on that: https://github.com/bitcoin/bitcoin/pull/31802#issuecomment-3191547244.
So with this new approach in mind, between mid 2024 until spring 2025, I introduced a new Mining interface (#30200 - #31785). At the same time Russ Ryanosky worked on more tight integration of [libmultiprocess](https://github.com/bitcoin-core/libmultiprocess), including making it a subtree in #31741. See [design/multiprocess.md](https://github.com/bitcoin/bitcoin/blob/master/doc/design/multiprocess.md).
Meanwhile I've been maintaining a fork of Bitcoin Core that includes the Template Provider, in the original integrated approach (https://github.com/Sjors/bitcoin/pull/68) as well as an IPC + sidecar variant (https://github.com/Sjors/bitcoin/pull/48). I've been shipping [regular releases](https://github.com/Sjors/bitcoin/releases), mostly after bug fixes or major rebases. The SRI team has been testing both variants, though the "official" [instruction on their web page](https://stratumprotocol.org/developers) is to stick to integrated version. Bug reports on [my repo fork](https://github.com/Sjors/bitcoin/issues?q=is%3Aissue) as well as on the [SRI repo](https://github.com/stratum-mining/stratum/issues?q=is%3Aissue%20%20label%3A%22template%20provider%22) are evidence of actual testing happening.
But as Pavlenex writes below:
> one recurring feedback I kept getting regardless of the size/type of miner is that the need to run a forked version of Bitcoin Core remains a significant barrier to adoption
This PR gets rids of that significant barrier. People can download a "pristine" version of Bitcoin Core and the only change is to start it with `bitcoin node -m -ipcconnect=unix` instead of the usual `bitcoind`.
Once that's released, I can dramatically simplify my sidecar codebase (https://github.com/Sjors/bitcoin/pull/48) by removing pretty much all Bitcoin Core code that it doesn't need. My plan is to then make that a separate repository, which should be much easier to contribute to. I can then also make (deterministically built) signed releases, while making it clear that sidecar code has nothing to do with Bitcoin Core. Perhaps later on SRI implements the same and I can stop maintaining that project.
Conceptually the situation will be a lot clearer;
- today: download forked version of `bitcoind` (or a forked version of `bitcoin-node`, plus `bitcoin-mine`), install SRI stuff
- tomorrow: download Bitcoin Core v30, install `bitcoin-mine` and SRI
- future: download Bitcoin Core v30 and SRI
<details>
<summary>
Guix hashes:
</summary>
```
find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
6dbf29baecb1d1593087ef1306ae7c78aa160c8beb04dc016e02549ae2d6d90d guix-build-ce7d94a492e6/output/aarch64-linux-gnu/SHA256SUMS.part
4b465e5e8f9652c176aa57cfe5c289267c28c3a3c684034a9ce471b529b95275 guix-build-ce7d94a492e6/output/aarch64-linux-gnu/bitcoin-ce7d94a492e6-aarch64-linux-gnu-debug.tar.gz
85bc6fa008b83419d96443d9dcc212b46f0992387fd58fd2dda5da76536ee22c guix-build-ce7d94a492e6/output/aarch64-linux-gnu/bitcoin-ce7d94a492e6-aarch64-linux-gnu.tar.gz
5ed9ea52a8bd55361d2d9c01fbd1b25ec9970530c2776e6c1959424ba1689f52 guix-build-ce7d94a492e6/output/arm-linux-gnueabihf/SHA256SUMS.part
2e483011fac64462d3aa000b577c3c05c825506032d879e39612e096d7a6c65b guix-build-ce7d94a492e6/output/arm-linux-gnueabihf/bitcoin-ce7d94a492e6-arm-linux-gnueabihf-debug.tar.gz
7ff1e3ba54944a2be89dd7d68cb91dff6f8950de9d7b521e15dfb746965f81bd guix-build-ce7d94a492e6/output/arm-linux-gnueabihf/bitcoin-ce7d94a492e6-arm-linux-gnueabihf.tar.gz
abdf89e701b21b8c1238a8cec46aeaa55e0c3a0b88ad718636e89cde9813ca08 guix-build-ce7d94a492e6/output/arm64-apple-darwin/SHA256SUMS.part
fb55cff0296cd5474811fe5cedcf28603628729dd085eeefa007c72582459b33 guix-build-ce7d94a492e6/output/arm64-apple-darwin/bitcoin-ce7d94a492e6-arm64-apple-darwin-codesigning.tar.gz
e9aa566b1e79c467d7987b7c68fa608db788e6ddf89c4d90e524cd47b4faaf86 guix-build-ce7d94a492e6/output/arm64-apple-darwin/bitcoin-ce7d94a492e6-arm64-apple-darwin-unsigned.tar.gz
bb428fc62a1230a55f49fa3b5c7ba8d588e8fed491357f890d5a6724a38b14e9 guix-build-ce7d94a492e6/output/arm64-apple-darwin/bitcoin-ce7d94a492e6-arm64-apple-darwin-unsigned.zip
5ef4b75e94b2c8265fbc588bbb42467a84438af969fddac0ea61ced3e4113345 guix-build-ce7d94a492e6/output/dist-archive/bitcoin-ce7d94a492e6.tar.gz
4f55d56a108c8f312a502cd5dfdf0840b091861a6d502df31caf4636a203697a guix-build-ce7d94a492e6/output/powerpc64-linux-gnu/SHA256SUMS.part
66c5b1242c60e37098885a00e24efe19baee4afcd2e3d6407207523d8872f055 guix-build-ce7d94a492e6/output/powerpc64-linux-gnu/bitcoin-ce7d94a492e6-powerpc64-linux-gnu-debug.tar.gz
d9dbbee7217544eda26e77158cd82caeaef2b40fb9fc7033323e7ffe64264109 guix-build-ce7d94a492e6/output/powerpc64-linux-gnu/bitcoin-ce7d94a492e6-powerpc64-linux-gnu.tar.gz
d9b808cc5685c819abcebb4ace65f003ebc4bfedf3fca046b34de37994358782 guix-build-ce7d94a492e6/output/riscv64-linux-gnu/SHA256SUMS.part
eeeea470b1cf76515bfae14c779a3ea356d89f719d1fef1a81e8f0d6b04ab747 guix-build-ce7d94a492e6/output/riscv64-linux-gnu/bitcoin-ce7d94a492e6-riscv64-linux-gnu-debug.tar.gz
9993da4eb51618b8bd25ec88cc576496720e5589315e9eba6f3ddab25f9c3e60 guix-build-ce7d94a492e6/output/riscv64-linux-gnu/bitcoin-ce7d94a492e6-riscv64-linux-gnu.tar.gz
1b5a676580e0e79598d182f6ebbb05fb8aee2381edc3c09c042cae2600f448ab guix-build-ce7d94a492e6/output/x86_64-apple-darwin/SHA256SUMS.part
9152122d95a34d5df75305c6883c87707e7b09033fffd08e264d703ed177ef12 guix-build-ce7d94a492e6/output/x86_64-apple-darwin/bitcoin-ce7d94a492e6-x86_64-apple-darwin-codesigning.tar.gz
2793f75730dbef6bdf12b5ed7135e22ed21178abff2926dee92843837d4ab544 guix-build-ce7d94a492e6/output/x86_64-apple-darwin/bitcoin-ce7d94a492e6-x86_64-apple-darwin-unsigned.tar.gz
e89aafd7e4a330a41f470e8f0a91ea876fad7d19547b404600867413f1a8ccb7 guix-build-ce7d94a492e6/output/x86_64-apple-darwin/bitcoin-ce7d94a492e6-x86_64-apple-darwin-unsigned.zip
955b27f881927a86da3c566357ad8ca68dbe17e9652bde8c482a57ceacba92cb guix-build-ce7d94a492e6/output/x86_64-linux-gnu/SHA256SUMS.part
fd012be97bdf5c75ac12ddef21526bfdb5e17ecc77cde9c34d832194b0dc3293 guix-build-ce7d94a492e6/output/x86_64-linux-gnu/bitcoin-ce7d94a492e6-x86_64-linux-gnu-debug.tar.gz
0ecf7f80e9049369760d0e27fe6c026391ab25eae0f42336bef43e51a2621726 guix-build-ce7d94a492e6/output/x86_64-linux-gnu/bitcoin-ce7d94a492e6-x86_64-linux-gnu.tar.gz
2e8085f5fecc246d841b0bf6f28ecd0684a6cee49252fc88c1019d7586c7b7a2 guix-build-ce7d94a492e6/output/x86_64-w64-mingw32/SHA256SUMS.part
c60041e8137eda352557254c5f67fb83eeb97ecfec342ee528451bd44ee4523a guix-build-ce7d94a492e6/output/x86_64-w64-mingw32/bitcoin-ce7d94a492e6-win64-codesigning.tar.gz
b1be6b2f4de1c69c2e0e4de6dd97a4891ae9eb50d89435ef47247b5a187915a9 guix-build-ce7d94a492e6/output/x86_64-w64-mingw32/bitcoin-ce7d94a492e6-win64-debug.zip
bfe143f41a20c537145c7044aca889b28efe19072b0150042a3bd865983b3d7e guix-build-ce7d94a492e6/output/x86_64-w64-mingw32/bitcoin-ce7d94a492e6-win64-setup-unsigned.exe
94a906b83d84db7b25f7e3cfdce2a2030243f2ee6cc70b1fc088459f0b2f382d guix-build-ce7d94a492e6/output/x86_64-w64-mingw32/bitcoin-ce7d94a492e6-win64-unsigned.zip
```
</details>
ACKs for top commit:
ryanofsky:
Code review ACK ce7d94a492. This was just rebased to fix a conflict since last review.
josibake:
ACK ce7d94a492
achow101:
ACK ce7d94a492
ismaelsadeeq:
ACK ce7d94a492 and tested again on macOS by building via depends and source.
janb84:
ACK ce7d94a492
Tree-SHA512: f7ab72933854e9dfce5746cdf764944bc26eec815f97cd0aa6b54fa499c3cccb1b678861ef5c1c793de28153d46bbb6e4d1b9aa0652163b74262e2d55ec8b813
1d9f1cb4bd kernel: improve BlockChecked ownership semantics (stickies-v)
9ba1fff29e kernel: refactor: ConnectTip to pass block pointer by value (stickies-v)
Pull request description:
Subscribers to the BlockChecked validation interface event may need access to the block outside of the callback scope. Currently, this is only possible by copying the block, which makes exposing this validation interface event publicly either cumbersome or with significant copy overhead.
By using shared_ptr, we make the shared ownership explicit and allow users to safely use the block outside of the callback scope. By using a const-ref shared_ptr, no atomic reference count cost is incurred if a subscriber does not require block ownership.
For example: in #30595, this would allow us to drop the `kernel_BlockPointer` handle entirely, and generalize everything into `kernel_Block`. This PoC is implemented in https://github.com/stickies-v/bitcoin/commits/kernel/remove-blockpointer/.
---
### Performance
I have added a benchmark in a [separate branch](https://github.com/stickies-v/bitcoin/commits/2025-07/validation-interface-ownership-benched/), to ensure this change does not lead to a problematic performance regression. Since most of the overhead comes from the subscribers, I have added scenarios for `One`, `Two`, and `Ten` subscribers. From these results, it appears there is no meaningful performance difference on my machine.
When `BlockChecked()` takes a `const CBlock&` reference _(master)_:
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 170.09 | 5,879,308.26 | 0.3% | 0.01 | `BlockCheckedOne`
| 1,603.95 | 623,460.10 | 0.5% | 0.01 | `BlockCheckedTen`
| 336.00 | 2,976,173.37 | 1.1% | 0.01 | `BlockCheckedTwo`
When `BlockChecked()` takes a `const std::shared_ptr<const CBlock>&` _(this PR)_:
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 172.20 | 5,807,155.33 | 0.1% | 0.01 | `BlockCheckedOne`
| 1,596.79 | 626,254.52 | 0.0% | 0.01 | `BlockCheckedTen`
| 333.38 | 2,999,603.17 | 0.3% | 0.01 | `BlockCheckedTwo`
ACKs for top commit:
achow101:
ACK 1d9f1cb4bd
w0xlt:
reACK 1d9f1cb4bd
ryanofsky:
Code review ACK 1d9f1cb4bd. These all seem like simple changes that make sense
TheCharlatan:
ACK 1d9f1cb4bd
yuvicc:
Code Review ACK 1d9f1cb4bd
Tree-SHA512: 7ed0cccb7883dbb1885917ef749ab7cae5d60ee803b7e3145b2954d885e81ba8c9d5ab1edb9694ce6b308235c621094c029024eaf99f1aab1b47311c40958095
cb173b8e93 test: use local `CBlockIndex` in block read hash mismatch test to avoid data race (Lőrinc)
Pull request description:
Avoid mutating the shared active tip `CBlockIndex` in the `blockmanager_readblock_hash_mismatch` test.
Instead, construct a local `CBlockIndex` with only the required fields set, ensuring the test remains self-contained and hopefully eliminating the data race reported in https://github.com/bitcoin/bitcoin/issues/33150.
ACKs for top commit:
stickies-v:
ACK cb173b8e93
maflcko:
lgtm ACK cb173b8e93
Tree-SHA512: 790528db0659f8cc5b87ed2b316bf274af68edc6158b0ce8821baccddf8d9bc4074afcb7260e3a61d5013d24ab51cc5c31e36693b8fb5ab913a44229fd6ad36b
0df2c3c42e qt: Update `src/qt/locale/bitcoin_en.xlf` translation source file (Hennadii Stepanov)
Pull request description:
This PR follows our [Release Process](53a996f122/doc/release-process.md).
It is required for the translation string freeze, as the following PRs introduced new translatable strings:
- #31296:7b4a1350df/src/wallet/wallet.h (L945)
- #31453:7b4a1350df/src/init.cpp (L1878-L1879)
- #32896:22e689587a/src/wallet/spend.cpp (L288-L292)
**Notes for reviewers:**
1. To reproduce the diff, run:
```
cmake --preset dev-mode
cmake --build build_dev_mode --target translate
```
2. The structure of `bitconstrings.cpp` has been altered due to #33209.
3. The diff in `bitcoin_en.xlf` contains many unrelated metadata changes, so it may be easier to verify the changes in `bitcoin_en.ts`.
ACKs for top commit:
janb84:
re ACK 0df2c3c42e
Tree-SHA512: be87c096ef99ce7148d046f30427bc1480cb72b080eb8537a4eda3dfe4e856eeaa50cf6efb9a1c6af3d15e1123ec87a07101c539c066a8d4dd6afb817cd95137
5dda364c4b test: modify logging_filesize_rate_limit params (Eugene Siegel)
Pull request description:
Change `time_window` from 20s to 1h so `Reset` is not accidentally called if the test takes a while.
Change `num_lines` from 1024 to 10 since `LogRateLimiter` is parameterized and does not require logging 1MiB of data.
Fixes#33195
ACKs for top commit:
stickies-v:
re-ACK 5dda364c4b for more helpful failure logging, no other changes
janb84:
re ACK 5dda364c4b
dergoegge:
utACK 5dda364c4b
Tree-SHA512: f781402a3a47abc26314ee7cdf6c74e77da9b9d0dde44ba52e3c42f6c400830147554d7875e7d1217a2a378383e56d87e9712c84e877bb448112f703b87a52b1
Since the only remaining isminetypes are ISMINE_NO and ISMINE_SPENDABLE,
this enum is now just a bool and can be removed. IsMine is changed to
return a bool and any usage of isminetypes and isminefilters are changed
to be the remaining ISMINE_SPENDABLE case.
This isminetype is not a real isminetype as it is never returned by
IsMine. This is only used for isminefilters in one function, which can
be better represented with a bool parameter avoid_reuse.
In descriptor wallets, we consider all outputs to be spendable as we no
longer have mixed mine and watchonly in a wallet. As such,
COutput::spendable is meaningless and can be removed.
Furthermore, CoinFilterParams::only_spendable can be removed as that was
essentially checking for COutput::spendable.
Lastly, AvailableCoinsListUnspent can also be removed as the wrapper is
now only setting the feerate to std::nullopt which is trivial enough that
a dedicated wrapper is not needed.
3aef38f44b test: exercise index reorg assertion failure (furszy)
acf50233cd index: fix wrong assert of current_tip == m_best_block_index (Hao Xu)
Pull request description:
In BaseIndex::Sync(), pindex in `Rewind(pindex, pindex_next->pprev)` isn't always equal to m_best_block_index since m_best_block_index is updated every SYNC_LOCATOR_WRITE_INTERVAL seconds, during which multiple pindex update could happen. Thus the assert here is wrong.
ACKs for top commit:
achow101:
ACK 3aef38f44b
furszy:
ACK 3aef38f
mzumsande:
Code Review ACK 3aef38f44b
Tree-SHA512: 3ef9cc6dfdec10a9f95d7414c6a11aa216e4cf5974440d80ab19fc919abd2a3bd4c875718c9dc94523c33826f8582ec5a016374deb8fb2d35cd2fb7799b5c82e
Change time_window from 20s to 1h so Reset is not accidentally called
if the test takes a while.
Change num_lines from 1024 to 10 since LogRateLimiter is parameterized
and does not require logging 1MiB of data.
Co-Authored-By: stickies-v <stickies-v@protonmail.com>
3c4a109aa8 cmake: Drop python dependency for translate (Daniel Pfeifer)
Pull request description:
Translate the `share/qt/extract_strings_qt.py` script to CMake. This removes the python dependency from the `translate` target.
Resolves#33146
ACKs for top commit:
hebasto:
re-ACK 3c4a109aa8.
janb84:
re ACK 3c4a109aa8
Tree-SHA512: 4fda8efd4301c49eef8bf2908073475fcff3f995cf6860187f8d08821559612303b303052c1e54a01ad31703fe63aea01e999d08f5471f2c479c97de8c240605
This change avoids hardcoding the "debug.log" filename, ensuring
compatibility with custom filenames provided via the `-debuglogfile`
command-line option.
5c8bf7b39e doc: add release notes for version 3 transactions (ishaanam)
4ef8065a5e test: add truc wallet tests (ishaanam)
5d932e14db test: extract `bulk_vout` from `bulk_tx` so it can be used by wallet tests (ishaanam)
2cb473d9f2 rpc: Support version 3 transaction creation (Bue-von-hon)
4c20343b4d rpc: Add transaction min standard version parameter (Bue-von-hon)
c5a2d08011 wallet: don't return utxos from multiple truc txs in AvailableCoins (ishaanam)
da8748ad62 wallet: limit v3 tx weight in coin selection (ishaanam)
85c5410615 wallet: mark unconfirmed v3 siblings as mempool conflicts (ishaanam)
0804fc3cb1 wallet: throw error at conflicting tx versions in pre-selected inputs (ishaanam)
cc155226fe wallet: set m_version in coin control to default value (ishaanam)
2e9617664e wallet: don't include unconfirmed v3 txs with children in available coins (ishaanam)
ec2676becd wallet: unconfirmed ancestors and descendants are always truc (ishaanam)
Pull request description:
This PR Implements the following:
- If creating a v3 transaction, `AvailableCoins` doesn't return unconfirmed v2 utxos (and vice versa)
- `AvailableCoins` doesn't return an unconfirmed v3 utxo if its transaction already has a child
- If a v3 transaction is kicked out of the mempool by a sibling, mark the sibling as a mempool conflict
- Throw an error if pre-selected inputs are of the wrong transaction version
- Allow setting version to 3 manually in `createrawtransaction` (uses commits from #31936)
- Limits a v3 transaction weight in coin selection
Closes#31348
To-Do:
- [x] Test a v3 sibling conflict kicking out one of our transactions from the mempool
- [x] Implement separate size limit for TRUC children
- [x] Test that we can't fund a v2 transaction when everything is v3 unconfirmed
- [x] Test a v3 sibling conflict being removed from the mempool
- [x] Test limiting v3 transaction weight in coin selection
- [x] Simplify tests
- [x] Add documentation
- [x] Test that user-input max weight is not overwritten by truc max weight
- [x] Test v3 in RPCs other than `createrawtransaction`
ACKs for top commit:
glozow:
reACK 5c8bf7b39e
achow101:
ACK 5c8bf7b39e
rkrux:
ACK 5c8bf7b39e
Tree-SHA512: da8aea51c113e193dd0b442eff765bd6b8dc0e5066272d3e52190a223c903f48788795f32c554f268af0d2607b5b8c3985c648879cb176c65540837c05d0abb5
05255d5d1e cmake: Drop dependency on sed for translate target (Daniel Pfeifer)
d5054beca5 cmake: Introduce translate.cmake script for translate target (Daniel Pfeifer)
Pull request description:
Using `file(GLOB)` in the generates step is discouraged because the globbing result may be out of date when the target is built. Performing the globbing in a script that is executed as the build target means the result is always reproducable and the overhead of globbing is only paid when used.
As a follow up, the dependency on `sed` may be removed by performing the replacement with cmake. Also, the logic from extract_strings_qt.py can be migrated to cmake.
ACKs for top commit:
hebasto:
ACK 05255d5d1e.
Tree-SHA512: ae55d9199e6294109b37e5e18f21f2b0e582c1f9903421cf22a237cfdbd215cc431706563b3caa03068cdba79f936b019526638fe3a1f83b4f01a72817e39be1
Using `file(GLOB)` in the generates step is discouraged because the
globbing result may be out of date when the target is built.
Performing the globbing in a script that is executed as the build
target means the result is always reproducable and the overhead
of globbing is only paid when used.
As a follow up, the dependency on `sed` may be removed by performing
the replacement with cmake. Also, the logic from extract_strings_qt.py
can be migrated to cmake.
60d1042b9a wallet: Remove unused `WalletFeature` enums (woltx)
66de58208a wallet: Remove `CWallet::nWalletVersion` and related functions (woltx)
7cda3d0f5b wallet: Remove `IsFeatureSupported()` and `CanSupportFeature()` (woltx)
ba01585229 wallet: `MigrateToDescriptor` no longer calls `CanSupportFeature` (woltx)
63acee2797 wallet: Remove `GetClosestWalletFeature()` (woltx)
e27da3150b wallet: Remove `GetVersion()` (woltx)
Pull request description:
This PR incorporates the suggestion provided by PRabahy and pablomartin4btc in https://github.com/bitcoin/bitcoin/pull/32944 of removing `CWallet::nWalletVersion` and several related functions, such as `SetMinVersion()`, `GetVersion()`, `GetClosestWalletFeature()`, `IsFeatureSupported()`, `CanSupportFeature()`, etc ...
This field is no longer used in the descriptor wallet and there is still a lot of code related to it, so the changes here provide a good cleanup in the wallet code.
Built on top of https://github.com/bitcoin/bitcoin/pull/32944
ACKs for top commit:
maflcko:
review ACK 60d1042b9a🐾
achow101:
ACK 60d1042b9a
pablomartin4btc:
ACK 60d1042b9a
Tree-SHA512: 1a7ad8e15d57df8f66545776e7d178a2cd5312c87769a29770588375e3de5f24247aab9919acf004ed3eca16d08ba595b5f1c7b2b3eef7752e89d9c295624583
fab2980bdc assumevalid: log every script validation state change (Lőrinc)
Pull request description:
The `-assumevalid` option skips script verification for a specified block and all its ancestors during Initial Block Download.
Many new [users are surprised](https://github.com/bitcoin/bitcoin/issues/32832) when this suddenly slows their node to a halt.
This commit adds a log message to clearly indicate when this optimization ends and full validation begins (and vice versa).
<details>
<summary>Testing instructions</summary>
The behavior can easily be tested by adding this before the new log:
```C++
// TODO hack to enable/disable script checks based on block height for testing purposes
if (pindex->nHeight < 100) fScriptChecks = false;
else if (pindex->nHeight < 200) fScriptChecks = true;
else if (pindex->nHeight < 300) fScriptChecks = false;
else if (pindex->nHeight < 400) fScriptChecks = true;
```
and exercise the new code with:
```bash
cmake -B build && cmake --build build && mkdir -p demo && build/bin/bitcoind -datadir=demo -stopatheight=500 | grep 'signature validation'
```
showing something like:
* Disabling signature validations at block #1 (00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048).
* Enabling signature validations at block #100 (000000007bc154e0fa7ea32218a72fe2c1bb9f86cf8c9ebf9a715ed27fdb229a).
* Disabling signature validations at block #200 (000000008f1a7008320c16b8402b7f11e82951f44ca2663caf6860ab2eeef320).
* Enabling signature validations at block #300 (0000000062b69e4a2c3312a5782d7798b0711e9ebac065cd5d19f946439f8609).
</details>
ACKs for top commit:
achow101:
ACK fab2980bdc
ajtowns:
crACK fab2980bdc
davidgumberg:
untested crACK fab2980bdc
Tree-SHA512: e90b66f7423b639356daace476942ce83e65e70466544394cbe2f15738bdbf716163eaf590c64c5448f9b41aeeaafe3342c48c6a7a478678a70b0310ca94e11d
In future commits we assume that coin_control.m_version has a
value when making sure that we follow truc rules, so we should
give it a default value of CTransaction::CURRENT_VERSION.
c0d91fc69c Add release note for #33050 and #33183 error string changes (Antoine Poinsot)
b3f781a0ef contrib: adapt max reject string size in tracing demo (Antoine Poinsot)
9a04635432 scripted-diff: validation: rename mandatory errors into block errors (Antoine Poinsot)
Pull request description:
This is a followup to #33050 now that it's merged. Using "block"/"mempool" as the error reason is clearer to a user than "mandatory"/"non-mandatory". The "non-mandatory" errors got renamed to "mempool" in #33050 (see https://github.com/bitcoin/bitcoin/pull/33050#discussion_r2230103371). This takes care of the second part of the renaming.
ACKs for top commit:
fjahr:
utACK c0d91fc69c
davidgumberg:
lgtm ACK c0d91fc69c
ajtowns:
utACK c0d91fc69c
Crypt-iQ:
utACK c0d91fc69c
janb84:
utACK c0d91fc69c
instagibbs:
ACK c0d91fc69c
Tree-SHA512: b463e633c57dd1eae7c49d23239a59066a672f355142ec194982eddc927a7646bc5cde583dc8d6f45075bf5cbb96dbe73f7e339e728929b0eff356b674d1b68c
ba84a25dee [doc] update mempool-replacements.md for incremental relay feerate change (glozow)
18720bc5d5 [doc] release note for min feerate changes (glozow)
6da5de58ca [policy] lower default minrelaytxfee and incrementalrelayfee to 100sat/kvB (glozow)
2e515d2897 [prep/test] make wallet_fundrawtransaction's minrelaytxfee assumption explicit (glozow)
457cfb61b5 [prep/util] help MockMempoolMinFee handle more precise feerates (glozow)
3eab8b7240 [prep/test] replace magic number 1000 with respective feerate vars (glozow)
5f2df0ef78 [miner] lower default -blockmintxfee to 1sat/kvB (glozow)
d6213d6aa1 [doc] assert that default min relay feerate and incremental are the same (glozow)
1fbee5d7b6 [test] explicitly check default -minrelaytxfee and -incrementalrelayfee (glozow)
72dc18467d [test] RBF rule 4 for various incrementalrelayfee settings (glozow)
85f498893f [test] check bypass of minrelay for various minrelaytxfee settings (glozow)
e5f896bb1f [test] check miner doesn't select 0fee transactions (glozow)
Pull request description:
ML post for discussion about the general concept, how this impacts the wider ecosystem, philosophy about minimum feerates, etc: https://delvingbitcoin.org/t/changing-the-minimum-relay-feerate/1886
This PR is inspired by #13922 and #32959 to lower the minimum relay feerate in response to bitcoin's exchange rate changes in the last ~10 years. It lowers the default `-minrelaytxfee` and `-incrementalrelayfee`, and knocks `-blockmintxfee` down to the minimum nonzero setting. Also adds some tests for the settings and pulls in #32750.
The minimum relay feerate is a DoS protection rule, representing a price on the network bandwidth used to relay transactions that have no PoW. While relay nodes don't all collect fees, the assumption is that if nodes on the network use their resources to relay this transaction, it will reach a miner and the attacker's money will be spent once it is mined. The incremental relay feerate is similar: it's used to price the relay of replacement transactions (the additional fees need to cover the new transactions at this feerate) and evicted transactions (following a trim, the new mempool minimum feerate is the package feerate of what was removed + incremental).
Also note that many nodes on the network have elected to relay/mine lower feerate transactions. Miners (some say up to 85%) are choosing to mine these low feerate transactions instead of leaving block space unfilled, but these blocks have extremely poor compact block reconstruction rates with nodes that rejected or didn't hear about those transactions earlier.
- https://github.com/bitcoin/bitcoin/pull/33106#issuecomment-3155627414
- https://x.com/caesrcd/status/1947022514267230302
- https://mempool.space/block/00000000000000000001305770e0aa279dcd8ba8be18c3d5cf736a26f77e06fd
- https://mempool.space/block/00000000000000000001b491649ec030aa8e003e1f4f9d3b24bb99ba16f91e97
- https://x.com/mononautical/status/1949452586391855121
While it wouldn't make sense to loosen DoS restrictions recklessly in response to these events, I think the current price is higher than necessary, and this motivates us changing the default soon. Since the minimum relay feerate defines an amount as too small based on what it costs the attacker, it makes sense to consider BTC's conversion rate to what resources you can buy in the "real world."
Going off of [this comment](https://github.com/bitcoin/bitcoin/pull/32959#issuecomment-3095260286) and [this comment](https://github.com/bitcoin/bitcoin/pull/33106#issuecomment-3142444090)
- Let's say an attacker wants to use/exhaust the network's bandwidth, and has the choice between renting resources from a commercial provider and getting the network to "spam" itself it by sending unconfirmed transactions. We'd like the latter to be more expensive than the former.
- The bandwidth for relaying a transaction across the network is roughly its serialized size (plus relay overhead) x number of nodes. A 1000vB transaction is 1000-4000B serialized. With 100k nodes, that's 0.1-0.4GB
- If the going rate for ec2 bandwidth is 10c/GB, that's like 1-4c per kvB of transaction data
- Then a 1000vB transaction should pay at least 4c
- $0.04 USD is 40 satoshis at 100k USD/BTC
- Baking in some margin for changes in USD/BTC conversion rate, number of nodes (and thus bandwidth), and commercial service costs, I think 50-100 satoshis is on the conservative end but in the right ballpark
- At least 97% of the recent sub-1sat/vB transactions would be accepted with a new threshold of 0.1sat/vB: https://github.com/bitcoin/bitcoin/pull/33106#issuecomment-3156213089
List of feerates that are changed and why:
- min relay feerate: significant conversion rate changes, see above
- incremental relay feerate: should follow min relay feerate, see above
- block minimum feerate: shouldn’t be above min relay feerate, otherwise the node accepts transactions it will never mine. I've knocked it down to the bare minimum of 1sat/kvB. Now that we no longer have coin age priority (removed in v0.15), I think we can leave it to the `CheckFeeRate` policy rule to enforce a minimum entry price, and the block assembly code should just fill up the block with whatever it finds in mempool.
List of feerates that are not changed and why:
- dust feerate: this feerate cannot be changed as flexibly as the minrelay feerate. A much longer record of low feerate transactions being mined is needed to motivate a decrease there.
- maxfeerate (RPC, wallet): I think the conversion rate is relevant as well, but out of scope for this PR
- minimum feerate returned by fee estimator: should be done later. In the past, we've excluded new policy defaults from fee estimation until we feel confident they represent miner policy (e.g. #9519). Also, the fee estimator itself doesn't have support for sub-1sat/vB yet.
- all wallet feerates (mintxfee, fallbackfee, discardfee, consolidatefeerate, WALLET_INCREMENTAL_RELAY_FEE, etc.): should be done later. Our standard procedure is to do wallet changes at least 1 release after policy changes.
ACKs for top commit:
achow101:
ACK ba84a25dee
gmaxwell:
ACK ba84a25dee
jsarenik:
Tested ACK ba84a25dee
darosior:
ACK ba84a25dee
ajtowns:
ACK ba84a25dee
davidgumberg:
crACK ba84a25dee
w0xlt:
ACK ba84a25dee
caesrcd:
reACK ba84a25dee
ismaelsadeeq:
re-ACK ba84a25dee
Tree-SHA512: b4c35e8b506b1184db466551a7e2e48bb1e535972a8dbcaa145ce3a8bfdcc70a8807dc129460f129a9d31024174d34077154a387c32f1a3e6831f6fa5e9c399e
5c74a0b397 config: add DEBUG_ONLY -logratelimit (Eugene Siegel)
9f3b017bcc test: logging_filesize_rate_limit improvements (stickies-v)
350193e5e2 test: don't leak log category mask across tests (stickies-v)
05d7c22479 test: add ReadDebugLogLines helper function (stickies-v)
3d630c2544 log: make m_limiter a shared_ptr (stickies-v)
e8f9c37a3b log: clean up LogPrintStr_ and Reset, prefix all logs with "[*]" when there are suppressions (Eugene Siegel)
3c7cae49b6 log: change LogLimitStats to struct LogRateLimiter::Stats (Eugene Siegel)
8319a13468 log: clarify RATELIMIT_MAX_BYTES comment, use RATELIMIT_WINDOW (Eugene Siegel)
5f70bc80df log: remove const qualifier from arguments in LogPrintFormatInternal (Eugene Siegel)
b8e92fb3d4 log: avoid double hashing in SourceLocationHasher (Eugene Siegel)
616bc22f13 test: remove noexcept(false) comment in ~DebugLogHelper (Eugene Siegel)
Pull request description:
Followups to #32604.
There are two behavior changes:
- prefixing with `[*]` is done to all logs (regardless of `should_ratelimit`) per [this comment](https://github.com/bitcoin/bitcoin/pull/32604#discussion_r2195710943).
- a DEBUG_ONLY `-disableratelimitlogging` flag is added by default to functional tests so they don't encounter rate limiting.
ACKs for top commit:
stickies-v:
re-ACK 5c74a0b397
achow101:
ACK 5c74a0b397
l0rinc:
Code review ACK 5c74a0b397
Tree-SHA512: d32db5fcc28bb9b2a850f0048c8062200a3725b88f1cd9a0e137da065c0cf9a5d22e5d03cb16fe75ea7494801313ab34ffec7cf3e8577cd7527e636af53591c4
The bitcoin-node binary is built on all platforms which have
multiprocess enabled, but for functional tests it's only used in
CentOS native (depends) job. The next commit will also add a
non-depends job.
This causes IPC binaries (bitcoin-node, bitcoin-gui) to be included
in releases.
The effect on CI is that this causes more depends builds to build IPC
binaries, but still the only build running functional tests with them
is the i686_multiprocess one.
Except for Windows.
2b00030af8 interfaces, chain, refactor: Remove inaccurate getActiveChainLocator (pablomartin4btc)
110a0f405c interfaces, chain, refactor: Remove unused getTipLocator (pablomartin4btc)
Pull request description:
Remove `Chain::getTipLocator`, `Chain::GetLocator()`, and `Chain::getActiveChainLocator`:
- `Chain::getTipLocator` is no longer used.
- `Chain::GetLocator`, replaced its call by `GetLocator()`, which uses `LocatorEntries`, avoiding direct access to the chain itself (change suggested by l0rinc while reviewing this PR to maintain consistency with the overall refactoring).
- `Chain::getActiveChainLocator`, whose name was misleading, has functionality redundant with Chain::findBlock.
- Additionally, the comment for getActiveChainLocator became inaccurate following changes in commit ed470940cd (from PR #25717).
This is a [follow-up](https://github.com/bitcoin/bitcoin/pull/29652#issuecomment-3151665095) to #29652.
ACKs for top commit:
achow101:
ACK 2b00030af8
furszy:
ACK 2b00030af8
stickies-v:
ACK 2b00030af8
w0xlt:
ACK 2b00030af8
Tree-SHA512: b12ba6a15feeaeec692d69204a6e155e3af43edfac25597dabf14cacca1e4a2152574816e58dc544f39043c5721f5e707acf544f4541d3b9c0f8c0c40069215e
67e186deb0 doc: update wallet build instruction (Sjors Provoost)
Pull request description:
Sqlite and the wallet are no longer optional, but they can still be opted out of. This PR updates the build instructions accordingly.
Updating this text now reduces churn in #31802.
ACKs for top commit:
jonatack:
ACK 67e186deb0
achow101:
ACK 67e186deb0
w0xlt:
reACK 67e186deb0
pablomartin4btc:
ACK 67e186deb0
janb84:
ACK 67e186deb0
Tree-SHA512: ea87a83c9fcb884f9ca380a006a93ede3cd1af5eb2bb5ccb54fc8124faa532e9473ae2461d775439da59e86178fbd04983113b43ce3d4c382cabe27866acd027
de0675f9de refactor: Move `transaction_identifier.h` to primitives (marcofleon)
6f068f65de Remove implicit uint256 conversion and comparison (marcofleon)
9c24cda72e refactor: Convert remaining instances from uint256 to Txid (marcofleon)
d2ecd6815d policy, refactor: Convert uint256 to Txid (marcofleon)
f6c0d1d231 mempool, refactor: Convert uint256 to Txid (marcofleon)
aeb0f78330 refactor: Convert `mini_miner` from uint256 to Txid (marcofleon)
326f244724 refactor: Convert RPCs and `merkleblock` from uint256 to Txid (marcofleon)
49b3d3a92a Clean up `FindTxForGetData` (marcofleon)
Pull request description:
This is the final leg of the [type safety refactor](https://github.com/bitcoin/bitcoin/pull/32189).
All of these changes are straightforward `uint256` --> `Txid` along with any necessary explicit conversions. Also, `transaction_identifier.h` is moved to primitives in the last commit, as `Txid` and `Wtxid` become fundamental types after this PR.
ACKs for top commit:
stickies-v:
re-ACK de0675f9de, no changes since a20724d926d5844168c6a13fa8293df8c8927efe except address review nits.
janb84:
re ACK de0675f9de
dergoegge:
re-ACK de0675f9de
theStack:
Code-review ACK de0675f9de
Tree-SHA512: 2413160fca7ab146a8d79d18ce3afcf7384cacc73c513d41928904aa453b4dd7a350064cee71e9c5d015da5904c7c81ac17603e50a47441ebc5b0c653235dd08
The Script errors were last touched in 2020. This value was calculated after that
in 2022 (commit 4b7aec2951). The previous commit
made the size of the largest reject reason string 4 characters smaller ("mandatory"
became "block"), so adapt the constant.
Using "block" or "mempool" as the prefix in place of "mandatory" or "non-mandatory" is clearer
to a user. "non-mandatory" was renamed into "mempool" as part of #33050. This takes care of the
other half of this renaming as a scripted diff.
-BEGIN VERIFY SCRIPT-
sed -i 's/mandatory-script-verify/block-script-verify/g' $(git grep -l mandatory-script-verify)
-END VERIFY SCRIPT-
776a163374 guix: increase maximum allowed (runtime) GCC to 7 (fanquake)
Pull request description:
Fixes:
```bash
/distsrc-base/distsrc-d3b8a54a8120-x86_64-linux-gnu/build/bin/bitcoind: symbol __divmodti4 from unsupported version GCC_7.0.0(14)
/distsrc-base/distsrc-d3b8a54a8120-x86_64-linux-gnu/build/bin/bitcoind: failed IMPORTED_SYMBOLS
```
which is occuring after #32750. I can't see any supported distro that is shipping a new enough glibc (2.31), but a GCC older than 7.0.
Fixes#33177.
ACKs for top commit:
hebasto:
ACK 776a163374.
Tree-SHA512: 8e5a77c509eb6164314fdb644ea199916e151eb0c7f48703f3a2bdedf0dea29b7f402ceacb2aaf42ebffba59080cefbb84253b2721047d973a851090447ba3b5
The getActiveChainLocator method name was misleading, and its functionality
duplicated `Chain::findBlock`. This commit removes the method and replaces
all its usages with direct `Chain::findBlock` calls.
Additionally, the comment of getActiveChainLocator has been outdated since
commit ed47094 from #25717.
Finally, in CWallet::ScanForWalletTransactions, the findBlock calls are now
unified into a single call at the start of the function.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: Matias Furszyfer <mfurszy@protonmail.com>
Also removed CChain::GetLocator() and replaced its call
with GetLocator() which uses LocatorEntries instead.
Co-authored-by: ryanofsky <ryan@ofsky.org>
Co-authored-by: l0rinc <l0rinc@users.noreply.github.com>
876dbdfb47 tests: drop expect_disconnect behaviour for tx relay (Anthony Towns)
b29ae9efdf validation: only check input scripts once (Anthony Towns)
266dd0e10d net_processing: drop MaybePunishNodeForTx (Anthony Towns)
Pull request description:
Because we do not discourage nodes for transactions we consider non-standard, we don't get any DoS protection from this check in adversarial scenarios, so remove the check entirely both to simplify the code and reduce the risk of splitting the network due to changes in tx relay policy.
Then, because we no longer make use of the distinction between consensus and standardness failures during script validation, don't re-validate each script with only-consensus rules, reducing the cost to us of transactions that we won't relay.
ACKs for top commit:
achow101:
ACK 876dbdfb47
darosior:
re-ACK 876dbdfb47
sipa:
re-ACK 876dbdfb47
glozow:
ACK 876dbdfb47
Tree-SHA512: 8bb0395766dde54fc48f7077b80b88e35581aa6e3054d6d65735965147abefffa7348f0850bb3d46f6c2541fd384ecd40a00a57fa653adabff8a35582e2d1811
db225cea56 wallet, refactor: Replace GetDisplayName() with LogName() (Ryan Ofsky)
01737883b3 wallet: Translate [default wallet] string in progress messages (Ryan Ofsky)
Pull request description:
Noticed while reviewing https://github.com/bitcoin/bitcoin/pull/31287#discussion_r1843809721 that the [default wallet] part of progress messages remains untranslated while the rest of the string is translated.
Fix this in all places where `CWallet::ShowProgress` (which has a cancel button) and `Chain::showProgress` (which doesn't have a cancel button) are called by making "default wallet" into a translated string.
ACKs for top commit:
achow101:
ACK db225cea56
pablomartin4btc:
ACK db225cea56
furszy:
utACK db225cea56
Tree-SHA512: 3e76e22ee692a7403d61c66615f56d0fa5f7883dd47553bcaec2f9ffd942daaa90ceb61830206bece50da53dcd737b6438c36bcb086030b2deb68c44172f3931
- Add helper functions and structs to improve readability and
reusability of test code
- Make tests more specific by comparing all produced log lines with
expected log lines instead of relying on approximations or proxies.
db3228042b util: detect and warn when using exFAT on macOS (willcl-ark)
Pull request description:
exFAT is known to cause intermittent corruption on MacOS.
Therefore we should warn when using this fs format for either the blocks or data directories.
See #28552 for more context.
ACKs for top commit:
l0rinc:
ACK db3228042b
marcofleon:
reACK db3228042b
ismaelsadeeq:
reACK db3228042b
Tree-SHA512: e4453a8e24b35c135e4eb0b4e47fe0c80f8b54700f458909c403aa37a0d2979ee165347bcd76e48e4d1ae5d3bae13f50e6afe714e33226a52f907b95df9d3b46
Fixes:
```bash
/distsrc-base/distsrc-d3b8a54a8120-x86_64-linux-gnu/build/bin/bitcoind: symbol __divmodti4 from unsupported version GCC_7.0.0(14)
/distsrc-base/distsrc-d3b8a54a8120-x86_64-linux-gnu/build/bin/bitcoind: failed IMPORTED_SYMBOLS
```
which is occuring after #32750. I can't see any supported distro that is
shipping a new enough glibc (2.31), but a GCC older than 7.0.
Let's say an attacker wants to use/exhaust the network's bandwidth, and
has the choice between renting resources from a commercial provider and
getting the network to "spam" itself it by sending unconfirmed
transactions. We'd like the latter to be more expensive than the former.
The bandwidth for relaying a transaction across the network is roughly
its serialized size (plus relay overhead) x number of nodes. A 1000vB
transaction is 1000-4000B serialized. With 100k nodes, that's 0.1-0.4GB
If the going rate for commercial services is 10c/GB, that's like 1-4c per kvB
of transaction data, so a 1000vB transaction should pay at least $0.04.
At a price of 120k USD/BTC, 100sat is about $0.12. This price allows us
to tolerate a large decrease in the conversion rate or increase in the
number of nodes.
Back when we implemented coin age priority as a miner policy, miners
mempools might admit transactions paying very low fees, but then want to
set a higher fee for block inclusion. However, since coin age priority
was removed in v0.15, the block assembly policy is solely based on fees,
so we do not need to apply minimum feerate rules in multiple places. In
fact, the block assembly policy ignoring transactions that are added to
the mempool is likely undesirable as we waste resources accepting and
storing this transaction.
Instead, rely on mempool policy to enforce a minimum entry feerate to
the mempool (minrelaytxfee). Set the minimum block feerate to the
minimum non-zero amount (1sat/kvB) so it collects everything it finds in
mempool into the block.
Moves the file from `src/util` to `src/primitives`. Now that the
refactor is complete, Txid and Wtxid are fundamental types, so it
makes sense for them to reside in `src/primitives`.
These remaining miscellaneous changes were identified by commenting out
the `operator const uint256&` conversion and the `Compare(const uint256&)`
method from `transaction_identifier.h`.
ca64b71ed5 test: fix scripts in `blockfilter_basic_test` (UdjinM6)
Pull request description:
`std::vector` fill ctor is like this:
```
// Constructs a vector with `count` copies of elements with value `value`.
explicit vector( size_type count, const T& value = T(), const Allocator& alloc = Allocator() ); // (until C++11)
vector( size_type count, const T& value, const Allocator& alloc = Allocator() ); // (since C++11)(constexpr since C++20)
```
https://en.cppreference.com/w/cpp/container/vector/vector.html
i.e. `std::vector<unsigned char>(0, 65)` means a vector with `0` copies of `65` which feels wrong. I believe `count` and `value` were swapped in `blockfilter_basic_test` scripts.
ACKs for top commit:
furszy:
ACK ca64b71ed5
pablomartin4btc:
ACK ca64b71ed5
janb84:
ACK ca64b71ed5
Tree-SHA512: 2cfc7f09788b0a1afdffc9cd6663204c7f1775dabdbe1046cdcd42936c479658c348cb46e0d8835645e6c508e8b40a598cbe6534084b6780a6b60378bcbd0f96
83950275ed qa: unit test sighash caching (Antoine Poinsot)
b221aa80a0 qa: simple differential fuzzing for sighash with/without caching (Antoine Poinsot)
92af9f74d7 script: (optimization) introduce sighash midstate caching (Pieter Wuille)
8f3ddb0bcc script: (refactor) prepare for introducing sighash midstate cache (Pieter Wuille)
9014d4016a tests: add sighash caching tests to feature_taproot (Pieter Wuille)
Pull request description:
This introduces a per-txin cache for sighash midstate computation to the script interpreter for legacy (bare), P2SH, P2WSH, and (as collateral effect, but not actually useful) P2WPKH. This reduces the impact of certain types of quadratic hashing attacks that use standard transactions. It is not known to improve the situation for attacks involving non-standard transaction attacks.
The cache works by remembering for each of the 6 sighash modes a `(scriptCode, midstate)` tuple, which gives a midstate `CSHA256` object right before the appending of the sighash type itself (to permit all 256, rather than just the 6 ones that match the modes). The midstate is only reused if the `scriptCode` matches. This works because - within a single input - only the sighash type and the `scriptCode` affect the actual sighash used.
The PR implements two different approaches:
* The initial commits introduce the caching effect always, for both consensus and relay relation validation. Despite being primarily intended for improving the situation for standard transactions only, I chose this approach as the code paths are already largely common between the two, and this approach I believe involves fewer code changes than a more targetted approach, and furthermore, it should not hurt (it may even help common multisig cases slightly).
* The final commit changes the behavior to only using the cache for non-consensus script validation. I'm open to feedback about whether adding this commit is worth it.
Functional tests are included that construct contrived cases with many sighash types (standard and non-standard ones) and `OP_CODESEPARATOR`s in all script types (including P2TR, which isn't modified by this PR).
ACKs for top commit:
achow101:
ACK 83950275ed
dergoegge:
Code review ACK 83950275ed
darosior:
re-ACK 83950275ed
Tree-SHA512: 65ae8635429a4d563b19969bac8128038ac2cbe01d9c9946abd4cac3c0780974d1e8b9aae9bb83f414e5d247a59f4a18fef5b37d93ad59ed41b6f11c3fe05af4
d3b8a54a81 Refactor CFeeRate to use FeeFrac internally (Pol Espinasa)
Pull request description:
The `FeeFrac` type represents a fraction, intended to be used for `sats/vbyte` or `sats/WU`. It was added to improve accuracy when evaluating fee rates in cluster mempool. [1]
But it can also be used to fix the precision issues that the current `CFeeRate` class has now.
At the moment, `CFeeRate` handles the fee rate as satoshis per kilovirtualbyte: `CAmount / kvB` using an integer.
This PR fix `CFeeRate` precision issues by encapsulating `FeeFrac` internally keeping backwards compatibility.
This PR can also be used as a based to use multiple units on RPC calls as detailed in this issue [2].
Some previous discussions:
[1] https://github.com/bitcoin/bitcoin/pull/30535
[2] https://github.com/bitcoin/bitcoin/issues/32093
ACKs for top commit:
achow101:
ACK d3b8a54a81
murchandamus:
code review, lightly tested ACK d3b8a54a81
ismaelsadeeq:
re-ACK d3b8a54a81📦
theStack:
Code-review ACK d3b8a54a81
Tree-SHA512: 5a8149d81e82ad4e60a0e76ff6a82a5b1c4e212cf5156c1cdd16bf9acbb351e7be458eac3f0a2ae89107f331062b299c1d9ca649d3b820ad0b68e6d1a14292e5
The `-assumevalid` option skips script verification for a specified block and all its ancestors during Initial Block Download.
Many new users are surprised when this suddenly slows their node to a halt.
This commit adds a log message to clearly indicate when this optimization ends and full validation begins (and vice versa).
When using `-assumeutxo`, logging is suppressed for the active assumed-valid chainstate and for the background validation chainstate to avoid the confusing toggles.
-------
> cmake -B build && cmake --build build && mkdir -p demo && build/bin/bitcoind -datadir=demo -stopatheight=500 | grep 'signature validation'
```
2025-08-08T20:59:21Z Disabling signature validations at block #1 (00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048).
2025-08-08T20:59:21Z Enabling signature validations at block #100 (000000007bc154e0fa7ea32218a72fe2c1bb9f86cf8c9ebf9a715ed27fdb229a).
2025-08-08T20:59:21Z Disabling signature validations at block #200 (000000008f1a7008320c16b8402b7f11e82951f44ca2663caf6860ab2eeef320).
2025-08-08T20:59:21Z Enabling signature validations at block #300 (0000000062b69e4a2c3312a5782d7798b0711e9ebac065cd5d19f946439f8609).
```
In LogPrintStr_:
- remove an unnecessary BCLog since we are in the BCLog namespace.
- remove an unnecessary \n when rate limiting is triggered since
FormatLogStrInPlace will add it.
- move the ratelimit bool into an else if block.
- prefix all log lines with [*] when suppressions exist. Previously this
was only done if should_ratelimit was true.
In Reset:
- remove an unnecessary \n since FormatLogStrInPlace will add it.
- Change Level::Info to Level::Warning.
Previously, we would check failing input scripts twice when considering
a transaction for the mempool, in order to distinguish policy failures
from consensus failures. This allowed us both to provide a different
error message and to discourage peers for consensus failures. Because we
are no longer discouraging peers for consensus failures during tx relay,
and because checking a script can be expensive, only do this once.
Also renames non-mandatory-script-verify-flag error to
mempool-script-verify-flag-failed.
Do not discourage nodes even when they send us consensus invalid
transactions.
Because we do not discourage nodes for transactions we consider
non-standard, we don't get any DoS protection from this check in
adversarial scenarios, so remove the check entirely both to simplify the
code and reduce the risk of splitting the network due to changes in tx
relay policy.
exFAT is known to cause corruption on macOS. See #28552.
Therefore we should warn when using this fs format for either the blocks
or data directories on macOS.
Co-authored-by: l0rinc <pap.lorinc@gmail.com>
27aefac425 validation: detect witness stripping without re-running Script checks (Antoine Poinsot)
2907b58834 policy: introduce a helper to detect whether a transaction spends Segwit outputs (Antoine Poinsot)
eb073209db qa: test witness stripping in p2p_segwit (Antoine Poinsot)
Pull request description:
Since it was introduced in 4eb515574e (#18044), the detection of a stripped witness relies on running the Script checks 3 times. In the worst case, this consists in running Script validation for every single input 3 times.
Detection of a stripped witness is necessary because in this case wtxid==txid, and the transaction's wtxid must not be added to the reject filter or it could allow a malicious peer to interfere with txid-based orphan resolution as used in 1p1c package relay.
However it is not necessary to run Script validation to detect a stripped witness (much less so doing it 3 times in a row). There are 3 types of witness program: defined program types (Taproot, P2WPKH and P2WSH), undefined types, and the Pay-to-anchor carve-out.
For defined program types, Script validation with an empty witness will always fail (by consensus). For undefined program types, Script validation is always going to fail regardless of the witness (by standardness). For P2A, an empty witness is never going to lead to a failure.
Therefore it holds that we can always detect a stripped witness without re-running Script validation. However this might lead to more "false positives" (cases where we return witness stripping for an otherwise invalid transaction) than the existing implementation. For instance a transaction with one P2PKH input with an invalid signature and one P2WPKH input with its witness stripped. The existing implementation would treat it as consensus invalid while the implementation in this PR would always consider it witness stripped.
h/t AJ: this essentially implements a variant of https://github.com/bitcoin/bitcoin/pull/33066#issuecomment-3135258539.
ACKs for top commit:
sipa:
re-ACK 27aefac425
Crypt-iQ:
re-ACK 27aefac425
glozow:
reACK 27aefac425
Tree-SHA512: 70cf76b655b52bc8fa2759133315a3f11140844b6b80d9de3c95f592050978cc01a87bd2446e3a9c25cc872efea7659d6da3337b1a709511771fece206e9f149
656e16aa5e qt: Update the `src/qt/locale/bitcoin_en.xlf` translation source file (Hennadii Stepanov)
a0eaa44925 Fix typos (Hennadii Stepanov)
8d4aaaec49 Update Transifex slug for 30.x (Hennadii Stepanov)
Pull request description:
This PR follows our [Release Process](53a996f122/doc/release-process.md).
It is required to open Transifex translations for v30.0, as scheduled in https://github.com/bitcoin/bitcoin/issues/32275.
For reference, see the previous similar PR: https://github.com/bitcoin/bitcoin/pull/31809.
**Note for reviewers:**
To reproduce the diff in the last commit, run:
```
cmake --preset dev-mode
cmake --build build_dev_mode --target translate
```
ACKs for top commit:
laanwj:
Code review ACK 656e16aa5e
stickies-v:
ACK 656e16aa5e , was able to reproduce
Tree-SHA512: 403b534329755079584fcdf98b696e3e75952dfc8d069f305843dbfa85de95f6816ee1d5dfc9b553c7c7f52cc296cb8d3cb03207051d26e0e76ff30d377f49e4
Since it was introduced in 4eb515574e (#18044), the detection of a
stripped witness relies on running the Script checks 3 times. In the worst case, this consists in
running Script validation 3 times for every single input.
Detection of a stripped witness is necessary because in this case wtxid==txid, and the transaction's
wtxid must not be added to the reject filter or it could allow a malicious peer to interfere with
txid-based orphan resolution as used in 1p1c package relay.
However it is not necessary to run Script validation to detect a stripped witness (much less so
doing it 3 times in a row). There are 3 types of witness program: defined program types (Taproot,
P2WPKH, P2WSH), undefined types, and the Pay-to-anchor carve-out.
For defined program types, Script validation with an empty witness will always fail (by consensus).
For undefined program types, Script validation is always going to fail regardless of the witness (by
standardness). For P2A, an empty witness is never going to lead to a failure.
Therefore it holds that we can always detect a stripped witness without re-running Script validation.
However this might lead to more "false positives" (cases where we return witness stripping for an
otherwise invalid transaction) than the existing implementation. For instance a transaction with one
P2PKH input with an invalid signature and one P2WPKH input with its witness stripped. The existing
implementation would treat it as consensus invalid while the implementation in this commit would
always consider it witness stripped.
f49840dd90 doc: Fix typo in files.md (Ryan Ofsky)
f5cf0b1ccc bitcoin wrapper: improve help output (Ryan Ofsky)
c810b168b8 doc: Add description of installed files to files.md (Ryan Ofsky)
94ffd01a02 doc: Add release notes describing libexec/ binaries (Ryan Ofsky)
cd97905ebc cmake: Move internal binaries from bin/ to libexec/ (Ryan Ofsky)
Pull request description:
This change moves binaries that are not typically invoked directly by users from the `bin/` directory to the `libexec/` directory in CMake installs and binary releases. The goal of the PR is to introduce a distinction between internal and external binaries so starting with #31802, we can use IPC to implement features in new binaries without adding those binaries to the CLI. The change also helps reduce clutter in `bin/`, making it easier for users to identify useful tools to run. Summary of changes:
- For **source builds** (i.e. developer builds) — There are no changes.
- For **source installs** (i.e. `cmake --install` result) — `test_bitcoin`, `test_bitcoin-qt`, and `bench_bitcoin` are installed in `${CMAKE_PREFIX_PATH}/libexec` instead of `${CMAKE_PREFIX_PATH}/bin`, so they are no longer on the system `PATH`. However, they can still be invoked from the `libexec/` directory, or from the CLI as `bitcoin test`, `bitcoin test-gui`, and `bitcoin bench`, respectively.
- For **binary releases** — Since `test_bitcoin` is the only test binary enabled in releases, the only change is moving `test_bitcoin` from `bin/` to `libexec/`.
<details><summary>Details</summary>
<p>
The table below shows the install location of each binary after this change, and the availability of each binary.
| Binary | Location | Availability | Change |
|----------------------|--------------|----------------------|-------------------------------|
| `bitcoin` | `bin/` | 📦 Binary release (since #31375) | Unchanged |
| `bitcoin-cli` | `bin/` | 📦 Binary release | Unchanged |
| `bitcoind` | `bin/` | 📦 Binary release | Unchanged |
| `bitcoin-qt` | `bin/` | 📦 Binary release | Unchanged |
| `bitcoin-tx` | `bin/` | 📦 Binary release | Unchanged |
| `bitcoin-util` | `bin/` | 📦 Binary release | Unchanged |
| `bitcoin-wallet` | `bin/` | 📦 Binary release | Unchanged |
| `bench_bitcoin` | `libexec/` | 🛠 Source build only | Moved from `bin/` |
| `bitcoin-chainstate` | `libexec/` | 🛠 Source build only | Newly installed (was built) |
| `bitcoin-gui` | `libexec/` | 🛠 Source build only (until #31802) | Moved from `bin/` |
| `bitcoin-node` | `libexec/` | 🛠 Source build only (until #31802) | Moved from `bin/` |
| `test_bitcoin` | `libexec/` | 📦 Binary release | Moved from `bin/` |
| `test_bitcoin-qt` | `libexec/` | 🛠 Source build only | Moved from `bin/` |
</p>
</details>
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
l0rinc:
re-ACK f49840dd90
Sjors:
re-ACK f49840dd90
achow101:
ACK f49840dd90
janb84:
re ACK f49840dd90
BrandonOdiwuor:
Tested ACK f49840dd90
hodlinator:
re-ACK f49840dd90
willcl-ark:
utACK f49840dd90
Tree-SHA512: 858a2e1a53db11ee3c5c759bfdeea566f242b9ce5e8a898fa435222e41662b8184577c0dc2c4c058294b4de41d8cb3ba3e5d24c748c280efa4a3f84e3ec4344d
9a5d29711a Squashed 'src/crc32c/' changes from b60d2b7334..efb8ea04e4 (fanquake)
Pull request description:
Sync the subtree with latest upstream. The changes here are a no-op, but pull them to fix the drive-by-typo-fixing: #33057.
Includes https://github.com/bitcoin-core/crc32c-subtree/pull/8.
ACKs for top commit:
maflcko:
lgtm ACK 8ef8dd6871
janb84:
ACK 8ef8dd6871
Tree-SHA512: b20a47514218206b934c4aa27ec667fb9b3ec7f7388a78725c52fc6e916358d2b9a2075a37808dbc2430b4c7816511ecf20e36bfe2fbd2d8a26bc8882a46d5e7
86e3a0a8cb refactor: standardize obfuscation memory alignment (Lőrinc)
13f00345c0 refactor: write `Obfuscation` object when new key is generated in dbwrapper (Lőrinc)
e5b1b7c557 refactor: rename `OBFUSCATION_KEY_KEY` (Lőrinc)
298bf95105 refactor: simplify `Obfuscation::HexKey` (Lőrinc)
2dea045425 test: make `obfuscation_serialize` more thorough (Lőrinc)
a17d8202c3 test: merge xor_roundtrip_random_chunks and xor_bytes_reference (Lőrinc)
Pull request description:
Follow up for https://github.com/bitcoin/bitcoin/pull/31144
Applied the remaining comments in separate commits - except for the last one where I could group them.
Please see the commit messages for more context.
ACKs for top commit:
achow101:
ACK 86e3a0a8cb
ryanofsky:
Code review ACK 86e3a0a8cb, just tweaking key write assert as suggested
hodlinator:
ACK 86e3a0a8cb
Tree-SHA512: 967510a141fbb57bf9d088d92b554cf2fffc2f6aa0eab756cbae3230f53e9b04ceebcc6fea5f3383c01ad41985ecde5b5686c64a771ca9deae3497b9b88c1c8b
18d1071dd1 init: replace deprecated PermissionsStartOnly systemd directive (Florian Schmaus)
1caaf65043 init: remove Group= as it will default to the user's default group (Florian Schmaus)
Pull request description:
> This removes the redundant 'Group=' directive and replaces the deprecated 'PermissionsStartOnly' directive.
Picks up #16994 / #19513. The concern in both of these PRs was changing this too early, while systemd v240 was still prelevant on supported systems. That was ~5 years ago, and from what I can see, no modern/supported OS is still using an older systemd.
Separately , I am wondering if we should move these files out to https://github.com/bitcoin-core/packaging/.
ACKs for top commit:
willcl-ark:
reACK 18d1071dd1
Tree-SHA512: a994e38099e68e8377ac820d3cd2047cbfca065ba617eff0d621e3c3b99b05bbd2329631aa8c885a83cf5d0066d97ff5be75bf5834e9f759d8f0d2c6c9b64851
fdbade6f8d kernel: create monolithic kernel static library (Cory Fields)
Pull request description:
Currently, consuming `libbitcoinkernel.a` requires all its dependency static libraries to be available. A switch to a monolithic variant, which contains object files from its dependencies, was discussed in the Kernel WG. The necessary preparations in the libsecp256k1 build scripts were completed in https://github.com/bitcoin-core/secp256k1/pull/1678, which are now available in this repository since https://github.com/bitcoin/bitcoin/pull/33036.
The changes in this PR were picked from https://github.com/theuni/bitcoin/commits/static_kernel/, with an additional adjustment in `libbitcoinkernel.pc.in`.
This PR can be tested as described in https://github.com/bitcoin/bitcoin/pull/30814#issue-2505698234.
ACKs for top commit:
TheCharlatan:
ACK fdbade6f8d
stickies-v:
tACK fdbade6f8d
Tree-SHA512: bd9e9dbb0b765bdcb162fb3f4ad3c4e01fe5fa0b7061f97d0bad64442b21db036cbe0e4341fd45d43a8862df76d62c9532ca8945f76423aca753c6b528f70873
b093a19ae2 cmake: Proactively avoid use of `SECP256K1_DISABLE_SHARED` (Hennadii Stepanov)
eb59a192d9 cmake, refactor: Encapsulate adding secp256k1 subtree in function (Hennadii Stepanov)
Pull request description:
The `SECP256K1_DISABLE_SHARED` CMake variable has been [removed](https://github.com/bitcoin-core/secp256k1/pull/1688) upstream.
This PR removes its usage ahead of the next `secp256k1` subtree update to prevent breakage and simplify integration.
ACKs for top commit:
fanquake:
ACK b093a19ae2
Tree-SHA512: a87cee71cf356f458f68d3163253ca5c4f86e56d268006b6b8e1d4b2c009ba436148a07a6b67b89ddbb2d0e3c1113ab4b4906c5fc5624cb3082b20e916e0e82b
3543bfdfec test: Fix 'getdescriptoractivity' RPCHelpMan, add test to verify 'spend_vin' is the correct field (Chris Stewart)
Pull request description:
Fixes bug in `getdescriptoractivity` RPC help manual.
Here is the line that pushes `spend_vin` field, there is no `spend_vout` json field.
https://github.com/bitcoin/bitcoin/blob/master/src/rpc/blockchain.cpp#L2757
ACKs for top commit:
nervana21:
tACK 3543bfd
luke-jr:
utACK 3543bfdfec
jonatack:
ACK 3543bfdfec
Tree-SHA512: 2cd543569a87261d8d804d9afe36f8e8ead55839c01da9c4831aea3ced7d1251e6885621e628898105700aae4d76cbb8a682f518f33c1c52163e66f75ec87a61
a26fbee38f qt: Translations update (Hennadii Stepanov)
ca04eebd72 cmake: Switch to generated `ts_files.cmake` file (Hennadii Stepanov)
95341de6ca cmake, refactor: Move handling of Qt TS files into `locale` directory (Hennadii Stepanov)
Pull request description:
This PR:
1. Moves handling of Qt TS files into the `locale` directory.
2. Switches from inferior globbing to the explicit file list generated by the [`bitcoin-maintainer-tools/update-translations.py`](https://github.com/bitcoin-core/bitcoin-maintainer-tools/blob/main/update-translations.py) script.
Closes#32653.
ACKs for top commit:
fanquake:
ACK a26fbee38f
Tree-SHA512: 10596768c120d9da21a2340b693c5f39e9e1e02976805e14284cf9785780756f953d73d3d2b4a7246ada37acafe3e5d0e8927a8f51bf9fce3bc93a3544231489
fa1d2f6380 ci: Pass CI_FAILFAST_TEST_LEAVE_DANGLING into container (MarcoFalke)
Pull request description:
After commit fd813bf863, the env var `CI_FAILFAST_TEST_LEAVE_DANGLING` is no longer passed into the container.
This is harmless, because it isn't needed for the Linux containers and macos doesn't use containers at all.
However, it would be nice to document it as an allowed setting and consistently pass it on, when set. So do that here.
ACKs for top commit:
fanquake:
ACK fa1d2f6380
Tree-SHA512: b61780a27f4c2e11359827b1360a34a132e15bff94f358cbf4d453805afcafb873e2c395908b17610fae026ead93226c19a8f6be40fbb87b623ee992f3e2b43e
3333d3f75f ci: Only pass documented env vars (MarcoFalke)
Pull request description:
The CI currently inherits almost all env vars from the host. This was problematic in the past and causing non-determinism, e.g. the fix in commit fa12558d21. It is still problematic today, see e.g. https://github.com/bitcoin/bitcoin/pull/31349#issuecomment-2586828644, or https://github.com/bitcoin/bitcoin/issues/32935
This fixes https://github.com/bitcoin/bitcoin/issues/32935 by only passing env vars documented in `./ci/test/00_setup_env.sh`.
Implementation-wise, instead of cramming the python code into the `python -c ""` statement, just start a fresh py file, which is easier to handle.
ACKs for top commit:
willcl-ark:
ACK 3333d3f75f
Tree-SHA512: f922e481a844128d7fbf773563278a3992c178ead60a3050eceb9ded2aad979afc815a5cbdb9f68494493c5d8d942cdd1111c21e32a5746d19505b87745cb84a
ad132761fc [allocators] Apply manual ASan poisoning to PoolResource (dergoegge)
Pull request description:
Currently ASan will not detect use-after-free issues for memory allocated by a `PoolResource`. This is because ASan is only aware of the memory chunks allocated by `PoolResource` but not the individual "sub-chunks" within.
E.g. this test will not produce an ASan error even though the referenced coin has been deallocated:
```c++
diff --git a/src/test/coins_tests.cpp b/src/test/coins_tests.cpp
index c46144b34b..aa6ca15ce1 100644
--- a/src/test/coins_tests.cpp
+++ b/src/test/coins_tests.cpp
@@ -508,6 +508,17 @@ BOOST_FIXTURE_TEST_CASE(updatecoins_simulation_test, UpdateTest)
BOOST_CHECK(spent_a_duplicate_coinbase);
}
+BOOST_AUTO_TEST_CASE(asan_uaf)
+{
+ CCoinsMapMemoryResource cache_coins_memory_resource{};
+ CCoinsMap map(0, SaltedOutpointHasher(/*deterministic=*/true), CCoinsMap::key_equal{}, &cache_coins_memory_resource);
+ COutPoint outpoint{};
+ map.emplace(outpoint, Coin{});
+ auto& coin = map.at(outpoint);
+ map.erase(outpoint);
+ coin.coin.nHeight = 1;
+}
+
BOOST_AUTO_TEST_CASE(ccoins_serialization)
{
// Good example
```
Fix this by applying [manual ASan poisoning](https://github.com/google/sanitizers/wiki/AddressSanitizerManualPoisoning) for memory allocated by `PoolResource`:
* Newly allocated chunks are poisoned as a whole
* "Sub-chunks" are unpoisoned/re-poisoned during allocation/deallocation
With the poisoning applied, ASan catches the issue in the test above:
```
$ ./build_unit/bin/test_bitcoin --run_test="coins_tests/asan_uaf"
Running 1 test case...
=================================================================
==366064==ERROR: AddressSanitizer: use-after-poison on address 0x7f99c3204870 at pc 0x55569dab6f8a bp 0x7ffe0210e4d0 sp 0x7ffe0210e4c8
READ of size 4 at 0x7f99c3204870 thread T0 (b-test)
```
ACKs for top commit:
achow101:
ACK ad132761fc
marcofleon:
code review ACK ad132761fc
Tree-SHA512: eb5e80bfa9509225e784151807bd8aa21fb0826ca1781dfe81b1d60bd3766019384ea3f9cb8e53398fde2f4e994a9c201b5a9962b4d279d7e52bb60e8961be11
1252eeb997 rpc: fix getpeerinfo ping duration unit docs (0xb10c)
Pull request description:
The docs have been incorrect since a3789c700b (released in v25; master since Sept. 2022). Noticed while setting up monitoring using getpeerinfo.
0cb1ed2b7c/src/rpc/net.cpp (L249-L257)
ACKs for top commit:
luke-jr:
utACK 1252eeb997
maflcko:
lgtm ACK 1252eeb997
jonatack:
ACK 1252eeb997
theStack:
ACK 1252eeb997
janb84:
ACK 1252eeb997
Tree-SHA512: 33f576336b2a4d9533f51f4641d564ee59ef692c5fa9a3cad239fc31465883d5da534bfd0e069be1e1d688e5f0dea3fe6850be19bf35335041b8f414d08f7f09
When the detailed peers list is requested, return the shortened services in the
-netinfo header in the same format as the "serv" column, instead of the full names
list in the report.
The getpeerinfo docs incorrectly specified the ping durations as
milliseconds. This was incorrectly changed in a3789c700b
(released in v25; master since Sept. 2022). The correct duration unit
is seconds.
Also, remove the documentation of the getpeerinfo RPC response from the
ping RPC since it's incomplete. Better to just reference the getpeerinfo
RPC and it's documenation for this.
A stripped witness is detected as a special case in mempool acceptance to make sure we do not add
the wtxid (which is =txid since witness is stripped) to the reject filter. This is because it may
interfere with 1p1c parent relay which currently uses orphan reconciliation (and originally it was
until wtxid-relay was widely adopted on the network.
This commit adds a test for this special case in the p2p_segwit function test, both when spending
a native segwit output and when spending a p2sh-wrapped segwit output.
Thanks to Eugene Siegel for pointing out the p2sh-wrapped detection did not have test coverage by
finding a bug in a related patch of mine.
e017ef3c7e init: make `-blockmaxweight` startup option debug-only (ismaelsadeeq)
Pull request description:
This PR updates `-blockmaxweight` startup option to be debug-only so that it will be hidden from help text.
The option is currently unlikely to be used on mainnet, after the addition of the new `blockreservedweight` option. however it can be useful for test and signet network see https://github.com/bitcoin/bitcoin/pull/32654#issuecomment-2925674473
ACKs for top commit:
Sjors:
tACK e017ef3c7e
fjahr:
ACK e017ef3c7e
polespinasa:
tACK e017ef3c7e
Tree-SHA512: 6c18781826b2f96b13b70b7f1624481f5971746a613079d0d9528366f274ba657a02611f134d7a64f35ecb7e5faf2e3cd025458b04574ac68f804372f6eb715f
a3cf623364 test: Test max_selection_weight edge cases (Murch)
57fe8acc8a test: Check max_weight_exceeded error (Murch)
Pull request description:
I tested all of the reported surviving mutants that @brunoerg reported in https://gist.github.com/brunoerg/834063398d5002f738506d741513e310.
I found that all Mutants except for 12, 14, 17, 37, and 39 were now being caught by one of the existing tests. This fixes Mutants 14, 37, and 39.
Mutant 17 is not fixed, because I consider it acceptable that running BnB for 100,001 instead of 100,000 comparisons doesn’t cause an issue, and Mutant 12 is not yet fixed, because at `fee` = `long_term_fee`, the waste of inputs is 0 and only excess matters, and I haven’t evaluated yet, whether it needs to be fixed.
ACKs for top commit:
achow101:
ACK a3cf623364
jlest01:
ACK a3cf623364
brunoerg:
code review ACK a3cf623364
Tree-SHA512: db67c52127ed98f809f64a903c6b3a012e56cf665a0cd851457af7c85c37ec3af8bb72035d7ad370dd883f99cf3014464e3576559899e37c1d6ee01230511754
This fixes an error reported by Antoine Poinsot <darosior@protonmail.com> in
https://github.com/bitcoin-core/libmultiprocess/issues/123 that does not happen
in master, but does happen with https://github.com/bitcoin/bitcoin/pull/10102
applied, where if Ctrl-C is pressed when `bitcoin-node` is started, it is
handled by both `bitcoin-node` and `bitcoin-wallet` processes, causing the
wallet to shutdown abruptly instead of waiting for the node and shutting down
cleanly.
This change fixes the problem by having the wallet process print to stdout when
it receives a Ctrl-C signal but not otherwise react, letting the node shut
everything down cleanly.
Currently this code is not called in unit tests. Calling should make it
possible to write tests for things like IPC exceptions being thrown during
shutdown.
Use EventLoopRef to avoid reference counting bugs and be more exception safe
and deal with removal of addClient/removeClient methods in
https://github.com/bitcoin-core/libmultiprocess/pull/160
A test update is also required due to
https://github.com/bitcoin-core/libmultiprocess/pull/160 to deal with changed
reference count semantics. In IpcPipeTest(), it is now necessary to destroy
the client Proxy object instead of just the client Connection object to
decrease the event loop reference count and allow the loop to exit so the test
does not hang on shutdown.
cc33e45789 test: improve assertion for SRD max weight test (yancy)
Pull request description:
Replace generic assertion with a result specific assertion showing the correctness of the solution found. If the max weight parameter is exceeded, the least valuable `UTXOs` are removed from the result. Therefore, only the most valued _encountered_ `UTXO's` are selected. While the smallest set would include all the most valued `UTXO's`, in the case of the test there is one high value `UTXO` that is never found before the target value is reached.
Correct the test comment to be more specific about why the assertion is a good result.
ACKs for top commit:
murchandamus:
ACK cc33e45789
furszy:
ACK cc33e45789
Tree-SHA512: bad224063ba830c27fba1b7b80e411ac7cd6c3edcb60bade4e6e3010f3b5d360a921de742c7c20efea8fa839d7939f338270658f66bbcebedebe5c5c8a3e8f9b
c0642e558a [fuzz] fix latency score check in txorphan_protected (glozow)
3d4d4f0d92 scripted-diff: rename "ann" variables to "latency_score" (monlovesmango)
3b92448923 [doc] comment fixups for orphanage changes (glozow)
1384dbaf6d [config] emit warning for -maxorphantx, but allow it to be set (glozow)
b10c55b298 fix up TxOrphanage lower_bound sanity checks (glozow)
cfd71c6704 scripted-diff: rename TxOrphanage outpoints index (glozow)
edb97bb3f1 [logging] add logs for inner loop of LimitOrphans (glozow)
8a58d0e87d scripted-diff: rename OrphanTxBase to OrphanInfo (glozow)
cc50f2f0df [cleanup] replace TxOrphanage::Size() with CountUniqueOrphans (glozow)
ed24e01696 [optimization] Maintain at most 1 reconsiderable announcement per wtxid (Pieter Wuille)
af7402ccfa [refactor] make TxOrphanage keep itself trimmed (glozow)
d1fac25ff3 [doc] 31829 release note (glozow)
Pull request description:
Followup to #31829:
- Release notes
- Have the orphanage auto-trim itself whenever necessary (and test changes) https://github.com/bitcoin/bitcoin/pull/31829#discussion_r2169508690
- Reduce duplicate reconsiderations by keeping track of which txns are already reconsiderable so we only mark it for reconsideration for 1 peer at a time https://github.com/bitcoin/bitcoin/pull/31829#issuecomment-3001627814
- Rename `OrphanTxBase` to `OrphanInfo`
- Get rid of `Size()` method by replacing all calls with `CountUniqueOrphans`
- Rename outpoints index since they point to wtxids, not iterators https://github.com/bitcoin/bitcoin/pull/31829#discussion_r2205557613
- Add more logging in the `LimitOrphans` inner loop to make it easy to see which peers are being trimmed https://github.com/bitcoin/bitcoin/pull/31829#issuecomment-3074385460
ACKs for top commit:
sipa:
utACK c0642e558a
marcofleon:
Nice, ACK c0642e558a
Tree-SHA512: f298eae92cf906ed5e4f15a24eeffa7b9e620bcff457772cd77522dd9f0b3b183ffc976871b1b0e6fe93009e64877d518e53d4b9e186e0df58fc16d17f6de90a
444dcb2f99 fuzz: txgraph: fix `real_is_optimal` flag propagation in `CommitStaging` (Sebastian Falbesoner)
Pull request description:
In the `txgraph` fuzz test, the `CommitStaging` step updates the `SimTxGraph` levels simply by erasing the front (=main) one in the `sims` vector, i.e. the staging level instance takes the place of the main level instance:
83a2216f52/src/test/fuzz/txgraph.cpp (L668-L672)
This also includes the `real_is_optimal` flag (reflecting whether the corresponding real graph is known to be optimally linearized), without taking into account that this flag should only be set if _both_ levels before the commiting are optimal.
E.g. in case of #33097, at this point the main level is not optimally linearized, while the staging level is, and due to the incorrect propagation of the latter the simulation incorrectly assumes that the main level is optimal after, leading to the assertion fail in the additional checks that are ran in this case[1]. Fix this by setting the flag in the resulting main level explicitly. This is done in a generic way, in case there will ever be more than two levels (not sure what is planned in this direction), a simpler alternative would be e.g. `main_optimal = sim[0].real_is_optimal && sim[1].real_is_optimal`.
Fixes#33097.
[1] see 0aedf09ccc for the printf-debug-session-clutter, if that is useful/interesting for anyone (most of the output turned out to be irrelevant to the actual cause of #33097, but it was an entertaining way to discover the interface and get a first glimpse of `TxGraph` internals as a cluster-mempool newbie).
ACKs for top commit:
sipa:
ACK 444dcb2f99
glozow:
ACK 444dcb2f99
Tree-SHA512: c20580e14628fcdc34dabb646a097e02e95b26c5740fcd5ce50f3472e4ee08f20b9a146c9ff16c85e19e57b05af1560e41a9220289c60c15083ad897dc62a0f0
The translations for the following languages, which appear to be the
result of a mistake or an act of vandalism, have been discarded:
- Greek (el)
- Vietnamese (vi)
In the `txgraph` fuzz test, the `CommitStaging` step updates the
`SimTxGraph` levels simply by erasing the front (=main) one in the
`sims` vector, i.e. the staging level instance takes the place of the
main level instance. This also includes the `real_is_optimal` flag
(reflecting whether the corresponding real graph is known to be
optimally linearized), without taking into account that this flag
should only be set if _both_ levels before the commiting are optimal.
E.g. in case of #33097, the main level is not optimally linearized,
while the staging level is, and due to the incorrect propagation of the
latter to the simulation incorrectly assumes that the main level is
optimal, leading to the assertion fail. Fix this by setting the flag
in the resulting main level explicitly.
Resolves the fuzzing assertion fail in issue #33097.
e07e2532b4 test: fix anti-fee-sniping off-by-one error (ishaanam)
Pull request description:
This fixes the off-by-one error in the anti-fee-sniping tests for `send` and `sendall`. `assert_greater_than` fails if the two values are equal.
Closes#33114
ACKs for top commit:
achow101:
ACK e07e2532b4
glozow:
utACK e07e2532b4
Tree-SHA512: 6c9c3d1256faf563361946703d9a51279777d73bc1a849873e03e5b5db52c3c2b9dea4bfe27b1f01b9c830ca246200a895b6a28484da6d822b93b0c7cba237c1
In BaseIndex::Sync(), pindex in `Rewind(pindex, pindex_next->pprev)` isn't always
equal to m_best_block_index since m_best_block_index is updated every
SYNC_LOCATOR_WRITE_INTERVAL seconds, during which multiple pindex update could
happen. Thus the assert here is wrong.
Signed-off-by: Hao Xu <hao.xu@linux.dev>
-BEGIN VERIFY SCRIPT-
sed -i 's/max_global_ann/max_global_latency_score/g' src/node/txorphanage.cpp
sed -i 's/max_global_ann/max_global_latency_score/g' src/node/txorphanage.h
sed -i 's/max_global_ann/max_global_latency_score/g' src/test/orphanage_tests.cpp
sed -i 's/max_global_ann/max_global_latency_score/g' src/test/fuzz/txorphan.cpp
sed -i 's/max_global_ann/max_global_latency_score/g' src/bench/txorphanage.cpp
sed -i 's/max_ann/max_lat/g' src/node/txorphanage.cpp
-END VERIFY SCRIPT-
This introduces an invariant that TxOrphanageImpl never holds more than one
announcement with m_reconsider=true for a given wtxid. This avoids duplicate
work, both in the caller might otherwise reconsider the same transaction multiple
times before it is ready, and internally in AddChildrenToWorkSet, which might
otherwise iterate over all announcements multiple times.
eb65f57f31 [test] setmocktime instead of waiting in 1p1c tests (glozow)
70772dd469 [test] cut the number of transactions involved in 1p1c DoS tests (glozow)
Pull request description:
It was brought to my attention that the runtime of this test is Too Damn High. The test is slow due to the many `wait_for_getdata`s with delays (inbound peer + txid request) and the large volume of messages sent in the dos-related tests. This PR cuts the runtime by about 60% by reducing the number of messages/transactions and using `setmocktime` instead of waiting.
On my machine, master:
```
84.51s user 1.55s system 57% cpu 2:28.53 total
```
After first commit (about 1min faster):
```
28.29s user 0.88s system 35% cpu 1:22.84 total
```
After second commit (about 30sec faster):
```
28.17s user 0.87s system 59% cpu 49.082 total
```
Reviewers should verify that the transactions in the DoS tests are still enough to cause evictions, and that the `bumpmocktime` amounts are not more than necessary.
Alternatives:
- If we don't like mocking the times, we can use outbound connections for all the peers. However, that approach won't improve the runtime as much because we impose a 2-second delay on all txid requests regardless of peer type.
- Note that `noban_tx_relay` is not relevant for this test because all delays are related to downloading, not announcing.
ACKs for top commit:
achow101:
ACK eb65f57f31
w0xlt:
ACK eb65f57f31
Tree-SHA512: 6ffe1f9e5144653e2ded744cec9ddb62ad728c587705542565400a0e8f1fba4843aced4e0d929843874ca7f56f670f5871b7e009ff6be58b791ab24d2e6fcc0e
This change offers a few advantages, such as:
- a more readable and cleaner `ts_files.cmake` (see the next commit);
- a scoped `ts_files` variable;
- improved code locality;
- no need to adjust the location of the resulting `*.qm` files.
ea17a9423f [doc] release note for relaxing requirement of all unconfirmed parents present (glozow)
12f48d5ed3 test: add chained 1p1c propagation test (Greg Sanders)
525be56741 [unit test] package submission 2p1c with 1 parent missing (glozow)
f24771af05 relax child-with-unconfirmed-parents rule (glozow)
Pull request description:
Broadens the package validation interface, see #27463 for wider context.
On master, package rules include that (1) the package topology must be child-wth-parents (2) all of the child's unconfirmed parents must be present. This PR relaxes the second rule, leaving the first rule untouched (there are plans to change that as well, but not here).
Original motivation for this rule was based on the idea that we would have a child-with-unconfirmed-parents package relay protocol, and this would verify that the peer provided the "correct" package. For various reasons, we're not planning on doing this. We could potentially do this for ancestor packages (with a similar definition that all UTXOs to make the tx valid are available in this package), but it's also questionable whether it's useful to enforce this.
This rule gets in the way of certain usage of 1p1c package relay currently. If a transaction has multiple parents, of which only 1 requires a package CPFP, this rule blocks the package from relaying. Even if all the non-low-feerate parents are already in mempool, when the p2p logic submits the 1p1c package, it gets rejected for not meeting this rule.
ACKs for top commit:
ishaanam:
re-utACK ea17a9423f
instagibbs:
ACK ea17a9423f
Tree-SHA512: c2231761ae7b2acea10a96735e7a36c646f517964d0acb59bacbae1c5a1950e0223458b84c6d5ce008f0c1d53c1605df0fb3cd0064ee535ead006eb7c0fa625b
PermissionsStartOnly is deprecated [1]. This removes the directives
and instead we prefixes the value of the ExecStartPre directive with
'!', which means the executable, 'chgrp' in this case, is run with
full privileges and able to change the group of /etc/bitcoin.
1: 60b45a80c1/NEWS (L2434)
Setting Group=bitcoin is redundant. It is typically the default group
of the user and if not explicitly specified, systemd will run the
service with the default group of the user.
Subscribers to the BlockChecked validation interface event may need
access to the block outside of the callback scope. Currently, this
is only possible by copying the block, which makes exposing this
validation interface event publicly either cumbersome or with significant
copy overhead.
By using shared_ptr, we make the shared ownership explicit and allow
users to safely use the block outside of the callback scope.
The `SECP256K1_DISABLE_SHARED` CMake variable has been removed upstream.
This change removes its usage ahead of the next `secp256k1` subtree
update to prevent breakage and simplify integration.
c157438116 qa: test that we do disconnect upon a second invalid compact block being announced (Antoine Poinsot)
fb2dcbb160 qa: test cached failure for compact block (Antoine Poinsot)
f12d8b104e qa: test a compact block with an invalid transaction (Antoine Poinsot)
d6c37b28a7 qa: remove unnecessary tx removal from compact block (Antoine Poinsot)
Pull request description:
In thinking about https://github.com/bitcoin/bitcoin/pull/33050 and https://github.com/bitcoin/bitcoin/pull/33012#issuecomment-3111631541, i went through the code paths for peer disconnection upon submitting an invalid block. It turns out that the fact we exempt a peer from disconnection upon submitting an invalid compact block was not properly tested, as can be checked with these diffs:
```diff
diff --git a/src/net_processing.cpp b/src/net_processing.cpp
index 0c4a89c44c..d243fb88d4 100644
--- a/src/net_processing.cpp
+++ b/src/net_processing.cpp
@@ -1805,10 +1805,10 @@ void PeerManagerImpl::MaybePunishNodeForBlock(NodeId nodeid, const BlockValidati
// The node is providing invalid data:
case BlockValidationResult::BLOCK_CONSENSUS:
case BlockValidationResult::BLOCK_MUTATED:
- if (!via_compact_block) {
+ //if (!via_compact_block) {
if (peer) Misbehaving(*peer, message);
return;
- }
+ //}
break;
case BlockValidationResult::BLOCK_CACHED_INVALID:
{
```
```diff
diff --git a/src/net_processing.cpp b/src/net_processing.cpp
index 0c4a89c44cb..e8e0c805367 100644
--- a/src/net_processing.cpp
+++ b/src/net_processing.cpp
@@ -1814,10 +1814,10 @@ void PeerManagerImpl::MaybePunishNodeForBlock(NodeId nodeid, const BlockValidati
{
// Discourage outbound (but not inbound) peers if on an invalid chain.
// Exempt HB compact block peers. Manual connections are always protected from discouragement.
- if (peer && !via_compact_block && !peer->m_is_inbound) {
+ //if (peer && !via_compact_block && !peer->m_is_inbound) {
if (peer) Misbehaving(*peer, message);
return;
- }
+ //}
break;
}
case BlockValidationResult::BLOCK_INVALID_HEADER:
```
We do have a test for this, but it actually uses a coinbase witness commitment error, which is checked much earlier in `FillBlock`. This PR adds coverage for the two exemptions in `MaybePunishNodeForBlock`.
ACKs for top commit:
kevkevinpal:
ACK [c157438](c157438116)
nervana21:
tACK [c157438](c157438116)
instagibbs:
crACK c157438116
stratospher:
ACK c157438116.
Tree-SHA512: a77d5a9768c0d73f122b06db2e416e80d0b3c3fd261dae8e340ecec2ae92d947d31988894bc732cb6dad2e338b3c82f33e75eb3280f8b0933b285657cf3b212c
4b80147feb test: Perform backup filename checks in migrate_and_get_rpc (Ava Chow)
Pull request description:
Some test cases were unnecessarily checking the backup filename, which involved setting the mocktime before `migrate_and_get_rpc`. However, this could cause a failure if the test was slow since `migrate_and_get_rpc` also sets the mocktime. Since it also already checks that the backup file is named correctly, there's no need for those tests to also do their own mocktime and filename check.
The CI failure can be reproduced locally by adding a sleep to `migrate_and_get_rpc`:
```diff
diff --git a/test/functional/wallet_migration.py b/test/functional/wallet_migration.py
index 704204425c7..e87a6100623 100755
--- a/test/functional/wallet_migration.py
+++ b/test/functional/wallet_migration.py
@@ -129,6 +129,7 @@ class WalletMigrationTest(BitcoinTestFramework):
assert_equal(w["warnings"], ["This wallet is a legacy wallet and will need to be migrated with migratewallet before it can be loaded"])
# Mock time so that we can check the backup filename.
+ time.sleep(1)
mocked_time = int(time.time())
self.master_node.setmocktime(mocked_time)
# Migrate, checking that rescan does not occur
```
Fixes#33096
ACKs for top commit:
fjahr:
reACK 4b80147feb
Sammie05:
tACK 4b80147
pablomartin4btc:
utACK 4b80147feb
rkrux:
ACK 4b80147feb
Tree-SHA512: 045d4acf2ad0b56a7083ff2ee5ef09f0d74ad097c01a290660daca096c71fc07109848024256d84f74abbc87dd52691d160f9968b3654726626d3dbd21a84ab6
Some test cases were unnecessarily checking the backup filename, which
involved setting the mocktime before `migrate_and_get_rpc`. However,
this could cause a failure if the test was slow since
`migrate_and_get_rpc` also sets the mocktime. Since it also already
checks that the backup file is named correctly, there's no need for
those tests to also do their own mocktime and filename check.
7aa5b67132 ci: remove DEBUG_LOCKORDER from TSAN job (fanquake)
b09af2ce50 ci: instrument libc++ in TSAN job (fanquake)
6653cafd0b ci: allow libc++ instrumentation other than msan (fanquake)
Pull request description:
Allow for instrumenting libc++ with a sanitizer other than MemoryWithOrigins.
Would also close#33087, as with the extra instrumentation, the issue from https://github.com/bitcoin/bitcoin/pull/32862#issuecomment-3114706601 is avoided (also see https://github.com/bitcoin/bitcoin/pull/33081), and we can drop `DEBUG_LOCKORDER`.
ACKs for top commit:
maflcko:
re-ACK 7aa5b67132🦀
dergoegge:
utACK 7aa5b67132
Tree-SHA512: 95f123e37da5e81d571244e4b1cd7658107676f1ea763ff16e5b69f4dfadb88236a577bb2ee52230ff542872c1da151c88fc50aba0f32540e765080120cec55e
9954d6c833 depends: hard-code necessary c(xx)flags rather than setting them per-host (Cory Fields)
Pull request description:
The per-host flag variables hold platform-specific defaults that are ignored when flag environment variables are set, so it was wrong for them to contain -std options relied on by package definitions.
Additionally, these flags (-pipe and -std=xx) will no longer be passed into the CMake build, meaning less duplication in the build summary.
Pulled out of #31920.
ACKs for top commit:
achow101:
ACK 9954d6c833
ryanofsky:
Code review ACK 9954d6c833. No changes since last review other than improving the commit message. Change overall makes sense because it deduplicates host definitions, stops dropping `-std` flags from package builds when custom CFLAGS/CXXFLAGS environment variables are set, and stops passing duplicate flags to cmake that have no effect.
theuni:
ACK 9954d6c833
Tree-SHA512: 62a2a21b741893cf708ca71fb5f0626c30d52685c845f9016f428a5e38fc8515acd4bf2c83635d6505b63830d1c296472026ec3d341c8069f1e490a991b6b5ef
fac90e5261 test: Check that the GUI interactive reindex works (MarcoFalke)
faaaddaaf8 init: [gui] Avoid UB/crash in InitAndLoadChainstate (MarcoFalke)
Pull request description:
`InitAndLoadChainstate` is problematic, when called twice in the GUI. This can happen when it returns a failure and the user selects an interactive reindex.
There are several bugs that have been introduced since the last time this was working correctly:
* The first one is a crash (assertion failure), which happens due to a cached tip block in the notifiications from the previous run. See https://github.com/bitcoin/bitcoin/pull/31346#discussion_r2207914726
* The second one is UB (use-after-free), which happens because the block index db in the blockmanager is not reset. See https://github.com/bitcoin/bitcoin/pull/30965#discussion_r2207822121
Fix both bugs by resetting any dirty state in `InitAndLoadChainstate`.
Also, add a test, because I don't really want to keep testing this manually every time. (A failing test run can be seen in https://github.com/bitcoin/bitcoin/pull/32979/checks)
ACKs for top commit:
achow101:
ACK fac90e5261
TheCharlatan:
ACK fac90e5261
mzumsande:
Tested ACK fac90e5261
Tree-SHA512: 9f744d36e7cdd3f5871764386ec5a5cca1ae144f1bacc26c07e60313c2bdacdc5fca351aa185cb51359540eea4534dda17e4fb6073ad90f91ba0a6936faeead8
3a03f07560 qt: Avoid header circular dependency (Anthony Towns)
25884bd896 qt, refactor: Move `FreespaceChecker` class into its own module (Hennadii Stepanov)
Pull request description:
For some reason, the MOC compiler in older versions of Qt 6 fails to parse `qt/intro.cpp`, as noted in [this comment](https://github.com/bitcoin/bitcoin/pull/32998#issuecomment-3082011233).
This PR proposes a move-only refactoring to simplify the source structure by eliminating the need for the inline `#include <qt/intro.moc>`, thereby effectively working around the issue.
Required for https://github.com/bitcoin/bitcoin/pull/32998.
ACKs for top commit:
ajtowns:
ACK 3a03f07560
Tree-SHA512: 4a7261f04fff9bd8edd4dc2df619c90e06417e19da672dd688a917cd0b9a324a6db7185a47c48f0385713b5e6c45d2204bef58cbe6c77299386136ed5682bd8d
c6e2c31c55 rpc: unhide waitfor{block,newblock,blockheight} (Sjors Provoost)
0786b7509a rpc: add optional blockhash to waitfornewblock (Sjors Provoost)
Pull request description:
The `waitfornewblock` is inherently racy as the tip may have changed since the last RPC call, and can even change during initial processing of this call.
Add an optional `blockhash` argument so the caller can specify their current tip. Return immediately if our tip is different.
I've made it fail if `LookupBlockIndex` fails. This should never happen if the user got the block hash from our RPC in the first place.
Finally, the `waitfor{block,newblock,blockheight}` RPC methods are no longer hidden in `help`:
- the changes in #30409 ensured these methods _could_ work in the GUI
- #31785 removed the guards that prevented GUI users from using them
- this PR makes `waitfornewblock` reliable
So there's no more reason to hide them.
ACKs for top commit:
TheCharlatan:
Re-ACK c6e2c31c55
ryanofsky:
Code review ACK c6e2c31c55. Just rebased and tweaked documentation since last review.
glozow:
utACK c6e2c31c55
Tree-SHA512: 84a0c94cb9a2e4449e7a395cf3dce1650626bd852e30e0e238a1aafae19d57bf440bfac226fd4da44eaa8d1b2fa4a8c1177b6c716235ab862a72ff5bf8fc67ac
cab6736b70 ci: remove ninja-build from MSAN jobs (fanquake)
Pull request description:
It is part of `CI_BASE_PACKAGES`.
ACKs for top commit:
maflcko:
review ACK cab6736b70 🕸
hebasto:
ACK cab6736b70, I have reviewed the code and it looks OK.
Tree-SHA512: 8da5f0b07310a1e003d405ade19408b390781121a317ecc0ebdf48cb63bb3abf39bcfb635e4a43200568e0debabb463c2a3a843705e625fa455609eb3f0ba416
c2ed576d2c fuzz: cover BanMan::IsDiscouraged (brunoerg)
Pull request description:
This PR adds fuzz coverage for the `IsDiscouraged` function in the banman target. This is the only function missing from `BanMan`.
ACKs for top commit:
maflcko:
lgtm ACK c2ed576d2c
marcofleon:
ACK c2ed576d2c
Tree-SHA512: 1dc5fc138f89413c46ed41195940f4c578ef996ce84595271b7433cae8a8f576205b649b493a7ec4804c712327d6c77b1004ba116b0144916377042adaaf6c5f
5888b4a2a5 doc: add note for watch-only wallet migration (rkrux)
Pull request description:
This was suggested in a previous PR #31423.
ACKs for top commit:
achow101:
ACK 5888b4a2a5
brunoerg:
reACK 5888b4a2a5
jonatack:
ACK 5888b4a2a5
Tree-SHA512: 96e51eda30a1f31cfd82ae3296ca97c9236599b18e19086dbde3a908f6fe66af8f2de7aa147bdb9ebccb2059c809a25ddfb0c23da57e1a84a35b62ca0a44e3c3
76fe0e59ec test: Migration of a wallet ending in `../` (David Gumberg)
f0bb3d50fe test: Migration of a wallet ending in `/` (David Gumberg)
41faef5f80 test: Migration fail recovery w/ `../` in path (David Gumberg)
63c6d36437 test: Migration of a wallet with `../` in path. (David Gumberg)
70f1c99c90 wallet: Fix migration of wallets with pathnames. (David Gumberg)
f6ee59b6e2 wallet: migration: Make backup in walletdir (David Gumberg)
e22c3599c6 test: wallet: Check direct file backup name. (David Gumberg)
Pull request description:
Support for wallets outside of the default wallet directory was added in #11687, and these external wallets can be specified with paths relative to the wallet directory, e.g. `bitcoin-cli loadwallet ../../mywallet`. In the RPC commands, there is no distinction between a wallet's 'name' and a wallet's 'path'. This PR fixes an issue with wallet backup during migration where the wallet's 'name-path' is used in the backup filename. This goes south when that filename is appended to the directory where we want to put the file and the wallet's 'name' actually gets treated as a path:
```cpp
fs::path backup_filename = fs::PathFromString(strprintf("%s_%d.legacy.bak", (wallet_name.empty() ? "default_wallet" : wallet_name), GetTime()));
fs::path backup_path = this_wallet_dir / backup_filename;
```
Attempting to migrate a wallet with the 'name' `../../../mywallet` results in a backup being placed in `datadir/wallets/../../../mywallet/../../../mywallet_1744683963.legacy.bak`.
If permissions don't exist to write to that folder, migration can fail.
The solution implemented here is to put backup files in the top-level of the node's `walletdir` directory, using the folder name (and in some rare cases the file name) of the wallet to name the backup file:
9fa5480fc4/src/wallet/wallet.cpp (L4254-L4268)
##### Steps to reproduce on master
Build and run `bitcoind` with legacy wallet creation enabled:
```bash
$ cmake -B build -DWITH_BDB=ON && cmake --build build -j $(nproc)
$ ./build/bin/bitcoind -regtest -deprecatedrpc=create_bdb
```
Create a wallet with some relative path specifiers (exercise caution with where this file may be written)
```bash
$ ./build/bin/bitcoin-cli -regtest -named createwallet wallet_name="../../../myrelativewallet" descriptors=false
```
Try to migrate the wallet:
```bash
$ ./build/bin/bitcoin-cli -regtest -named migratewallet wallet_name="../../../myrelativewallet"
```
You will see a message in `debug.log` about trying to backup a file somewhere like: `/home/user/.bitcoin/regtest/wallets/../../../myrelativewallet/../../../myrelativewallet_1744686627.legacy.bak` and migration might fail because `bitcoind` doesn't have permissions to write the backup file.
ACKs for top commit:
pablomartin4btc:
tACK 76fe0e59ec
achow101:
ACK 76fe0e59ec
ryanofsky:
Code review ACK 76fe0e59ec. Nice changes that (1) fix potential errors when names of wallets being migrated contain slashes, and (2) store migration backups in the top-level `-walletdir` instead of in individual wallet subdirectories.
Tree-SHA512: 5cf6ed9f44ac7d204e4e9854edd3fb9b43812e930f76343b142b3c19df3de2ae5ca1548d4a8d26226d537bca231e3a50b3ff0d963c200303fb761f2b4eb3f0d9
fa45ccc15d doc: Add legacy wallet removal release notes (MarcoFalke)
Pull request description:
This spans over several pulls, so add a single note for all of them.
ACKs for top commit:
glozow:
lgtm ACK fa45ccc15d
achow101:
ACK fa45ccc15d
pablomartin4btc:
ACK fa45ccc15d
janb84:
re ACK fa45ccc15d
Tree-SHA512: e753cc3afbd66a88099ff62c2591aa31d32d784098e433e392c20a8dfd40d5c85807e955b264a287c3778d68605cd7022278886a43cd1635c080d563c88fc0cc
aac0b6dd79 test: test sendall and send do anti-fee-sniping (ishaanam)
20802c7b65 wallet, rpc: add anti-fee-sniping to `send` and `sendall` (ishaanam)
Pull request description:
Currently, `send` and `sendall` don't do anti-fee-sniping because they don't use `CreateTransaction`. This PR adds anti-fee-sniping to these RPCs by calling `DiscourageFeeSniping` from `FinishTransaction` when the user does not specify a locktime.
ACKs for top commit:
achow101:
ACK aac0b6dd79
murchandamus:
ACK aac0b6dd79
glozow:
ACK aac0b6dd79
Tree-SHA512: d4f1b43b5bda489bdba46b0af60e50bff0de604a35670e6ea6e1de2b539f16b3f68805492f51d6d2078d421b63432ca22a561a5721d1a37686f2e48284e1e646
1bed0f734b guix: warn SOURCE_DATE_EPOCH set in guix-codesign (will)
Pull request description:
#32678 added a sanity check for this environment variable when running `guix-build` but missed that `guix-codesign` also relies on `SOURCE_DATE_EPOCH`, which can result in non-determinism in the codesigning step: https://github.com/bitcoin-core/guix.sigs/pull/1720#issuecomment-3124332676
To avoid repeating the logic move common functionality into the prelude and call the function in both guix actions.
ACKs for top commit:
fanquake:
ACK 1bed0f734b
Tree-SHA512: ad3de8ab06e7f4ffcee5c02e8185b20879d63a02a614a706ea54da5087cca4ba75817ca1aa95301572c34723317fcc44e4478261ac73dd223ee9fa827e6b35b3
3b23f95e34 ci: limit max stack size to 512 KiB (dergoegge)
2931a87477 ci: limit stack size to 512kb in native macOS jobs (fanquake)
Pull request description:
Picks up #31367.
Closes#29840.
Limit adjustment is moved until after compilation, otherwise compilation might not succeed.
I've used compilation flags to limit the stack size in the native macOS jobs, because trying to use `ulimit` resulted in:
```bash
+ '[' -n 1 ']'
+ ulimit -s 262144
/Users/runner/work/bitcoin/bitcoin/ci/test/03_test_script.sh: line 17: ulimit: stack size: cannot modify limit: Operation not permitted
```
See example failures (`ulimit -s 64`) here: https://github.com/bitcoin/bitcoin/runs/46861548042.
ACKs for top commit:
dergoegge:
utACK 3b23f95e34
Tree-SHA512: 7e00626f3ca9e860d79a301af2427008ce27c329b618e24f95e7a55b284459a446216d2859c2e63be50abb9d4f0d343c12ff50445231652d354f225477928a35
6757052fc4 doc: move `cmake -B build -LH` up in Unix build docs (Bufo)
Pull request description:
#32269 rebased.
> I had trouble building bitcoin core the way I wanted since now more features require a flag while building. IMO it makes sense to make it a bit more prominent in the build docs how to get the needed flags.
> Related issue: https://github.com/bitcoin/bitcoin/issues/32258
ACKs for top commit:
maflcko:
lgtm ACK 6757052fc4
stickies-v:
ACK 6757052fc4
janb84:
ACK 6757052fc4
Tree-SHA512: 8e6dc1e432c067f862560776176112d5c24c4009bdf8e9a4e8d1ea3328b88732188fc4d8a7cd29f8a9ed8a1809a7a8a86d63b7ae3ec4ebae74be466727c8d730
This was in fact untested until now. This can be checked with the following diff.
```diff
diff --git a/src/net_processing.cpp b/src/net_processing.cpp
index 0c4a89c44cb..f8b9adf910a 100644
--- a/src/net_processing.cpp
+++ b/src/net_processing.cpp
@@ -1822,7 +1822,7 @@ void PeerManagerImpl::MaybePunishNodeForBlock(NodeId nodeid, const BlockValidati
}
case BlockValidationResult::BLOCK_INVALID_HEADER:
case BlockValidationResult::BLOCK_INVALID_PREV:
- if (peer) Misbehaving(*peer, message);
+ if (!via_compact_block && peer) Misbehaving(*peer, message);
return;
// Conflicting (but not necessarily invalid) data or different policy:
case BlockValidationResult::BLOCK_MISSING_PREV:
```
62ed1f92ef txgraph: check that DoWork finds optimal if given high budget (tests) (Pieter Wuille)
f3c2fc867f txgraph: add work limit to DoWork(), try optimal (feature) (Pieter Wuille)
e96b00d99e txgraph: make number of acceptable iterations configurable (feature) (Pieter Wuille)
cfe9958852 txgraph: track amount of work done in linearization (preparation) (Pieter Wuille)
6ba316eaa0 txgraph: 1-or-2-tx split-off clusters are optimal (optimization) (Pieter Wuille)
fad0eb091e txgraph: reset quality when merging clusters (bugfix) (Pieter Wuille)
Pull request description:
Part of #30289. Builds on top of #31553.
So far, the `TxGraph::DoWork()` function took no parameters, and just made all clusters reach the "acceptable" internal quality level by performing a minimum number of improvement iterations on it, but:
* Did not attempt to go beyond that.
* Was broken, as the QualityLevel of optimal clusters that merge together was not being reset.
Fix this by adding an argument to `DoWork()` to control how much work it is allowed to do right now, which will first be used to get all clusters to the acceptable level, and if more budget remains, use it to try to get some or all clusters optimal. The function will now return `true` if all clusters are known to be optimal (and thus no further work remains). This is verified in the tests, by remembering whether the graph is optimal, and if it is at the end of the simulation run, verify that the overall linearization cannot be improved further.
ACKs for top commit:
instagibbs:
ACK 62ed1f92ef
ismaelsadeeq:
Code review ACK 62ed1f92ef
glozow:
ACK 62ed1f92ef
Tree-SHA512: 5f57d4052e369f3444e72e724f04c02004e0f66e365faa59c9f145323e606508380fc97bb038b68783a62ae9c10757f1b628b3b00b2ce9a46161fca2d4336d73
The per-host flag variables hold platform-specific defaults that are ignored
when flag environment variables are set, so it was wrong for them to contain
-std options relied on by package definitions.
Additionally, these flags (-pipe and -std=xx) will no longer be passed into
the CMake build, meaning less duplication in the build summary.
Pulled out of #31920.
0ce041ea88 tracing: fix pointer argument handling in mempool_monitor.py (deadmanoz)
Pull request description:
The BPF code was incorrectly passing pointer variables by value to `bpf_usdt_readarg()`, causing the function to fail silently and resulting in transaction hashes showing as zeros and reason strings displaying empty strings.
This fix adds the missing reference operator (&) when passing pointer variables to `bpf_usdt_readarg()`, allowing the function to properly write the pointer values and enabling correct display of transaction hashes and removal/rejection reasons.
Fixes the regression introduced in [ec47ba349d](ec47ba349d) where `bpf_usdt_readarg_p` was replaced with `bpf_usdt_readarg` but the calling convention wasn't properly updated for pointer arguments.
**Before: "0000000000000000000000000000000000000000000000000000000000000000" tx hashes, and missing reasons (empty strings) for removal.**
<img width="1683" height="1330" alt="Screenshot 2025-07-29 at 4 30 03 PM" src="https://github.com/user-attachments/assets/71ba88be-dbcc-43a6-bfe7-bd49ae082b13" />
**After: tx hashes show, reasons for removal showing.**
<img width="1682" height="1330" alt="Screenshot 2025-07-29 at 4 29 23 PM" src="https://github.com/user-attachments/assets/03738c75-5526-4c1e-82c2-ba100cdf128a" />
ACKs for top commit:
0xB10C:
tested ACK 0ce041ea88
Tree-SHA512: cb50748fa2cd38be4b0abed36723917c2c82a92f588928bb0650eed0049c121df89b33d53421037b12836a497f30b449fe3d041ff7755a1fd9da43544392cf40
b6d4688f77 [doc] reword comments in test_mid_package_replacement (glozow)
f3a613aa5b [cleanup] delete brittle test_mid_package_eviction (glozow)
c3cd7fcb2c [doc] remove references to now-nonexistent Finalize() function (glozow)
d8140f5f05 don't make a copy of m_non_base_coins (glozow)
98ba2b1db2 [doc] MemPoolAccept coins views (glozow)
ba02c30b8a [doc] always CleanupTemporaryCoins after a mempool trim (glozow)
Pull request description:
Deletes `test_mid_package_eviction` that is brittle and already covered in other places. It was introduced in #28251 addressing 2 issues: (1) calling `LimitMempoolSize()` in the middle of package validation and (2) not updating coins view cache when the mempool contents change, leading to "disappearing coins."
(1) If you let `AcceptSingleTransaction` call `LimitMempoolSize` in the middle of package validation, you should get a failure in `test_mid_package_eviction_success` (the package is rejected):
```
diff --git a/src/validation.cpp b/src/validation.cpp
index f2f6098e214..4bd6f059849 100644
--- a/src/validation.cpp
+++ b/src/validation.cpp
@@ -1485,7 +1485,7 @@ MempoolAcceptResult MemPoolAccept::AcceptSingleTransaction(const CTransactionRef
FinalizeSubpackage(args);
// Limit the mempool, if appropriate.
- if (!args.m_package_submission && !args.m_bypass_limits) {
+ if (!args.m_bypass_limits) {
LimitMempoolSize(m_pool, m_active_chainstate.CoinsTip());
// If mempool contents change, then the m_view cache is dirty. Given this isn't a package
// submission, we won't be using the cache anymore, but clear it anyway for clarity.
```
Mempool modifications have a pretty narrow interface since #31122 and `TrimToSize()` cannot be called while there is an outstanding mempool changeset. So I think there is a low likelihood of accidentally reintroducing this problem and not immediately hitting e.g. a fuzzer crash on this line b53fab1467/src/txmempool.cpp (L1143)
(2) If you remove the `CleanupTemporaryCoins()` call from `ClearSubPackageState()` you should get a failure from `test_mid_package_replacement`:
```
diff --git a/src/validation.cpp b/src/validation.cpp
index f2f6098e214..01b904b69ef 100644
--- a/src/validation.cpp
+++ b/src/validation.cpp
@@ -779,7 +779,7 @@ private:
m_subpackage = SubPackageState{};
// And clean coins while at it
- CleanupTemporaryCoins();
+ // CleanupTemporaryCoins();
}
};
```
I also added/cleaned up the documentation about coins views to hopefully make it extremely clear when people should `CleanupTemporaryCoins`.
ACKs for top commit:
instagibbs:
reACK b6d4688f77
sdaftuar:
utACK b6d4688f77
marcofleon:
ACK b6d4688f77
Tree-SHA512: 79c68e263013b1153520f5453e6b579b8fe7e1d6a9952b1ac2c3c3c017034e6d21d7000a140bba4cc9d2ce50ea3a84cc6f91fd5febc52d7b3fa4f797955d987d
The BPF code was incorrectly passing pointer variables by value to
bpf_usdt_readarg(), causing the function to fail silently and resulting
in transaction hashes and reason strings displaying as zeros or garbage.
This fix adds the missing reference operator (&) when passing pointer
variables to bpf_usdt_readarg(), allowing the function to properly
write the pointer values and enabling correct display of transaction
hashes and removal/rejection reasons.
Fixes the regression introduced in ec47ba349d where bpf_usdt_readarg_p
was replaced with bpf_usdt_readarg but the calling convention wasn't
properly updated for pointer arguments.
Submit the block with an invalid transaction Script again, leading to
CACHED_INVALID being returned by AcceptBlockHeader(). Ensure that also this
code path does not lead to a disconnection.
This was previously untested, as can be checked with the following diff:
```diff
diff --git a/src/net_processing.cpp b/src/net_processing.cpp
index 0c4a89c44cb..e8e0c805367 100644
--- a/src/net_processing.cpp
+++ b/src/net_processing.cpp
@@ -1814,10 +1814,10 @@ void PeerManagerImpl::MaybePunishNodeForBlock(NodeId nodeid, const BlockValidati
{
// Discourage outbound (but not inbound) peers if on an invalid chain.
// Exempt HB compact block peers. Manual connections are always protected from discouragement.
- if (peer && !via_compact_block && !peer->m_is_inbound) {
+ //if (peer && !via_compact_block && !peer->m_is_inbound) {
if (peer) Misbehaving(*peer, message);
return;
- }
+ //}
break;
}
case BlockValidationResult::BLOCK_INVALID_HEADER:
```
The current test to exercise a block with an invalid transaction actually
creates a block with an invalid coinbase witness, which is checked early and
for which MaybePunishNodeForBlock() is not called.
Add a test case with an invalid regular transaction, which will lead
CheckInputScripts to return a CONSENSUS error and MaybePunishNodeForBlock() to
be called, appropriately not disconnecting upon an invalid compact block. This
was until now untested as can be checked with the following diff:
```diff
diff --git a/src/net_processing.cpp b/src/net_processing.cpp
index 0c4a89c44cb..d243fb88d4b 100644
--- a/src/net_processing.cpp
+++ b/src/net_processing.cpp
@@ -1805,10 +1805,10 @@ void PeerManagerImpl::MaybePunishNodeForBlock(NodeId nodeid, const BlockValidati
// The node is providing invalid data:
case BlockValidationResult::BLOCK_CONSENSUS:
case BlockValidationResult::BLOCK_MUTATED:
- if (!via_compact_block) {
+ //if (!via_compact_block) {
if (peer) Misbehaving(*peer, message);
return;
- }
+ //}
break;
case BlockValidationResult::BLOCK_CACHED_INVALID:
{
```
Finally, note this failure is cached (unlike the malleated witness failure),
which will be used in the following commits.
The error being checked here is BLOCK_MUTATED, as returned by IsBlockMutated()
in FillBlock(). Dropping the fourth transaction from the block is unnecessary
and would make testing of other block validation failures in following commits
more verbose.
* parameter name uses underscores
* commit message typo fixed and compacted
* used `10_MiB` to avoid having to comment
* swapped order of operands in (9 * x / 10) to make it obvious that we're calculating 90%
* inlined return value
Move-only commit, enabled reusing the large cache size calculation logic later. The only difference is the removal of the `static` keyword (since in a constexpr function it's a C++23 extension)
Simplifies `m_tx_inventory_to_send` a bit by making it a set of Wtxids.
Wtxid relay is prevalent throughout the network, so the complexity of
dealing with a GenTxid in this data structure isn't necessary.
For peers that aren't wtxid relay, the txid is now retrieved from our
mempool entry when the inv is constructed.
1c10b7351e RPC: Return permitbaremultisig and maxdatacarriersize in getmempoolinfo (Kristaps Kaupe)
Pull request description:
Other node relay settings like `fullrbf` and `minrelaytxfee` are already returned, makes sense to add these two too.
ACKs for top commit:
ajtowns:
ACK 1c10b7351e
maflcko:
lgtm ACK 1c10b7351e
theStack:
ACK 1c10b7351e
Tree-SHA512: 1750d7d12de511f0ac34922ea9c58c4b9b55c3aaf22109abfd7dbe01ad1eb7b48fb4a6b074a0baf0e55ee2270fcc969b6830e499ff33adbcd0b9c761fb25e563
251d020846 init, wallet: replace hardcoded output types with `FormatAllOutputTypes` (Sebastian Falbesoner)
e3ba0757a9 rpc, wallet: replace remaining hardcoded output types with `FormatAllOutputTypes` (Sebastian Falbesoner)
Pull request description:
This PR takes use of the `FormatAllOutputTypes` helper (introduced in PR #32432, commit 8cc9845b8d) to get rid of the remaining hardcoded output types in wallet RPC and command line arguments documentation [1]. Note that it can't be used in the [`createmultisig` RPC](fc162299f0/src/rpc/output_script.cpp (L100)), as this one is only for pre-taproot output types and hence doesn't contain "bech32m" in the list.
[1] instances were found via `$ git grep legacy.*p2sh-segwit ./src/rpc/ ./src/wallet/`
ACKs for top commit:
nervana21:
tACK [251d020](251d020846)
maflcko:
review ACK 251d020846 🌨
pablomartin4btc:
re-utACK 251d020846
rkrux:
crACK 251d020846
Tree-SHA512: 23d1025d068f3a44b115a34b217b808fcae59fc574e35a899f0d43a85512935c90675d2e98c621287e02adc3a9f4a08289a26596c66e2122262af0cd2dfbde72
8aed477c33 test: fix RPC coverage check (Brandon Odiwuor)
2630b64f81 test: add abortrescan RPC test (Brandon Odiwuor)
Pull request description:
This is #27593 cleaned up / rebased, now that the legacy wallet has been dropped.
Closes#27593.
ACKs for top commit:
maflcko:
lgtm ACK 8aed477c33
cedwies:
ACK 8aed477
Tree-SHA512: 14a28b1ef0c1f63236d04c2ff6c11adddc40642e4a23d30398e8a03fc47f911465af91affc6e66ee2d548515ef4f65fb1cb5d69985c5a771a17b1c9c009f48ad
Currently there is a warning for this in guix-build, but we also need
one in guix-codesign, otherwise the codesigned hashes are not
reproducible.
Move common functionality into prelude and call the function in both
guix actions.
Previously, the assertion only showed that a result was found, however
made no assertion about the quality of the result.
Remove comment about what UTXOs are selected and what are not
since the test does not reflect that.
Co-authored-by: Mark "Murch" Erhardt <murch@murch.one>
faa1c3e80d Revert "Merge bitcoin/bitcoin#32343: common: Close non-std fds before exec in RunCommandJSON" (MarcoFalke)
Pull request description:
After a fork() in a multithreaded program, the child can safely
call only async-signal-safe functions (see [signal-safety(7)](https://www.man7.org/linux/man-pages/man7/signal-safety.7.html))
until such time as it calls execv.
The standard library (`std` namespace) is not async-signal-safe. Also, `throw`, isn't.
There was an alternative implementation using `readdir` (https://github.com/bitcoin/bitcoin/pull/32529), but that isn't async-signal-safe either, and that implementation was still using `throw`.
So temporarily revert this feature.
A follow-up in the future can add it back, using only async-signal-safe functions, or by using a different approach.
Fixes https://github.com/bitcoin/bitcoin/issues/32524
Fixes https://github.com/bitcoin/bitcoin/issues/33015
Fixes https://github.com/bitcoin/bitcoin/issues/32855
For reference, a failure can manifest in the GCC debug mode:
* While `fork`ing, a debug mode mutex is held (by any other thread).
* The `fork`ed child tries to use the stdard libary before `execv` and deadlocks.
This may look like the following:
```
(gdb) thread apply all bt
Thread 1 (Thread 0xf58f4b40 (LWP 774911) "b-httpworker.2"):
#0 0xf7f4f589 in __kernel_vsyscall ()
#1 0xf79e467e in ?? () from /lib32/libc.so.6
#2 0xf79eb582 in pthread_mutex_lock () from /lib32/libc.so.6
#3 0xf7d93bf2 in ?? () from /lib32/libstdc++.so.6
#4 0xf7d93f36 in __gnu_debug::_Safe_iterator_base::_M_attach(__gnu_debug::_Safe_sequence_base*, bool) () from /lib32/libstdc++.so.6
#5 0x5668810a in __gnu_debug::_Safe_iterator_base::_Safe_iterator_base (this=0xf58f13ac, __seq=0xf58f13f8, __constant=false) at /bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/debug/safe_base.h:91
#6 0x56ddfb50 in __gnu_debug::_Safe_iterator<__gnu_cxx::__normal_iterator<int*, std::__cxx1998::vector<int, std::allocator<int> > >, std::__debug::vector<int, std::allocator<int> >, std::forward_iterator_tag>::_Safe_iterator (this=0xf58f13a8, __i=3, __seq=0xf58f13f8) at /bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/debug/safe_iterator.h:162
#7 0x56ddfacb in __gnu_debug::_Safe_iterator<__gnu_cxx::__normal_iterator<int*, std::__cxx1998::vector<int, std::allocator<int> > >, std::__debug::vector<int, std::allocator<int> >, std::bidirectional_iterator_tag>::_Safe_iterator (this=0xf58f13a8, __i=3, __seq=0xf58f13f8) at /bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/debug/safe_iterator.h:539
#8 0x56ddfa5b in __gnu_debug::_Safe_iterator<__gnu_cxx::__normal_iterator<int*, std::__cxx1998::vector<int, std::allocator<int> > >, std::__debug::vector<int, std::allocator<int> >, std::random_access_iterator_tag>::_Safe_iterator (this=0xf58f13a8, __i=3, __seq=0xf58f13f8) at /bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/debug/safe_iterator.h:687
#9 0x56ddd3f6 in std::__debug::vector<int, std::allocator<int> >::begin (this=0xf58f13f8) at /bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/debug/vector:300
#10 0x57d83701 in subprocess::detail::Child::execute_child (this=0xf58f156c) at ./util/subprocess.h:1372
#11 0x57d80a7c in subprocess::Popen::execute_process (this=0xf58f1cd8) at ./util/subprocess.h:1231
#12 0x57d6d2b4 in subprocess::Popen::Popen<subprocess::input, subprocess::output, subprocess::error, subprocess::close_fds> (this=0xf58f1cd8, cmd_args="fake.py enumerate", args=..., args=..., args=..., args=...) at ./util/subprocess.h:964
#13 0x57d6b597 in RunCommandParseJSON (str_command="fake.py enumerate", str_std_in="") at ./common/run_command.cpp:27
#14 0x57a90547 in ExternalSigner::Enumerate (command="fake.py", signers=std::__debug::vector of length 0, capacity 0, chain="regtest") at ./external_signer.cpp:28
#15 0x56defdab in enumeratesigners()::$_0::operator()(RPCHelpMan const&, JSONRPCRequest const&) const (this=0xf58f2ba0, self=..., request=...) at ./rpc/external_signer.cpp:51
...
(truncated, only one thread exists)
```
ACKs for top commit:
fanquake:
ACK faa1c3e80d
darosior:
ACK faa1c3e80d
Tree-SHA512: 602da5f2eba08d7fe01ba19baf411e287ae27fe2d4b82f41734e05b7b1d938ce94cc0041e86ba677284fa92838e96ebee687023ff28047e2b036fd9a53567e0a
The current `prevector` size of 28 bytes (chosen to fill the `sizeof(CScript)` aligned size) was introduced in 2015 (https://github.com/bitcoin/bitcoin/pull/6914) before SegWit and TapRoot.
However, the increasingly common `P2WSH` and `P2TR` scripts are both 34 bytes, and are forced to use heap (re)allocation rather than efficient inline storage.
The core trade-off of this change is to eliminate heap allocations for common 34-36 byte scripts at the cost of increasing the base memory footprint of all `CScript` objects by 8 bytes (while still respecting peak memory usage defined by `-dbcache`).
Increasing the `prevector` size allows these scripts to be stored inline, avoiding extra heap allocations, reducing potential memory fragmentation, and improving performance during cache flushes. Massif analysis confirms a lower stable memory usage after flushing, suggesting the elimination of heap allocations outweighs the larger base size for common workloads.
Due to memory alignment, increasing the `prevector` size to 36 bytes doesn't change the overall `sizeof(CScript)` compared to an increase to 34 bytes, allowing us to include `P2PK` scripts as well at no additional memory cost.
Performance benchmarks for AssumeUTXO load and flush show:
* Small dbcache (450MB): ~1-3% performance improvement (despite more frequent flushes)
* Large dbcache (4500MB): ~6-8% performance improvement due to fewer heap allocations (and basically the number of flushes)
* Very large dbcache (4500MB): ~5-6% performance improvement due to fewer heap allocations (and memory limit not being reached, so there's no memory penalty)
Full IBD and reindex-chainstate with larger `dbcache` values also show an overall ~3-4% speedup.
Co-authored-by: Ava Chow <github@achow101.com>
Co-authored-by: Andrew Toth <andrewstoth@gmail.com>
Co-authored-by: maflcko <6399679+maflcko@users.noreply.github.com>
Verifies that script types are correctly allocated using prevector's direct or indirect storage based on their size:
Direct allocated script types (size ≤ 28 bytes):
* OP_RETURN (small)
* P2WPKH
* P2SH
* P2PKH
Indirect allocated script types (size > 28 bytes):
* P2WSH
* P2TR
* P2PK
* MULTISIG (small)
This test provides a baseline for verifying changes to prevector's inline capacity.
The `CHECK_SCRIPT_STATIC_SIZE` and `CHECK_SCRIPT_DYNAMIC_SIZE` macros were added to differentiate the two cases - while preserving the correct source code line in case of failure.
c5c1960f93 doc: Add release notes for changes in RPCs (pablomartin4btc)
90fd5acbe5 rpc, test: Fix error message in getdescriptoractivity (pablomartin4btc)
39fef1d203 test: Add missing logging info for each test (pablomartin4btc)
53ac704efd rpc, test: Fix error message in unloadwallet (pablomartin4btc)
1fc3a8e8e7 rpc, test: Add EnsureUniqueWalletName tests (pablomartin4btc)
b635bc0896 rpc, util: Add EnsureUniqueWalletName (pablomartin4btc)
Pull request description:
Currently, `unloadwallet` RPC call fails with a JSON parsing error when no `wallet_name` argument is provided. This behavior is misleading because the error originates from a low-level JSON type mismatch, rather than clearly indicating that the wallet name or RPC endpoint (`-rpcwallet=...`) is missing. Also, found out that the [issue](https://github.com/bitcoin/bitcoin/pull/13111#issuecomment-398831543) was noticed during its implementation but never addressed.
In addition, I've verified all RPC commands calls finding that `getdescriptoractivity` had the same problem, but related to the array input types (blockhashes & descriptors), so I've corrected that RPC as well. For consistency I've added the missing logging info for each test case in `test/functional/rpc_getdescriptoractivity.py` in preparation for the new test.
**_-Before_**
```
./build/bin/bitcoin-cli -regtest -datadir=/tmp/btc unloadwallet
error code: -3
error message:
JSON value of type number is not of expected type string
```
```
./build/bin/bitcoin-cli -regtest -datadir=/tmp/btc getdescriptoractivity
error code: -3
error message:
JSON value of type null is not of expected type array
```
```
./build/bin/bitcoin-cli -regtest -datadir=/tmp/btc getdescriptoractivity '[]'
error code: -3
error message:
JSON value of type null is not of expected type array
```
**_-After_**
```
./build/bin/bitcoin-cli -regtest -datadir=/tmp/btc unloadwallet
error code: -8
error message:
Either the RPC endpoint wallet or the wallet name parameter must be provided
```
```
./build/bin/bitcoin-cli -regtest -datadir=/tmp/btc getdescriptoractivity
error code: -1
error message:
getdescriptoractivity ["blockhash",...] [scanobjects,...] ( include_mempool )
Get spend and receive activity associated with a set of descriptors for a set of blocks. This command pairs well with the `relevant_blocks` output of `scanblocks()`.
This call may take several minutes. If you encounter timeouts, try specifying no RPC timeout (bitcoin-cli -rpcclienttimeout=0)
Arguments:
1. blockhashes (json array, required) The list of blockhashes to examine for activity. Order doesn't matter. Must be along main chain or an error is thrown.
[
"blockhash", (string) A valid blockhash
...
]
2. scanobjects (json array, required) Array of scan objects. Every scan object is either a string descriptor or an object:
[
"descriptor", (string) An output descriptor
{ (json object) An object with output descriptor and metadata
"desc": "str", (string, required) An output descriptor
"range": n or [n,n], (numeric or array, optional, default=1000) The range of HD chain indexes to explore (either end or [begin,end])
},
...
]
3. include_mempool (boolean, optional, default=true) Whether to include unconfirmed activity
...
```
```
./build/bin/bitcoin-cli -regtest -datadir=/tmp/btc getdescriptoractivity '[]'
error code: -1
error message:
getdescriptoractivity ["blockhash",...] [scanobjects,...] ( include_mempool )
...
```
ACKs for top commit:
achow101:
ACK c5c1960f93
stickies-v:
re-ACK c5c1960f93
furszy:
ACK c5c1960f93
Tree-SHA512: e831ff1acbfd15d2ce3a69bb408cce94664c0b63b2aa2f4627a05c6c052241ae3b5cc238219ef1b30afb489a4a3f4c3030e2168b0c8f08b4d20805d050d810f5
Slays Mutant 37 from Bruno’s report:
https://gist.github.com/brunoerg/834063398d5002f738506d741513e310
diff --git a/src/wallet/coinselection.cpp b/muts/coinselection.mutant.37.cpp
index cee558088f..9747cd26c9 100644
--- a/src/wallet/coinselection.cpp
+++ b/muts/coinselection.mutant.37.cpp
@@ -128,7 +128,7 @@ util::Result<SelectionResult> SelectCoinsBnB(std::vector<OutputGroup>& utxo_pool
curr_value > selection_target + cost_of_change || // Selected value is out of range, go back and try other branch
(curr_waste > best_waste && is_feerate_high)) { // Don't select things which we know will be more wasteful if the waste is increasing
backtrack = true;
- } else if (curr_selection_weight > max_selection_weight) { // Selected UTXOs weight exceeds the maximum weight allowed, cannot find more solutions by adding more inputs
+ } else if (curr_selection_weight >= max_selection_weight) { // Selected UTXOs weight exceeds the maximum weight allowed, cannot find more solutions by adding more inputs
max_tx_weight_exceeded = true; // at least one selection attempt exceeded the max weight
backtrack = true;
} else if (curr_value >= selection_target) { // Selected value is within range
This slays the mutants 14 and 39 Bruno reported via
https://gist.github.com/brunoerg/834063398d5002f738506d741513e310,
that changing the intial or subsequent value of
`max_tx_weight_exceeded` in BnB would not fail any tests:
diff --git a/src/wallet/coinselection.cpp b/muts/coinselection.mutant.14.cpp
index cee558088f..947bf7b642 100644
--- a/src/wallet/coinselection.cpp
+++ b/muts/coinselection.mutant.14.cpp
@@ -118,7 +118,7 @@ util::Result<SelectionResult> SelectCoinsBnB(std::vector<OutputGroup>& utxo_pool
CAmount best_waste = MAX_MONEY;
bool is_feerate_high = utxo_pool.at(0).fee > utxo_pool.at(0).long_term_fee;
- bool max_tx_weight_exceeded = false;
+ bool max_tx_weight_exceeded = true;
// Depth First search loop for choosing the UTXOs
for (size_t curr_try = 0, utxo_pool_index = 0; curr_try < TOTAL_TRIES; ++curr_try, ++utxo_pool_index) {
diff --git a/src/wallet/coinselection.cpp b/muts/coinselection.mutant.39.cpp
index cee558088f..bbfdc23889 100644
--- a/src/wallet/coinselection.cpp
+++ b/muts/coinselection.mutant.39.cpp
@@ -129,7 +129,7 @@ util::Result<SelectionResult> SelectCoinsBnB(std::vector<OutputGroup>& utxo_pool
(curr_waste > best_waste && is_feerate_high)) { // Don't select things which we know will be more wasteful if the waste is increasing
backtrack = true;
} else if (curr_selection_weight > max_selection_weight) { // Selected UTXOs weight exceeds the maximum weight allowed, cannot find more solutions by adding more inputs
- max_tx_weight_exceeded = true; // at least one selection attempt exceeded the max weight
+ max_tx_weight_exceeded = false; // at least one selection attempt exceeded the max weight
backtrack = true;
} else if (curr_value >= selection_target) { // Selected value is within range
curr_waste += (curr_value - selection_target); // This is the excess value which is added to the waste for the below comparison
This commit takes use of the `FormatAllOutputTypes` helper
(introduced in PR #32432, commit 8cc9845b8d)
to get rid of the hardcoded output types in wallet RPC documentation.
Note that it can't be used in the `createmultisig` RPC, as this one is
only for pre-taproot output types and hence doesn't contain "bech32m" in the list.
1cb2399703 doc: clarify the GetAddresses/GetAddressesUnsafe documentation (Daniela Brozzoni)
e5a7dfd79f p2p: rename GetAddresses -> GetAddressesUnsafe (Daniela Brozzoni)
Pull request description:
Rename GetAddresses to GetAddressesUnsafe to make it clearer that this function should only be used in trusted contexts. This helps avoid accidental privacy leaks by preventing the uncached version from being used in non-trusted scenarios, like P2P.
Additionally, better reflect in the documentation that the two methods should be used in different contexts.
Also update the outdated "call the function without a parameter" phrasing in the cached version. This wording was accurate when the cache was introduced in #18991, but became outdated after later commits (f26502e9fc, 81b00f8780) added parameters to each
function, and the previous commit changed the function naming completely.
ACKs for top commit:
stickies-v:
re-ACK 1cb2399703
l0rinc:
ACK 1cb2399703
luisschwab:
ACK 1cb2399703
brunoerg:
ACK 1cb2399703
theStack:
Code-review ACK 1cb2399703
mzumsande:
Code review ACK 1cb2399703
Tree-SHA512: 02c05d88436abcdfabad994f47ec5144e9ba47668667a2c1818f57bf8710727505faf8426fd0672c63de14fcf20b96f17cea2acc39fe3c1f56abbc2b1a9e9c23
fa1fd07468 ci: Enable more shellcheck (MarcoFalke)
Pull request description:
shellcheck is often the main "reviewer" of CI code written in Bash, so it seems odd to disable it by putting commands into `bash -c "cmd..."`.
Fix that by removing `bash -c`, where it isn't intended and where the removal is easily possible.
ACKs for top commit:
hebasto:
ACK fa1fd07468.
Tree-SHA512: 6412dd3f8d702bca7762a8f1be3f9d2782132936fcc7ae5c31690b594e04f69708110e6f6233d5a61901289d13c7089ab5646a2c3ef2266fffc36d0543f4b7ae
face8123fd log: [refactor] Use info level for init logs (MarcoFalke)
fa183761cb log: Remove function name from init logs (MarcoFalke)
Pull request description:
Many of the normal, and expected init logs, which are run once after startup use the deprecated alias of `LogInfo`.
Fix that by using `LogInfo` directly, which is a refactor, except for a few log lines that also have `__func__` removed.
(Also remove the unused trailing `\n` char while touching those logs)
ACKs for top commit:
stickies-v:
re-ACK face8123fd
fanquake:
ACK face8123fd
Tree-SHA512: 28c296129c9a31dff04f529c53db75057eae8a73fc7419e2f3068963dbb7b7fb9a457b2653f9120361fdb06ac97d1ee2be815c09ac659780dff01d7cd29f8480
fa1a14a13a fuzz: Reset chainman state in process_message(s) targets (MarcoFalke)
fa9a3de09b fuzz: DisableNextWrite (MarcoFalke)
aeeeeec9f7 fuzz: Reset dirty connman state in process_message(s) targets (MarcoFalke)
fa11eea405 fuzz: Avoid non-determinism in process_message(s) target (PeerMan) (MarcoFalke)
Pull request description:
`process_message(s)` are the least stable fuzz targets, according to OSS-Fuzz.
Tracking issue: https://github.com/bitcoin/bitcoin/issues/29018.
### Testing
Needs coverage compilation, as explained in `./contrib/devtools/README.md`. And then, using 32 threads:
```
cargo run --manifest-path ./contrib/devtools/deterministic-fuzz-coverage/Cargo.toml -- $PWD/bld-cmake/ $PWD/../b-c-qa-assets/fuzz_corpora/ process_messages 32
```
Each commit can be reverted to see more non-determinism re-appear.
ACKs for top commit:
marcofleon:
ReACK fa1a14a13a
dergoegge:
reACK fa1a14a13a
Tree-SHA512: 37b5b6dbdde6a39b4f83dc31e92cffb4a62a4b8f5befbf17029d943d0b2fd506f4a0833570dcdbf79a90b42af9caca44e98e838b03213d6bc1c3ecb70a6bb135
It is redundant with -logsourcelocations and the log messages are
clearer without it.
Also, remove a double-space.
Also, add braces around `if` touched in the next commit.
This tiny behavior change requires a test fixup.
6135e0553e wallet, rpc: Move (Un)LockCoin WalletBatch creation out of RPC (Ava Chow)
Pull request description:
If the locked coin needs to be persisted to the wallet database, insteead of having the RPC figure out when to create a WalletBatch and having LockCoin's behavior depend on it, have LockCoin take whether to persist as a parameter so it makes the batch.
Since unlocking a persisted locked coin requires a database write as well, we need to track whether the locked coin was persisted to the wallet database so that it can erase the locked coin when necessary.
Keeping track of whether a locked coin was persisted is also useful information for future PRs.
Split from #32489
ACKs for top commit:
rkrux:
ACK 6135e05
Sjors:
ACK 6135e0553e
w0xlt:
ACK 6135e0553e
Tree-SHA512: 0e2367fc4d50c62ec41443374b64c4c5ecf679998677df47fb8776cfb44704713bc45547e32e96cd30d1dbed766f5d333efb6f10eb0e71271606638e07e61a01
This rule was originally introduced along with a very early proposal for
package relay as a way to verify that the "correct"
child-with-unconfirmed-parents package was provided for a transaction,
where correctness was defined as all of the transactions unconfirmed
parents. However, we are not planning to introduce a protocol where
peers would be asked to send these packages.
This rule has downsides: if a transaction has multiple parents but only
1 that requires package CPFP to be accepted, the current rule prevents
us from accepting that package. Even if the other parents are already in
mempool, the p2p logic will only submit the 1p1c package, which fails
this check. See the test in p2p_1p1c_network.py
faa3171ff2 ci: Use optimized Debug build type in test-each-commit (MarcoFalke)
fa21c3401e ci: [doc] reword debug log message (MarcoFalke)
Pull request description:
An optimized debug build is mostly as fast as a release build, because hot loops of heavy debug-only code are rare. So use that setting in the test-each-commit CI, to enable more checks almost "for free".
ACKs for top commit:
Prabhat1308:
re-ACK [`faa3171`](faa3171ff2)
willcl-ark:
crACK faa3171ff2
Tree-SHA512: ca041cf7f79d7abb6f93e17b58b2aea730f3bb9fc51256c1ca1b9f7ce7e7188d18fd99d3754cdbe3f504f4e08d560e72d4b7a75409c214ee2c3771c9a8ba7f96
060695c22a test: Failed load after migrate should restore backup (MarcoFalke)
8a4cfddf23 wallet: Set migrated wallet name only on success (Ava Chow)
Pull request description:
After a wallet is migrated and we are trying to load it, if it could not be loaded, don't try to set the wallet name. Otherwise we have a segfault.
This can be tested by migrated a legacy wallet from another network (e.g. trying to migrate a testnet wallet on mainnet). The fixed behavior is return an error and restore the backup.
ACKs for top commit:
davidgumberg:
ACK 060695c22a
furszy:
ACK 060695c22a
rkrux:
ACK 060695c22a
w0xlt:
reACK 060695c22a
pablomartin4btc:
ACK 060695c22a
Tree-SHA512: f4289e0b3dedef0a3d734c18604f2fd0df0dc65e9641bc342cfa45088d2540a3f6631bbea8bdd394b2631fa83b38e8ac37c83cfc4b53b19dcbd0b820a9beb6e4
b59dc21847 doc: Fix typos in asmap README (nervana21)
Pull request description:
This minor PR fixes some spelling mistakes found while reviewing #33026.
ACKs for top commit:
fanquake:
ACK b59dc21847
Tree-SHA512: e76f7f97c10f3e506d024da0cbf804f4975cf07f31f0dd0abad6fcb97a5fa1032087459dba46de3715f6275d47e2fde4d8db3d38341540110d87fd5669855359
We just need enough transactions to push us above the orphanage limits
and trigger trimming. Reducing the number of transactions cuts the
runtime of this test significantly.
faa3e68411 test: Log KeyboardInterrupt as exception (MarcoFalke)
fa30b34026 test: Do not pass tests on unhandled exceptions (MarcoFalke)
Pull request description:
Currently the functional tests are problematic, because they pass, even if they encounter an unhanded exception.
Fix this by handling all exceptions: Catch `BaseException` as fallback and mark it as failure.
Can be tested via:
```diff
diff --git a/test/functional/wallet_disable.py b/test/functional/wallet_disable.py
index da6e5d408f..ecc41fb041 100755
--- a/test/functional/wallet_disable.py
+++ b/test/functional/wallet_disable.py
@@ -19,6 +19,7 @@ class DisableWalletTest (BitcoinTestFramework):
self.wallet_names = []
def run_test (self):
+ import sys;sys.exit("fatal error")
# Make sure wallet is really disabled
assert_raises_rpc_error(-32601, 'Method not found', self.nodes[0].getwalletinfo)
x = self.nodes[0].validateaddress('3J98t1WpEZ73CNmQviecrnyiWrnqRhWNLy')
```
Previously, the test would pass. With this patch, it would fail.
ACKs for top commit:
enirox001:
Looks good to me—ACK faa3e68
stickies-v:
re-ACK faa3e68411
pablomartin4btc:
tACK faa3e68411
Tree-SHA512: 11ecd5201982e2c776e48d98834b17c15a415306a95524bc702daeba20a316aac797748e9592be8db575597804f149ee7ef104416037cc9e5891758625810e2d
Mark blockhashes and scanobjects arguments as required, so the user receives
a clear help message when either is missing.
Added a new functional test for this use case.
Co-authored-by: stickies-v <stickies-v@users.noreply.github.com>
Add self.log.info(...) calls at the beginning of each test
in GetBlocksActivityTest.
This improves readability and provides debugging information
by logging the purpose of each test upon its correct
execution.
This is in preparation for the next commit, which adds a new test
with log info, and it would look inconsistent without this commit.
The unloadwallet RPC previously failed with a low-level JSON parsing error
when called without any arguments (wallet_name).
Although this issue was first identified during review of the original unloadwallet
implementation in #13111, it was never addressed.
Raise RPC_INVALID_PARAMETER instead describing that either the RPC endpoint or wallet
name must be provided.
Adding a new functional test for this use case.
Refactor migratewallet to use the same logic as the wallet_name argument handling
is identical.
Co-authored-by: maflcko <maflcko@users.noreply.github.com>
2dfeb6668c wallet: remove outdated `pszSkip` arg of database `Rewrite` func (rkrux)
Pull request description:
This argument might have been used in the legacy wallets, but I don't see any implementation using this argument in the SQLite wallets. Removing it cleans up the code a bit.
ACKs for top commit:
achow101:
ACK 2dfeb6668c
brunoerg:
code review ACK 2dfeb6668c
Tree-SHA512: de2178ad6862125f084434ec6a7271d567544870c474c5ea2e75a4f69f3f5eb2170ff46947e098f58e1fa47c35bbe4ebafcd8180581d1f100f1f8d177b32dd91
06ab3a394a tests: speed up coins_tests by parallelizing (Anthony Towns)
Pull request description:
Updates the cmake logic to generate a separate test for each BOOST_FIXTURE_TEST_SUITE declaration in a file, and splits coins_tests.cpp into three separate suites so that they can be run in parallel. Also updates the convention enforced by test/lint/lint-tests.py.
ACKs for top commit:
l0rinc:
reACK 06ab3a394a
maflcko:
lgtm ACK 06ab3a394a
achow101:
ACK 06ab3a394a
Tree-SHA512: 940d9aa31dab60d1000b5f57d8dc4b2c5b4045c7e5c979ac407aba39f2285d53bc00c5e4d7bf2247551fd7e1c8681144e11fc8c005a874282c4c59bd362fb467
065e42976a test: IsFinalTx returns true when there is no locktime (brunoerg)
Pull request description:
According to https://corecheck.dev/mutation/src/consensus/tx_verify.cpp, there is no proper test for the `tx.nLockTime == 0` check in the `IsFinalTx` function, which is understandable, since this check will only be useful for a specific case where the `nBlockHeight` (block height) is zero. Otherwise, the following check `if ((int64_t)tx.nLockTime < ((int64_t)tx.nLockTime < LOCKTIME_THRESHOLD ? (int64_t)nBlockHeight : nBlockTime))` would catch any of it. This PR adds a test case for it.
ACKs for top commit:
maflcko:
lgtm ACK 065e42976a
enirox001:
ACK 065e429: Valuable test case that explicitly demonstrates `IsFinalTx` behavior when nLockTime is 0
achow101:
ACK 065e42976a
darosior:
utACK 065e42976a
Tree-SHA512: e44a7c060bd4c3d746fab166442cadc3fd449ddd8b02cabf22024a5dde6f438f24c6e1bff2a6dc49b57c8e01234aa0fd393fbfe6194df9d9b6c3d4fa2655c99b
Add a new function called EnsureUniqueWalletNamet that returns the
selected wallet name across the RPC request endpoint and wallet_name.
Supports the case where the wallet_name argument may be omitted—either
when using a wallet endpoint, or when not provided at all. In the latter
case, if no wallet endpoint is used, an error is raised.
Internally reuses the existing implementation to avoid redundant URL
decoding and logic duplication.
This is a preparatory change for upcoming refactoring of unloadwallet
and migratewallet, which will adopt EnsureUniqueWalletName for improved
clarity and consistency.
Better reflect in the documentation that the two methods should be
used in different contexts.
Also update the outdated "call the function without a parameter" phrasing
in the cached version. This wording was accurate when the cache was
introduced in #18991, but became outdated after later commits
(f26502e9fc,
81b00f8780) added parameters to each
function, and the previous commit changed the function naming completely.
Co-Authored-By: stickies-v <stickies-v@protonmail.com>
Rename GetAddresses to GetAddressesUnsafe to make it clearer that this
function should only be used in trusted contexts. This helps avoid
accidental privacy leaks by preventing the uncached version from being
used in non-trusted scenarios, like P2P.
Updates the cmake logic to generate a separate test for each
BOOST_FIXTURE_TEST_SUITE declaration in a file, and splits coins_tests.cpp
into three separate suites so that they can be run in parallel. Also
updates the convention enforced by test/lint/lint-tests.py.
249889bee6 orphanage: avoid vtx iteration when no orphans (furszy)
41ad2be434 mempool: Avoid expensive loop in `removeForBlock` during IBD (Lőrinc)
Pull request description:
During Initial Block Download, the mempool is usually empty, but `CTxMemPool::removeForBlock` is still called for every connected block where we:
* iterate over every transaction in the block even though none will be found in the empty `mapTx`, always leaving `txs_removed_for_block` empty...
* which is pre-allocated regardless with `40 bytes * vtx.size()`, even though it will always remain empty.
Similarly to https://github.com/bitcoin/bitcoin/pull/32730#discussion_r2140691354, this change introduces a minor performance & memory optimization by only executing the loop if any of the affected mempool maps have any contents. The second commit is cherry-picked from there since it's related to this change as well.
ACKs for top commit:
optout21:
ACK 249889bee6
glozow:
ACK 249889bee6
ismaelsadeeq:
reACK 249889bee6
Tree-SHA512: 80d06ff1515164529cdc3ad21db3041bb5b2a1d4b72ba9e6884cdf40c5f1477fee7479944b8bca32a6f0bf27c4e5501fccd085f6041a2dbb101438629cfb9e4b
31c4e77a25 test: fix ReadTopologicalSet unsigned integer overflow (ismaelsadeeq)
Pull request description:
This PR is a simple fix for a potential unsigned integer overflow in ReadTopologicalSet.
We obtain the value of `mask` from fuzz input, which can be the maximum representable value.
Adding 1 to it would then cause an overflow.
The fix skips the addition when the read value is already the maximum.
See https://github.com/bitcoin/bitcoin/pull/30605#discussion_r2215338569 for more context
ACKs for top commit:
maflcko:
lgtm ACK 31c4e77a25
Tree-SHA512: f58d7907f66a0de0ed8d4b1cad6a4971f65925a99f3c030537c21c4d84126b643257c65865242caf7d445b9cbb7a71a1816a9f870ab7520625c4c16cd41979cb
fad040a578 ci: Use APT_LLVM_V in msan task (MarcoFalke)
Pull request description:
This skips compilation of clang by using the apt.
ACKs for top commit:
m3dwards:
ACK fad040a578
willcl-ark:
ACK fad040a578
Tree-SHA512: cc8977a0e97f731b15a2bb9321442d4fc935e310a9cd1993c4ec08ddfd8d7f08a128bbe51ad4d820627bbdcdc748dd58feeec00dee6ee0723e528c546d209f92
During Initial Block Download, the mempool is usually empty, but `CTxMemPool::removeForBlock` is still called for every connected block where we:
* iterate over every transaction in the block even though none will be found in the empty `mapTx`, always leaving `txs_removed_for_block` empty...
* which is pre-allocated regardless with 40 bytes * vtx.size(), even though it will always remain empty.
This change introduces a minor performance optimization by only executing the loop if any of the core mempool maps have any contents.
The call to `MempoolTransactionsRemovedForBlock` and the updates to the rolling fee logic remain unchanged.
The `removeForBlock` was also updated stylistically to match the surrounding methods and a clarification was added to clarify that it affects fee estimation as well.
96da68a38f qa: functional test a transaction running into the legacy sigop limit (Antoine Poinsot)
367147954d qa: unit test standardness of inputs packed with legacy sigops (Antoine Poinsot)
5863315e33 policy: make pathological transactions packed with legacy sigops non-standard. (Antoine Poinsot)
Pull request description:
The Consensus Cleanup soft fork proposal includes a limit on the number of legacy signature
operations potentially executed when validating a transaction. If this change is to be implemented
here and activated by Bitcoin users in the future, we should make transactions that are not valid
according to the new rules non-standard first because it would otherwise be a trivial DoS to
potentially unupgraded miners after the soft fork activates.
ML post: https://gnusha.org/pi/bitcoindev/49dyqqkf5NqGlGdinp6SELIoxzE_ONh3UIj6-EB8S804Id5yROq-b1uGK8DUru66eIlWuhb5R3nhRRutwuYjemiuOOBS2FQ4KWDnEh0wLuA=@protonmail.com/T/#u
ACKs for top commit:
instagibbs:
reACK 96da68a38f
maflcko:
review ACK 96da68a38f🚋
achow101:
ACK 96da68a38f
glozow:
light code review ACK 96da68a38f, looks correct to me
Tree-SHA512: 106ffe62e48952affa31c5894a404a17a3b4ea8971815828166fba89069f757366129f7807205e8c6558beb75c6f67d8f9a41000be2f8cf95be3b1a02d87bfe9
50024620b9 [bench] worst case LimitOrphans and EraseForBlock (glozow)
45c7a4b56d [functional test] orphan resolution works in the presence of DoSy peers (glozow)
835f5c77cd [prep/test] restart instead of bumpmocktime between p2p_orphan_handling subtests (glozow)
b113877545 [fuzz] Add simulation fuzz test for TxOrphanage (Pieter Wuille)
03aaaedc6d [prep] Return the made-reconsiderable announcements in AddChildrenToWorkSet (Pieter Wuille)
ea29c4371e [p2p] bump DEFAULT_MAX_ORPHANAGE_LATENCY_SCORE to 3,000 (glozow)
24afee8d8f [fuzz] TxOrphanage protects peers that don't go over limit (glozow)
a2878cfb4a [unit test] strengthen GetChildrenFromSamePeer tests: results are in recency order (glozow)
7ce3b7ee57 [unit test] basic TxOrphanage eviction and protection (glozow)
4d23d1d7e7 [cleanup] remove unused rng param from LimitOrphans (glozow)
067365d2a8 [p2p] overhaul TxOrphanage with smarter limits (glozow)
1a41e7962d [refactor] create aliases for TxOrphanage Count and Usage (glozow)
b50bd72c42 [prep] change return type of EraseTx to bool (glozow)
3da6d7f8f6 [prep/refactor] make TxOrphanage a virtual class implemented by TxOrphanageImpl (glozow)
77ebe8f280 [prep/test] have TxOrphanage remember its own limits in LimitOrphans (glozow)
d0af4239b7 [prep/refactor] move DEFAULT_MAX_ORPHAN_TRANSACTIONS to txorphanage.h (glozow)
51365225b8 [prep/config] remove -maxorphantx (glozow)
8dd24c29ae [prep/test] modify test to not access TxOrphanage internals (glozow)
44f5327824 [fuzz] add SeedRandomStateForTest(SeedRand::ZEROS) to txorphan (glozow)
15a4ec9069 [prep/rpc] remove entry and expiry time from getorphantxs (glozow)
08e58fa911 [prep/refactor] move txorphanage to node namespace and directory (glozow)
bb91d23fa9 [txorphanage] change type of usage to int64_t (glozow)
Pull request description:
This PR is part of the orphan resolution project, see #27463.
This design came from collaboration with sipa - thanks.
We want to limit the CPU work and memory used by `TxOrphanage` to avoid denial of service attacks. On master, this is achieved by limiting the number of transactions in this data structure to 100, and the weight of each transaction to 400KWu (the largest standard tx) [0]. We always allow new orphans, but if the addition causes us to exceed 100, we evict one randomly. This is dead simple, but has problems:
- It makes the orphanage trivially churnable: any one peer can render it useless by spamming us with lots of orphans. It's possible this is happening: "Looking at data from node alice on 2024-09-14 shows that we’re sometimes removing more than 100k orphans per minute. This feels like someone flooding us with orphans." [1]
- Effectively, opportunistic 1p1c is useless in the presence of adversaries: it is *opportunistic* and pairs a low feerate tx with a child that happens to be in the orphanage. So if nothing is able to stay in orphanages, we can't expect 1p1cs to propagate.
- This number is also often insufficient for the volume of orphans we handle: historical data show that overflows are pretty common, and there are times where "it seems like [the node] forgot about the orphans and re-requested them multiple times." [1]
Just jacking up the `-maxorphantxs` number is not a good enough solution, because it doesn't solve the churnability problem, and the effective resource bounds scale poorly.
This PR introduces numbers for {global, per-peer} {memory usage, announcements + number of inputs}, representing resource limits:
- The (constant) **global latency score limit** is the number of unique (wtxid, peer) pairs in the orphanage + the number of inputs spent by those (deduplicated) transactions floor-divided by 10 [2]. This represents a cap on CPU or latency for any given operation, and does not change with the number of peers we have. Evictions must happen whenever this limit is reached. The primary goal of this limit is to ensure we do not spend more than a few ms on any call to `LimitOrphans` or `EraseForBlock`.
- The (variable) **per-peer latency score limit** is the global latency score limit divided by the number of peers. Peers are allowed to exceed this limit provided the global announcement limit has not been reached. The per-peer announcement limit decreases with more peers.
- The (constant) **per-peer memory usage reservation** is the amount of orphan weight [3] reserved per peer [4]. Reservation means that peers are effectively guaranteed this amount of space. Peers are allowed to exceed this limit provided the global usage limit is not reached. The primary goal of this limit is to ensure we don't oom.
- The (variable) **global memory usage limit** is the number of peers multiplied by the per-peer reservation [5]. As such, the global memory usage limit scales up with the number of peers we have. Evictions must happen whenever this limit is reached.
- We introduce a "Peer DoS Score" which is the maximum between its "CPU Score" and "Memory Score." The CPU score is the ratio between the number of orphans announced by this peer / peer announcement limit. The memory score is the total usage of all orphans announced by this peer / peer usage reservation.
Eviction changes in a few ways:
- It is triggered if either limit is exceeded.
- On each iteration of the loop, instead of selecting a random orphan, we select a peer and delete 1 of its announcements. Specifically, we select the peer with the highest DoS score, which is the maximum between its CPU DoS score (based on announcements) and Memory DoS score (based on tx weight). After the peer has been selected, we evict the oldest orphan (non-reconsiderable sorted before reconsiderable).
- Instead of evicting orphans, we evict announcements. An orphan is still in the orphanage as long as there is 1 peer announcer. Of course, over the course of several iteration loops, we may erase all announcers, thus erasing the orphan itself. The purpose of this change is to prevent a peer from being able to trigger eviction of another peer's orphans.
This PR also:
- Reimplements `TxOrphanage` as single multi-index container.
- Effectively bounds the number of transactions that can be in a peer's work set by ensuring it is a subset of the peer's announcements.
- Removes the `-maxorphantxs` config option, as the orphanage no longer limits by unique orphans.
This means we can receive 1p1c packages in the presence of spammy peers. It also makes the orphanage more useful and increases our download capacity without drastically increasing orphanage resource usage.
[0]: This means the effective memory limit in orphan weight is 100 * 400KWu = 40MWu
[1]: https://delvingbitcoin.org/t/stats-on-orphanage-overflows/1421
[2]: Limit is 3000, which is equivalent to one max size ancestor package (24 transactions can be missing inputs) for each peer (default max connections is 125).
[3]: Orphan weight is used in place of actual memory usage because something like "one maximally sized standard tx" is easier to reason about than "considering the bytes allocated for vin and vout vectors, it needs to be within N bytes..." etc. We can also consider a different formula to encapsulate more the memory overhead but still have an interface that is easy to reason about.
[4]: The limit is 404KWu, which is the maximum size of an ancestor package.
[5]: With 125 peers, this is 50.5MWu, which is a small increase from the existing limit of 40MWu. While the actual memory usage limit is higher (this number does not include the other memory used by `TxOrphanage` to store the outpoints map, etc.), this is within the same ballpark as the old limit.
ACKs for top commit:
marcofleon:
ReACK 50024620b9
achow101:
light ACK 50024620b9
instagibbs:
ACK 50024620b9
theStack:
Code-review ACK 50024620b9
Tree-SHA512: 270c11a2d116a1bf222358a1b4e25ffd1f01e24da958284fa8c4678bee5547f9e0554e87da7b7d5d5d172ca11da147f54a69b3436cc8f382debb6a45a90647fd
5fa34951ea test: avoid unneeded block header hash -> integer conversions (Sebastian Falbesoner)
2118301d77 test: rename CBlockHeader `.hash` -> `.hash_hex` for consistency (Sebastian Falbesoner)
23be0ec2f0 test: rename CBlockHeader `.rehash()`/`.sha256` -> `.hash_int` for consistency (Sebastian Falbesoner)
8b09cc350a test: remove bare CBlockHeader `.rehash()`/`.calc_sha256()` calls (Sebastian Falbesoner)
0716382c20 test: remove header hash caching in CBlockHeader class (Sebastian Falbesoner)
0f044e82bd test: avoid direct block header modification in feature_block.py (Sebastian Falbesoner)
f3c791d2e3 test: refactor: dedup `CBlockHeader` serialization (Sebastian Falbesoner)
Pull request description:
Similar to what #32421 did for `CTransaction` instances, this PR aims to improve the block hash determination of `CBlockHeader`/`CBlock` (the latter is a subclass of the former) instances by removing the block header caching mechanism and introducing consistent naming. Without the statefulness, sneaky testing bugs like #32742 and #32823 are less likely to happen in the future. Note that performance is even less of an issue here compared to `CTransaction`, as we only need to hash 80 bytes, which is less than typical standard transaction sizes [2].
The only instance where the testing logic was relying on caching (i.e. we want to return an outdated value) is tackled in the second commit, the rest should be straight-forward to review, especially for contributors who already reviewed #32421.
Summary table showing block hash determaination before/after this PR:
| Task | master | PR |
|:-----------------------------------|:-------------------------|:-------------|
| get block header hash (hex string) | `.hash`[1] | `.hash_hex` |
| get block header hash (integer) | `rehash()`, `.sha256`[1] | `.hash_int` |
[1] = returned value might be `None` or out-of-date, if rehashing function wasn't called after modification
[2] = the only exception I could think of are transaction with pay-to-anchor (P2A) outputs
ACKs for top commit:
rkrux:
re-ACK 5fa34951ea modulo failing CI due to silent merge conflict.
maflcko:
re-ACK 5fa34951ea🎩
danielabrozzoni:
reACK 5fa34951ea
Tree-SHA512: 3d13540012654effa063846958a3166d56c1bcb58e1321f52ca4d5c3bcb7abdea72c54d1fb566d04e636d84d06a41d293e16232dbe5d5e78a73c903bb6ffc80d
This is required in the process_message(s) fuzz targets to avoid leaking
the next write time from one run to the next. Also, disable it
completely because it is not needed and due to leveldb-internal
non-determinism.
The PeerManager has several members, such as the FastRandomContext,
which need to be reset before every run to avoid leaking state from one
run into the next.
Also, style fixups in p2p_handshake.cpp, where this code is copied from.
This adds a missing catch for BaseException (e.g. SystemExit), which
would otherwise be silently ignored.
Also, remove the redundant other catches, which are just calling
log.exception with a redundant log message.
It's useful to have an end-to-end test in addition to the unit test to sanity check the RPC error as
well as making sure the transaction is otherwise fully standard.
The Consensus Cleanup soft fork proposal includes a limit on the number of legacy signature
operations potentially executed when validating a transaction. If this change is to be implemented
here and activated by Bitcoin users in the future, we should prevent the ability for someone to
broadcast a transaction through the p2p network that is not valid according to the new rules. This
is because if it was possible it would be a trivial DoS to potentially unupgraded miners after the
soft fork activates.
We do not know for sure whether users will activate the Consensus Cleanup. However if they do such
transactions must have been made non-standard long in advance, due to the time it takes for most
nodes on the network to upgrade. In addition this limit may only be run into by pathological
transactions which pad the Script with sigops but do not use actual signatures when spending, as
otherwise they would run into the standard transaction size limit.
Note that we unfortunately can't use a scripted diff here, as the
`sha256` symbol is also used for other instances (e.g. as function
in hashlib, or in the `UTXO` class in p2p_segwit.py).
Since the previous commit, CBlockHeader/CBlock object calls to the
methods `.rehash()` and `.calc_sha256()` are effectively no-ops
if the returned value is not used, so we can just remove them.
Rather than block hashes (represented by the fields `.sha256` and
`.hash`) being stateful, simply compute them on-the-fly. This ensures
that the correct values are always returned and takes the burden of
rehashing from test writers, making the code shorter overall. In a
first step, the fields are kept at the same name with @property
functions as drop-in replacements, for a minimal diff. In later commits,
the names are changed to be more descriptive and indicating the return
type of the block hash.
This is a preparatory commit for removing the header hash
caching in the CBlockHeader class. In order to not lose the
old block hash, necessary for updating the internal state of
the test (represented by `self.block_heights` and `self.blocks`),
we should only modify it within the `update_block` method.
Note that we can't call `.serialize()` directly in
the `.calc_sha256()` method, as this could wrongly lead
to the serialization of the derived class (CBlock) if
called from an instance there.
This check ensures that when migrating a legacy wallet with a direct
filename, the backup file is named as expected.
Co-authored-by: Ava Chow <github@achow101.com>
Benchmarks indicated that obfuscating multiple bytes already gives an order of magnitude speed-up, but:
* GCC still emitted scalar code;
* Clang’s auto-vectorized loop ran on the slow unaligned-load path.
Fix contains:
* peeling the misaligned head enabled the hot loop starting at an 8-byte address;
* `std::assume_aligned<8>` tells the optimizer the promise holds - required to keep Apple Clang happy;
* manually unrolling the body to 64 bytes enabled GCC to auto-vectorize.
Note that `target.size() > KEY_SIZE` condition is just an optimization, the aligned and unaligned loops work without it as well - it's why the alignment calculation still contains `std::min`.
> C++ compiler .......................... GNU 14.2.0
| ns/byte | byte/s | err% | ins/byte | cyc/byte | IPC | bra/byte | miss% | total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
| 0.03 | 32,464,658,919.11 | 0.0% | 0.50 | 0.11 | 4.474 | 0.08 | 0.0% | 5.29 | `ObfuscationBench`
> C++ compiler .......................... Clang 20.1.7
| ns/byte | byte/s | err% | ins/byte | cyc/byte | IPC | bra/byte | miss% | total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
| 0.02 | 41,231,547,045.17 | 0.0% | 0.30 | 0.09 | 3.463 | 0.02 | 0.0% | 5.47 | `ObfuscationBench`
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
All former `std::vector<std::byte>` keys were replaced with `uint64_t` (we still serialize them as vectors but convert immediately to `uint64_t` on load).
This is why some tests still generate vector keys and convert them to `uint64_t` later instead of generating them directly.
In `Obfuscation::Unserialize` we can safely throw an `std::ios_base::failure` since during mempool fuzzing `mempool_persist.cpp#L141` catches and ignored these errors.
> C++ compiler .......................... GNU 14.2.0
| ns/byte | byte/s | err% | ins/byte | cyc/byte | IPC | bra/byte | miss% | total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
| 0.04 | 28,365,698,819.44 | 0.0% | 0.34 | 0.13 | 2.714 | 0.07 | 0.0% | 5.33 | `ObfuscationBench`
> C++ compiler .......................... Clang 20.1.7
| ns/byte | byte/s | err% | ins/byte | cyc/byte | IPC | bra/byte | miss% | total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
| 0.08 | 13,012,464,203.00 | 0.0% | 0.65 | 0.28 | 2.338 | 0.13 | 0.8% | 5.50 | `ObfuscationBench`
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
This is meant to focus the usages to narrow the scope of the obfuscation optimization.
`Obfuscation::Xor` is mostly a move.
Co-authored-by: maflcko <6399679+maflcko@users.noreply.github.com>
Since `FastRandomContext` delegates to `GetRandBytes` anyway, we can simplify new key generation to a Write/Read combo, unifying the flow of enabling obfuscation via `Read`.
The comments were also adjusted to clarify that the `m_obfuscation` field affects the behavior of `Read` and `Write` methods.
These changes are meant to simplify the diffs for the riskier optimization commits later.
Mechanical refactor of the low-level "xor" wording to signal the intent instead of the implementation used.
The renames are ordered by heaviest-hitting substitutions first, and were constructed such that after each replacement the code is still compilable.
-BEGIN VERIFY SCRIPT-
sed -i \
-e 's/\bGetObfuscateKey\b/GetObfuscation/g' \
-e 's/\bxor_key\b/obfuscation/g' \
-e 's/\bxor_pat\b/obfuscation/g' \
-e 's/\bm_xor_key\b/m_obfuscation/g' \
-e 's/\bm_xor\b/m_obfuscation/g' \
-e 's/\bobfuscate_key\b/m_obfuscation/g' \
-e 's/\bOBFUSCATE_KEY_KEY\b/OBFUSCATION_KEY_KEY/g' \
-e 's/\bSetXor(/SetObfuscation(/g' \
-e 's/\bdata_xor\b/obfuscation/g' \
-e 's/\bCreateObfuscateKey\b/CreateObfuscation/g' \
-e 's/\bobfuscate key\b/obfuscation key/g' \
$(git ls-files '*.cpp' '*.h')
-END VERIFY SCRIPT-
The two tests are doing different things - `xor_roundtrip_random_chunks` does black-box style property-based testing to validate that certain invariants hold - that deobfuscating an obfuscation results in the original message (higher level, it doesn't have to know about the implementation details).
The `xor_bytes_reference` test makes sure the optimized xor implementation behaves in every imaginable scenario exactly as the simplest possible obfuscation - with random chunks, random alignment, random data, random key.
Since we're touching the file, other related small refactors were also applied:
* `nullpt` typo fixed;
* manual byte-by-byte xor key creations were replaced with `_hex` factories;
* since we're only using 64 bit keys in production, smaller keys were changed to reflect real-world usage;
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Since 31 byte xor-keys are not used in the codebase, using the common size (8 bytes) makes the benchmarks more realistic.
Co-authored-by: maflcko <6399679+maflcko@users.noreply.github.com>
This test was introduced in #28251 to ensure that the mempool is not
trimmed in the middle of a package evaluation and the m_view cache
is updated when evictions and replacements happen so coins are no longer
visible in subsequent package transactions. These two things have
coverage in other tests as well, and are pretty unlikely to happen.
This test is also brittle: it requires evaluation of the parents in a
particular order, and creates a transaction that itself is not
enough to trigger eviction but will be pushed out immediately by the
package spending from it. While the current magic number 2000 works, we
do not have a way to query remaining space in the mempool if mempool
data structures change, and it can differ across platforms.
f5647c6c5a depends: fix libevent _WIN32_WINNT usage (fanquake)
Pull request description:
Starting with version 13.x, the mingw headers will define the value of
`NTDDI_VERSION`, based on the value of `_WIN32_WINNT`, if that version is <
Windows 10. Given that libevent was undefining our `_WIN32_WINNT`, and
redefining it to a value < Windows 10 (`0x0501`), `NTDDI_VERSION` was also
being defined to that value, leading to functions not being exposed in
the mingw-w64 headers; see here: 9c2668ef77/mingw-w64-headers/include/iphlpapi.h (L36-L41).
Imports a commit from usptream ([a14ff91254f40cf36e0fee199e26fb11260fab49](a14ff91254)).
Fixes#32707.
ACKs for top commit:
willcl-ark:
crACK f5647c6c5a
Tree-SHA512: eb429457a4af6191dd27ef3d1087667c5304ff0f49d4c6824883651e3c2dbab5d9784fa1f170402f23cd9238005c5214e0a71a4160562a59dfa35618dc702132
This argument might have been used in the legacy wallets, but I don't
see any implementation using this argument in the SQLite wallets.
Removing it cleans up the code a bit.
4bb4c86599 test: document HOST for get_previous_releases.py (Sjors Provoost)
609203d507 test: stop signing previous releases >= v28.2 (Sjors Provoost)
c6dc2c29f8 test: replace v28.0 with notarized v28.2 (Sjors Provoost)
5bd73d96a3 test: fix macOS detection (Sjors Provoost)
Pull request description:
Since https://github.com/bitcoin/bitcoin/pull/31407 macOS guix builds are signed and notarized. This was included in v29 and backported to 28.x.
This PR bumps the v28.0 previous release binary to v28.2 and adjusts the test that uses it. Additionally it no longer manually code signs binaries >= v28.2.
While testing on an M4 mac and redownloading all the binaries, I noticed that `platform == "arm64-apple-darwin"` doesn't actually work. This initially used `args.platform` in #26694, but that was changed to just `platform` in #32219.
So the first commit switches this to use `args.host`. I manually tested on Intel macOS 13.7.6 that code-signing still isn't needed there (when downloading using a script).
Also documented that you can set `HOST`.
ACKs for top commit:
m3dwards:
ACK 4bb4c86599
maflcko:
review ACK 4bb4c86599🚏
Tree-SHA512: b4803d39a21cb622fd2388a0528b76d2b502956e2505385d3da201143b0afcf6f9d71c8c28937f27b70d2588fb6da677da058bdcd67b90fb53617acc3a727818
61e800e75c test: headers sync timeout (stringintech)
Pull request description:
When reviewing PR #32051 and considering which functional tests might need to be adapted/extended accordingly, I noticed there appears to be limited functional test coverage for header sync timeouts and thought it might be helpful to add one.
This test attempts to cover two scenarios:
1. **Normal peer timeout behavior:** When a peer fails to respond to initial getheaders requests within the timeout period, it should be disconnected and the node should attempt to sync headers from the next available peer.
2. **Noban peer behavior:** When a peer with noban privileges times out, it should remain connected while the node still attempts to sync headers from other peers.
ACKs for top commit:
maflcko:
re-ACK 61e800e75c 🗝
stratospher:
reACK 61e800e7.
Tree-SHA512: b8a867e7986b6f0aa00d81a84b205f2bf8fb2e6047a2e37272e0244229d1f43020e9031467827dabbfe7849a91429f2685e00a25356e2ed477fa1a035fa0b1fd
28416f367a test: fix intermittent failure in rpc_invalidateblock.py (stratospher)
Pull request description:
resolves#32965.
node1 (with 24 blocks) causes node0 (with 6 blocks + 1 extra header) to silently reorg. so move the subtest to a point before the 20 blocks are generated so that node1's state doesn't cause node0 to silently reorg.
ACKs for top commit:
maflcko:
lgtm ACK 28416f367a
mzumsande:
Code Review ACK 28416f367a
Tree-SHA512: f6cc682b8e5416125f887c094d5e291dd37f0bfc41d7c0de218f3e24fa1ea0cd642f7a1e362f3127f68cde725a67f3054501326b9bd25f0caa9a05de7d0052b0
This adds a large simulation fuzz test for all TxOrphanage public interface
functions, using a mix of comparison with expected behavior (in case it is
fully specified), and testing of properties exhibited otherwise.
This is preparation for the simulation fuzz test added in a later commit. Since
AddChildrenToWorkSet consumes randomness, there is no way for the simulator to
exactly predict its behavior. By returning the set of made-reconsiderable announcements
instead, the simulator can instead test that it is *a* valid choice, and then
apply it to its own data structures.
For the default number of peers (125), allows each to relay a default
descendant package (up to 25-1=24 can be missing inputs) of small (9
inputs or fewer) transactions out of order.
This limit also gives acceptable bounds for worst case LimitOrphans iterations.
Functional tests aren't changed to check for larger cap because it would
make the runtime too long.
Also deletes the now-unused DEFAULT_MAX_ORPHAN_TRANSACTIONS.
This is largely a reimplementation using boost::multi_index_container.
All the same public methods are available. It has an index by outpoint,
per-peer tracking, peer worksets, etc.
A few differences:
- Limits have changed: instead of a global limit of 100 unique orphans,
we have a maximum number of announcements (which can include duplicate
orphans) and a global memory limit which scales with the number of
peers.
- The maximum announcements limit is 100 to match the original limit,
but this is actually a stricter limit because the announcement count
is not de-duplicated.
- Eviction strategy: when global limits are reached, a per-peer limit
comes into play. While limits are exceeded, we choose the peer whose
“DoS score” (max usage / limit ratio for announcements and memory
limits) is highest and evict announcements by entry time, sorting
non-reconsiderable ones before reconsiderable ones. Since announcements
are unique by (wtxid, peer), as long as 1 announcement remains for a
transaction, it remains in the orphanage.
- This eviction strategy means no peer can influence the eviction of
another peer’s orphans.
- Also, since global limits are a multiple of per-peer limits, as long
as a peer does not exceed its limits, its orphans are protected from
eviction.
- Orphans no longer expire, since older announcements are generally
removed before newer ones.
- GetChildrenFromSamePeer returns the transactions from newest to
oldest.
Co-authored-by: Pieter Wuille <pieter@wuille.net>
Move towards a model where TxOrphanage is initialized with limits that
it remembers throughout its lifetime.
Remove the param. Limiting by number of unique orphans will be removed
in a later commit.
Now that -maxorphantx is gone, this does not change the node behavior.
The parameter is only used in tests.
c18bf0bd9b refactor: cleanup index logging (Sjors Provoost)
Pull request description:
This PR removes the use of `__func__` from index logging, since we have `-logsourcelocations`.
It also improves readability by putting `GetName()` in a more logical place.
Before
> coinstatsindex: best block of the index not found. Please rebuild the index.
After:
> best block of coinstatsindex not found. Please rebuild the index.
I found myself maintaining this commit as part of https://github.com/Sjors/bitcoin/pull/86, but since that might never land here, it seemed better to split it into its own PR (or get rid of it).
ACKs for top commit:
l0rinc:
Lightweight code review ACK c18bf0bd9b
maflcko:
review ACK c18bf0bd9b🚣
Tree-SHA512: 755948371e3ff7a5515b63ce48075631ec7868d69c3c1469176d5be0e8b28e1c071e206ae3f7320f87d8c441f815894acfef61621f05795b5ff6b8a5a3031e3b
This adds an `iters` parameter to DoWork(), which controls how much work it is
allowed to do right now.
Additionally, DoWork() won't stop at just getting everything ACCEPTABLE, but if
there is work budget left, will also attempt to get every cluster linearized
optimally.
node1 (with 24 blocks) causes node0 (with 6 blocks) to silently
reorg. so move the subtest to a point before the 20 blocks are
generated so that node1's state doesn't cause node0 to silently
reorg.
44f3bae300 depends: Force `CMAKE_EXPORT_NO_PACKAGE_REGISTRY=TRUE` (Hennadii Stepanov)
Pull request description:
When using CMake policies 3.14 and below, the `export(PACKAGE)` command by default populates the user package registry, which is stored outside the build tree. Setting the `CMAKE_EXPORT_NO_PACKAGE_REGISTRY` variable disables this side effect.
In CMake 3.15 and later, this behavior is disabled by default, and the variable has no effect.
This PR forces `CMAKE_EXPORT_NO_PACKAGE_REGISTRY=TRUE` globally, rather than managing it for each dependency package individually rather. It may be reverted once all CMake-based packages have updated to policies 3.15 or newer.
Fixes https://github.com/bitcoin/bitcoin/issues/32938.
ACKs for top commit:
fanquake:
ACK 44f3bae300
Tree-SHA512: 0aac398b7182e80185b064d59f81aece4d8477a609fad9cc3fee317da2aff43b66ef7db1efec0135b4f0feaad23b1db664e33bd035fe659712c5b2a9bf2d6fb6
fad191ff48 ci: Avoid cd into build dir (MarcoFalke)
Pull request description:
Changing into the build dir is confusing and brittle, because the following commands implicitly assume it. So they could break on unrelated changes.
The changes are required for stuff like:
* cmake presets (see https://github.com/bitcoin/bitcoin/pull/30871#issuecomment-2344031208)
* meta ci tests (like https://github.com/bitcoin/bitcoin/pull/32874)
So remove the `cd` and just make the build dir explicit.
ACKs for top commit:
hebasto:
ACK fad191ff48, I have reviewed the code and it looks OK.
Tree-SHA512: a88a9341445ffe28a0dac3815f235ec8eb0459d10a91a80829fd3184762d3c807d0f68c56243b20c04a6efa5becd8a7fad568f43c2b1e6af1ff8ba07b140ef87
12a6959892 cmake: Drop no longer necessary "cmakeMinimumRequired" object (Hennadii Stepanov)
Pull request description:
The minimum required CMake version is 3.22:6a13a6106e/CMakeLists.txt (L10)
ACKs for top commit:
fanquake:
ACK 12a6959892 - has been unneeded since it was introduced (minimum was already 3.22).
Tree-SHA512: 26f97662bfe52986e19e38dbf4ab8e1e7558bc78c3a65593cbecd1f35887bba7a9f7d8a3d08ccfab8396f41c2334cdad5b0e503999a759cfa158d3bb8d0d14d7
84ef5524d5 fix spelling in tor.md docs (stutxo)
Pull request description:
This PR is to fix some spelling mistakes i found of the word occurrences! there are two occurrences of this mistake.
thanks!
ACKs for top commit:
maflcko:
lgtm ACK 84ef5524d5
willcl-ark:
ACK 84ef5524d5
delta1:
ACK 84ef5524d5
Tree-SHA512: 4ba71b772fdc8cf36ada7493d29fb5b312a7a6085099347162eb3495db4de984b0417b7861f2927c617cbd552741356e26688479601bdf7e835c15e097aa28f3
8f766f39df ci: enable -Werror=dev (fanquake)
7b420ca834 guix: configure with -Werror=dev (fanquake)
44097ddb19 cmake: enable -Werror=dev in dev-mode preset (fanquake)
Pull request description:
Pass `-Werror=dev` in the CI, Guix and the `dev-mode` preset.
https://cmake.org/cmake/help/latest/manual/cmake.1.html#cmdoption-cmake-Werror:
> Make developer warnings errors.
> Make warnings that are meant for the author of the CMakeLists.txt files errors. By default this will also turn on deprecated warnings as errors.
Pulled out of #32865.
ACKs for top commit:
Sjors:
re-ACK 8f766f39df
hebasto:
ACK 8f766f39df, tested on Ubuntu 24.04.
Tree-SHA512: 0fa321b77d2519b5249d90590664c4e5938ac86209b068658647adf97ab55ea4d54c913aae2f622385fe2f41d7c851cd5d7371905fdad38b66cb124371e16ac7
Expiry is going away in a later commit.
This is only an RPC change. Behavior of the orphanage does not change.
Note that getorphantxs is marked experimental.
a60f863d3e scripted-diff: Replace GenTxidVariant with GenTxid (marcofleon)
c8ba199598 Remove old GenTxid class (marcofleon)
072a198ea4 Convert remaining instances of GenTxid to GenTxidVariant (marcofleon)
1b528391c7 Convert `txrequest` to GenTxidVariant (marcofleon)
bde4579b07 Convert `txdownloadman_impl` to GenTxidVariant (marcofleon)
c876a892ec Replace GenTxid with Txid/Wtxid overloads in `txmempool` (marcofleon)
de858ce2be move-only: make GetInfo a private CTxMemPool member (stickies-v)
eee473d9f3 Convert `CompareInvMempoolOrder` to GenTxidVariant (marcofleon)
243553d590 refactor: replace get_iter_from_wtxid with GetIter(const Wtxid&) (stickies-v)
fcf92fd640 refactor: make CTxMemPool::GetIter strongly typed (marcofleon)
11d28f21bb Implement GenTxid as a variant (marcofleon)
Pull request description:
Part of the [type safety refactor](https://github.com/bitcoin/bitcoin/pull/32189).
This PR changes the GenTxid class to a variant, which holds both Txids and Wtxids. This provides compile-time type safety and eliminates the manual type check (bool m_is_wtxid). Variables that can be either a Txid or a Wtxid are now using the new GenTxid variant, instead of uint256.
ACKs for top commit:
w0xlt:
ACK a60f863d3e
dergoegge:
Code review ACK a60f863d3e
maflcko:
review ACK a60f863d3e🎽
theStack:
Code-review ACK a60f863d3e
Tree-SHA512: da9b73b7bdffee2eb9281a409205519ac330d3336094d17681896703fbca8099608782c9c85801e388e4d90af5af8abf1f34931f57bbbe6e9674d802d6066047
fa894b0f3e log: Properly log warnings with warn loglevel in addrdb (MarcoFalke)
Pull request description:
The logging in addrdb is confusing, because it uses `LogPrintf` (info level) to log warnings.
Fix this by properly using the `warn` level, where needed. Also, drop unused trailing `\n` while touching the lines.
ACKs for top commit:
stickies-v:
ACK fa894b0f3e
dergoegge:
utACK fa894b0f3e
Tree-SHA512: 96d3823623ea8e1698e8cb541ca97cbab7b2a9934b2f894884171045abbca7be796f07965082e997001c97d06d1e0c4d13b29354eb4fe71c3a2ee680eada5516
- don't log function name
- take into account that GetName() always ends with " index"
- replace deprecated LogPrintf with LogInfo
- remove trailing \n
- adjusted log level where needed
fa8862723c fuzz: CheckGlobals in init (MarcoFalke)
fa26bfde98 test: Avoid resetting mocktime in testing setup (MarcoFalke)
fa6b45fa8e Add SetMockTime for time_point types (MarcoFalke)
Pull request description:
(Tracking issue https://github.com/bitcoin/bitcoin/issues/29018)
During fuzzing, `AppInitParameterInteraction` may actually disable a previously set mocktime. This is confusing and can also cause non-determinism.
Fix this issue, by
* fixing the erroneous `-mocktime` parsing in `AppInitParameterInteraction`.
* adding the missing `SetMockTime` calls to the affected fuzz init functions.
* adding a `CheckGlobals` to the fuzz init, to prevent this issue in the future.
This can be tested by
* Cherry-picking the `CheckGlobals`-commit onto current master and observing a fuzz failure in the touched fuzz targets.
* Reverting the touched fuzz fixups and observing a fuzz failure for each target.
ACKs for top commit:
w0xlt:
ACK fa8862723c
dergoegge:
utACK fa8862723c
Tree-SHA512: 5a9400f0467c82fa224713af4cc2b525afbefefc7c3f419077110925ad7af6c7fda3dcd2b50f7facf0ee7df2547c6ac20336906d707adcdfd1d652a9d9a735fe
Starting with version 13.x, the mingw headers will define the value of
NTDDI_VERSION, based on the value of _WIN32_WINNT, if that version is <
Windows 10. Given that libevent was undefining our _WIN32_WINNT, and
redefining it to a value < Windows 10 (0x0501), NTDDI_VERSION was also
being defined to that value, leading to functions not being exposed in
the mingw-w64 headers; see here:
9c2668ef77/mingw-w64-headers/include/iphlpapi.h (L36-L41).
Imports a commit from usptream (a14ff91254f40cf36e0fee199e26fb11260fab49).
Fixes#32707.
f43571010e Resolve guix non-determinism with emplace_back instead of push_back (Ava Chow)
Pull request description:
For some reason, building x86_64-w64-mingw32 on x86_64 and aarch64 results in a single instruction difference which can be traced down to prevector.h:174. The ultimate caller of this is the copy constructor for a prevector that ends up being called by std::vector::push_back in walletmodel.cpp:183. By replacing the push_back with an emplace_back, somehow this non-determinism goes away.
Closes#32923
ACKs for top commit:
l0rinc:
code review ACK f43571010e
Sjors:
utACK f43571010e
maflcko:
lgtm ACK f43571010e
Tree-SHA512: 5bf0571f32cb72efc0c533e16d2704cfc3a79bcef2943f0892743572808610fb00ca8ab41223897536f8e5090bf4030735be910942de8116652d02bc3f231e2e
83bb414557 test: less ambiguous error if bitcoind is missing (Sjors Provoost)
Pull request description:
Before this change, when a functional test is run without building the source, the error message suggested that previous release binaries were missing.
When no previous release version is set, make the error message more specifically about bitcoind.
To test, try this before and after:
```sh
git clean -dfx
cmake -B build
build/test/functional/mining_basic.py
cmake --build build
build/test/functional/mining_basic.py
build/test/functional/wallet_backwards_compatibility.py
test/get_previous_releases.py
build/test/functional/wallet_backwards_compatibility.py
```
ACKs for top commit:
achow101:
ACK 83bb414557
janb84:
ACK 83bb414557
w0xlt:
ACK 83bb414557
Tree-SHA512: c6df65019de99d6c214951cf70944c4ddca9b635c5ab60ac2c47e4589478e9c65d5e079c394ace9b470a7eaeea3c9cf68b7246dd413e802c4a1e071913a7fc32
150b5c99ca wallet: replace `reload_wallet` with inline functionality (rkrux)
0f86da382d wallet: remove dead code in legacy wallet migration (rkrux)
Pull request description:
A discussion on a previous [PR 32481](https://github.com/bitcoin/bitcoin/pull/32481#discussion_r2145152084) related to legacy wallet dead
code removal made me realize that checking if the legacy
wallet was loaded prior to the start of the migration is not
required ever since legacy wallets can't be loaded in the first
place. I also verified that the `load_on_start` persistent
setting can also not cause the legacy wallets to be loaded, which
further makes the case for removal of the above mentioned checks
during migration.
The current test coverage also shows these lines uncovered.
ACKs for top commit:
achow101:
ACK 150b5c99ca
furszy:
ACK 150b5c99ca
Tree-SHA512: 9bc7043cac1f4051228557208895e43648de3c7ffae6860c0676d1aa2db3a8ed3a09d1f9defacd96ca50bbb9699ba86652ccb0c5e55cc88be248a1fe727c13d9
d7fca5c171 clusterlin: add big comment explaning the relation between tests (Pieter Wuille)
b64e61d2de clusterlin: abstract try-permutations into ExhaustiveLinearize function (Pieter Wuille)
1fa55a64ed clusterlin tests: verify that chunks are minimal (Pieter Wuille)
da23ecef29 clusterlin tests: support non-empty ReadTopologicalSubset() (Pieter Wuille)
94f3e17c33 clusterlin tests: compare with fuzz-provided linearizations (Pieter Wuille)
5f92ebee0d clusterlin tests: compare with fuzz-provided topological sets (Pieter Wuille)
6e37824ac3 clusterlin tests: optimize clusterlin_simple_linearize (Pieter Wuille)
98c1c88b6f clusterlin tests: separate testing of SimpleLinearize and Linearize (Pieter Wuille)
10e90f7aef clusterlin tests: make SimpleCandidateFinder always find connected (Pieter Wuille)
a38c38951e clusterlin tests: separate testing of Search- and SimpleCandidateFinder (Pieter Wuille)
77a432ee70 clusterlin tests: count SimpleCandidateFinder iterations better (Pieter Wuille)
Pull request description:
Part of the cluster mempool project: #30289
The current cluster linearization fuzz tests contain two tests which combine testing of production code with testing of the test code itself:
* `clusterlin_search_finder`: establishes the correctness of `SearchCandidateFinder` by comparing against both `SimpleCandidateFinder` and `ExhaustiveCandidateFinder` (which is even more simple than `SimpleCandidateFinder`). If `SimpleCandidateFinder` works correctly, then this comparison with `ExhaustiveCandidateFinder` is redundant. If it isn't, we ought to find that in a test specific to `SimpleCandidateFinder` rather than as a side-effect of testing `SearchCandidateFinder`. Split this functionality out into a new `clusterlin_simple_finder`.
* `clusterlin_linearize`: establishes the correctness of `Linearize` by comparing against both `SimpleLinearize` and literally every valid linearization for the cluster. Again, if `SimpleLinearize` works correctly, then this comparison with all valid linearizations is redundant, and if it isn't we should find it in a test for `SimpleLinearize`. Do so by splitting off that functionality into `clusterlin_simple_linearize`.
After that, a few general improvements to the affected tests are made (comparing with linearizations and subsets read from the fuzz input, plus a performance improvement).
ACKs for top commit:
marcofleon:
Re ACK d7fca5c171
ismaelsadeeq:
re-ACK d7fca5c171
monlovesmango:
ACK d7fca5c171
Tree-SHA512: 33cb76bd9b9547a5f3ee231fa452e928f064ad03af98e3d9e64246eb972f2b026c13e7367257ccdac1ae57982ee8ef98c907684588ecbb4bc4c82cbec160b3e8
For some reason, building x86_64-w64-mingw32 on x86_64 and aarch64
results in a single instruction difference which can be traced down to
prevector.h:174. The ultimate caller of this is the copy constructor for
a prevector that ends up being called by std::vector::push_back in
walletmodel.cpp:183. By replacing the push_back with an emplace_back,
somehow this non-determinism goes away.
When using CMake policies 3.14 and below, the `export(PACKAGE)` command
by default populates the user package registry, which is stored outside
the build tree. Setting the `CMAKE_EXPORT_NO_PACKAGE_REGISTRY` variable
disables this side effect.
In CMake 3.15 and later, this behavior is disabled by default, and the
variable has no effect.
Changing into the build dir is confusing and brittle.
This can be reviewed using the git option `--word-diff-regex=.`.
Also:
* add missing -j1 to the fallback that prints a verbose build failure
* remove quotes around $GOAL in the fallback
fa2fbaa4a2 bench: Avoid tmp files in pwd (MarcoFalke)
Pull request description:
It is a bit confusing that one bench run, when aborted, could leave behind temp files in the current working directory. It is similarly confusing to delete those files in the next run of bench.
Fix all issues by using `BasicTestingSetup`, which provides a proper temp folder to use and also cleans up after itself.
Can be tested via:
```
( echo 'my file content' > streams_tmp ) && ls streams_tmp && ./bld-cmake/bin/bench_bitcoin --filter=FindByte && ls streams_tmp
```
Previously the file would be deleted, now it is kept.
ACKs for top commit:
stickies-v:
ACK fa2fbaa4a2
Tree-SHA512: 33798030f990d1b4c95be4682d8dbfad95e8716d5fc0b99d65937196f2ced1ba649193c2adba4155f4eec9fd06e16be6667f3c3705af1880f47b2ff57a76243b
fa0528479d test: Add missing convert_to_json_for_cli (MarcoFalke)
Pull request description:
Currently the tests are failing on current master, if they use the `--usecli` flag. See https://github.com/bitcoin/bitcoin/runs/45676472375, https://cirrus-ci.com/task/5707897310543872.
This can be reproduced locally via:
```
./bld-cmake/test/functional/wallet_reorgsrestore.py --usecli
```
Fix it by adding the missing `hash_or_height=self.convert_to_json_for_cli(tip)` for the value that could either be a string (needs quotes in json), or a number (does not need quotes in json).
ACKs for top commit:
fanquake:
ACK fa0528479d
Tree-SHA512: 3d6deafca1249b2266cfabcd883edc9daaf985c417035a4b0223da4693f4165f8c9ce91a0e128d626000c10c32fe31f323f4b3f6ea0d0b3a771237a4f1d4cf44
fa4d68cf97 Turn rpcauth.py test into functional test (MarcoFalke)
Pull request description:
Currently the `rpcauth-test.py` is problematic, because:
* The boilerplate for the test runner is duplicate or inconsistent with the other (functional) tests. Specifically `ConfigParser`.
* The cmake/ci behavior is brittle and can silently fail, as explained in https://github.com/bitcoin/bitcoin/issues/31476.
* Outside of ctest, this single test has to be run manually and separately, which is easy to forget.
* If the test is manually called, it runs single threaded, when it could just run in parallel with the other functional tests.
* It is also the only "unit" test written in Python, but not called by the functional test runner.
Fix all issues by turning it into a functional test.
ACKs for top commit:
l0rinc:
ACK fa4d68cf97
janb84:
LGTM ACK fa4d68cf97
w0xlt:
ACK fa4d68cf97
Tree-SHA512: a3b2b03be31c33288dee23c544b33ec43295e796c2047777597ceb86acce9f697478e32d891aa986c1d7d5749d62eded65eeb858e9d7479bda7a400eb1167040
8cc3ac6c23 validation: Don't use IsValid() to filter for invalid blocks (Martin Zumsande)
86d98b94e5 test: verify that ancestors of a reconsidered block can become the chain tip (stratospher)
3c39a55e64 validation: Add ancestors of reconsiderblock to setBlockIndexCandidates (Martin Zumsande)
Pull request description:
When we call `reconsiderblock` for some block, `Chainstate::ResetBlockFailureFlags` puts the descendants of that block into `setBlockIndexCandidates` (if they meet the criteria, i.e. have more work than the tip etc.), but never put any ancestors into the set even though we do clear their failure flags.
I think that this is wrong, because `setBlockIndexCandidates` should always contain all eligible indexes that have at least as much work as the current tip, which can include ancestors of the reconsidered block. This is being checked by `CheckBlockIndex()`, which could fail if it was invoked after `ActivateBestChain` connects a block and releases `cs_main`:
``` diff
diff --git a/src/validation.cpp b/src/validation.cpp
index 7b04bd9a5b..ff0c3c9f58 100644
--- a/src/validation.cpp
+++ b/src/validation.cpp
@@ -3551,6 +3551,7 @@ bool Chainstate::ActivateBestChain(BlockValidationState& state, std::shared_ptr<
}
}
// When we reach this point, we switched to a new tip (stored in pindexNewTip).
+ m_chainman.CheckBlockIndex();
if (exited_ibd) {
// If a background chainstate is in use, we may need to rebalance our
```
makes `rpc_invalidateblock.py` fail on master.
Even though we don't currently have a `CheckBlockIndex()` in that place, after `cs_main` is released other threads could invoke it, which is happening in the rare failures of #16444 where an invalid header received from another peer could trigger a `CheckBlockIndex()` call that would fail.
Fix this by adding eligible ancestors to `setBlockIndexCandidates` in `Chainstate::ResetBlockFailureFlags` (also simplifying that function a bit).
Fixes#16444
ACKs for top commit:
achow101:
ACK 8cc3ac6c23
TheCharlatan:
Re-ACK 8cc3ac6c23
stratospher:
reACK 8cc3ac6.
Tree-SHA512: 53f27591916246be4093d64b86a0494e55094abd8c586026b1247e4a36747bc3d6dbe46dc26ee4a22f47b8eb0d9699d13e577dee0e7198145f3c9b11ab2a30b7
1b5c545e82 wallet, test: best block locator matches scan state follow-ups (rkrux)
Pull request description:
Few follows-ups from #30221: Use `SetLastBlockProcessedInMem` more in `AttachChain`, add not null locator check in `WriteBestBlock`. Add log and few assertions in `wallet_reorgstore` test.
ACKs for top commit:
achow101:
ACK 1b5c545e82
pablomartin4btc:
cr-ACK 1b5c545e82
Tree-SHA512: 34edde55beef5714cea2e1131c29b57da2dc32ea091cd81878014de503c128f02c3ab88aee1e456541d7937e033dca5a81b03e9e2888cf781d71b62ad9b5ca5c
922adf66ac mempool: use `FeeFrac` for calculating regular score (Sebastian Falbesoner)
3322b3a059 mempool: use `FeeFrac` for calculating ancestor score (Sebastian Falbesoner)
ac9c113bd2 mempool: use `FeeFrac` for calculating descendant score (Sebastian Falbesoner)
Pull request description:
Rather than determining fee-rates for the mempool index scores and comparators manually in a rather tedious way (even involving floating-points), use the `FeeFrac` class [1] to simplify and deduplicate the code. Note that though this is intended to be a refactoring PR, there might be subtle differences in behaviour due to floating-point arithmetic involved in the original code (to avoid overflows at the cost of precision loss), but these shouldn't matter.
[1] introduced in PR #29242, commit ce8e22542e
ACKs for top commit:
ismaelsadeeq:
Code review ACK 922adf66ac
glozow:
ACK 922adf66ac
Tree-SHA512: 6c3a9436f2be668aa8561b40c1b93efa7dc97b4ef354e98233ac3d3286a88804668164a55f2fcce4239fee5830e4e70f520e6285b667b87baa65c7cec09159cf
4c772cbd83 doc: add release notes for new rate limiting logging behavior (Eugene Siegel)
d541409a64 log: Add rate limiting to LogPrintf, LogInfo, LogWarning, LogError, LogPrintLevel (Eugene Siegel)
a6a35cc0c2 log: use std::source_location in place of __func__, __FILE__, __LINE__ (Eugene Siegel)
afb9e39ec5 log: introduce LogRateLimiter, LogLimitStats, Status (Eugene Siegel)
df7972a6cf test: Mark ~DebugLogHelper as noexcept(false) (Eugene Siegel)
Pull request description:
This revives the work done by dergoegge in https://github.com/bitcoin/bitcoin/pull/21603. The approach is similar — this PR uses `std::source_location` under the hood now that we can use c++20 features. It also resets the rate limiting statistics via the `CScheduler`. The logging functions have also changed slightly since that PR was opened, so work has been done to preserve the intent of the original rate limiting change. I have tried to give commit attribution where possible.
**Approach:**
Each source code location is given an hourly logging quota of 1MiB of logging per hour. Logging is only dropped from source locations that exceed the quota.
- Only logging to disk is rate limited. Logging to console is not rate limited.
- Printing with the category argument is not rate limited.
- `UpdateTip: new best=[…]` is logged without rate limiting. High log volume is expected for that source location during IBD.
- When logging is restarted a tally of how many bytes were dropped is printed.
- All logs will be prefixed with [*] if there is at least one source location that is currently being suppressed.
I've repurposed the old logging rpc mentioned in #21603 in another branch for testing [here](https://github.com/Crypt-iQ/bitcoin/tree/log_ratelimiting_05192025_rpc). This can be used to log from source locations and test out the new changes in logging behavior. Note that the `setmocktime` RPC needs to be used to set the mock time past the current clock time to reset the logging messages.
Example usage:
```
bitcoin-cli -regtest excessivelog 1 1048500 # log just under 1MiB
bitcoin-cli -regtest excessivelog 1 100 # this should get the total amount logged above 1MiB
# and the rate limiting logic should kick in
bitcoin-cli -regtest excessivelog 2 1048500
bitcoin-cli -regtest excessivelog 2 100 # trigger rate limiting from another location
bitcoin-cli -regtest mockscheduler 3600 # fast-forward the scheduler
bitcoin-cli -regtest excessivelog 1 100 # this should trigger logging to resume and will log the source locations that were reset
```
Example output:
```
2025-07-02T22:03:56Z [warning] Excessive logging detected from rpc/node.cpp:142 (RPCHelpMan excessivelog()): >1048576 bytes logged during the last time window of 3600s. Suppressing logging to disk from this source location until time window resets. Console logging unaffected. Last log entry.
[*] 2025-07-02T22:03:56Z aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
2025-07-02T22:04:58Z (mocktime: 2262-04-11T23:47:15Z) Restarting logging from rpc/node.cpp:142 (RPCHelpMan excessivelog()): 121 bytes were dropped during the last 3600s.
2025-07-02T22:04:58Z (mocktime: 2262-04-11T23:47:15Z) Restarting logging from rpc/node.cpp:139 (RPCHelpMan excessivelog()): 121 bytes were dropped during the last 3600s.
```
ACKs for top commit:
maflcko:
re-ACK 4c772cbd83🕚
glozow:
reACK 4c772cb
stickies-v:
re-ACK 4c772cbd83, no changes except release notes update
Tree-SHA512: d07087cd0f2b188100b51c9b8c3da376fa24ec3612a2a284bd83f650bba0ea409f9fa0acd5f3b10f45e664ef4fdf3abc97ed3da08098d2beb599cc83e3fc4504
To mitigate disk-filling attacks caused by unsafe usages of LogPrintf and
friends, we rate-limit them by passing a should_ratelimit bool that
eventually makes its way to LogPrintStr which may call
LogRateLimiter::Consume. The rate limiting is accomplished by
adding a LogRateLimiter member to BCLog::Logger which tracks source
code locations for the given logging window.
Every hour, a source location can log up to 1MiB of data. Source
locations that exceed the limit will have their logs suppressed for the
rest of the window determined by m_limiter.
This change affects the public LogPrintLevel function if called with
a level >= BCLog::Level::Info.
The UpdateTipLog function has been changed to use the private LogPrintLevel_
macro with should_ratelimit set to false. This allows UpdateTipLog to log
during IBD without hitting the rate limit.
Note that on restart, a source location that was rate limited before the
restart will be able to log until it hits the rate limit again.
Co-Authored-By: Niklas Gogge <n.goeggi@gmail.com>
Co-Authored-By: stickies-v <stickies-v@protonmail.com>
The std::source_location conveniently stores the file name, line number,
and function name of a source code location. We switch to using it instead
of the __func__ identifier and the __FILE__ and __LINE__ macros.
BufferedLog is changed to have a std::source_location member, replacing the
source_file, source_line, and logging_function members. As a result,
MemUsage no longer explicitly counts source_file or logging_function as the
std::source_location memory usage is included in the MallocUsage call.
This also changes the behavior of -logsourcelocations as std::source_location
includes the entire function signature. Because of this, the functional test
feature_config_args.py must be changed to no longer include the function
signature as the function signature can differ across platforms.
Co-Authored-By: Niklas Gogge <n.goeggi@gmail.com>
Co-Authored-By: stickies-v <stickies-v@protonmail.com>
LogRateLimiter will be used to keep track of source locations and our
current time-based logging window. It contains an unordered_map and a
m_suppressions_active bool to track source locations. The map is keyed
by std::source_location, so a custom Hash function (SourceLocationHasher)
and custom KeyEqual function (SourceLocationEqual) is provided.
SourceLocationHasher uses CSipHasher(0,0) under the hood to get a
uniform distribution.
A public Reset method is provided so that a scheduler (e.g. the
"b-scheduler" thread) can periodically reset LogRateLimiter's state when
the time window has elapsed.
The LogRateLimiter::Consume method checks if we have enough available
bytes in our rate limiting budget to log an additional string. It
returns a Status enum that denotes the rate limiting status and can
be used by the caller to emit a warning, skip logging, etc.
The Status enum has three states:
- UNSUPPRESSED (logging was successful)
- NEWLY_SUPPRESSED (logging was succcesful, next log will be suppressed)
- STILL_SUPPRESSED (logging was unsuccessful)
LogLimitStats counts the available bytes left for logging per source
location for the current logging window. It does not track actual source
locations; it is used as a value in m_source_locations.
Also exposes a SuppressionsActive() method so the logger can use
that in a later commit to prefix [*] to logs whenenever suppressions
are active.
Co-Authored-By: Niklas Gogge <n.goeggi@gmail.com>
Co-Authored-By: stickies-v <stickies-v@protonmail.com>
We mark ~DebugLogHelper as noexcept(false) to be able to catch the
exception it throws. This lets us use it in test in combination with
BOOST_CHECK_THROW and BOOST_CHECK_NO_THROW to check that certain log
messages are (not) logged.
Co-Authored-By: Niklas Gogge <n.goeggi@gmail.com>
Now that we are storing `CTxMemPool::CompareDepthAndScore` parameters using
`std::variant` we have no portable zero-overhead way of accessing them,
so use `std::visit` and drop `bool wtxid` in-parameter.
Co-authored-by: stickies-v <stickies-v@protonmail.com>
6d19815cd4 rest: replace `rf_names[0].rf` by `RESTResponseFormat::UNDEF` for code clarity (Eval EXEC)
Pull request description:
I'm reviewing the bitcoin's rest.cpp source code.
In the function: `ParseDataFormat`, `rf_names[0].rf` is actualy `RESTResponseFormat::UNDEF`:
e3f416dbf7/src/rest.cpp (L48-L57)
so it would be more clarity and code readability to use `return RESTResponseFormat::UNDEF;` to replace `return rf_names[0].rf;`
ACKs for top commit:
maflcko:
lgtm ACK 6d19815cd4
brunoerg:
code review ACK 6d19815cd4
Tree-SHA512: 420454f1cc09db44c1d76423d8623a0b8865d41d6c34015844ff83d78a9373e3e26f3f62818d1502b33eb063caf904750e858b74ddecd76750577ae82b64b0c1
Before this change, when a functional test is run without building
the source, the error message suggested that previous release binaries
were missing.
When no previous release version is set, make the error message more
specifically about bitcoind.
Also, update related comments because a reload is not happening
anymore. It is done because the legacy wallets could not have been
loaded prior to migration, so I don't think a reload is happening
post a successful migration, it's just load IMO.
4e69aa5701 doc: fix `BlockConnected` incorrect comment (ismaelsadeeq)
Pull request description:
This is a simple PR that fixes the `BlockConnected` validation interface notification comment, which incorrectly states that a vector of transactions removed from the mempool is as a parameter of the method.
Originally, this was the case when the method was first introduced in https://github.com/bitcoin/bitcoin/pull/9725
However, the method has since changed, and this is no longer accurate. Keeping the outdated comment is now misleading.
This PR removes the information about the method parameters from the docstring, aligning it with the style of other notifications methods. As noticed in this PR, comments listing parameters can become stale and go uncorrected.
Therefore, this PR simply removes the inaccurate comment without listing the current returned values.
ACKs for top commit:
l0rinc:
ACK 4e69aa5701
maflcko:
lgtm ACK 4e69aa5701
Tree-SHA512: 3737313f7a9da55c67c78ce01bab5005946f4e1fccbb471560ff3af8c8275cb5cf876f6c53400c93f0ba1fdf134f28766ed573cbe62903127a3129ca8ce88db6
fcfd3db563 remove RPCTimerInterface and RPCRunLater (Matthew Zipkin)
8a1765795f use WalletContext scheduler for walletpassphrase callback (Matthew Zipkin)
Pull request description:
This removes the dependency on libevent for events scheduled by RPC commands, like re-locking a wallet some time after decryption with walletpassphrase. Since walletpassphrase is currently the only RPC that does this, `RPCRunLater`, `RPCTimerInterface` and all related methods are left unused, and deleted in the second commit. Any future RPC that needs to execute a callback in the future can follow the pattern in this PR and just use a scheduler from node or wallet context.
This is an alternative approach to #32796, described in https://github.com/bitcoin/bitcoin/pull/32796#issuecomment-3014309449
ACKs for top commit:
fjahr:
Code Review ACK fcfd3db563
achow101:
ACK fcfd3db563
furszy:
ACK fcfd3db563
Tree-SHA512: 04f5e9c3f73f598c3d41d6e35bb59c64c7b93b03ad9fce3c40901733147ce7764f41f475fef1527d44af18f722759996a31ca83b48cb52153795d5022fecfd14
fa946520d2 refactor: Use structured binding for-loop (MarcoFalke)
eeeec1579e rpc: Use type-safe exception to pass RPC help (MarcoFalke)
Pull request description:
The current "catch-all" `catch (const std::exception& e)` in `CRPCTable::help` is problematic, because it could catch exceptions unrelated to passing the help string up.
Fix this by using a dedicated exception type.
ACKs for top commit:
l0rinc:
tested ACK fa946520d2 (edited)
achow101:
ACK fa946520d2
rkrux:
re-ACK fa946520d2
Tree-SHA512: 23dac6e0fe925561bfbf421e6a7441d546eed8c1492ac41ca4ed7dfcd12f4d2ef39c35f105a0291aac511365d98f08fbdc9a4f0bf627172873b8f23c2be45e76
4207d9bf82 test: feature_init, ensure indexes are synced prior to perturbing files (furszy)
abd07cf733 test: feature_init, only init what's needed per perturbation/deletion round (furszy)
Pull request description:
Aims to solve #32600. Found it while working on #26966 (this was really annoying there).
This ensures the node is index-synced before perturbing files.
If the index sync gets interrupted before it starts, the database could be empty,
making any following perturbation ineffective (which explains why the node
does not abort during startup in the #32600 logs).
Also, the first commit avoids initializing components not under test.
This reduces log flooding, which helped in understanding the issue.
Patch to reproduce the issue on master using `feature_init.py` (this simulates
a node shutting down before the index starts syncing):
```
diff --git a/src/index/base.cpp b/src/index/base.cpp
--- a/src/index/base.cpp(revision 1e03052c3fefb188f047e72548f2c6b0cc019e50)
+++ b/src/index/base.cpp(date 1751293306725)
@@ -185,6 +185,7 @@
void BaseIndex::Sync()
{
const CBlockIndex* pindex = m_best_block_index.load();
+ m_interrupt();
if (!m_synced) {
std::chrono::steady_clock::time_point last_log_time{0s};
std::chrono::steady_clock::time_point last_locator_write_time{0s};
```
ACKs for top commit:
maflcko:
lgtm ACK 4207d9bf82🍄
achow101:
ACK 4207d9bf82
hodlinator:
ACK 4207d9bf82
Tree-SHA512: c8c89c7af9d473a12756b6a59b97f8fb473500181620eb96ecc10da954fe185d13fbb1d00a4ecb181e8daf149ec93cc547e292da0877522a4d23425fa7fd646b
afaaba69ed test: refactor out same-txid-diff-wtxid tx to reuse in other tests (stratospher)
Pull request description:
It's useful to easily create transactions with same txid, different wtxid and valid witness for testing scenarios in other places in the codebase (ex: private broadcast connections, see https://github.com/bitcoin/bitcoin/pull/29415#discussion_r2055915118)
So refactor out the current `same-txid-diff-wtxid` transaction in `mempool_accept_wtxid.py` so that it can be reused.
ACKs for top commit:
maflcko:
review ACK afaaba69ed📎
theStack:
ACK afaaba69ed
vasild:
ACK afaaba69ed
Tree-SHA512: 0fc51ac326725d4abe76a15b6b5be55d070b96c303c444f4dd31c2b0a82f266836382389a123a7f6a71aa35e61fbfae27f843b31cc19474e49f3dc82f36ebf73
b1a8ac07e9 doc: Release note for removed watchonly parameters and results (Ava Chow)
15710869e1 wallet: Remove ISMINE_WATCH_ONLY (Ava Chow)
4439bf4b41 wallet, spend: Remove fWatchOnly from CCoinControl (Ava Chow)
1337c72198 wallet, rpc: Remove watchonly from RPCs (Ava Chow)
e81d95d435 wallet: Remove watchonly balances (Ava Chow)
d20dc9c6aa wallet: Wallets without private keys cannot grind R (Ava Chow)
9991f49c38 test: Watchonly wallets should estimate larger size (Ava Chow)
Pull request description:
Descriptor wallets do not use the watchonly behavior as it is not possible to mix watchonly and non-watchonly in a descriptor wallet. With legacy wallets now removed, all of the watchonly handling and reporting code is no longer needed. This PR removes watchonly options and results from the RPCs and the handling of watchonly things from the wallet's internals.
With all of the watchonly things removed, ISMINE_WATCH_ONLY is removed as well.
Split from #32523
Depends on #32594 for tests that are easier to read
ACKs for top commit:
Eunovo:
ACK b1a8ac07e9
maflcko:
re-ACK b1a8ac07e9🌈
rkrux:
ACK b1a8ac07e9
furszy:
light code review ACK b1a8ac07e9
Tree-SHA512: bc87f37a13294f7208991be8f93899b49e5bdf87c70e0f66d9c4cb09c03be6c202320406f27e9a35aa2f57319d19a3f0c07d5e5ddbc97c7edab165b1656d6612
1632fc104b txgraph: Track multiple potential would-be clusters in Trim (improvement) (Pieter Wuille)
4608df37e0 txgraph: add Trim benchmark (benchmark) (Pieter Wuille)
9c436ff01c txgraph: add fuzz test scenario that avoids cycles inside Trim() (tests) (Pieter Wuille)
938e86f8fe txgraph: add unit test for TxGraph::Trim (tests) (glozow)
a04e205ab0 txgraph: Add ability to trim oversized clusters (feature) (Pieter Wuille)
eabcd0eb6f txgraph: remove unnecessary m_group_oversized (simplification) (Greg Sanders)
19b14e61ea txgraph: Permit transactions that exceed cluster size limit (feature) (Pieter Wuille)
c4287b9b71 txgraph: Add ability to configure maximum cluster size/weight (feature) (Pieter Wuille)
Pull request description:
Part of cluster mempool (#30289).
During reorganisations, it is possible that dependencies get added which would result in clusters that violate policy limits (cluster count, cluster weight), when linking the new from-block transactions to the old from-mempool transactions. Unlike RBF scenarios, we cannot simply reject the changes when they are due to received blocks. To accommodate this, add a `TxGraph::Trim()`, which removes some subset of transactions (including descendants) in order to make all resulting clusters satisfy the limits.
Conceptually, the way this is done is by defining a rudimentary linearization for the entire would-be too-large cluster, iterating it from beginning to end, and reasoning about the counts and weights of the clusters that would be reached using transactions up to that point. If a transaction is encountered whose addition would violate the limit, it is removed, together with all its descendants.
This rudimentary linearization is like a merge sort of the chunks of the clusters being combined, but respecting topology. More specifically, it is continuously picking the highest-chunk-feerate remaining transaction among those which have no unmet dependencies left. For efficiency, this rudimentary linearization is computed lazily, by putting all viable transactions in a heap, sorted by chunk feerate, and adding new transactions to it as they become viable.
The `Trim()` function is rather unusual compared to the `TxGraph` functionality added in previous PRs, in that `Trim()` makes it own decisions about what the resulting graph contents will be, without good specification of how it makes that decision - it is just a best-effort attempt (which is improved in the last commit). All other `TxGraph` mutators are simply to inform the graph about changes the calling mempool code decided on; this one lets the decision be made by txgraph.
As part of this, the "oversized" property is expanded to also encompass a configurable cluster weight limit (in addition to cluster count limit).
ACKs for top commit:
instagibbs:
reACK 1632fc104b
glozow:
reACK 1632fc104b via range-diff
ismaelsadeeq:
reACK 1632fc104b🛰️
Tree-SHA512: ccacb54be8ad622bd2717905fc9b7e42aea4b07f824de1924da9237027a97a9a2f1b862bc6a791cbd2e1a01897ad2c7c73c398a2d5ccbce90bfbeac0bcebc9ce
de4eef52d1 threading: use correct mutex name in reverse_lock fatal error messages (Cory Fields)
Pull request description:
"Now that REVERSE_LOCK requires the name of the actual mutex, it can be used for better error messages." - theuni
This is a follow-up to this comment https://github.com/bitcoin/bitcoin/pull/32465#issuecomment-2981287545
I just cherry-picked the commit 85c2848eb575f4abaa81fdd4e8f3b2048693dd98
ACKs for top commit:
theuni:
Re-ACK de4eef52d1
TheCharlatan:
ACK de4eef52d1
Tree-SHA512: 1109381e1f0589093f7c737cb1ebd1c43324a9e1ea34b5f05a9171d06ab44cca0c5ead43c581f6e37ded1f0463ab8a280f3319c288d39a4625109b5c08a7cb68
9b75cfda4d test: retain the intended behavior of `feature_fee_estimation.py` nodes (ismaelsadeeq)
5c1236f04a test: fix incorrect subtest in `feature_fee_estimation.py` (ismaelsadeeq)
Pull request description:
Attempt to fix#32461
In the `estimatesmartfee` RPC, we return the maximum of the following: the feerate estimate for the target, `minrelaytxfee`, and `mempoolminfee`.
9a05b45da6/src/rpc/fees.cpp (L85)
The test `test_feerate_mempoolminfee`, originally introduced in ea31caf6b4, is incorrect.
It should calculate the fee rate ceiling by taking the maximum of the custom `minrelaytxfee`, `mempoolminfee`, and the highest fee rate observed during the test (`check_smart_estimates`). This is necessary because:
* There is no guarantee that the generated fee rates will exceed both `minrelaytxfee` and `mempoolminfee`.
* Users can start a node with custom fee settings.
Due to the non-deterministic nature of the `feature_fee_estimation.py` test, it often passes by chance. The randomly generated fees typically include a value higher than the custom `minrelaytxfee`, inadvertently hiding the issue.
Issue #32461 identified a random seeds that consistently fails the test because the generated fees never exceed the custom `minrelaytxfee`:
e.g
```
build/test/functional/feature_fee_estimation.py --random=3450808900320758527
```
This PR has two commits which :
* Correctly fixes the test by calculating the fee rate ceiling as the maximum of the node `minrelaytxfee`, `mempoolminfee`, and the highest seen fee rate, when verifying smart fee estimates.
* Improves the subtest name and comment for clarity.
* Restores the original test behavior by appending 4000 WU to the custom `blockmaxweight`.
ACKs for top commit:
achow101:
ACK 9b75cfda4d
glozow:
ACK 9b75cfda4d
theStack:
Light ACK 9b75cfda4d
Tree-SHA512: 0f7fb0496b50a399b58f6fb1afd95414fad454795fbc0046e22dfc54a2062ae0c519a12ebfeb6ad7ef547438868d99eca2351c0d19d0346adaadb500eff6f15f
c10e382d2a flatfile: check whether the file has been closed successfully (Vasil Dimov)
4bb5dd78ea util: check that a file has been closed before ~AutoFile() is called (Vasil Dimov)
8bb34f07df Explicitly close all AutoFiles that have been written (Vasil Dimov)
a69c4098b2 rpc: take ownership of the file by WriteUTXOSnapshot() (Hodlinator)
Pull request description:
`fclose(3)` may fail to flush the previously written data to disk, thus a failing `fclose(3)` is as serious as a failing `fwrite(3)`.
Previously the code ignored `fclose(3)` failures. This PR improves that by changing all users of `AutoFile` that use it to write data to explicitly close the file and handle a possible error.
---
Other alternatives are:
1. `fflush(3)` after each write to the file (and throw if it fails from the `AutoFile::write()` method) and hope that `fclose(3)` will then always succeed. Assert that it succeeds from the destructor 🙄. Will hurt performance.
2. Throw nevertheless from the destructor. Exception within the exception in C++ I think results in terminating the program without a useful message.
3. (this is implemented in the latest incarnation of this PR) Redesign `AutoFile` so that its destructor cannot fail. Adjust _all_ its users 😭. For example, if the file has been written to, then require the callers to explicitly call the `AutoFile::fclose()` method before the object goes out of scope. In the destructor, as a sanity check, assume/assert that this is indeed the case. Defeats the purpose of a RAII wrapper for `FILE*` which automatically closes the file when it goes out of scope and there are a lot of users of `AutoFile`.
4. Pass a new callback function to the `AutoFile` constructor which will be called from the destructor to handle `fclose()` errors, as described in https://github.com/bitcoin/bitcoin/pull/29307#issuecomment-2243842400. My thinking is that if that callback is going to only log a message, then we can log the message directly from the destructor without needing a callback. If the callback is going to do more complicated error handling then it is easier to do that at the call site by directly calling `AutoFile::fclose()` instead of getting the `AutoFile` object out of scope (so that its destructor is called) and inspecting for side effects done by the callback (e.g. set a variable to indicate a failed `fclose()`).
ACKs for top commit:
l0rinc:
ACK c10e382d2a
achow101:
ACK c10e382d2a
hodlinator:
re-ACK c10e382d2a
Tree-SHA512: 3994ca57e5b2b649fc84f24dad144173b7500fc0e914e06291d5c32fbbf8d2b1f8eae0040abd7a5f16095ddf4e11fe1636c6092f49058cda34f3eb2ee536d7ba
Try to make extra commands more obvious based on a suggestion from Sjors:
https://github.com/bitcoin/bitcoin/pull/31679#issuecomment-2922787970i
When `bitcoin` is invoked with no arguments, still show short help output, but
now explicitly state that more commands are available and `bitcoin help` will
list them.
Also:
- Get rid of -a/--all option. Just show all commands when `bitcoin help` or
`bitcoin --help` is used. It maybe a helpful to add an option like this if
more commands are added in the future, but right now there are not very many.
- Just show name of executable, not full path of executable in help output. This
can be a little easier to read if the path is long.
4f10a57671 depends: Override host compilers for FreeBSD and OpenBSD (Hennadii Stepanov)
Pull request description:
When building depends on FreeBSD/OpenBSD `aarch64`, the host compilers default to `default_host_{CC,CXX}`, which resolves to `gcc`/`g++`. This is incorrect on these systems, where Clang is the default system compiler.
To ensure proper compiler selection, this PR adopts the same approach used for `darwin`:c1d4253d31/depends/builders/darwin.mk (L12-L14)
Fixes https://github.com/bitcoin/bitcoin/issues/32691.
ACKs for top commit:
Sjors:
ACK 4f10a57671
Tree-SHA512: 66ace60b63671f2ab7facb84776298e1489aa0a984559c9a22323adaaedcc86f67158078b18c333cb7bdc2b17a4573e321ee7361596bc941db060c5659014069
67dc7523f3 cmake, test: Disable tests instead of ignoring them (Hennadii Stepanov)
bb9157db5d cmake, refactor: Switch to `Python3::Interpreter` imported target (Hennadii Stepanov)
Pull request description:
This PR:
1. Switches to a modern CMake approach by using the `Python3::Interpreter` imported target, which is more robust than using variables.
2. Disables the `util_rpcauth_test` test explicitly instead of silently ignoring it.
A build and test log for the case when Python is unavailable is provided below:
```
$ cmake -B build
$ cmake --build build -j 16
$ ctest --test-dir build -j $(nproc) -R "^util"
Internal ctest changing into directory: /bitcoin/build
Test project /bitcoin/build
Start 115: util_tests
Start 117: util_trace_tests
Start 114: util_string_tests
Start 116: util_threadnames_tests
Start 1: util_rpcauth_test
1/5 Test #1: util_rpcauth_test ................***Not Run (Disabled) 0.00 sec
2/5 Test #114: util_string_tests ................ Passed 0.11 sec
3/5 Test #117: util_trace_tests ................. Passed 0.11 sec
4/5 Test #116: util_threadnames_tests ........... Passed 0.11 sec
5/5 Test #115: util_tests ....................... Passed 0.13 sec
100% tests passed, 0 tests failed out of 4
Total Test time (real) = 0.13 sec
The following tests did not run:
1 - util_rpcauth_test (Disabled)
```
ACKs for top commit:
purpleKarrot:
ACK 67dc7523f3
janb84:
tACK 67dc7523f3
Tree-SHA512: 5fc7ebe31ac03f4b8a53ecfcfc1cace0f647a1d2c989651988edae96bdfbbe2dee171714e57cb028e65ead1bb40806a82d9821746451dbf005538601fd33ea88
666016e56b ci: use --usecli in one of the CI jobs (Martin Zumsande)
7ea248a020 test: Disable several (sub)tests with cli (Martin Zumsande)
f420b6356b test: skip subtests that check for wrong types with cli (Martin Zumsande)
6530d0015b test: add function to convert to json for height_or_hash params (Martin Zumsande)
54d28722ba test: Don't send empty named args with cli (Martin Zumsande)
cca422060e test: convert tuple to json for cli (Martin Zumsande)
af34e98086 test: make rpc_psbt.py usable with --usecli (Martin Zumsande)
8f8ce9e174 test: rename .rpc to ._rpc and remove unnecessary uses (Martin Zumsande)
5b08885986 test: enable functional tests with large rpc args for cli (Martin Zumsande)
7d5352ac73 test: use -stdin for large rpc commands (Martin Zumsande)
6c364e0c10 test: Enable various tests for usage with cli (Martin Zumsande)
Pull request description:
Fixes#32264
I looked into all current failures listed in the issue, as well all tests that are already disabled for the cli with `self.supports_cli = False`. There are several reasons why existing tests fail with `--usecli` on many systems, the most important ones are:
- Most common reason is that the test executes a RPC call with a large arg that exceeds `MAX_ARG_STRLEN` of the OS, which is usually 128kb on linux: This is fixed by using `-stdin` for these large calls (idea by 0xB10C)
- they test specifically the rpc interface - nothing to do there except disabling.
- Some functional test submit wrong types to params on purpose to test the error message (which is different when using the cli) - deactivated these specific subtests locally for the cli when there is just one or two of them, deactivated the entire tests when there are more spots
- When python sets `None` for an arg, the cli converts this to 'null' in `arg_to_cli`. This is fine e.g. for boolean args, but doesn't work for strings where it's interpreted as the string 'null'. Bypass this for named args by not including args in case the value is `None` for the cli is used (it's effectively the same as leaving the optional arg out).
- the `height_or_hash` param used in some RPC needs to be converted to a JSON (effectively adding full quotes).
- Some tests were marked with `self.supports_cli = False` in the past but run fine on master today - enabled those.
In total, this PR fixes all tests that fail on master and reduces the number of tests that are deactivated (`self.supports_cli = False`) from 40 to 21.
It also adds `--usecli` to one CI job (multiprocess, i686, DEBUG) to detect regressions.
ACKs for top commit:
maflcko:
re-ACK 666016e56b🔀
pinheadmz:
re-ACK 666016e56b
Tree-SHA512: 7a1efd212649ca100b236a1239294d40ecd36e2720e3b173a230b14545bb40b135111db7fed8a0d1448120f5387da146a03f1912e2028c8d03a0b6a3ca8761b0
A discussion on a previous PR 32481 related to legacy wallet dead
code removal made me realize that checking if the legacy
wallet was loaded prior to the start of the migration is not
required ever since legacy wallets can't be loaded in the first
place. I also verified that the `load_on_start` persistent
setting can also not cause the legacy wallets to be loaded, which
further makes the case for removal of the above mentioned checks
during migration.
The current test coverage also shows these lines uncovered.
d6aaffcb11 test: check P2SH sigop count for coinbase tx (brunoerg)
Pull request description:
We currently do not test that `GetP2SHSigOpCount` returns 0 for coinbase transactions (see line L129 at https://corecheck.dev/mutation/src/consensus/tx_verify.cpp). This PR addresses it.
ACKs for top commit:
darosior:
That said, i guess unit-tested dead consensus code is better than not-unit-tested dead consensus code. utACK d6aaffcb11
theStack:
ACK d6aaffcb11
w0xlt:
ACK d6aaffcb11
ishaanam:
ACK d6aaffcb11
pablomartin4btc:
ACK d6aaffcb11
Tree-SHA512: a7d7306f064bb2ec7e93e92625848ae38e150ebb67bde37cd15be1038816b154e867ad21ecd2685d8de5341b67e3b768d30b7654e27b541f33e8f9d63e52261d
Avoids initializing and syncing components not under test.
This not only speeds up execution a bit but also helps isolate
and debug issues more easily, as logs aren't flooded with
unrelated details.
ec004cdb86 test: Use rehash() in outbound eviction block-relay (pablomartin4btc)
26598ed21e test: Clarify roles in outbound eviction comments (pablomartin4btc)
Pull request description:
This change avoids relying on `tip_header.hash`, which is `None` when the header is deserialized from hex during `CBlockHeader()` construction.
Instead, `tip_header.rehash()` explicitly computes the hash, making the test behavior more robust.
Using the explicit `rehash()` avoids depending on `wait_for_getheaders()` falling back to any received message, thus making the test more deterministic.
This is a follow-up to #32742.
Also, as noted in a previous review [comment](https://github.com/bitcoin/bitcoin/pull/32742#pullrequestreview-2923802386), "_the hash field is wrong either way, simply due to being the wrong type (it is an optional hex string), as opposed to an optional int_".
---
The first commit intention is to improve clarity around the tests purpose, helping reviewers follow what's being verified and why. What started as a small comment during review of #32742 led me reviewing and try to improve most relevant tests comments for consistency and correctness.
ACKs for top commit:
achow101:
ACK ec004cdb86
theStack:
lgtm ACK ec004cdb86#️⃣
yuvicc:
ACK ec004cdb86
danielabrozzoni:
ACK ec004cdb86
Tree-SHA512: 6a14dedfdc425cd806f63443b3b9f79df69a7717452739f5d7fef1b2bdba23402670d63cf1d6b66c9f1a6b460d4d4a6f185426d0a4982fa95115a234cd6baef7
b789907346 wallet: migration, avoid creating spendable wallet from a watch-only legacy wallet (furszy)
e86d71b749 wallet: refactor, dedup wallet re-loading code (furszy)
1de423e0a0 wallet: introduce method to return all db created files (furszy)
d04f6a97ba refactor: remove sqlite dir path back-and-forth conversion (furszy)
Pull request description:
Currently, the migration process creates a brand-new descriptor wallet with no
connection to the user's legacy wallet when the legacy wallet lacks key material
and contains only watch-only scripts. This behavior is not aligned with user
expectations. If the legacy wallet contains only watch-only scripts, the migration
process should only generate a watch-only wallet instead.
TODO List:
* Explain that `migratewallet` renames the watch-only after migration, and
also that the wallet will not have keys enabled.
ACKs for top commit:
achow101:
ACK b789907346
pablomartin4btc:
tACK b789907346
rkrux:
LGTM ACK b789907346
Tree-SHA512: 1d583ac4b206fb477e9727daf4b5ad9c3e18b12d40e1ab4a61e8565da44c3d0327c892b51cf47b4894405d122e414cefb6b6366c357e02a74a7ca96e06762d83
In the existing Trim function, as soon as the set of accepted transactions
would exceed the max cluster size or count limit, the acceptance loop is
stopped, removing all later transactions. However, it is possible that by
excluding some of those transactions the would-be cluster splits apart into
multiple would-clusters. And those clusters may well permit far more
transactions before their limits are reached.
Take this into account by using a union-find structure inside TrimTxData to
keep track of the count/size of all would-be clusters that would be formed
at any point, and only reject transactions which would cause these resulting
partitions to exceed their limits.
This is not an optimization in terms of CPU usage or memory; it just
improves the quality of the transactions removed by Trim().
Trim internally builds an approximate dependency graph of the merged cluster,
replacing all existing dependencies within existing clusters with a simple
linear chain of dependencies. This helps keep the complexity of the merging
operation down, but may result in cycles to appear in the general case, even
though in real scenarios (where Trim is called for stitching re-added mempool
transactions after a reorg back to the existing mempool transactions) such
cycles are not possible.
Add a test that specifically targets Trim() but in scenarios where it is
guaranteed not to have any cycles. It is a special case, is much more a
whitebox test than a blackbox test, and relies on randomness rather than
fuzz input. The upside is that somewhat stronger properties can be tested.
Co-authored-by: Greg Sanders <gsanders87@gmail.com>
During reorganisations, it is possible that dependencies get add which
result in clusters that violate limits (count, size), when linking the
new from-block transactions to the old from-mempool transactions.
Unlike RBF scenarios, we cannot simply reject these policy violations
when they are due to received blocks. To accomodate this, add a Trim()
function to TxGraph, which removes transactions (including descendants)
in order to make all resulting clusters satisfy the limits.
In the initial version of the function added here, the following approach
is used:
- Lazily compute a naive linearization for the to-be-merged cluster (using
an O(n log n) algorithm, optimized for far larger groups of transactions
than the normal linearization code).
- Initialize a set of accepted transactions to {}
- Iterate over the transactions in this cluster one by one:
- If adding the transaction to the set makes it exceed the max cluster size
or count limit, stop.
- Add the transaction to the set.
- Remove all transactions from the cluster that were not included in the set
(note that this necessarily includes all descendants too, because they
appear later in the naive linearization).
Co-authored-by: Greg Sanders <gsanders87@gmail.com>
This removes the restriction added in the previous commit that individual
transactions do not exceed the max cluster size limit.
With this change, the responsibility for enforcing cluster size limits can
be localized purely in TxGraph, without callers (and in particular, tests)
needing to duplicate the enforcement for individual transactions.
This is integrated with the oversized property: the graph is oversized when
any connected component within it contains more than the cluster count limit
many transactions, or when their combined size/weight exceeds the cluster size
limit.
It becomes disallowed to call AddTransaction with a size larger than this limit,
though this limit will be lifted in the next commit.
In addition, SetTransactionFeeRate becomes SetTransactionFee, so that we do not
need to deal with the case that a call to this function might affect the
oversizedness.
4be81e9746 feature_taproot: sample tx version border values more (Greg Sanders)
Pull request description:
Currently if the version 3 is selected for an otherwise standard spender, the test will fail. It's unlikely but possible, so change the test to update expectations and sample more aggressively on border values to instigate failures much quicker in the future if another version is made standard.
ACKs for top commit:
maflcko:
lgtm ACK 4be81e9746
darosior:
ACK 4be81e9746
Tree-SHA512: 53267a201aaa495bea9d624930a19e40af3633427b6715965f43b9e1a060b2c9f19c8b10c8168778349fa50715e44cb8e5e9d2ce477d5f324ca8ed28ff7996cd
Few follows-ups from #30221: Use `SetLastBlockProcessedInMem` more in
`AttachChain`, add not null locator check in `WriteBestBlock`. Add log
and few assertions in `wallet_reorgstore` test.
6efbd1e1dc refactor: CTransaction equality should consider witness data (Cory Fields)
cbf9b2dab1 mempool: codify existing assumption about duplicate txids during removal (Cory Fields)
e9331cd6ab wallet: IsEquivalentTo should strip witness data in addition to scriptsigs (Cory Fields)
Pull request description:
I stumbled upon the `CTransaction` comparison operators while refactoring some nearby code. I found it surprising and not at all obvious that two transactions would test equal even if their witness data differed. It seems like an unnecessary potential footgun. Fix that by comparing against wtxid rather than txid.
Outside of tests, there were only 3 users of these functions in the code-base:
- Its use in the mempool has been replaced with an explicit txid comparison, as that's a tighter constraint and matches the old behavior. glozow suggested also upgrading this to an `Assume()`.
- Its use in the wallet was accidentally doing the correct thing by ignoring witness data. I've changed that to an explicit witness removal so that `IsEquivalentTo` continues to work as-intended.
- Its use in `getrawtransaction` is indifferent to the change.
ACKs for top commit:
maflcko:
review ACK 6efbd1e1dc🦋
achow101:
ACK 6efbd1e1dc
glozow:
ACK 6efbd1e1dc
Tree-SHA512: 89be424889f49e7e26dd2bdab7fbc8b2def59bf002ae8b94989b349ce97245f007d6c96e409a626cbf0de9df83ae2485b4815b40a70f7aa5b6c720eb34a6c017
941b8f54c0 ci: run get_previous_releases as part of test cross win job (Max Edwards)
5e2182140b test: increment mocked time for migrating wallet backups (Max Edwards)
5174565802 ci: disable feature_unsupported_utxo_db functional test (Max Edwards)
3dc90d69a6 test: remove mempool.dat before copying (Max Edwards)
67a6b20d50 test: add windows support to get previous releases script (Max Edwards)
1a1b478ca3 scripted-diff: rename tarball to archive (Max Edwards)
4f06dc8484 test: remove building from source from get prev releases script (Max Edwards)
Pull request description:
This PR updates the `test/get_previous_releases.py` script to also work on Windows by changing to be pure python rather than using unix tools such as `curl` and `tar`.
This enables additional functional tests to run such as `wallet_migration.py`, `mempool_compatability.py` and `wallet_backwards_compatibility.py`.
Unfortunately `feature_unsupported_utxo_db.py` _could_ run but this test requires Bitcoin `v0.14.3` which will not run under windows with emojis in the data directory (as the functional test runner has by default) . This test could be run as it's own step in the ci workflow file and would pass but as it's quite an old version / feature I have assumed it's not worth worrying about and best just to exclude.
Two tests needed to be slightly modified to run under windows. Both were issues with trying to overwrite a file that already exists which windows seems to be more strict on than the unix based systems.
Finally, building from source has been dropped from the `get_previous_releases.py` script. This had not been updated after the move to cmake and so it was assumed that nobody could have been using that feature.
ACKs for top commit:
maflcko:
re-ACK 941b8f54c0🍪
achow101:
ACK 941b8f54c0
hodlinator:
re-ACK 941b8f54c0
Tree-SHA512: 22933d0ec278b9b0ffcd2a8e90026e1a3631b00186e7f78bd65be925049021e319367d488c36a82ab526a07b264bac18c2777f87ca1174b231ed49fed56d11cb
8cc9845b8d wallet, rpc: Use `OUTPUT_TYPES` to describe the output types instead of hardcoding them (w0xlt)
Pull request description:
Follow-up to https://github.com/bitcoin/bitcoin/pull/32429, built on top of it.
This PR addresses the https://github.com/bitcoin/bitcoin/pull/32429#discussion_r2076251627 that the RPC documentation does not use `OUTPUT_TYPES`, but rather hardcodes them, as is already the case for the `getnewaddress` command.
So here the output types are changed from `std::string` to `std::string_view` so that the values are known at compile time or during the early stages of program startup, before main() execution.
It also updates `wallet/rpc/addresses.cpp` to write the RPC docs according to `OUTPUT_TYPES` instead of using hardcoded version.
It also updates the documentation in outputtypes.h, adding Doxygen comments,
ACKs for top commit:
maflcko:
lgtm ACK 8cc9845b8d
achow101:
ACK 8cc9845b8d
Tree-SHA512: e86d813d6d158dd2f6c62519a7ecaa878f2e4f686b5bae82028a106bd6671a13b10fb366f9bb7b94974777217e1852f38e8aa05bba00cd27f94f4412167a3562
0e9f409db3 doc: clarify that the "-j N" goes after the "--build build" part (Salvatore Ingala)
Pull request description:
I was surprised that something like `cmake -j 4 --build build` doesn't work, so this might help others to not make the same mistake.
ACKs for top commit:
achow101:
ACK 0e9f409db3
hebasto:
ACK 0e9f409db3.
stickies-v:
ACK 0e9f409db3
pablomartin4btc:
ACK 0e9f409db3
theStack:
ACK 0e9f409db3
brunoerg:
ACK 0e9f409db3
w0xlt:
ACK 0e9f409db3
Tree-SHA512: 5283f87a68abae4905b773e73aa4529f52d2302d804165b560b22f300a449d8ce1ae3194d2e1fe69be6f7e4d279dd32b8ef0daa18af4ac6a76a51f5abc39ec63
Test that when a watchonly wallet and the wallet with private keys fund
the same tx, the watchonly wallet should use a higher fee since it
should be estimating the size to be larger as it assumes the signer
cannot grind the R value.
215e5999e2 wallet: Remove unused CachedTxGet{Available,Immature}Credit (Ava Chow)
49675de035 wallet: Have GetDebit use the wallet's TXO set (Ava Chow)
17d453cb3a wallet: Recompute wallet TXOs after descriptor migration (Ava Chow)
764016eb22 wallet: Retrieve TXO directly in FetchSelectedInputs (Ava Chow)
c1801b78f1 wallet: Use wallet's TXO set in AvailableCoins (Ava Chow)
dde7cbe105 wallet: Change balance calculation to use m_txos (Ava Chow)
96e7a89c5e wallet: Recalculate the wallet's txos after any imports (Ava Chow)
ae888c38d0 wallet: Exit IsTrustedTx early if wtx is already in trusted_parents (Ava Chow)
ae0876ec42 wallet: Keep track of transaction outputs owned by the wallet (Ava Chow)
0f269bc48c walletdb: Load Txs last (Ava Chow)
5cc32ee2a7 test: Test for balance update due to untracked output becoming spendable (Ava Chow)
8222341d4f wallet: MarkDirty after AddWalletDescriptor (Ava Chow)
e02f2d331c bench: Have AvailableCoins benchmark include a lot of unrelated utxos (Ava Chow)
Pull request description:
Currently, the wallet is not actually aware about its own transaction outputs. Instead, it will iterate all of the transactions stored in `mapWallet`, and then all of the outputs of those transactions, in order to figure out what belongs to it for the purposes of coin selection and balance calculation. For balance calculation, there is caching that results in it only iterating all of the transactions, but not all of the outputs. However when the cache is dirty, everything is iterated. This is especially problematic for wallets that have a lot of transactions, or transactions that have a lot of unrelated outputs (as may occur with coinjoins or batched payments).
This PR helps to resolve this issue by making the wallet track all of the outputs that belong to it in a new member `m_txos`. Note that this includes outputs that may have already been spent. Both balance calculation (`GetBalance`) and coin selection (`AvailableCoins`) are updated to iterate `m_txos`. This is generally faster since it ignores all of the unrelated outputs, and it is not slower as in the worst case of wallets containing only single output transactions, it's exactly the same number of outputs iterated.
`m_txos` is memory only, and it is populated during wallet loading. When each transaction is loaded, all of its outputs are checked to see if it is `IsMine`, and if so, an entry added to `m_txos`. When new transactions are received, the same procedure is done.
Since imports can change the `IsMine` status of a transaction (although they can only be "promoted" from watchonly to spendable), all of the import RPCs will be a bit slower as they re-iterate all transactions and all outputs to update `m_txos`.
Each output in `m_txos` is stored in a new `WalletTXO` class. This class contains references to the parent `CWalletTx` and the `CTxOut` itself. It also caches the `IsMine` value of the txout. This should be safe as `IsMine` should not change unless there are imports. This allows us to have additional performance improvements in places that use these `WalletTXO`s as they can use the cached `IsMine` rather than repeatedly calling `IsMine` which can be expensive.
The existing `WalletBalance` benchmark demonstrates the performance improvement that this PR makes. The existing `WalletAvailableCoins` benchmark doesn't as all of the outputs used in that benchmark belong to the test wallet. I've updated that benchmark to have a bunch of unrelated outputs in each transaction so that the difference is demonstrated.
This is part of a larger project to have the wallet actually track and store a set of its UTXOs.
Built on #24914 as it requires loading the txs last in order for `m_txos` to be built correctly.
***
## Benchmarks:
Master:
| ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
| 34,590,013.00 | 28.91 | 0.0% | 812,669,269.00 | 148,360,642.50 | 5.478 | 18,356,853.00 | 0.2% | 0.76 | `WalletAvailableCoins`
| 3,193.46 | 313,139.91 | 0.4% | 96,868.06 | 13,731.82 | 7.054 | 26,238.01 | 0.1% | 0.01 | `WalletBalanceClean`
| 26,871.18 | 37,214.59 | 3.3% | 768,179.50 | 115,544.39 | 6.648 | 154,171.09 | 0.1% | 0.01 | `WalletBalanceDirty`
| 3,177.30 | 314,732.47 | 0.2% | 96,868.06 | 13,646.20 | 7.099 | 26,238.01 | 0.1% | 0.01 | `WalletBalanceMine`
| 10.73 | 93,186,952.53 | 0.1% | 157.00 | 46.14 | 3.403 | 36.00 | 0.0% | 0.01 | `WalletBalanceWatch`
| 590,497,920.00 | 1.69 | 0.1% |12,761,692,005.00 |2,536,899,595.00 | 5.030 | 129,124,399.00 | 0.7% | 6.50 | `WalletCreateEncrypted`
| 182,929,529.00 | 5.47 | 0.0% |4,199,271,397.00 | 785,477,302.00 | 5.346 | 75,363,377.00 | 1.1% | 2.01 | `WalletCreatePlain`
| 699,337.20 | 1,429.93 | 0.7% | 18,054,294.00 | 3,005,072.20 | 6.008 | 387,756.60 | 0.3% | 0.04 | `WalletCreateTxUseOnlyPresetInputs`
| 32,068,583.80 | 31.18 | 0.5% | 562,026,110.00 | 137,457,635.60 | 4.089 | 90,667,459.40 | 0.3% | 1.78 | `WalletCreateTxUsePresetInputsAndCoinSelection`
| 36.62 | 27,306,578.40 | 0.5% | 951.00 | 157.05 | 6.056 | 133.00 | 0.0% | 0.01 | `WalletIsMineDescriptors`
| 35.00 | 28,569,989.42 | 0.7% | 937.00 | 150.33 | 6.233 | 129.00 | 0.0% | 0.01 | `WalletIsMineMigratedDescriptors`
| 203,284,889.00 | 4.92 | 0.0% |4,622,691,895.00 | 872,875,275.00 | 5.296 | 90,345,002.00 | 1.2% | 1.02 | `WalletLoadingDescriptors`
| 1,165,766,084.00 | 0.86 | 0.0% |24,139,316,211.00 |5,005,218,705.00 | 4.823 |2,664,455,775.00 | 0.1% | 1.17 | `WalletMigration`
PR:
| ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
| 33,975,750.50 | 29.43 | 0.1% | 794,719,150.50 | 145,763,550.00 | 5.452 | 16,036,630.50 | 0.2% | 0.75 | `WalletAvailableCoins`
| 2,442.01 | 409,498.46 | 0.2% | 60,782.04 | 10,500.60 | 5.788 | 9,492.01 | 0.3% | 0.01 | `WalletBalanceClean`
| 2,763.12 | 361,909.21 | 0.2% | 61,493.05 | 11,859.48 | 5.185 | 9,625.01 | 0.2% | 0.01 | `WalletBalanceDirty`
| 2,347.98 | 425,898.72 | 0.3% | 60,782.04 | 10,082.73 | 6.028 | 9,492.01 | 0.2% | 0.01 | `WalletBalanceMine`
| 11.67 | 85,654,630.36 | 0.2% | 176.00 | 50.18 | 3.508 | 40.00 | 0.0% | 0.01 | `WalletBalanceWatch`
| 590,119,519.00 | 1.69 | 0.1% |12,754,398,258.00 |2,534,998,522.00 | 5.031 | 129,078,027.00 | 0.7% | 6.50 | `WalletCreateEncrypted`
| 183,124,790.00 | 5.46 | 0.1% |4,199,212,926.00 | 786,323,886.00 | 5.340 | 75,354,437.00 | 1.1% | 2.02 | `WalletCreatePlain`
| 669,643.00 | 1,493.33 | 0.1% | 17,213,904.20 | 2,877,336.40 | 5.983 | 394,292.80 | 0.3% | 0.04 | `WalletCreateTxUseOnlyPresetInputs`
| 26,205,987.80 | 38.16 | 0.8% | 365,551,340.80 | 112,376,905.20 | 3.253 | 65,684,276.20 | 0.4% | 1.44 | `WalletCreateTxUsePresetInputsAndCoinSelection`
| 34.75 | 28,778,846.38 | 0.1% | 937.00 | 149.41 | 6.271 | 129.00 | 0.0% | 0.01 | `WalletIsMineDescriptors`
| 29.91 | 33,428,072.85 | 0.2% | 920.00 | 128.63 | 7.152 | 126.00 | 0.0% | 0.01 | `WalletIsMineMigratedDescriptors`
| 202,437,985.00 | 4.94 | 0.1% |4,626,686,256.00 | 869,439,274.00 | 5.321 | 90,961,305.00 | 1.1% | 1.02 | `WalletLoadingDescriptors`
| 1,158,394,152.00 | 0.86 | 0.0% |24,143,589,972.00 |4,971,946,380.00 | 4.856 |2,665,355,654.00 | 0.1% | 1.16 | `WalletMigration`
ACKs for top commit:
davidgumberg:
untested reACK 215e599
murchandamus:
reACK 215e5999e2
ishaanam:
reACK 215e5999e2
w0xlt:
reACK 215e5999e2
Tree-SHA512: d6b929de56f67930678db654e46f15fb71008390189c701a026b2d76af8f14a7c9769e49835ce7e2b6515d2934a77aad8de0b1a82231a2e1de5337de25db9629
6bb38bf37f Update p2p_monitor.py (leopardracer)
Pull request description:
Fix typo in variable name in p2p_monitor.py.
ACKs for top commit:
maflcko:
lgtm ACK 6bb38bf37f
Tree-SHA512: 3d18b56766acd35f86e002fda7460f57ef8d09dd8948cc5aa172f720789624f52e1a2e4f7b095fcee7e524bdc63228903056ab2339d206efd5fbbb4189eeb969
dd99cedc0b doc: add `/spenttxouts` to REST-interface.md (Sebastian Falbesoner)
Pull request description:
Seems like adding the `spenttxouts` endpoint to the REST interface description was forgotten in #32540.
ACKs for top commit:
maflcko:
lgtm ACK dd99cedc0b
pablomartin4btc:
ACK dd99cedc0b
Tree-SHA512: 9b1da9cbab914664217cc7f8792092e672518ec7f79c7670eb1c54ef94e6cd52b139e1051035ce33ad62b7b74a169e3abc793d1804760787a11a0dc269d26402
This allows adding a GetIter(const Wtxid&) overload in a next
commit, making it easier to visit this function from a variant.
Co-authored-by: stickies-v <stickies-v@protonmail.com>
45b1d39757 doc: Add fetching single PRs from upstream (will)
Pull request description:
Current recommendation is to add a new remote fetching all PRs, but this is resource-intensive.
Document a better way to fetch a single PR, and to update a PR which has been force-pushed.
Follows up on a [comment from 32774](https://github.com/bitcoin/bitcoin/pull/32774#discussion_r2156728913)
ACKs for top commit:
pablomartin4btc:
re-ACK 45b1d39
achow101:
ACK 45b1d39757
janb84:
re ACK 45b1d39757
theStack:
ACK 45b1d39757
Tree-SHA512: 3af02aa1335fd941538fabaa527bcfa92907dc6c272e72bc37ca38211b8aeebf32dd1837f976308058360ed1364fec749b49213f2b8bc4e35542da55a7bd30e1
Currently if the version 3 is selected for an otherwise
standard spender, the test will fail. It's unlikely but
possible, so change the test to update expectations and
sample more aggressively on border values to instigate
failures much quicker in the future if another version is
made standard.
fa21631595 test: Use self.log (MarcoFalke)
fa346f7797 test: Move error string into exception (MarcoFalke)
fa1986181f test: Remove useless catch-throw (MarcoFalke)
fa2f1c55b7 move-only util data to test/functional/data/util (MarcoFalke)
faa18bf287 test: Turn util/test_runner into functional test (MarcoFalke)
fa955154c7 test: Add missing skip_if_no_bitcoin_tx (MarcoFalke)
fac9db6eb0 test: Add missing tx util to Binaries (MarcoFalke)
fa91835ec6 test: Use lowercase env var as attribute name (MarcoFalke)
fac49094cd test: Remove duplicate ConfigParser (MarcoFalke)
Pull request description:
The `test/util/test_runner.py` has many issues:
* The boilerplate for the test runner is duplicate or inconsistent with the other (functional) tests. For example, logging options, `ConfigParser` handling, `Binaries` handling ...
* The cmake/ci behavior is brittle and can silently fail, as explained in https://github.com/bitcoin/bitcoin/issues/31476
* corecheck (and likely other places that manually run the tests) completely forget to run it
* If the test is manually called, it runs single threaded, when it could just run in parallel with the other functional tests
Fix all issues by removing the util test_runner and moving the test logic into a new functional test file.
ACKs for top commit:
janb84:
re ACK fa21631595
brunoerg:
re-ACK fa21631595
hebasto:
re-ACK fa21631595, additional feedback has been addressed since my previous [review](https://github.com/bitcoin/bitcoin/pull/32697#pullrequestreview-2940350432).
Tree-SHA512: 694e647887801f002843a74011035d5ed3dfed091d3f0ae18e812a16a4680e04e60e50de0a92af7e047e8ddd6ff5a7834c690f16fd42b74ebc1674bf9989406f
6967e8e8ab add more bad p2p ports (Jameson Lopp)
Pull request description:
Add a few more ports used by extremely well adopted services that require authentication and really ought not be used by bitcoin nodes for p2p traffic.
ACKs for top commit:
Sjors:
utACK 6967e8e8ab
l0rinc:
ACK 6967e8e8ab
glozow:
ACK 6967e8e8ab
Tree-SHA512: bbe86aef2be9727338712ded8f90227f5d12f633ab5d324c8907c01173945d1c4d9899e05565f78688842bbf5ebb010d22173969e4168ea08d4e33f01fe9569d
28299ce776 p2p: remove vestigial READ_STATUS_CHECKBLOCK_FAILED (Greg Sanders)
bac9ee4830 p2p: Add witness mutation check inside FillBlock (Greg Sanders)
Pull request description:
Since #29412, we have not allowed mutated blocks to continue being processed immediately the block is received, but this is only done for the legacy BLOCK message.
Extend these checks as belt-and-suspenders to not allow similar mutation strategies to affect relay by honest peers by applying the check inside `PartiallyDownloadedBlock::FillBlock`, immediately before returning `READ_STATUS_OK`.
ACKs for top commit:
Crypt-iQ:
ACK 28299ce776
achow101:
ACK 28299ce776
stratospher:
ACK 28299ce7.
dergoegge:
Code review ACK 28299ce776
Tree-SHA512: 883d7c12ca096234b425e6fe12e46b0611607600916e6ac8d1c8112224aa76924b7b074754910163ac2ec15379075d618a9ece3642649ac7629cddb0d4e432ea
fa3f100010 test: Use msg_generic in p2p_ping.py (MarcoFalke)
Pull request description:
It seems odd to derive `msg_pong_corrupt` from `msg_pong`, but then overwrite the serialize method, when one can just directly use `msg_generic` to pass the raw bytes to send over the wire.
Fix that by using `msg_generic`. This also serves as a regression test against the fix in commit 33480573cb.
(Can be tested by reverting that commit to observe a failure)
ACKs for top commit:
dergoegge:
utACK fa3f100010
theStack:
ACK fa3f100010
Tree-SHA512: 53d7d2289f27646fdf7d3b86c53e8e707fa4ca4b006d232850f9dc27409d79b7abe1dece95ccef429d4b52c6a89579c0cc5c0ee37046375c3c0310a2d6f9ddd5
7dc43ea503 test: Add msgtype to msg_generic slots (dergoegge)
Pull request description:
`msg_generic` can't be used unless `msgtype` is listed in `__slots__`
Example from a [custom test](6329ce979f/test/functional/p2p_bug28676.py):
```
2025-06-30T10:14:55.418000Z TestFramework (INFO): PRNG seed is: 3137163719543762151
2025-06-30T10:14:55.418000Z TestFramework (INFO): Initializing test directory /tmp/nix-shell-110135-0/bitcoin_func_test_7lmiemmp
2025-06-30T10:14:55.675000Z TestFramework (INFO): Setting up connections & mining some blocks...
2025-06-30T10:14:56.511000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
File "/home/dergoegge/workspace/bitcoin/worktrees/master/test/functional/test_framework/test_framework.py", line 189, in main
self.run_test()
File "/home/dergoegge/workspace/bitcoin/worktrees/master/./build/test/functional/p2p_bug28676.py", line 46, in run_test
self.connections[0].send_without_ping(msg_generic(b"block", bytes.fromhex("0500000006226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f06b816712ffab8c59299a6abb58ccefa1995a1368ca4348782ef268cbc9bacacdbe5494dffff7f200200000001010000000001010000000000000000000000000000000000000000000000000000000000000000ffffffff075105ffffffff00ffffffff0200f90295000000002200204ae81572f06e1b88fd5ced7a1a000945432e83e1551e6f721ee9c00b8cc332600000000000000000266a24aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf90120000000000000000000000000000000000000000000000000000000000000000000000000")))
File "/home/dergoegge/workspace/bitcoin/worktrees/master/test/functional/test_framework/messages.py", line 1386, in __init__
self.msgtype = msgtype
AttributeError: 'msg_generic' object has no attribute 'msgtype'
```
ACKs for top commit:
maflcko:
lgtm ACK 7dc43ea503
theStack:
ACK 7dc43ea503
Tree-SHA512: 8c634d50a884b063117e8ae29510ffd013e73dda9f8b0f73d098e80038b610ef8d80bd2e576c37f0cedfb4b6baa3d4ebeceb0902f29f90d59e1525f418f712fe
Due to a bug in `boost_headers-config.cmake`, the `Boost::headers`
target's `INTERFACE_INCLUDE_DIRECTORIES` property is set to "/usr",
which is incorrect.
9341b5333a blockstorage: make block read hash checks explicit (Lőrinc)
2371b9f4ee test/bench: verify hash in `ComputeFilter` reads (Lőrinc)
5d235d50d6 net: assert block hash in `ProcessGetBlockData` and `ProcessMessage` (Lőrinc)
Pull request description:
A follow-up to https://github.com/bitcoin/bitcoin/pull/32487#discussion_r2094072165, after which validating the hash of a read block from disk doesn't incur the cost of calculating its hash anymore.
### Summary
This PR adds explicit checks that the read block header's hash matches the one we were expecting.
### Context
After the previous PR, validating a block's hash during read operations became essentially free. This PR leverages that by requiring callers to provide a block's expected hash (or `std::nullopt`), preventing silent failures caused by corrupted or mismatched data. Most `ReadBlock` usages were updated with expected hashes and now fail on mismatch.
### Changes
* added hash assertions in `ProcessGetBlockData` and `ProcessMessage` to validate that the block read from disk matches the expected hash;
* updated tests and benchmark to pass the correct block hash to `ReadBlock()`, ensuring the hash validation is tested - or none if we already expect PoW failure;
* removed the default value for `expected_hash`, requiring an explicit hash for all block reads.
### Why is the hash still optional (but no longer has a default value)
* for header-error tests, where the goal is to trigger failures early in the parsing process;
* for out-of-order orphan blocks, where the child hash isn't available before the initial disk read.
ACKs for top commit:
maflcko:
review ACK 9341b5333a🕙
achow101:
ACK 9341b5333a
hodlinator:
ACK 9341b5333a
janb84:
re ACK 9341b5333a
Tree-SHA512: cf1d4fff4c15e3f8898ec284929cb83d7e747125d4ee759e77d369f1716728e843ef98030be32c8d608956a96ae2fbefa0e801200c333b9eefd6c086ec032e1f
Ensure that tip_header.rehash() is used instead of tip_header.hash, which is None when the header is deserialized from hex.
This avoids depending on wait_for_getheaders() falling back to any received message, making the test more explicit and robust.
Some ambiguous uses of "we" referring to either the node or the peer are replaced with clearer phrasing.
Also rephrase some comments for consistency and readability.
Applies to all relevant outbound eviction tests in p2p_eviction_logic.py.
14653b869b build: Find Boost in config mode (Hennadii Stepanov)
Pull request description:
The `FindBoost` module has been removed by policy [CMP0167](https://cmake.org/cmake/help/latest/policy/CMP0167.html).
ACKs for top commit:
purpleKarrot:
ACK 14653b869b
Tree-SHA512: 5ec88647af83158f9bc04b41a3b72d4da7d84a7c81af351b8dac61cdf7f2f3b34bedd6ff164f21c229f2fd442918aaf21ba7c2c81c346b64de9032aae27b10ce
9f8e7b0b3b node: cap -dbcache to 1GiB on 32-bit architectures (Antoine Poinsot)
2c43b6adeb init: cap -maxmempool to 500 MB on 32-bit systems (Antoine Poinsot)
Pull request description:
32-bit architecture is limited to 4GiB of RAM, so it doesn't make sense to set a too high value. A too high value could cause an OOM unbeknownst to the user a while after startup as mempool / dbcache fills.
ACKs for top commit:
achow101:
ACK 9f8e7b0b3b
instagibbs:
utACK 9f8e7b0b3b
dergoegge:
Code review ACK 9f8e7b0b3b
glozow:
utACK 9f8e7b0b3b
Tree-SHA512: cc7541b2c0040fc21a43916caec464dfb443af808f4e85deffa1187448ffff6edb0d69f9ebdb43915d145b8b4694d8465afe548f88da53ccebc9ce4b7c34b735
173394d951 depends: Build `qt` package for FreeBSD hosts (Hennadii Stepanov)
Pull request description:
This PR continues the work started in https://github.com/bitcoin/bitcoin/pull/23948.
Here is an excerpt from the log:
```
$ ./build/bin/bitcoin-qt -printtoconsole
2025-06-12T01:06:56Z Bitcoin Core version v29.99.0-15de25ba2a28 (release build)
2025-06-12T01:06:56Z Qt 6.7.3 (static), plugin=xcb
2025-06-12T01:06:56Z Static plugins:
2025-06-12T01:06:56Z QMinimalIntegrationPlugin, version 395008
2025-06-12T01:06:56Z QXcbIntegrationPlugin, version 395008
2025-06-12T01:06:56Z Style: fusion / QFusionStyle
2025-06-12T01:06:56Z System: FreeBSD 14.3-RELEASE, x86_64-little_endian-lp64
```
And here are the screenshots:


ACKs for top commit:
vasild:
ACK 173394d951
Tree-SHA512: 42a0bd11e4ef1a23efcfe6c4ab179dc667a076e65060891ce8358b3fe78de4e3ea33f975387d4236cc2ac620e2935b0a29c278065a47f038c66658106bf36755
8800b5acc1 cmake: Explicitly specify `Boost_ROOT` for Homebrew's package (Hennadii Stepanov)
Pull request description:
On macOS, this PR ensures that the Boost package is located at its real path rather than via the symlink in the default prefix.
A backport to 29.x is required for https://github.com/bitcoin/bitcoin/pull/32804, as this change prevents contamination of include directories by broad locations such as `/usr/local/include` or `/opt/homebrew/include`, which take precedence over Qt’s `-iframework` flags.
Below is the relevant change in the configuration logs on my macOS 15.5 `x64`:
- master branch @ ead4468748:
```
% cmake -B build
<snip>
-- Found Boost: /usr/local/include (found suitable version "1.88.0", minimum required is "1.73.0")
<snip>
```
- this PR:
```
% cmake -B build
<snip>
-- Found Boost: /usr/local/opt/boost/include (found suitable version "1.88.0", minimum required is "1.73.0")
<snip>
```
This PR is forward-compatible with the changes proposed in https://github.com/bitcoin/bitcoin/pull/32667.
ACKs for top commit:
fanquake:
ACK 8800b5acc1 Checked that this plus #32805fixes#31009
Tree-SHA512: 114bd945ec0c06a8d15b565e5b9aafc3bcfdf2a4ba4400e072b8e31053dff0f9ef423b941ee1dff2113f83e08f7fada728383ae88b3ec380b5c3e40553205f7d
6c2538d5bf depends: Bump boost to 1.88.0 and use new CMake buildsystem (Cory Fields)
Pull request description:
Originally #30434.
This has a few advantages over the old method of simply copying headers:
- Installs proper CMake files which can be picked up by our buildsystem
- Only installs necessary headers, not all of Boost
Pulls in upstreamed https://github.com/boostorg/test/pull/445.
ACKs for top commit:
willcl-ark:
tACK 6c2538d5bf
hebasto:
re-ACK 6c2538d5bf, only rebased since my previous [review](https://github.com/bitcoin/bitcoin/pull/32665#pullrequestreview-2891203225).
Tree-SHA512: fc3fce77b21c8ea500370841f44f1cc87e0bb09cdde55f75d2f90853cb06a6f8c73ac6ca9ca3e91a879e9f95dd59aa40254c1b04e7a168c52fa1b31cc5b7f537
ead4468748 cmake: Use `HINTS` instead of `PATHS` in `find_*` commands (Hennadii Stepanov)
Pull request description:
According to the CMake documentation, `HINTS` "should be paths computed by system introspection, such as a hint provided by the location of another item already found", which is precisely the case in the `FindQRencode` module.
Entries in `HINTS` are searched before those in `PATHS`. On macOS, Homebrew’s `libqrencode` will therefore be located at its real path rather than via the symlink in the default prefix.
A backport to 29.x is required for https://github.com/bitcoin/bitcoin/pull/32804, as this change prevents contamination of include directories by broad locations such as `/usr/local/include` or `/opt/homebrew/include`, which take precedence over Qt’s `-iframework` flags.
Below is the relevant change in the configuration logs on my macOS 15.5 `x64`:
- master branch @ ead4468748:
```
% cmake -B build -DBUILD_GUI=ON
<snip>
-- Found QRencode: /usr/local/lib/libqrencode.dylib (found version "4.1.1")
<snip>
```
- this PR:
```
% cmake -B build -DBUILD_GUI=ON
<snip>
-- Found QRencode: /usr/local/Cellar/qrencode/4.1.1/lib/libqrencode.dylib (found version "4.1.1")
<snip>
```
ACKs for top commit:
fanquake:
ACK ead4468748
Tree-SHA512: 1f0b04e3efeb7fe3efbb969be911abbcf56030d715acd87c0fbaf24422cdf1122f169e32242571256916c96a159212842e1e73092145c63ecc495ce429c6e587
- Increase block weight by 4000 for all nodes with custom -blockmaxweight.
Prior to this commit, we generated blocks with 4000 weight units less worth of transactions.
See https://github.com/bitcoin/bitcoin/issues/32461#issuecomment-2925282272 for details.
This commit fixes it by increasing the block weight by 4000.
- Update `check_smart_estimates` to calculate the fee rate ceiling
by taking the maximum of fees seen, minrelaytxfee, and mempoolminfee.
- Improve the subtest name and comments.
dd8447f70f test: fix catchup loop in outbound eviction functional test (Sebastian Falbesoner)
Pull request description:
In the course of working on an equivalent of #32421 for the `CBlockHeader` class, I noticed that the [catchup loop in the outbound eviction functional test](19765dca19/test/functional/p2p_outbound_eviction.py (L86-L103)) currently has a small flaw: the contained waiting for a `getheaders` message
19765dca19/test/functional/p2p_outbound_eviction.py (L98-L99)
only waits for _any_ such message instead of one with the intended block hash after the first iteration. The reason is that the `prev_prev_hash` variable is set incorrectly, since the `tip_header` instance is not updated and its field `.hash` is None [1]. Fix that by updating `tip_header` after generating a new block and also use the correct field on it -- we want the tip header's previous hash (`.hashPrevBlock`), which will be the previous-previous hash in the next iteration as intended.
Can be demonstrated by adding a debug output for `prev_prev_hash`, e.g.
```diff
diff --git a/test/functional/p2p_outbound_eviction.py b/test/functional/p2p_outbound_eviction.py
index 30ac85e32f..9886a49512 100755
--- a/test/functional/p2p_outbound_eviction.py
+++ b/test/functional/p2p_outbound_eviction.py
@@ -85,6 +85,7 @@ class P2POutEvict(BitcoinTestFramework):
self.log.info("Keep catching up with the old tip and check that we are not evicted")
for i in range(10):
+ print(f"i={i}, prev_prev_hash={prev_prev_hash}")
# Generate an additional block so the peers is 2 blocks behind
prev_header = from_hex(CBlockHeader(), node.getblockheader(best_block_hash, False))
best_block_hash = self.generateblock(node, output="raw(42)", transactions=[])["hash"]
```
master branch
```
...
i=0, prev_prev_hash=21722572577213525620063947414919931742473663114977483853465070858884938201585
i=1, prev_prev_hash=None
i=2, prev_prev_hash=None
i=3, prev_prev_hash=None
i=4, prev_prev_hash=None
i=5, prev_prev_hash=None
i=6, prev_prev_hash=None
i=7, prev_prev_hash=None
i=8, prev_prev_hash=None
i=9, prev_prev_hash=None
...
```
PR branch
```
...
i=0, prev_prev_hash=21722572577213525620063947414919931742473663114977483853465070858884938201585
i=1, prev_prev_hash=23204083306104595181276643925327085197417756603258684897360269464191973063397
i=2, prev_prev_hash=18117007775254206852722585270408843074799046031613422902091537272077477361634
i=3, prev_prev_hash=30556804635951812756130312631227721973553160707632138130845362630877961299882
i=4, prev_prev_hash=16476515948153779819467376247405243058769281687868039119037064816106574626111
i=5, prev_prev_hash=14965506521435221774966695805624206855826023174786191695076697927307467053159
i=6, prev_prev_hash=14510815979277079515923749862202324542606166669768865640616202929053689167149
i=7, prev_prev_hash=15360268707191667685151951417759114642582372006627142890517655217275478262166
i=8, prev_prev_hash=55984929479619644661389829786223559362979512070332438490054115824374865094074
i=9, prev_prev_hash=6591573629906616262191232272909118561529534571119028248829355592878183757083
...
```
[1] that's in my opinion another example how caching hashes is confusing and easy to be misused; it's better to remove it and just compute the hash on-the-fly, so returning None is not even possible anymore
ACKs for top commit:
maflcko:
lgtm ACK dd8447f70f
mzumsande:
Code Review ACK dd8447f70f
pablomartin4btc:
cr-ACK dd8447f70f
Tree-SHA512: bd8e786b52e3e96661453006140d6b8fad5a35f1c8d38243c61df52b19c97cd3800404745a2f9603bcdf0006e9780b4f15f8f7e4fa34ff07d52dba04d87b68d0
Reimplements the GenTxid class as a variant for better type safety.
Also adds two temporary functions to the old GenTxid class that
convert to and from the new variant.
When a legacy wallet has been migrated to contain descriptors, but
before the transactions have been updated to match, we need to recompute
the wallet TXOs so that the transaction update will work correctly.
Instead of searching mapWallet for the preselected inputs, search
m_txos.
wallet_fundrawtransaction.py spends external inputs and needs the change
output to also belong to the test wallet for the oversized tx test.
Instead of iterating every transaction and every output stored in wallet
when trying to figure out what outputs can be spent, iterate the TXO set
which should be a lot smaller.
Since we track the outputs owned by the wallet with m_txos, we can now
calculate the balance of the wallet by iterating m_txos and summing up
the amounts of the unspent txos.
As ISMINE_USED is not an actual isminetype that we attach to outputs and
was just passed into `CachedTxGetAvailableCredit` for convenience, we
pull the same determining logic from that function into `GetBalances` in
order to preserve existing behavior.
After adding a wallet descriptor (typically by import), mark all balance
caches dirty. This allows transactions that the wallet already knows
about that have outputs that are now ISMINE_SPENDABLE after the import
to actually be shown in balance calculations. Legacy wallet imports
would do this, but importdescriptors did not.
One of the main issues with AvailableCoins is its performance when txs
have unrelated outputs, so update the benchmark to check the performance
of that.
c3fe85e2d6 wallet, rpc, test: Remove deprecated getunconfirmedbalance (Ava Chow)
0ec255139b wallet, rpc: Remove deprecated balances from getwalletinfo (Ava Chow)
Pull request description:
`getwalletinfo` result fields `balance`, `immature_balance`, and `unconfirmed_balance`, and the `getunconfirmedbalance` RPC have all been deprecated since 0.19.0. It's been long enough that they should either be removed or undeprecated. The functionality provided by these RPCs is provided by `getbalances`.
ACKs for top commit:
davidgumberg:
ACK c3fe85e2d6
rkrux:
ACK c3fe85e2d6
BrandonOdiwuor:
ACK c3fe85e2d6 removing the deprecated `balance, unconfirmed_balance, immature_balance` fields from `getwalletinfo` and `getunconfirmedbalance` RPCs, as this infomation can be found on the `getbalances` RPC
w0xlt:
reACK c3fe85e2d6
Tree-SHA512: c7c4acfd9cabc7517ba813b95281a6c6a717a417312afd9346298669b4f7bd37724ad977148ce42db7fd47fc3d1f5a8482d8ff2e7b9cb74756b171a5b8b91ef2
47237cd193 wallet, rpc: Output wallet flags in getwalletinfo (Ava Chow)
bc2a26b296 wallet: Add GetWalletFlags (Ava Chow)
69f588a99a wallet: Set upgraded descriptor cache flag for newly created wallets (Ava Chow)
Pull request description:
Newly created wallets will always have an upgraded descriptor cache, so set those.
Also, to verify this behavior, add a new `flags` field to `getwalletinfo` and check that in the functional tests.
Split from #32489
ACKs for top commit:
Sjors:
ACK 47237cd193
w0xlt:
ACK 47237cd193
rkrux:
ACK 47237cd193
Tree-SHA512: 97c7f85b858efe5ced9b8aafb6cd7c1a547de6f8013b82bfc75bc567cf73c9db5e168e3980355756541305520022fd776b8d4d240d3fb34ed86c27d2acaf4863
The wallet backups performed before migration use the time as part of
their filename. As the time is mocked, increment it between migration
attempts to prevent file name conflicts which is a problem on Windows.
This test required v0.14.3 which doesn't support special characters like
emojis in the datapath on windows. Functional test runner includes a few
emojis in it's default data directory.
9eb2c82e7c walletdb: Remove unused upgraded_txs (Ava Chow)
c668033709 wallet: Remove unused fTimeReceivedIsTxTime (Ava Chow)
Pull request description:
`CWalletTx::fTimeReceivedIsTxTime` is no longer used and can be removed. This additionally allows the removal of the `upgraded_txs` loop in `LoadWallet`.
ACKs for top commit:
maflcko:
lgtm ACK 9eb2c82e7c
Eunovo:
ACK 9eb2c82e7c
davidgumberg:
ACK 9eb2c82e7c
PeterWrighten:
ACK 9eb2c82e7c
rkrux:
ACK 9eb2c82e7c
w0xlt:
ACK 9eb2c82e7c
Tree-SHA512: 05cf3a50f0d8ab6ef423ad1113c5ce6f45bfdc90e2c0dcf61c2dceced2465502e574b4b5b0091fcbb4bdd2182f8d69224f1e5516c7c505de07102b84a5f40e9c
Windows zip files are added in the next commit which are not tarballs so
renaming tarball to the more generic term archive which can cover both.
-BEGIN VERIFY SCRIPT-
sed -i 's/tarball/archive/g' test/get_previous_releases.py
-END VERIFY SCRIPT-
Using the get_previous_releases.py script to build from source only works for
releases prior to v29 due to removal of Autotools (in favor of CMake). It also
does not support building on Windows, and we are adding support for downloading
Windows release binaries in later commits of this PR.
As there were no complaints during review, it is assumed nobody uses this
functionality.
Current recommendation is to add a new remote fetching all PRs, but this
is resource-intensive.
Document a better way to fetch a single PR, and to update a PR which has
been force-pushed.
This has a few advantages over the old method of simply copying headers:
- Installs proper cmake files which can be picked up by our buildsystem
- Only installs necessary headers, not all of boost
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
e27a94596f build: add root dir to CMAKE_PREFIX_PATH (will)
Pull request description:
Fixes: #32428
Nix patches `cmake` to remove the root directory `/` from `CMAKE_PREFIX_PATH`:
428b49b28e/pkgs/by-name/cm/cmake/001-search-path.diff (L10)
Without this, and when using the toolchain for depends builds, cmake's `find_path()` and `find_package()` do not know where to find dependencies, causing issues like as seen in #32428
Adding this path back is harmless on other systems, and fixes the toolchain for Nix users.
As described in https://github.com/bitcoin/bitcoin/issues/32428#issuecomment-2991258328 I think this can be taken as a temporary fix whilst a longer-term solution is worked on.
ACKs for top commit:
josibake:
reACK e27a94596f
hebasto:
ACK e27a94596f, I have reviewed the code and it looks OK.
janb84:
reACK e27a94596f
Tree-SHA512: f299f2bab2620179518da866cbb7992d41d142ad42e79c14496e72f725a1dc60698b3e4b1daf45d28f71f32a23f0c8d7b4f6c6cf33aeedf322b7ef565b70b4af
According to the CMake documentation, `HINTS` "should be paths computed
by system introspection, such as a hint provided by the location of
another item already found", which is precisely the case in the
`FindQRencode` module.
Entries in `HINTS` are searched before those in `PATHS`. On macOS,
Homebrew’s `libqrencode` will therefore be located at its real path
rather than via the symlink in the default prefix.
fa68dcb207 ci: Add missing errexit to lint CI install (MarcoFalke)
fa535a6de7 ci: Allow running CI in worktrees (MarcoFalke)
faf6a04597 ci: Clean UID/GID mismatch (MarcoFalke)
Pull request description:
Fixes#30028 (modulo lint and tidy CI).
The error on current master in a worktree is:
```
$ git worktree add ./main origin/master && cd ./main
$ MAKEJOBS="-j$(nproc)" FILE_ENV="./ci/test/00_setup_env_mac_cross.sh" ./ci/test_run_all.sh
...
+ git config --global ci.base-install-done true
fatal: not a git repository: /root/b-c-ci/.git/worktrees/main
```
So just use a plain file, instead of git. Also, enable pipefail while touching this bash script.
ACKs for top commit:
willcl-ark:
tACK fa68dcb207
Tree-SHA512: 0ce360a80883b4aa655fe8a99c38eb54a465b17c7cdb0a69a2d886ff78da32d6af996412ffc5b0db0322acafa9650619838573484de8243dc41594a04a6e17ec
Nix patches cmake to remove the root directory `/` from
`CMAKE_SYSTEM_PREFIX_PATH`:
428b49b28e/pkgs/by-name/cm/cmake/001-search-path.diff (L10)
Without this, and when using the toolchain for depends builds, cmake's
`find_path()` and `find_package()` do not know where to find
dependencies, causing issues like:
https://github.com/bitcoin/bitcoin/issues/32428
Adding this path back via CMAKE_PREFIX_PATH is harmless on other
systems, and fixes the toolchain for Nix users.
We append the `/` dir a maximum of once, as the toolchain may be called
repeatedly during builds.
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: josibake <josibake@protonmail.com>
Reason for each test:
rpc_whitelist.py: Relies on direct RPC calls
wallet_encryption.py: Null characters cannot be passed to suprocess.Popen
wallet_fundrawtransaction.py: multiple checks for wrong types, which have different error messages with cli
wallet_send.py: multiple checks for wrong types
If python passed None for an optional (i.e. 'null' is
sent), this will lead to the arg being interpreted as not
provided by bitcoind - except for string args, for which the arg is
interpreted as as 'null' string. Bypass this by not sending
named args to bitcoin-cli - so that the default value will
actually be used.
Also drops an unnecessary str() conversion, kwargs keys
are always strings.
The psbt string would include a "=" sign, which would
make the cli interpret this as a named argument.
Fix this by making it an actual named arg with the
correct name.
Also, the following tests (for which self.supports_cli = False was not
set) will now work with --usecli:
feature_fastprune.py
feature_fee_estimation.py
feature_reindex_readonly.py
feature_taproot.py
mempool_package_rbf.py
p2p_net_deadlock.py
p2p_tx_download.py
rpc_packages.py
Because of the MAX_ARG_STRLEN limit (128kb on most systems)
for args, these would usually fail. As a workaround, use
-stdin for these large calls. Idea by 0xB10C.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
3473986fe1 contrib: tracing: Correctly read msg type in p2p_monitor.py (David Gumberg)
Pull request description:
This fixes a bug in the contrib tracing script `p2p_monitor.py`. currently the script fails to read the `msg_type` of inbound and outbound messages, which is useful in the per-peer message view.
<details>
<summary>Screenshot of p2p_monitor.py on master</summary>

</details>
<details>
<summary>Screenshot of p2p_monitor.py on this branch</summary>

</details>
ACKs for top commit:
yuvicc:
ACK 3473986fe1
janb84:
ut ACK 3473986fe1
0xB10C:
ACK 3473986fe1
Tree-SHA512: 94da0dc35072933a20ef693024855b3c382fc6f5ae0a3108d092d7aa5a4004df478f5de07b80f675be13e00f3f4596b0f34c49ec1d8d2c38a15797dcf86c2a56
578ea3eedb test: round difficulty and networkhashps (Sjors Provoost)
Pull request description:
Both are rational numbers. Client software should only use them to display information to humans. Followup calculations should use the underlying values such as target.
Therefore it's not necessary to test the handling of these floating point values. Round them down to avoid spurious test failures.
Fixes#32515
ACKs for top commit:
Prabhat1308:
Code Review ACK [`578ea3e`](578ea3eedb)
achow101:
ACK 578ea3eedb
w0xlt:
Code review ACK 578ea3eedb
janb84:
ACK 578ea3eedb
Tree-SHA512: 5fc63c73ad236b7cd55c15da0f1d1e6b45e4289d252147a86717bf77d79f897f42c3e38aa514df6a4a8deca10c87a8710b61b454c533ad56b0daf738365f426c
b184f5c87c test: update BIP340 test vectors and implementation (variable-length messages) (Sebastian Falbesoner)
Pull request description:
This PR updates the Schnorr signatures implementation in the functional test framework to the latest BIP changes (see https://github.com/bitcoin/bips/pull/1446,commit 200f9b26fe0a2f235a2af8b30c4be9f12f6bc9cb) and syncs the [test vectors](https://github.com/bitcoin/bips/blob/master/bip-0340/test-vectors.csv) accordingly. Practically, we probably don't need non-32-bytes message signing/verifying any time soon, but it seems good practice anyways to update.
ACKs for top commit:
stratospher:
ACK b184f5c.
achow101:
ACK b184f5c87c
real-or-random:
utACK b184f5c87c
jonasnick:
utACK b184f5c87c
Tree-SHA512: b566823aa0f1cd7151215178c57551d772b338d022ccb2807a0df2670df6d59c4b63a6fc936708ccf2922c7e59f474f544adaafc4aea731bfd896250c0d45fa6
272cd09b79 log: Use warning level while scanning wallet dir (MarcoFalke)
1777644367 qa, wallet: Verify warning when failing to scan (Hodlinator)
893e51ffeb wallet: Correct dir iteration error handling (Hodlinator)
Pull request description:
Make wallet DB properly detect and report failure to scan wallet directory. Seems to have been broken since moving from Boost to `std::filesystem`.
Found while reviewing: https://github.com/bitcoin/bitcoin/pull/31410#pullrequestreview-2604068753
ACKs for top commit:
achow101:
ACK 272cd09b79
maflcko:
re-ACK 272cd09b79 🍽
rkrux:
tACK 272cd09b79
Tree-SHA512: 969afde2e37f885ed0c823dc36d2dbeaa0378639849c6a26f8ac67b4f1997eea95bbcae6d58aef5b716807210f37eb166c0cda7ba1d6caffd34249970833af3a
This change ensures that the `script_assets_tests` test case is
explicitly reported as "Skipped" when it is not run, making it clearer
when running the test suite with `ctest`.
cd1ae1b4df fuzz: wallet: remove FundTx from FuzzedWallet (brunoerg)
Pull request description:
`FundTx` was used by the `wallet_notifications` target which we recently removed. So it's now unused and can be removed.
ACKs for top commit:
maflcko:
lgtm ACK cd1ae1b4df
kevkevinpal:
ACK [cd1ae1b](cd1ae1b4df)
dergoegge:
utACK cd1ae1b4df
Tree-SHA512: 909cc4c8a0ac2a5f8844993ccf0e725021932888da3591925799145daf9196eadfcd0ebbc74a44f4a245074ded4cb8c3c099513f109ce2681dceff36b5f74bcc
e285e691b7 test: Fix list index out of range error in feature_bip68_sequence.py (zaidmstrr)
Pull request description:
Fixes [#32334](https://github.com/bitcoin/bitcoin/issues/32334)
The test `feature_bip68_sequence.py` fails with `IndexError: list index out of range` error due to a mismatch between the number of inputs requested (at random) and the number of UTXOs available. The error is reproducible with the randomseed:
```
$ ./build/test/functional/feature_bip68_sequence.py --randomseed 6169832640268785903
```
This PR adds a valid upper bound to randomly select the inputs.
ACKs for top commit:
maflcko:
lgtm ACK e285e691b7
Prabhat1308:
re-ACK [`e285e69`](e285e691b7)
theStack:
ACK e285e691b7
Tree-SHA512: 2e5e19d5db2880915f556ed4444abed94e9ceb1ecee5f857df5616040c850dae682aaa4ade3060c48acb16676df92ba81c3af078c1958965e9e874e7bb489388
fa94fd53c9 doc: Explain how to fetch commits directly (MarcoFalke)
Pull request description:
This is often needed, and works better than the existing refspec documentation, because even commits that have been force-pushed away can be fetched (as long as they are not garbage collected on the remote).
ACKs for top commit:
Sjors:
ACK fa94fd53c9
l0rinc:
ACK fa94fd53c9
willcl-ark:
ACK fa94fd53c9
rkrux:
ACK fa94fd53c9
janb84:
ACK fa94fd53c9
Tree-SHA512: b68c0c612e13f501ad4c1c709502e060b0a2d0eb55ef888c7466e2a10bdf3ca63d81b8bd7927de49cde9e29f0b06f8233d51b99d015ae0b39d556854be542b8a
53a996f122 doc: fix transifex 404s (fanquake)
Pull request description:
https://www.transifex.com/bitcoin/bitcoin/ is now a 404.
ACKs for top commit:
maflcko:
lgtm ACK 53a996f122
hebasto:
ACK 53a996f122, I've verified all the links.
Tree-SHA512: 8e698c83095a3d3a225b0bf2ee9c39ad434b2917ead4271ff39a282cea6283710091d1e8b91edafd280bf356dec2bdbe42981aafe4d64f623a975232c5ca848c
8ee8a951c2 doc: taproot became always active in v24.0 (Sjors Provoost)
Pull request description:
Split from #26201.
ACKs for top commit:
maflcko:
lgtm ACK 8ee8a951c2
janb84:
ACK 8ee8a951c2
Tree-SHA512: 1ac6994c6775ca5423f022d1e02e3d531fb7fa295be9940355b8aa9d173787a8d65945a0cf976ab344bcaa3ea8a0f3aa6f8da851325bf475e59375981b115cab
a18e572328 test: more template verification tests (Sjors Provoost)
10c908808f test: move gbt proposal mode tests to new file (Sjors Provoost)
94959b8dee Add checkBlock to Mining interface (Sjors Provoost)
6077157531 ipc: drop BlockValidationState special handling (Sjors Provoost)
74690f4ed8 validation: refactor TestBlockValidity (Sjors Provoost)
Pull request description:
This PR adds the IPC equivalent of the `getblocktemplate` RPC in `proposal` mode.
In order to do so it has `TestBlockValidity` return error reasons as a string instead of `BlockValidationState`. This avoids complexity in IPC code for handling the latter struct.
The new Mining interface method is used in `miner_tests`.
It's not used by the `getblocktemplate` and `generateblock` RPC calls, see https://github.com/bitcoin/bitcoin/pull/31981#discussion_r2096473337
The `inconclusive-not-best-prevblk` check is moved from RPC
code to `TestBlockValidity`.
Test coverage is increased by `mining_template_verification.py`.
Superseedes #31564
## Background
### Verifying block templates (no PoW)
Stratum v2 allows miners to generate their own block template. Pools may wish (or need) to verify these templates. This typically involves comparing mempools, asking miners to providing missing transactions and then reconstructing the proposed block.[^0] This is not sufficient to ensure a proposed block is actually valid. In some schemes miners could take advantage of incomplete validation[^1].
The Stratum Reference Implementation (SRI), currently the only Stratum v2 implementation, collects all missing mempool transactions, but does not yet fully verify the block.[^2]. It could use the `getblocktemplate` RPC in `proposal` mode, but using IPC is more performant, as it avoids serialising up to 4 MB of transaction data as JSON.
(although SRI could use this PR, the Template Provider role doesn't need it, so this is _not_ part of #31098)
[^0]: https://github.com/stratum-mining/sv2-spec/blob/main/06-Job-Declaration-Protocol.md
[^1]: https://delvingbitcoin.org/t/pplns-with-job-declaration/1099/45?u=sjors
[^2]: https://github.com/stratum-mining/stratum/blob/v1.1.0/roles/jd-server/src/lib/job_declarator/message_handler.rs#L196
ACKs for top commit:
davidgumberg:
reACK a18e572328
achow101:
ACK a18e572328
TheCharlatan:
ACK a18e572328
ryanofsky:
Code review ACK a18e572328 just adding another NONFATAL_UNREACHABLE since last review
Tree-SHA512: 1a6c29f45a1666114f10f55aed155980b90104db27761c78aada4727ce3129e6ae7a522d90a56314bd767bd7944dfa46e85fb9f714370fc83e6a585be7b044f1
rsync --archive will preserve owner and group, which is then required to
be handled by adding a git safe.directory workaround.
Remove the need for the workaround by only preserving permissions during
the recursive rsync copy.
a201a99f8c thread-safety: fix annotations with REVERSE_LOCK (Cory Fields)
aeea5f0ec1 thread-safety: add missing lock annotation (Cory Fields)
832c57a534 thread-safety: modernize thread safety macros (Cory Fields)
Pull request description:
This is one of several PRs to cleanup/modernize our threading primitives.
While replacing the old critical section locks in the mining code with a `REVERSE_LOCK`, I noticed that our thread-safety annotations weren't hooked up to it. This PR gets `REVERSE_LOCK` working properly.
Firstly it modernizes the attributes as-recommended by the [clang docs](https://clang.llvm.org/docs/ThreadSafetyAnalysis.html) (ctrl+f for `USE_LOCK_STYLE_THREAD_SAFETY_ATTRIBUTES`). There's a subtle difference between the old `unlock_function` and new `release_capability`, where our `reverse_lock` only works with the latter. I believe this is an upstream bug. I've [reported and attempted a fix here](https://github.com/llvm/llvm-project/pull/139343), but either way it makes sense to me to modernize.
The second adds a missing annotation pointed out by a fixed `REVERSE_LOCK`. Because clang's thread-safety annotations aren't passed through a reference to `UniqueLock` as one may assume (see [here](https://clang.llvm.org/docs/ThreadSafetyAnalysis.html#no-alias-analysis) for more details), `cs_main` has to be listed explicitly as a requirement.
The last commit actually fixes the `reverse_lock` by making it a `SCOPED_LOCK` and using the pattern [found in a clang test](https://github.com/llvm/llvm-project/blob/main/clang/test/SemaCXX/warn-thread-safety-analysis.cpp#L3126). Though the docs don't describe how to accomplish it, the functionality was added [in this commit](6a68efc959). Due to aliasing issues (see link above), in order to work correctly, the original mutex has to be passed along with the lock, so all existing `REVERSE_LOCK`s have been updated. To ensure that the mutexes actually match, a runtime assertion is added.
ACKs for top commit:
fjahr:
re-ACK a201a99f8c
davidgumberg:
reACK a201a99f8c
theuni:
Ok, done. Those last pushes can be ignored. ACKs on a201a99 are still fresh.
ryanofsky:
Code review ACK a201a99f8c. Just dropping 0065b9673db5da2994b0b07c1d50ebfb19af39d0 and fixing incorrect `reverse_lock::lockname` initialization since last review.
TheCharlatan:
Re-ACK a201a99f8c
Tree-SHA512: 2755fae0c41021976a1a633014a86d927f104ccbc8014c01c06dae89af363f92e5bc5d4276ad6d759302ac4679fe02a543758124d48318074db1c370989af7a7
9dfc61d95f test: detect no external signer connected (Sjors Provoost)
0a4ee93529 wallet: use PSBTError::EXTERNAL_SIGNER_NOT_FOUND (Sjors Provoost)
8ba2f9b7c8 refactor: use util::Result for GetExternalSigner() (Sjors Provoost)
Pull request description:
When attempting to sign a transaction involving an external signer, if the device isn't connected we throw an `std::runtime_error`. This prevents the (mainly GUI) code that's actually supposed to handle this case from running.
This PR returns a `PSBTError::EXTERNAL_SIGNER_NOT_FOUND` instead of throwing.
The first commit is a refactor to have `GetExternalSigner()` return a `util::Result<ExternalSigner>` so the caller can decide how to handle the error. There are two other places where call `GetExternalSigner()` which this PR doesn't change (which I think is fine there).
Before:

After (the translation already exist):

Fixes#32426
Additionally use `LogWarning` instead of `std::cerr` for both a missing signer and failure to sign.
ACKs for top commit:
achow101:
ACK 9dfc61d95f
brunoerg:
code review ACK 9dfc61d95f
Tree-SHA512: 22515f4f0b4f50cb0ef532b729e247f11a68be9c90e384942d4277087b2e76806a1cdaa57fb51d5883dacf0a428e5279674aab37cce8c0d3d7de0f96346b8233
Without proper annotations, clang thinks that mutexes are still held for the
duration of a reverse_lock. This could lead to subtle bugs as
EXCLUSIVE_LOCKS_REQUIRED(foo) passes when it shouldn't.
As mentioned in the docs [0], clang's thread-safety analyzer is unable to deal
with aliases of mutexes, so it is not possible to use the lock's copy of the
mutex for that purpose. Instead, the original mutex needs to be passed back to
the reverse_lock for the sake of thread-safety analysis, but it is not actually
used otherwise.
[0]: https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
c7eaac326a depends: capnp 1.2.0 (fanquake)
Pull request description:
See https://github.com/capnproto/capnproto/compare/release-1.1.0...release-1.2.0. We can drop all the patches we are currently applying.
ACKs for top commit:
Sjors:
ACK c7eaac326a
theStack:
ACK c7eaac326a
ryanofsky:
Code review ACK c7eaac326a. Just checked hashes, compared tarball to git and diffed 1.1.0 and 1.2.0 tarballs which showed only minor and expected changes.
Tree-SHA512: 75085ec96952e9693c67531c3d04cd0d7df580dd1df35ce50dff618b29f651674c17a84e9089c6b7ed230e2b4fd0a7f24e2220e983ec00235db9a9d1ee2d7116
If an `AutoFile` has been written to, then expect callers to have closed
it explicitly via the `AutoFile::fclose()` method. This is because if
the destructor calls `std::fclose()` and encounters an error, then it
is too late to indicate this to the caller in a meaningful way.
There is no way to report a close error from `AutoFile` destructor.
Such an error could be serious if the file has been written to because
it may mean the file is now corrupted (same as if write fails).
So, change all users of `AutoFile` that use it to write data to
explicitly close the file and handle a possible error.
useful to easily create transactions with same txid, different
wtxid and valid witness for testing scenarios in other places
(ex: private broadcast connections)
6ecb9fc65f chore: use `std::vector<std::byte>` for `BlockManager::ReadRawBlock()` (Roman Zeyde)
Pull request description:
Following [this comment](https://github.com/bitcoin/bitcoin/pull/32540#discussion_r2135820932), this PR changes `BlockManager::ReadRawBlock()` to accept a `std::vector<std::byte>` instead of `std::vector<uint8_t>`, in order to avoid casts during its invocations.
It also adds a new `SpanReader` constructor to allow reading from a span of `std::byte`s (in addition to span of `uint8_t`).
ACKs for top commit:
l0rinc:
ACK 6ecb9fc65f
maflcko:
re-ACK 6ecb9fc65f
TheCharlatan:
Re-ACK 6ecb9fc65f
Tree-SHA512: b0976c34b8da4fa1e6d805a89de2883f48ba431a71069e8c1ae450f48e425cc41aff1a5d479a7d40312a972aaf1f92e9478a985a14a1357c6b3e564e988d03e5
Rather than this exhaustive linearization check happening inline inside
clusterlin_simple_linearize, abstract it out into a Linearize()-like
function for clarity.
Note that this isn't exactly a refactor, because the old code would compare the
found linearization against all (valid) permutations, while the new code instead
first computes the best linearization from all valid permutations, and then
compares it with the found one.
In several call sites for ReadTopologicalSubset, a non-empty result is
expected, necessitating a special case at the call site for empty results.
Fix this by adding a bool non_empty argument, which does this special
casing (more efficiently) inside ReadTopologicalSubset itself.
Whenever a non-topological permutation is encountered, fast forward to the
last permutation with the same non-topological prefix, skipping over
potentially many permutations that are non-topological for the same reason.
With that, increase the checking of all permutations to clusters of size 8
instead of 7.
The separates the existing fuzz test into:
* clusterlin_linearize: establishes the correctness of Linearize() using the
simpler SimpleLinearize() function.
* clusterlin_simple_linearize: establishes the correctness of SimpleLinearize() by
comparing with all valid linearizations computed by
std::next_permutation.
rather than combining the first two into a single fuzz test.
This separates the existing fuzz test into:
* clusterlin_search_finder: establishes SearchCandidateFinder's correctness using the
simpler SimpleCandidateFinder.
* clusterlin_simple_finder: establishes SimpleCandidateFinder's correctness using the
(even) simpler ExhaustiveCandidateFinder.
rather than trying to do both at once.
Only count the number of actual new subsets added. If the queue contains
a work item that completely covers a component, no transaction can be added
to it without creating a disconnected component. In this case, also don't
count it as an iteration.
With this, the number of iterations performed by SimpleCandidateFinder is
bounded by the number of distinct connected topologically-valid subsets of
the cluster.
Additionally this commit gives each test its
own function.
The assert_submitblock helper is absorbed into
assert_template.
Review hint:
git show --color-moved=dimmed-zebra
Comments are expanded.
Return BlockValidationState instead of passing a reference.
Lock Chainman mutex instead of cs_main.
Remove redundant chainparams and pindexPrev arguments.
Drop defaults for checking proof-of-work and merkle root.
The ContextualCheckBlockHeader check is moved to after CheckBlock,
which is more similar to normal validation where context-free checks
are done first.
Validation failure reasons are no longer printed through LogError(),
since it depends on the caller whether this implies an actual bug
in the node, or an externally sourced block that happens to be invalid.
When called from getblocktemplate, via BlockAssembler::CreateNewBlock(),
this method already throws an std::runtime_error if validation fails.
Additionally it moves the inconclusive-not-best-prevblk check from RPC
code to TestBlockValidity.
There is no behavior change when callling getblocktemplate with proposal.
Previously this would return a BIP22ValidationResult which can throw for
state.IsError(). But CheckBlock() and the functions it calls only use
state.IsValid().
The final assert is changed into Assume, with a LogError.
Co-authored-by: <Ryan Ofsky <ryan@ofsky.org>
ce90f0c99f rpc, wallet, refactor: Remove non-descriptor errors (pablomartin4btc)
573bcd75d7 wallet, refactor: Remove unused SetupGeneration (pablomartin4btc)
5431f2dc21 wallet, refactor: Remove Legacy warnings and errors (pablomartin4btc)
Pull request description:
Remove dead code due to legacy wallet support removal.
These changes have no impact on functionality. They are transparent to the end user, as legacy wallets can't be created or loaded anymore, so these checks are no longer reached. The legacy-to-descriptor wallet migration flow is not affected either, as these removals are not part of its process.
ACKs for top commit:
achow101:
ACK ce90f0c99f
rkrux:
utACK ce90f0c99f
Tree-SHA512: 9229ad9dda9ff1dece73b5b15a20d69c6ab1ff2c75b2ec430ddbbaeb3467f6a850f53df527bcb4a8114ccbf1aa9c794462d71a8d516aed6f9a9da74edae16feb
0def84d407 test: Verify parent_desc in RPCs (Ava Chow)
2554cee988 test: Enable default wallet for wallet_descriptor.py (Ava Chow)
3fc9d9f241 wallet, rpc: Push the normalized parent descriptor (Ava Chow)
Pull request description:
Instead of prividing the descriptor string as stored in the db, use the normalized descriptor as is done for getaddressinfo's parent_desc field.
Split from #32489
ACKs for top commit:
Sjors:
re-utACK 0def84d407
rkrux:
ACK 0def84d407
w0xlt:
reACK 0def84d407
Tree-SHA512: 575c5b545d6f0aa7e135696b7a55c004e754fca4dd35dd9cf71b0b45b49a2e86e7b20570e768534d587005953bb893645379ec1ba4f98cfd26811f9c2f17de2d
130a922980 wallet, interfaces: Use BERKELEY_RO in isEncrypted (Ava Chow)
Pull request description:
The GUI uses `WalletLoader::isEncrypted()` to detect whether a wallet file is encrypted so that it knows whether to prompt for a passphrase when migrating a legacy wallet. However, legacy wallets need to be opened with `options.require_format = BERKELEY_RO`. Since this wasn't being provided, following #28710, encrypted legacy wallets could not be migrated.
This fixes the issue by detecting when a wallet file is for a legacy wallet, and re-attempting with `options.require_format = BERKELEY_RO` in that case.
Depends on #32449 for `DatabaseStatus::FAILED_LEGACY_DISABLED`
ACKs for top commit:
davidgumberg:
Tested ACK 130a922980
furszy:
utACK 130a922980
pablomartin4btc:
tACK 130a922980
w0xlt:
Code review ACK 130a922980
rkrux:
utACK 130a922980
Tree-SHA512: aa70defc3b5f41635333a4d83c46ecdb5cd3cb129d590b4c0fe7a5f16e8aeaba1592f932ead242ed5f84524b146d87319154f4a1820bb34d9e80f63d24fc6b20
206bc05e62 test: remove unnecessary m_best_header setting hack in feature_assumeutxo.py (Sebastian Falbesoner)
Pull request description:
The mentioned bug (issue #26245) has been fixed in PR #30666, so the workaround is not needed anymore.
ACKs for top commit:
fjahr:
ACK 206bc05e62
janb84:
ACK 206bc05e62
mzumsande:
utACK 206bc05e62
Tree-SHA512: 92ad6c09c80318a02e00d8c2a4ca1ec8a6caa6aa310916ff705871c91fe5c960c15ed394ead8ce9c7466b592dba176aa87a635b3228ddc93f57082124f04613f
61ea5f348d fuzz: doc: add info about `afl-system-config` for macOS (brunoerg)
Pull request description:
`afl-system-config` adjusts the shared memory segment size limits and configures kernel parameters for better fuzzing performance. Since macOS has more conservative values on shared memory, it's necessary to run `afl-system-config`, or manually adjust the values to fuzz with AFL++.
e.g.:
```sh
kern.sysv.shmmax: 524288000
kern.sysv.shmmin: 1
kern.sysv.shmseg: 48
kern.sysv.shmall: 131072000
```
ACKs for top commit:
janb84:
reACK [61ea5f3](61ea5f348d)
w0xlt:
ACK 61ea5f348d
Crypt-iQ:
ACK 61ea5f3
Tree-SHA512: 8fd9fcb397a6b22e344ac37d3d18052ce8b6b32a5118ab3554c49116d7dd8769cdcbad41f0784d9ab525d4284a82e3d7a89b2c66966b725bf3781d9389d7454d
Seems to have been broken since conversion from Boost in #20744. The std::filesystem iteration aborts upon failure while Boost might have allowed skipping over faulty entries.
d7c37906e7 build: patch cmake min version on freetype (josibake)
Pull request description:
## Problem
While doing a depends build with CMake 4.0.1, I got the following error:
```
Extracting freetype...
/root/bitcoin/depends/sources/freetype-2.11.0.tar.xz: OK
Preprocessing freetype...
Configuring freetype...
CMake Error at CMakeLists.txt:100 (cmake_minimum_required):
Compatibility with CMake < 3.5 has been removed from CMake.
Update the VERSION argument <min> value. Or, use the <min>...<max> syntax
to tell CMake that the project requires at least <min> but has been updated
to work with policies introduced by <max> or earlier.
Or, add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to try configuring anyway.
-- Configuring incomplete, errors occurred!
make: *** [funcs.mk:343: /root/bitcoin/depends/x86_64-pc-linux-gnu/.freetype_stamp_configured] Error 1
make: Leaving directory '/root/bitcoin/depends'
```
.. which led me to https://cmake.org/cmake/help/latest/release/4.0.html#deprecated-and-removed-features, which states compatibility with CMake versions less than 3.5 has been removed in 4.0.
## Fix
Based on the suggestion from the error message (and from reading the CMake docs), I added `-DCMAKE_POLICY_VERSION_MINIMUM=3.22`. I picked `3.22` (as opposed to 3.5) since that is the minimum version of CMake we specify in `doc/dependencies.md`. Would be nice if there was a way to pipe the min version in as a variable (since presumably we'd want to update this to be in lock step with the minimum CMake version of the whole project), but I couldn't think of a simple way to do this. Open to suggestions on a more robust way to do this if this is deemed too brittle.
ACKs for top commit:
fanquake:
ACK d7c37906e7
hebasto:
ACK d7c37906e7. Tested on Ubuntu 25.04 with both cmake 4.0.2 and the default cmake 3.31.6.
Tree-SHA512: fb664ec73bfffc504f1dcc9076072307f443d056d14325de41c4a29f3ee4077f1922e79b5895b49e7354f45ad6a35be4973c153c2baf3376df6c0d209efc9c54
Dropped the default expected_hash parameter from `ReadBlock()`.
In `blockmanager_flush_block_file` tests, we pass {} since the tests would already fail at PoW validation for corrupted blocks.
In `ChainstateManager::LoadExternalBlockFile`, we pass {} when processing child blocks because their hashes aren't known beforehand.
Switch to the index-aware `ReadBlock()` overload in `ComputeFilter` so that filter creation will abort if the stored block header hash doesn't match the expected one.
In the `readwriteblock` benchmark, pass the expected hash to `ReadBlock()` to match the new signature without affecting benchmark performance.
The non-recent-block code path in `ProcessGetBlockData` already has `inv.hash` available (equaling `pindex->GetBlockHash()`).
Pass it to `ReadBlock()` and assert that the on-disk header matches the requested hash.
The `GETBLOCKTXN` message handler in `ProcessMessage` receives `req.blockhash` from the peer (equaling `pindex->GetBlockHash()`).
Pass this hash to `ReadBlock()` for verification and assert that the index lookup matches.
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
The catchup loop in the outbound eviction functional test currently has
a small flaw, as the contained waiting for a `getheaders` message just
waits for any such message instead of one with the intended block hash.
The reason is that the `prev_prev_hash` variable is set incorrectly,
since the `tip_header` instance is not updated and its field `.hash` is
None. Fix that by updating `tip_header` and use the correct field -- we
want the tip header's previous hash (`.hashPrevBlock`).
029ba1a21d index: remove CBlockIndex access from CustomAppend() (furszy)
91b7ab6c69 refactor: index, simplify CopyHeightIndexToHashIndex to process single block (furszy)
6f1392cc42 indexes, refactor: Remove remaining CBlockIndex* uses in index Rewind methods (Ryan Ofsky)
0a248708dc indexes, refactor: Stop requiring CBlockIndex type to call IsBIP30Unspendable (Ryan Ofsky)
331a25cb16 test: indexes, avoid creating threads when sync runs synchronously (furszy)
Pull request description:
Combining common refactors from #24230 and #26966, aiming to move both efforts forward while reducing their size and review burden.
Broadly, #24230 focuses on enabling indexes to run in a separate process, and #26966 aims to parallelize the indexes initial synchronization process. A shared prerequisite for both is ensuring that only the base index class interacts with the node’s chain internals - child index classes should instead operate solely through chain events.
This PR moves disk read lookups from child index classes to the base index class. It also includes a few documentation improvements and a test-only code cleanup.
ACKs for top commit:
maflcko:
review ACK 029ba1a21d👡
achow101:
ACK 029ba1a21d
TheCharlatan:
Re-ACK 029ba1a21d
davidgumberg:
ACK 029ba1a21d
mzumsande:
Code Review ACK 029ba1a21d
Tree-SHA512: f073af407fc86f228cb47a32c7bcf2241551cc89ff32059317eb81d5b86fd5fda35f228d2567e0aedbc9fd6826291f5fee05619db35ba44108421ae04d11e6fb
IsValid() also returns false for blocks that have not been
validated yet up to the default validity level of BLOCK_VALID_TRANSACTIONS but
are not marked as invalid - e.g. if we only know the header.
Here, we specifically want to filter for invalid blocks.
Also removes the default arg from IsValid() which is now unused outside
of tests, to prevent this kind of misuse for the future.
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
when we reconsiderblock, previously only block and it's
descendants were considered as chain tip candidates/inserted into
setBlockIndexCandidates
ex: on this chain, with block 4 invalidated
1 -> 2 -> 3 -> 4 -> 5 -> 6 -> header 7
blocks 4, 5, 6, header 7 have BLOCK_FAILED_* flags set
previously:
- if we reconsiderblock header 7, the chain would have all the
BLOCK_FAILED_* flags cleared but would report chain tip as block 3.
- after restart, it reports correct chain tip block 6.
now:
- if we reconsiderblock header 7, the correct chain tip block 6 is
reported since ancestors are also considered as chain tip
candidates/inserted into setBlockIndexCandidates.
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
When we call reconsiderblock for some block, ResetBlockFailureFlags puts the descendants of that block
into setBlockIndexCandidates (if they meet the criteria, i.e. have more work than the tip etc.)
We also clear the failure flags of the ancestors, but we never put any of those into setBlockIndexCandidates
this is wrong and could lead to failures in CheckBlockIndex().
fa9ca13f35 refactor: Sort includes of touched source files (MarcoFalke)
facb152697 scripted-diff: Bump copyright headers after include changes (MarcoFalke)
fae71d30f7 clang-tidy: Apply modernize-deprecated-headers (MarcoFalke)
Pull request description:
Bitcoin Core is written in C++, so it is confusing to sometimes use the deprecated C headers (with the `.h` extension). For example, it is less clear whether `string.h` refers to the file in this repo or the cstring stdlib header (https://github.com/bitcoin/bitcoin/pull/31308#discussion_r2121492797).
The check is currently disabled for headers, to exclude subtree headers.
ACKs for top commit:
l0rinc:
ACK fa9ca13f35
achow101:
ACK fa9ca13f35
janb84:
ACK fa9ca13f35
stickies-v:
ACK fa9ca13f35
Tree-SHA512: 6639608308c598d612e24435aa519afe92d71b955874b87e527245291fb874b67f3ab95d3a0a5125c6adce5eb41c0d62f6ca488fbbfd60a94f2063d734173f4d
4ef6253017 test: avoid unneeded (w)txid hex -> integer conversions (Sebastian Falbesoner)
472f3770ae scripted-diff: test: rename CTransaction `.getwtxid()` -> `wtxid_hex` for consistency (Sebastian Falbesoner)
81af4334e8 test: rename CTransaction `.sha256` -> `.txid_int` for consistency (Sebastian Falbesoner)
ce83924237 test: rename CTransaction `.rehash()`/`.hash` -> `.txid_hex` for consistency (Sebastian Falbesoner)
e9cdaefb0a test: introduce and use CTransaction `.wtxid_int` property (Sebastian Falbesoner)
9b3dce24a3 test: remove bare CTransaction `.rehash()`/`.calc_sha256()` calls (Sebastian Falbesoner)
a2724e3ea3 test: remove txid caching in CTransaction class (Sebastian Falbesoner)
Pull request description:
In the functional test framework, determining a (w)txid for a `CTransaction` instance is currently rather confusing and footgunny due to inconsistent naming/interfaces (see table below) and statefulness involved. This PR aims to improve that by:
* removing the (w)txid caching mechanism, in order to avoid the need to call additional rehashing functions (`.rehash()`/`.calculate_sha256()`, see first two commits and https://github.com/bitcoin/bitcoin/pull/32050#discussion_r1993286997). This change in theory decreases the performance, as the involved serialization and hashing involved might be called more often than previously, but I couldn't find a functional test where this leads to a measurable run-time increase on my machine.
* introduce consistent naming that shows the type of the returned txid, i.e. hex string vs. test-framework-internal representation [currently integers] (see remaining commits)
Summary table showing (w)txid determaination before/after this PR:
| Task | master | PR |
|:-----------------------|:-----------------------|:-------------|
| get TXID (hex string) | `.rehash()` / `.hash`[1] | `.txid_hex` |
| get TXID (integer) | `.sha256`[1] | `.txid_int` |
| get WTXID (hex string) | `.getwtxid()` | `.wtxid_hex` |
| get WTXID (integer) | `.calc_sha256(True)` | `.wtxid_int` |
Unfortunately, most renames can't be done with a scripted-diff, as the property names (`.hash`, `.sha256`) are also used for blocks and other message types. The PR is rather invasive and touches a lot of files, but I think it's worth to do it, also to make life easier for new contributors. Future tasks like e.g. doing the same overhaul for block (header) objects or getting rid of the integer representation (see https://github.com/bitcoin/bitcoin/pull/32050) become easier should become easier after this one.
[1] = returned value might be out-of-date, if rehashing function wasn't called after modification
ACKs for top commit:
maflcko:
re-ACK 4ef6253017🏈
achow101:
ACK 4ef6253017
marcofleon:
code review ACK 4ef6253017
Tree-SHA512: 4b472c31d169966b6f6878911a8404d25bf3e503b6e8ef30f36a7415d21ad4bc1265083af2d3ead6edfcd9fac9ccb0a8be57e1b0739ad431b836413070d7d583
f6b782f3aa doc: Improve m_best_header documentation (Martin Zumsande)
ee673b9aa0 validation: remove m_failed_blocks (Martin Zumsande)
ed764ea2b4 validation: Add more checks to CheckBlockIndex() (Martin Zumsande)
9a70883002 validation: in invalidateblock, calculate m_best_header right away (Martin Zumsande)
8e39f2d20d validation: in invalidateblock, mark children as invalid right away (Martin Zumsande)
4c29326183 validation: cache all headers with enough PoW in invalidateblock (Martin Zumsande)
15fa5b5a90 validation: call InvalidBlockFound also from AcceptBlock (Martin Zumsande)
Pull request description:
Some fields in validation are set opportunistically by "best effort":
- The `BLOCK_FAILED_CHILD` status (which means that the block index has an invalid predecessor)
- `m_best_header` (the most-work header not known to be invalid).
This means that there are known situations in which these fields are not set when they should be, or set to wrong values. This is tolerated because the fields are not used for anything consensus-critical and triggering these situations involved creating invalid blocks with valid PoW header, so would have a cost attached. Also, having stricter guarantees for these fields requires iterating over the entire block index, which has some DoS potential, especially with any header above the checkpoint being accepted int he past (see e.g. #11531).
However, there are reasons to change this now:
- RPCs use these fields and can report wrong results
- There is the constant possibility that someone could add code that expects these fields to be correct, especially because it is not well documented that these fields cannot always be relied upon.
- DoS concerns have become less of an issue after #25717 - now an attacker would need to invest much more work because they can't fork off the last checkpoint anymore
This PR continues the work from #30666 to ensure that `BLOCK_FAILED_CHILD` status and `m_best_header` are always correct:
- it adds a call to `InvalidChainFound()` in `AcceptBlock()`.
- it adds checks for `BLOCK_FAILED_CHILD` and `m_best_header` to `CheckBlockIndex()`. In order to be able to do this, the existing cache in the RPC-only `InvalidateBlock()` is adjusted to handle these as well. These are performance optimizations with the goal of avoiding having a call of `InvalidChainFound()` / looping over the block index after each disconnected block.
I also wrote a fuzz test to find possible edge cases violating `CheckBlockIndex`, which I will PR separately soon.
- it removes the `m_failed_blocks` set, which was a heuristic necessary when we couldn't be sure if a given block index had an invalid predecessor or not. Now that we have that guarantee, the set is no longer needed.
ACKs for top commit:
stickies-v:
re-ACK f6b782f3aa
achow101:
reACK f6b782f3aa
ryanofsky:
Code review ACK f6b782f3aa with only minor code & comment updates
TheCharlatan:
Re-ACK f6b782f3aa
Tree-SHA512: 1bee324216eeee6af401abdb683abd098b18212833f9600dbc0a46244e634cb0e6f2a320c937a5675a12af7ec4a7d10fabc1db9e9bc0d9d0712e6e6ca72d084f
secp256k1 provides us secp256k1_musig_keyagg_cache objects which we are
used as part of session info and to get the aggregate pubkey. These
helper functions help us convert to/from the secp256k1 C objects into
the Bitcoin Core C++ objects.
When splitting a string, sometimes the separator needs to be included.
Split will now optionally include the separator at the end of the left
side of the splits, i.e. it appears at the end of the splits, except
for the last one.
Specifically, for musig() descriptors, Split is used to separate a
musig() from any derivation path that follows it by splitting on the
closing parentheses. Since that parentheses is needed for Func() and
Expr(), Split() needs to preserve the end parentheses instead of
discarding it.
When parsing a descriptor, it is useful to be able to check whether a
string begins with a substring without consuming that substring as
another function such as Func() will be used later which requires that
substring to be present at the beginning.
Specifically, for MuSig2, this modified Const will be used to determine
whether a an expression begins with "musig(" before a subsequent
Func("musig", ...) is used.
fa0b766f43 test: Remove intermittent and presumed fixed tsan race suppressions (MarcoFalke)
fa4b659dcd test: Explain how to reproduce zmq:: upstream race (MarcoFalke)
Pull request description:
An explanation makes it easier to reproduce, if needed.
ACKs for top commit:
fanquake:
ACK fa0b766f43
Tree-SHA512: 4857cc1e2c97e3d8c194fd12d0bb2a3293136c51ae1b89e0320161d1b8f22ef5122519e099288e52e42bb828ee4a56bfdfbe80717d95178748b76dd7209e12db
8713e8060d depends: fix SHA256SUM command on OpenBSD (use GNU mode output) (Sebastian Falbesoner)
2d938720bd depends: add patch to fix capnp build on OpenBSD (Sebastian Falbesoner)
Pull request description:
This PR fixes the multiprocess depends build for OpenBSD by applying upstream patch https://github.com/capnproto/capnproto/pull/2308 and switching the SHA256SUM command to output hash sums in the expected format (the default is BSD format [1], but we need GNU format [2], see commit message for details). Note that the hashing issue is only prevailing for packages defining the `$(package)_local_dir` variable (introduced in 5d105fb8c3, part of #31741), where the following line of the `fetch_local_dir_sha256` function leads to the wrong output:
ae024137bd/depends/funcs.mk (L57)
The first commit can be replaced with a simple capnp version bump once this is available in a release.
Tested on OpenBSD 7.7 (x86_64) via
```
$ gmake -C depends MULTIPROCESS=1 NO_BOOST=1 NO_LIBEVENT=1 NO_QT=1 NO_QR=1 NO_WALLET=1 NO_ZMQ=1 NO_USDT=1
```
[1] example output: `SHA256 (/home/thestack/.vimrc) = 6ba69d100e8c5ca0488ded6293d4e5f740a6a5d5ace96cbcf0599c18d27389e4`
[2] example output: `6ba69d100e8c5ca0488ded6293d4e5f740a6a5d5ace96cbcf0599c18d27389e4 /home/thestack/.vimrc`
ACKs for top commit:
Sjors:
ACK 8713e8060d
hebasto:
ACK 8713e8060d.
fanquake:
ACK 8713e8060d
Tree-SHA512: 178b8b41156e1f1eea101849110167d2636c3093b6a68c88a91a994f0750831aa02e415eb2793c522682c92cb3085de025300e0e2dee894e112dd7e1f495cc08
Both are rational numbers. Client software should only use them to
display information to humans. Followup calculations should use the
underlying values such as target.
Therefore it's not necessary to test the handling of these floating
point values. Round them down to avoid spurious test failures.
Fixes#32515
239fc4d62e doc, windows: CompanyName "Bitcoin" => "Bitcoin Core project" (Hodlinator)
Pull request description:
Brings Windows executables in line with */share/setup.nsi.in:14* used by the installer.
Discovered while reviewing tangential PR: https://github.com/bitcoin/bitcoin/pull/32634#discussion_r2112641918
ACKs for top commit:
maflcko:
lgtm ACK 239fc4d62e
Sjors:
utACK 239fc4d62e
janb84:
utACK 239fc4d62e
hebasto:
ACK 239fc4d62e.
Tree-SHA512: 5855e78c32e15a1e4e9b1a6bdefd29c45676a64b3eb4470cb98fa0eea02701edadbde7153143757b525e9a66eb3b49bbba926e8e322307ae6ea4a44ac23eeffb
Moved CBlockUndo disk read lookups from child index classes to
the base index class.
The goal is for child index classes to synchronize only through
events, without directly accessing the chain database.
This change will enable future parallel synchronization mechanisms,
reduce database access (when batched), and contribute toward the
goal of running indexes in a separate process (with no chain
database access).
Besides that, this commit also documents how NextSyncBlock() behaves.
It is not immediately clear this function could return the first
block after the fork point during a reorg.
Some of the primary changes are:
- lief.EXE_FORMATS became lief.Binary.FORMATS IN 0.14.0
- 494f116c6b/doc/sphinx/changelog.rst (L702)
- lief.ARCHITECTURES became lief.Header.ARCHITECTURES in 0.16.0
- 494f116c6b/doc/sphinx/changelog.rst (L226C18-L227C18)
- lief.ELF.ARCH.x86_64 became lief.ELF.ARCH.X86_64
This commit includes a workaround for the bug fixed in
https://github.com/lief-project/LIEF/pull/1218, but the workaround can
be kept, since it makes `has_nx` checks stricter by enforcing both heap
and stack are non-executable.
This change also requires a patch to partially revert a commit to LIEF
(f23ced2f4f)
which broke compatibility with versions of scikit-build-core <= 0.10.x.
This patch can be dropped once the guix time machine advances to or
beyond 35c5f07e96,
which bumps the scikit-build-core version in guix from 0.9.3 to 0.10.7.
Co-authored-by: willcl-ark <will8clark@gmail.com>
Co-authored-by: fanquake <fanquake@gmail.com>
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
Rather than determining a CTransaction's (w)txid as an integer by
converting it's hex value, it can be directly accessed via the
introduced `.{w,}txid_int` property.
It is not at all obvious that two transactions with differing witness data
should test equal to each other.
There was only a single instance of a caller relying on this behavior, and that
one appears accidental (left-over from before segwit). That caller (in the
wallet) has been fixed.
Change the definition of transaction equality (and inequality) to use the wtxid
instead.
Also explicitly check for txid equality rather than transaction equality as the
former is a tighter constraint if witness data is included when comparing the
full transactions.
Co-authored-by: glozow <gloriajzhao@gmail.com>
e98c51fcce doc: update tor.md to mention the new -proxy=addr:port=tor (Vasil Dimov)
ca5781e23a config: allow setting -proxy per network (Vasil Dimov)
Pull request description:
`-proxy=addr:port` specifies the proxy for all networks (except I2P). Previously only the Tor proxy could have been specified separately via `-onion=addr:port`.
Make it possible to specify separately the proxy for IPv4, IPv6, Tor and CJDNS by e.g. `-proxy=addr:port=ipv6`. Or remove the proxy for a given network, e.g. `-proxy=0=cjdns`.
Resolves: https://github.com/bitcoin/bitcoin/issues/24450
ACKs for top commit:
pinheadmz:
ACK e98c51fcce
caesrcd:
reACK e98c51fcce
danielabrozzoni:
Code Review ACK e98c51fcce
1440000bytes:
ACK e98c51fcce
Tree-SHA512: 0cb590cb72b9393cc36357e8bd7861514ec4c5bc044a154e59601420b1fd6240f336ab538ed138bc769fca3d17e03725d56de382666420dc0787895d5bfec131
fac00d4ed3 doc: Move CI-must-pass requirement into readme section (MarcoFalke)
fab79c1a25 doc: Clarify and move "hygienic commit" note (MarcoFalke)
fac8b05197 doc: Clarify strprintf size specifier note (MarcoFalke)
faaf34ad72 doc: Remove section about RPC alias via function pointer (MarcoFalke)
2222d61e1c doc: Remove section about RPC arg names in table (MarcoFalke)
fa00b8c02c doc: Remove section about include guards (MarcoFalke)
fad6cd739b doc: Remove dev note section on includes (MarcoFalke)
fa6623d85a doc: Remove file name section (MarcoFalke)
7777fb8bc7 doc: Remove shebang section (MarcoFalke)
faf65f0531 doc: Remove .gitignore section (MarcoFalke)
faf2094f25 doc: Remove note about removed ParsePrechecks (MarcoFalke)
fa69c5b170 doc: Remove -disablewallet from dev notes (MarcoFalke)
Pull request description:
This removes sections that I've been collecting as stale or overly redundant over the years. The rationale for each removal is in the commit message.
ACKs for top commit:
yuvicc:
ACK fac00d4ed3
janb84:
LGTM ACK fac00d4ed3
glozow:
ACK fac00d4ed3, all lgtm
Tree-SHA512: 17a5b4277fb30d265959d1230a705b36d8501a64c0f4a7f272ea5d9c22031421f95c491144f6d6f714dc7927df667d96ece9ceb43e0a07317d76fdcc4769aaa7
It is not possible to load a legacy/ non-descriptor wallet anymore
so no need to check for WALLET_FLAG_DESCRIPTORS in RPC calls, even when
passing -rpcwallet/ JSON `/wallet/<walletname>/` endpoint, that searches
for the wallets loaded already in the context.
SetupGeneration was supposed to be the function that all SPKMs used
to setup automatic generation, but it didn't work out that way and
ended up being legacy only. It should be deleted at this point.
Move ReadBlock code from CoinStatsIndex::CustomRewind to BaseIndex::Rewind
Move ReadUndo code from CoinStatsIndex::ReverseBlock to BaseIndex::Rewind
This commit does change behavior slightly. Since the new CustomRemove
methods only take a single block at a time instead of a range of
disconnected blocks, when they call CopyHeightIndexToHashIndex they will
now do an index seek for each removed block instead of only seeking once
to the height of the earliest removed block. Seeking instead of scanning
is a little worse for performance if there is a >1 block reorg, but
probably not noticeable unless the reorg is very deep.
32d4e92b9a doc: make `-DWITH_ZMQ=ON` explicit on `build-unix.md` (Luis Schwab)
Pull request description:
ZMQ support is not built by default on Linux, and the docs don't make that clear. This PR makes it explicit that the `-DWITH_ZMQ=ON` flag is required to build with ZMQ support on `build-unix.md`.
ACKs for top commit:
maflcko:
lgtm ACK 32d4e92b9a
Tree-SHA512: 322d0dd86bb80aa5a5640a5510cbeeec29f490c33b8f7360e3a202147a02c303064e6761ceb42e38e26982c61f35c9b048804c705a0d95c5737ebd2109febead
5c4a0f8009 guix: warn and abort when SOURCE_DATE_EPOCH is set (will)
Pull request description:
Fixes: #29935
Current behaviour will by-default use SOURCE_DATE_EPOCH from the environment without warning. This breaks the default reproducibility from a guix build.
Warn when and exit when this variable is set, and
FORCE_SOURCE_DATE_EPOCH is unset.
ACKs for top commit:
maflcko:
lgtm ACK 5c4a0f8009
fanquake:
ACK 5c4a0f8009
Tree-SHA512: fdd6095a91bd87ffdc22918dc43869edc2380501d1b047e95caadd8a6624928691bfe5b7af9693177cbc28e69366e3397e43a06f2f346cc3a9fe233b7fb9588f
a39b7071cf doc: fuzz: fix AFL++ link (brunoerg)
Pull request description:
Fix link about selecting the best AFL compiler.
ACKs for top commit:
maflcko:
lgtm ACK a39b7071cf
Tree-SHA512: 6366f18767f6c60f806faa374bd4a6a3bb71a1c74b7040867177ca9ded946707a75b4f39bf698530ba9c5ff708394d88f5b29f4a92f9e95aaca7b37c62b72093
Since #29412, we have not allowed mutated blocks to continue
being processed immediately the block is received, but this
is only done for the legacy BLOCK message.
Extend these checks as belt-and-suspenders to not allow
similar mutation strategies to affect relay by honest peers
by applying the check inside
PartiallyDownloadedBlock::FillBlock, immediately before
returning READ_STATUS_OK.
This also removes the extraneous CheckBlock call.
When building depends on FreeBSD/OpenBSD `aarch64`, the host compilers
default to `default_host_{CC,CXX}`, which resolves to `gcc`/`g++`. This
is incorrect on these systems, where Clang is the default system
compiler.
b44514b876 rpc, doc: update `listdescriptors` RCP help (rkrux)
Pull request description:
This RPC lists all the descriptors present in the wallet, not only the ones that were imported, but also the ones generated when a new wallet is created.
It can be verified by creating a new wallet and calling the `listdescriptors` RPC, which will contain 8 ranged descriptors that are created for every new wallet.
Also, update the description to get rid of "descriptor-enabled" because this is the only wallet type available now after removal of legacy wallets.
ACKs for top commit:
maflcko:
lgtm ACK b44514b876
achow101:
ACK b44514b876
pablomartin4btc:
ACK b44514b876
theStack:
ACK b44514b876
Tree-SHA512: d1018dd42fc4de12793f3e4f3be79ecb3fdee46fbc93ec8adb62b29a86e74aba2605d9908632107061f48ef8ee6f39ef6d0e34cc5e91acd93bc02242a2cee3eb
f16c8c67bf tests: Expand HTTP coverage to assert libevent behavior (Matthew Zipkin)
Pull request description:
These commits are cherry-picked from #32061 and part of a project to [remove libevent](https://github.com/bitcoin/bitcoin/issues/31194).
This PR only adds functional tests to `interface_http` to cover some HTTP server behaviors we inherit from libevent, in order to maintain those behaviors when we replace libevent with our own HTTP server.
1. Pipelining: The server must respond to requests from a client in the order in which they were received [RFC 7230 6.3.2](https://www.rfc-editor.org/rfc/rfc7230#section-6.3.2)
2. `-rpcservertimeout` config option which sets the amount of time the server will keep an idle client connection alive
3. "Chunked" Transfer-Encoding: Allows a client to send a request in pieces, without the `Content-Length` header [RFC 7230 4.1](https://www.rfc-editor.org/rfc/rfc7230#section-4.1)
ACKs for top commit:
achow101:
ACK f16c8c67bf
vasild:
ACK f16c8c67bf
polespinasa:
ACK f16c8c67bf
fjahr:
utACK f16c8c67bf
Tree-SHA512: 405b59431b4d2bf118fde04b270865dee06ef980ab120d9cc1dce28e5d65dfd880a57055b407009d22f4de614bc3eebdb3e203bcd39e86cb14fbfd62195ed06a
Since the previous commit, CTransaction object calls to the
methods `.rehash()` and `.calc_sha256()` are effectively no-ops
if the returned value is not used, so we can just remove them.
Rather than txids (represented by the fields `.sha256` and `.hash`)
being stateful, simply compute them on-the-fly. This ensures that
the correct values are always returned and takes the burden of
rehashing from test writers, making the code shorter overall.
In a first step, the fields are kept at the same name with @property
functions as drop-in replacements, for a minimal diff. In later commits,
the names are changed to be more descriptive and indicating the return
type of the txid.
LoadWallet was added in commit d77170d526, which
points to a traceback with BerkeleyBatch in it. Now that BDB is removed,
this can be removed as well.
The race in DatabaseBatch was added in commit
fd59670642, which does not point to a
traceback. This was likely also fixed with the BDB removal.
If not, the suppressions should be added back, mentioning that they are
intermittent and including a traceback and possibly steps to reproduce.
a189d63618 add release note for datacarriersize default change (Greg Sanders)
a141e1bf50 Add more OP_RETURN mempool acceptance functional tests (Peter Todd)
0b4048c733 datacarrier: deprecate startup arguments for future removal (Greg Sanders)
63091b79e7 test: remove unnecessary -datacarriersize args from tests (Greg Sanders)
9f36962b07 policy: uncap datacarrier by default (Greg Sanders)
Pull request description:
Retains the `-datacarrier*` args, marks them as deprecated, and does not require another startup argument for multiple OP_RETURN outputs.
If a user has set `-datacarriersize` the value is "budgeted" across all seen OP_RETURN output scriptPubKeys. In other words the total script bytes stays the same, but can be spread across any number of outputs. This is done to not introduce an additional argument to support multiple outputs.
I do not advise people use the option with custom arguments and it is marked as deprecated to not mislead as a promise to offer it forever. The argument itself can be removed in some future release to clean up the code and minimize footguns for users.
ACKs for top commit:
stickies-v:
re-ACK a189d63618
Sjors:
re-ACK a189d63618
polespinasa:
re-ACK a189d63618
hodlinator:
re-ACK a189d63618
ajtowns:
reACK a189d63618
mzumsande:
re-ACK a189d63618
petertodd:
ACK a189d63618
theStack:
re-ACK a189d63618
1440000bytes:
re-ACK a189d63618
willcl-ark:
ACK a189d63618
dergoegge:
ACK a189d63618
fanquake:
ACK a189d63618
murchandamus:
ACK a189d63618
darosior:
Concept ACK a189d63618.
Tree-SHA512: 3da2f1ef2f50884d4da7e50df2121bf175cb826edaa14ba7c3068a6d5b2a70beb426edc55d50338ee1d9686b9f74fdf9e10d30fb26a023a718dd82fa1e77b038
This RPC lists all the descriptors present in the wallet, not only
the ones that were imported, but also the ones generated when a
new wallet is created.
It can be verified by creating a new wallet and calling the
`listdescriptors` RPC, which will contain 8 ranged descriptors that
are created for every new wallet.
Also, update the description to get rid of "descriptor-enabled"
because this is the only wallet type available now after removal of
legacy wallets.
12ff4be9c7 test: ensure -rpcallowip is compatible with RFC4193 (Matthew Zipkin)
c02bd3c187 config: Explain RFC4193 and CJDNS interaction in help and init error (Matthew Zipkin)
f728b6b111 init: Configure reachable networks before we start the RPC server (Matthew Zipkin)
Pull request description:
Closes https://github.com/bitcoin/bitcoin/issues/32433
`MaybeFlipIPv6toCJDNS()` relies on `g_reachable_nets` to distinguish between CJDNS addresses and other IPv6 addresses. In particular, [RFC4193](https://www.rfc-editor.org/rfc/rfc4193#section-3.1) address or "Unique Local Address" with the L-bit unset also begins with the `fc` prefix. #32433 highlights a use case for these addresses that have nothing to do with CJDNS.
On master we don't parse init flags like `-cjdnsreachable` until *after* the HTTP server has started, causing conflicts with `-rpcallowip` because CJDNS doesn't support subnets.
This PR ensures that `NET_CJDNS` is only present in the reachable networks list if set by `-cjdnsreachable` *before* `-rpcallowip` is checked. If it is set all `fc` addresses are assumed to be CJDNS, can not have subnets, and can't be set for `-rpcallowip`.
I also noted this specific parameter interaction in the init help as well as the error message if configured incorrectly.
This can be tested locally:
`bitcoind -regtest -rpcallowip=fc00:dead:beef::/64 -rpcuser=u -rpcpassword=p`
On master this will just throw an error that doesn't even mention IPv6 at all.
On the branch, this will succeed and can be tested by adding the ULA to a local interface.
On linux: `sudo ip -6 addr add fc00:dead:beef::1/64 dev lo`
On macos: `sudo ifconfig lo0 inet6 fc00:dead:beef::1/128 add`
then: `curl -v -g -6 --interface fc00:dead:beef::1 u:p@[::1]:18443 --data '{"method":"getblockcount"}'`
If the `rpcallowip` option is removed, the RPC request will fail to authorize.
Finally, adding `-cjdnsreachable` to the start up command will throw an error and specify the incompatibility:
> RFC4193 is allowed only if -cjdnsreachable=0.
ACKs for top commit:
achow101:
ACK 12ff4be9c7
tapcrafter:
tACK 12ff4be9c7
ryanofsky:
Code review ACK 12ff4be9c7
willcl-ark:
ACK 12ff4be9c7
Tree-SHA512: a4dd70ca2bb9f6ec2c0a9463fd73985d1ed80552c674a9067ac9a86662d1c018cc275ba757cebb2993c5f3971ecf4778b95d35fe7a7178fb41b1d18b601c9960
7cfbb8575e test: wallet: cover wallet passphrase with a null char (brunoerg)
Pull request description:
This PR adds test coverage for the `walletpassphrase`/`walletpassphrasechange` RPC when the passphrase is incorrect due to a null character.
For reference: https://github.com/bitcoin/bitcoin/pull/27068 introduced the usage of `SecureString` to allow null characters.
ACKs for top commit:
maflcko:
lgtm ACK 7cfbb8575e
achow101:
ACK 7cfbb8575e
w0xlt:
Code review ACK 7cfbb8575e
BrandonOdiwuor:
Code Review ACK 7cfbb8575e
theStack:
ACK 7cfbb8575e
pablomartin4btc:
cr ACK 7cfbb8575e
Tree-SHA512: ecdb48662ceb6c55c4b301ca7f537c3159ece7b66ee40ea977583ffb74bd3d06e334ab3a5639a9cde3aa6443129f412f9aea0ee5a8b73b31dba0728d0890b7f1
The indexes test call StartBackgroundSync(), which spawns a thread to run Sync(),
only for the test thread to wait for it to complete by calling IndexWaitSynced().
So, since the sync is performed synchronously, we can skip the extra thread creation
entirely and call Sync() directly.
After changes in previous commits, we now mark all blocks that descend from an invalid block
immediately as the block is found invalid. This happens both in the AcceptBlock
and ConnectBlock stages of block validation.
As a result, the pindexPrev->nStatus check in AcceptBlockHeader is now sufficient to detect
invalid blocks and checking m_failed_blocks there is no longer necessary.
This adds checks that
1) Descendants of invalid block indexes are also marked invalid
2) m_best_header cannot be invalid, and there can be no valid
block with more work than it.
Before, m_best_header would be calculated only after disconnecting
multiple blocks, letting go of cs_main in the meantime.
This is in preparation for adding checks to CheckBlockIndex()
requiring that m_best_header is the most-work header not known to be invalid.
Co-authored-by: stringintech <stringintech@gmail.com>
Before, they would be marked as invalid only after disconnecting
multiple blocks, letting go of cs_main in the meantime.
This is in preparation for adding a check to
CheckBlockIndex() requiring that descendants of invalid block indexes
are always marked as invalid.
Entries from highpow_outofchain_headers are now only removed if made invalid,
no longer after inserting into setBlockIndexCandidates, because they
might still become invalid later in the second case.
This means that blocks could be inserted multiple times now into
setBlockIndexCandidates - this won't have any effect, so
behavior isn't changed.
We now include blocks without HaveNumChainTxs() / lower validation status
than BLOCK_VALID_TRANSACTIONS. These checks are still performed at the
spot where we use the cache to insert into setBlockIndexCandidates.
This is in preparation for using the cache for more things than
just setBlockIndexCandidates candidates in the following commits.
Co-authored-by: stickies-v <stickies-v@protonmail.com>
When a block it found invalid during acceptance (but before connection)
we now mark its descendants with BLOCK_FAILED_CHILD and update
m_best_header when these things weren't done reliably before.
This does not fix a serious bug because the flags and m_best_header were being set on a best-effort basis before
and not used for anything critical.
Setting these reliably has a slight performance cost (iterating over the
entire block index) but leads to more consistency in validation and allows removing m_failed_blocks in a later commit.
This can only be triggered by providing a block with sufficient PoW
that is otherwise invalid, so it is not a DoS vector.
On OpenBSD, the `sha256` command by default outputs hashsums on files in
"BSD" mode, looking like this:
$ sha256 ~/.vimrc
SHA256 (/home/thestack/.vimrc) = 6ba69d100e8c5ca0488ded6293d4e5f740a6a5d5ace96cbcf0599c18d27389e4
This is not compatible with our depends commands, which expect the
hashes to be on the first column (to be extracted via `cut -d" " -f1`).
Fix this by switching to GNU mode output, looking like this:
$ sha256 -r ~/.vimrc
6ba69d100e8c5ca0488ded6293d4e5f740a6a5d5ace96cbcf0599c18d27389e4 /home/thestack/.vimrc
Without this change, the multiprocess depends build fails with the following output:
$ gmake -C depends MULTIPROCESS=1 NO_BOOST=1 NO_LIBEVENT=1 NO_QT=1 NO_QR=1 NO_WALLET=1 NO_ZMQ=1 NO_USDT=1
[ ..... ]
Extracting native_libmultiprocess...
sha256: /home/thestack/bitcoin/depends/work/build/x86_64-unknown-openbsd7.7/native_libmultiprocess/-2bc902f4693/.src-ipc-libmultiprocess.tar.hash: no properly formatted checksum lines found
gmake: *** [funcs.mk:342: /home/thestack/bitcoin/depends/work/build/x86_64-unknown-openbsd7.7/native_libmultiprocess/-2bc902f4693/.stamp_extracted] Error 1
Instead of throwing a runtime error, let the caller decide how to handle a missing signer.
GUI code was already in place to handle this, but it was unused until this commit.
Fixes#32426
Additionally use LogWarning instead of std::cerr.
9653ebc053 depends: remove support for Windows Qt LTO builds (fanquake)
Pull request description:
The related Windows patches were dropped in 5e794e6202, and "Cross-compiling does not support LTO." (from #30997).
ACKs for top commit:
maflcko:
lgtm ACK 9653ebc053
Tree-SHA512: 40ae7b17669bf87f2e848055e85e1a6c946f0bb0bc1674e18f1622ec4a0613fe955a4daf83928c9375035dac289ce2a72dd7f347b15f86d108157f9da9499945
4ce53495e5 doc: update tor docs to use bitcoind binary from path (ismaelsadeeq)
Pull request description:
I noticed this while trying to run a node over Tor.
Using `./bitcoind` as the executable path is incorrect.
This is a simple documentation update PR that fixes the path by removing the prefix and just
having `bitcoind` as the usage example targeting those who have Bitcoin Core in their PATH.
ACKs for top commit:
davidgumberg:
ACK 4ce53495e5
janb84:
ACK 4ce53495e5
jonatack:
ACK 4ce53495e5
Tree-SHA512: a23c94a175f77d66ee1a81599a15a809ad768090eebb619c8e4a67b8a020a2256da4f40cec3c00ec35775b265d3c53cdb70c09fbed48d399416fbc9156ebff31
cfc42ae5b7 fuzz: add a target for the coins database (Antoine Poinsot)
46e14630f7 fuzz: move the coins_view target's body into a standalone function (Antoine Poinsot)
56d878c465 fuzz: avoid underflow in coins_view target (Antoine Poinsot)
Pull request description:
This reopens https://github.com/bitcoin/bitcoin/pull/28216.
The current `coins_view` target only tests `CCoinsViewCache` using a basic `CCoinsView` instance. The addition of the `coins_view_db` target enables testing with an actual `CCoinsViewDB` as the backend.
ACKs for top commit:
maflcko:
lgtm ACK cfc42ae5b7
l0rinc:
code review ACK cfc42ae5b7
TheCharlatan:
ACK cfc42ae5b7
Tree-SHA512: d3a92f122629f075767453a1abd9819a1c9716db53b997418993fef62d27683324740d0a8f84df76d8a7a45e508ccadeb69553b6f69e29a1238cd7c0be5276ca
ed179e0a65 test: apply microsecond precision to test framework logging (Martin Zumsande)
Pull request description:
When analyzing functional test logs (produced with `combine_logs.py`), entries sometimes sort slightly out of order because even though python prints 6 digits for microsecond precision, it fills up the last 3 digits with zeroes. For example, it may look like a message was received by the test framework before it was sent by the node.
Change this to actually use microsecond precision - this should make combined logs a little bit easier to analyze.
ACKs for top commit:
davidgumberg:
Tested ACK ed179e0a65
achow101:
ACK ed179e0a65
maflcko:
review ACK ed179e0a65 🗳
janb84:
ACK ed179e0a65
Tree-SHA512: 55cdb5024e8e910c5a5ce741ce512eb88f4f82f11f378ba0fe7a5a2b1c97d2e7b540bdf5603c76aab837d35798610b165f087fbeb7c9dc90aaad890bf4d0323d
f98e1aaf34 rpc: Note in fundrawtransaction doc, fee rate is for package (benthecarman)
Pull request description:
Accidentally made some transactions with a much higher fee rate than I wanted because I did not know this would do it for the package rather than the individual tx.
ACKs for top commit:
achow101:
ACK f98e1aaf34
rkrux:
re-ACK f98e1aaf34
danielabrozzoni:
ACK f98e1aaf34
Tree-SHA512: 9f961de1200803ec4d1c6901fd606bb6cf707ffd03942d9dc0d4b6554c827075f99d693b93e892f728679d67e63e12c71da4426dab091b3311d1605bc37251a2
Current behaviour will by-default use SOURCE_DATE_EPOCH from the
environment without warning. This breaks the default reproducibility
from a guix build.
Warn when and exit when this variable is set, and
FORCE_SOURCE_DATE_EPOCH is unset.
83bfe1485c build: add -Wthread-safety-pointer (fanquake)
240a4fb95d Squashed 'src/leveldb/' changes from 113db4962b..aba469ad6a (fanquake)
Pull request description:
This will become available in Clang 21:
> ThreadSafetyAnalysis now supports -Wthread-safety-pointer, which
> enables warning on passing or returning pointers to guarded variables
> as function arguments or return value respectively. Note that
> ThreadSafetyAnalysis still does not perform alias analysis. The
> feature will be default-enabled with -Wthread-safety in a future release.
See https://github.com/llvm/llvm-project/blob/main/clang/docs/ReleaseNotes.rst.
Also updates the leveldb subtree to pull: https://github.com/bitcoin-core/leveldb-subtree/pull/54.
ACKs for top commit:
davidgumberg:
Tested ACK 83bfe1485c
maflcko:
lgtm ACK 83bfe1485c
theuni:
utACK 83bfe1485c
Tree-SHA512: 9bc80bd04a9cebed8aca20bc23a17e52a6a89a1fb042993322f43dbf7bd93de509c091ebb69255063833b098ab11a64285eccf61e17b9f94f974c734a20ad8da
df9ebbf659 depends: use "mkdir -p" when installing xproto (fanquake)
Pull request description:
It looks like the mkdir detection in xproto is broken on Alpine. Ensure we always use `mkdir -p`.
Fixes#32494.
ACKs for top commit:
hebasto:
ACK df9ebbf659, I have reviewed the code and it looks OK.
janb84:
ACK df9ebbf659
willcl-ark:
ACK df9ebbf659
Tree-SHA512: 0f23b1096ffdf5ffa13115665dc42b65835b78bb0ab04a8be8f210980356953ab518e1273302fe4c9239361201f4f9ac737c0ebf10625f4817f81b65b3b25572
Historically, the headers have been bumped some time after a file has
been touched. Do it now to avoid having to touch them again in the
future for that reason.
-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended 's;( 20[0-2][0-9])(-20[0-2][0-9])? The Bitcoin Core developers;\1-present The Bitcoin Core developers;g' $( git show --pretty="" --name-only HEAD~0 )
-END VERIFY SCRIPT-
This can be reproduced according to the developer notes with something
like
( cd ./src/ && ../contrib/devtools/run-clang-tidy.py -p ../bld-cmake -fix -j $(nproc) )
Also, the header related changes were done manually.
e50312eab0 doc: fix typos (fanquake)
c797e50dda ci: update codespell to 2.4.1 (fanquake)
21ee656337 doc: Remove obselete link in notificator.cpp (strmfos)
ee4406c04a doc: update URLs (fanquake)
Pull request description:
A round up of #32629 + some other changes that had previously been PR'd.
ACKs for top commit:
maflcko:
review ACK e50312eab0🥗
Tree-SHA512: 8fa3e14fdfa0cf65a42debc9cbb1f8b379aba44aa185e2e27337431e884d169bf1e811655c3a884d918e65ea28c5767ddaabaf25c862ebd9b4b38a0229ec5a93
6ee32aaaca test: signet tool genpsbt and solvepsbt commands (Sjors Provoost)
0a99d99fe4 signet: miner skips PSBT step for OP_TRUE (Sjors Provoost)
cdfb70e5a6 signet: split decode_psbt miner helper (Sjors Provoost)
Pull request description:
[BIP325](https://github.com/bitcoin/bips/blob/master/bip-0325.mediawiki) mentions the following rule:
> In the special case where an empty solution is valid (ie scriptSig and scriptWitness are both empty) this additional commitment can optionally be left out. This special case is to allow non-signet-aware block generation code to be used to test a custom signet chain where the challenge is trivially true.
Such a signet can be created using e.g. `-signetchallenge=51` (`OP_TRUE`). However `contrib/signet/miner` won't omit the commitment.
This PR improves the miner by skipping the PSBT for known trivial scripts (just `OP_TRUE` and trivial pushes for now). This prevents it from appending the 4 byte signet header to the witness commitment, as allowed by the above rule.
---
Previously the script would fail with `PSBT signing failed`, making it difficult to mine. This is no longer the case.
ACKs for top commit:
achow101:
ACK 6ee32aaaca
theStack:
re-ACK 6ee32aaaca
danielabrozzoni:
ACK 6ee32aaaca
Tree-SHA512: e47fbf471f2909286a6c1c073799ea388b9c19551afcce96cf9af45cc48d25c02f1e48e08861a88b604361e2c107a759d5baf393da8a37360de419f31651758a
86e1111239 test: verify node skips loading legacy wallets during startup (furszy)
9f94de5bb5 wallet: init, don't error out when loading legacy wallets (furszy)
Pull request description:
Instead of failing during initialization and shutting down the app when encountering a legacy wallet, skip loading the wallet and notify the user accordingly.
This allows users to access migration functionalities without needing to manually remove the wallet from settings.json or resort to using the bitcoin-wallet utility.
This means that GUI users will be able to use the migration button, and bitcoin-cli users will be able to call the migratewallet RPC directly after init.
ACKs for top commit:
achow101:
ACK 86e1111239
w0xlt:
ACK 86e1111239
Tree-SHA512: 85d594a503ee7a833a23754b71b6ba4869ca34ed802c9ac0cd7b2fa56978f5fcad84ee4bd3acdcc61cf8e7f08f0789336febc5d76beae1eebf7bd51462512b78
If the locked coin needs to be persisted to the wallet database,
insteead of having the RPC figure out when to create a WalletBatch and
having LockCoin's behavior depend on it, have LockCoin take whether to
persist as a parameter so it makes the batch.
Since unlocking a persisted locked coin requires a database write as
well, we need to track whether the locked coin was persisted to the
wallet database so that it can erase the locked coin when necessary.
Keeping track of whether a locked coin was persisted is also useful
information for future PRs.
getaddressinfo, listunspent, listtransactions, listsinceblock, and
gettransaction all include parent_desc(s). Make sure that these are
consistent with each other, as well as being in normalized form.
The GetDisplayName() method name was confusing because it suggested the return
value could be used for display, while documentation and implementation
indicated it only meant to be used for logging. Also the name didn't suggest
that it was formatting the wallet names, which made it harder understand how
messages were formatted in the places it was called. Fix these issues by
splitting up the GetDisplayName() method and replacing it with LogName() /
DisplayName() methods.
This commit is a refactoring that does not change any behavior.
4b1b36acb4 doc: Remove build instruction for running `clang-tidy` (Hennadii Stepanov)
Pull request description:
One of the benefits of using a compilation database, which is available after the CMake build system generation step, is that it is not necessary to actually build the code in order to run `clang-tidy`.
ACKs for top commit:
TheCharlatan:
ACK 4b1b36acb4
janb84:
ACK 4b1b36acb4
Tree-SHA512: cf28fb1bcff83016b927522f1c719f3b91df7d107a310250c550308c8544b212fa6d2e8a5502d69fa424421acdf952469edd67504ac2a8465a2c1520593a1f26
Noticed while reviewing #31287
(https://github.com/bitcoin/bitcoin/pull/31287#discussion_r1843809721) that the
[default wallet] part of progress messages remains untranslated while the rest
of the string is translated. Fix this in all places where Wallet::ShowProgress
(which has a cancel button) and chain::showProgress (which doesn't have a
cancel button) are called by making "default wallet" into a translated string.
To minimize scope of this bugfix, this introduces a new wallet DisplayName()
method which behaves differently than the existing GetDisplayName() method. The
existing method will be cleaned up in the following commit.
One of the benefits of using a compilation database, which is available
after the CMake build system generation step, is that it is not
necessary to actually build the code in order to run `clang-tidy`.
This will become available in Clang 21:
> ThreadSafetyAnalysis now supports -Wthread-safety-pointer, which
> enables warning on passing or returning pointers to guarded variables
> as function arguments or return value respectively. Note that
> ThreadSafetyAnalysis still does not perform alias analysis. The
> feature will be default-enabled with -Wthread-safety in a future release.
See
https://github.com/llvm/llvm-project/blob/main/clang/docs/ReleaseNotes.rst.
Datacarrier output script sizes and output counts are now
uncapped by default.
To avoid introducing another startup argument, we modify the
OP_RETURN accounting to "budget" the spk sizes.
If a user has set a custom default, this results in that
budget being spent over the sum of all OP_RETURN outputs'
scripts in the transaction, no longer capping the number
of OP_RETURN outputs themselves. This should allow a
superset of current behavior while respecting the passed
argument in terms of total arbitrary data storage.
Co-authored-by: Anthony Towns <aj@erisian.com.au>
ad9a13fc42 walletdb: Log additional exception error messages for corrupted wallets (Ava Chow)
Pull request description:
Many exceptions thrown for corruption are `std::runtime_error`; we should catch those and log the message to help with debugging.
Split from #32489
ACKs for top commit:
davidgumberg:
ACK ad9a13fc42
furszy:
ACK ad9a13fc42
rkrux:
ACK ad9a13fc42
Sjors:
utACK ad9a13fc42
Tree-SHA512: 107b938d67346804733ea27c44ed38822db0e020e5b1ac889ee35280d812ec56dcc9af7b3eab7a521d72cdd9cb4a8d6d35f3a3dfbcb2a6fd170a981f34fbdfc2
f3a444c45f gui: Disallow loading legacy wallets (Ava Chow)
09955172f3 wallet, rpc: Give warning in listwalletdir for legacy wallets (Ava Chow)
Pull request description:
A new field `warnings` is added for each wallet in `listwalletdir`. If a legacy wallet is detected, the warning will contain a message that the wallet is a legacy wallet and will need to be migrated before it can be loaded.
In the GUI, the "Open Wallet" menu is changed to show legacy wallets greyed out with "(needs migration)" appended to their name to indicate to the user that the legacy wallet will need to be migrated.
ACKs for top commit:
maflcko:
lgtm ACK f3a444c45f
adyshimony:
Test ACK [f3a444c](f3a444c45f)
furszy:
Code review ACK f3a444c45f
w0xlt:
Code Review ACK f3a444c45f
Tree-SHA512: 496caec0ca37845487bd709e592240315eb23461fbd697e68a7fde8e4d9b74b48aab1212c88dbbcc8a107a896b824c2e1f69691068641812ae903f873fa2f22b
24e5fd3bed fs: remove _POSIX_C_SOURCE defining (fanquake)
Pull request description:
On Linux systems, `_POSIX_C_SOURCE` will default to `200809L` (since glibc 2.10). There's currently no reason for us to undefine it, and then set it to an earlier value. Also tested with musl libc.
I think if anything, the project should be settings macros like `_POSIX_C_SOURCE`, globally.
ACKs for top commit:
hebasto:
re-ACK 24e5fd3bed, only rebased since my recent [review](https://github.com/bitcoin/bitcoin/pull/32460#pullrequestreview-2854183748).
Tree-SHA512: 920d60058821992193616e0c73c2f7e4230a9e3ccb9d71d16493ae69696c868f4325d3dd2d4e8388749080c187aa7b205493b3e2c6986ad37440e591ebe107e1
b1ea542ae6 test: test MAX_SCRIPT_SIZE for block validity (Greg Sanders)
Pull request description:
I don't believe there are direct tests for this.
ACKs for top commit:
achow101:
ACK b1ea542ae6
TheCharlatan:
ACK b1ea542ae6
theStack:
ACK b1ea542ae6
Tree-SHA512: 1d7d3eab9c54977844bf2ca1aa403b070aae0f818db2fb5cae367d1c4d12f1e403b6fdec224af769a2ebb648cbca8bfd0d7df5db2a89fccf256c9c244484eba2
This change moves binaries that are not typically invoked directly by users
from the `bin/` directory to the `libexec/` directory in CMake installs and
binary releases. The goal is to simplify the contents of `bin/` for end users
while still making all binaries available when needed. After this change, the
binaries remaining in `bin/` are:
- bitcoin
- bitcoin-cli
- bitcoind
- bitcoin-qt
- bitcoin-tx
- bitcoin-util
- bitcoin-wallet
And the binaries that are moved to `libexec/` are:
- bench_bitcoin
- bitcoin-chainstate(*)
- bitcoin-gui(***)
- bitcoin-node(***)
- test_bitcoin(**)
- test_bitcoin-qt
(*) bitcoin-chainstate was previously missing an install rule and was actually
not installed even when it was enabled.
(**) test_bitcoin is the only libexec/ binary that is currently included in
bitcoin binary releases. The others are only installed when building from
source with relevant cmake options enabled.
(***) bitcoin-node and bitcoin-gui are not currently built by default or
included in binary releases but both of these changes are planned and
implemented in #31802
It reuses the logic from the `coins_view` target, except it uses an
in-memory CCoinsViewDB as the backend.
Note `CCoinsViewDB` will assert the best block hash is never null, so we
slightly modify the coins_view fuzz logic to take care of this.
83df64d749 log: Stats when fulfilling GETBLOCKTXN (David Gumberg)
3733ed2dae log: Size of missing tx'es when reconstructing compact block (David Gumberg)
36bcee05dc log: Log start of compact block initialization. (David Gumberg)
Pull request description:
This PR adds some additional logging to help measure performance of compact block reconstruction.
1. Adds a message to the beginning of `PartiallyDownloadedBlock::InitData()` so that that the logs indicate the amount of time it takes to populate a compact block from mempool transactions.
2. Logs the size of the transactions which a node did not have in its mempool and was forced to request.
3. Logs the size and number of transactions that a node sends to it's peer in a `BLOCKTXN` to fulfill a compact block `GETBLOCKTXN` request.
Relevant to this discussion on delving bitcoin: https://delvingbitcoin.org/t/stats-on-compact-block-reconstructions/1052
ACKs for top commit:
instagibbs:
reACK 83df64d749
w0xlt:
reACK 83df64d749
1440000bytes:
ACK 83df64d749
Tree-SHA512: 92c3c7d55005dd47dad90ddb54e4127482260cea5390f7696e8b3b9defb337f5fb09166af6b12eb2ab8151d04dae08b0a570e3509a86509b0ab3151d84387e06
84aa484d45 test: fix transaction_graph_test reorg test (Greg Sanders)
eaf44f3767 test: check chainlimits respects on reorg (Greg Sanders)
47894367b5 functional test: add MAX_DISCONNECTED_TX_POOL_BYTES coverage (Greg Sanders)
Pull request description:
`DisconnectedBlockTransactions::LimitMemoryUsage()` has unit test coverage, but the default value end to end doesn't have coverage.
This test adds exercised coverage of memory limiting of the disconnect pool, and some basic behavior sanity checks.
Another test added is making sure chainlimits are being respected on reorg, and the expected transactions pruned.
Lastly, fix the existing test case which was using a deficient test via directly inducing reorgs with `invalidateblock`
ACKs for top commit:
maflcko:
re-ACK 84aa484d45🚋
TheCharlatan:
ACK 84aa484d45
Tree-SHA512: f5cdb9647fadc8eb30352ce38de44064103825e5358787dfccd6416fa8faf6ceea42552fe2250b37d56271a6c3898b3912e1c028652da122f5c99304aafddb64
dbb2d4c3d5 windows: Add application manifest to `bitcoin.exe` (Hennadii Stepanov)
df82c2dc17 windows: Add resource file for `bitcoin.exe` (Hennadii Stepanov)
Pull request description:
This PR is a follow up to https://github.com/bitcoin/bitcoin/pull/31375, which:
1. Adds a resource file for `bitcoin.exe` for consistency with other Windows executables.
2. Adds an application manifest to `bitcoin.exe`, which has been required for release binaries since https://github.com/bitcoin/bitcoin/pull/32396.
ACKs for top commit:
davidgumberg:
ACK dbb2d4c3d5
hodlinator:
ACK dbb2d4c3d5
Tree-SHA512: 853c9e578bfd74bfd2e1f0fa39f978638723c8e061456caa165fca6f10497517f9503ae12dfb88e7229a02de593ccf22126f3362ca0d75c74becbb727e80c9ad
Currently, the migration process creates a brand-new descriptor wallet with no
connection to the user's legacy wallet when the legacy wallet lacks key material
and contains only watch-only scripts. This behavior is not aligned with user
expectations. If the legacy wallet contains only watch-only scripts, the migration
process should only generate a watch-only wallet instead.
4df4df45d7 test: fix sync function in rpc_psbt.py (Martin Zumsande)
Pull request description:
Even though the block is created on `node2`, the sync is only between `node1` and `node0`. Accordingly the test fails if I put a sleep in `msg_type == NetMsgType::HEADERS` processing: In this case, `node1` and `node0` do not hear about the new block, the sync still passes because they are in sync with each other, and later on in the `test_input_confs_control` subtest, `node1` would generate a forked block instead of building on the previous one, leading to test failure.
Haven't seen this in the CI, but I ran into it on an experimental branch.
ACKs for top commit:
maflcko:
lgtm ACK 4df4df45d7
achow101:
ACK 4df4df45d7
Tree-SHA512: 1211ba0ad263ebcd0aa6ef7c856dec7ec6ca6010e1df705e7243f6c9d950ccca6df1275c36a73a83034f49ea8401e8f9800c05cdb74c39e860e7ebcaf2ce6ada
fab1e02086 refactor: Pass verification_progress into block tip notifications (MarcoFalke)
fa76b378e4 rpc: Round verificationprogress to exactly 1 for a recent tip (MarcoFalke)
faf6304bdf test: Use mockable time in GuessVerificationProgress (MarcoFalke)
Pull request description:
Some users really seem to care about this. While it shouldn't matter much, the diff is so trivial that it is probably worth doing.
Fixes#31127
One could also consider to split the field into two dedicated ones (https://github.com/bitcoin/bitcoin/issues/28847#issuecomment-1807115357), but this is left for a more involved follow-up and may also be controversial.
ACKs for top commit:
achow101:
ACK fab1e02086
pinheadmz:
ACK fab1e02086
sipa:
utACK fab1e02086
Tree-SHA512: a3c24e3c446d38fbad9399c1e7f1ffa7904490a3a7d12623b44e583b435cc8b5f1ba83b84d29c7ffaf22028bc909c7cec07202b825480449c6419d2a190938f5
3e6ac5bf77 refactor: validation: mark CheckBlockIndex as const (stickies-v)
61a51eccbb validation: don't use GetAll() in CheckBlockIndex() (stickies-v)
d05481df64 refactor: validation: mark SnapshotBase as const (stickies-v)
Pull request description:
While reviewing another PR, I [noticed](https://github.com/bitcoin/bitcoin/pull/31405#discussion_r2056509235) that `ChainstateManager::CheckBlockIndex()` is not a `const` method. To try and assert that this method was not causing any side-effects, I modified the method to make it `const`. It did not surface any errors, but I think it would be good to merge this change regardless, even if `CheckBlockIndex` is only used in regtest.
This PR removes `CheckBlockIndex()`'s calls to non-const `ChainstateManager` methods by marking `SnapshotBase` `const` and ~inlining the `GetAll()` calls (thereby also performing consistency checks on invalid or fully validated `m_disabled==true` chainstates, as slight behaviour change), and finally marks `CheckBlockIndex()` as `const`.
ACKs for top commit:
achow101:
ACK 3e6ac5bf77
mzumsande:
Code Review ACK 3e6ac5bf77
TheCharlatan:
ACK 3e6ac5bf77
Tree-SHA512: 3d3cd351f5af1fab9a9498218ec62dba6e397fc7b5f4868ae0a77dc2b7c813d12c4f53f253f209101a3f6523695014e20c82dfac27cf0035611d5dd29feb80b5
The current test directly uses invalidatblock to trigger
mempool re-entry of transactions. Unfortunately, the
behavior doesn't match what a real reorg would look like. As
a result you get surprising behavior such as the mempool
descendant chain limits being exceeded, or if a fork is
greater than 10 blocks deep, evicted block transactions stop
being submitted back into in the mempool.
Fix this by preparing an empty fork chain, and then
continuing with the logic, finally submitting the fork chain
once the rest of the test is prepared. This triggers a more
typical codepath.
Also, extend the descendant limit to 100, like ancestor
limit.
09ee8b7f27 node: avoid recomputing block hash in `ReadBlock` (Lőrinc)
2bf173210f test: exercise `ReadBlock` hash‑mismatch path (Lőrinc)
Pull request description:
Eliminate one block header hash calculation per block-read by reusing the hash for:
* proof‑of‑work verification;
* (optional) integrity check against the supplied hash.
This part of the code wasn't covered by tests either, so the first commit exercises this part first, before pushing the validation to the delegate method.
ACKs for top commit:
maflcko:
lgtm ACK 09ee8b7f27
achow101:
ACK 09ee8b7f27
jonatack:
ACK 09ee8b7f27
pinheadmz:
ACK 09ee8b7f27
Tree-SHA512: 43fe51b478ea574b6d4c952684b13ca54fb8cbd67c3b6c136f460122d9ee953cc70b88778537117eecea71ccb8d88311faeac21b866e11d117f1145973204ed4
a5ac43d98d doc: Add release notes describing bitcoin wrapper executable (Ryan Ofsky)
258bda80c0 doc: Mention bitcoin wrapper executable in documentation (Ryan Ofsky)
d2739d75c9 build: add bitcoin.exe to windows installer (Sjors Provoost)
ba649c0006 ci: Run multiprocess tests through wrapper executable (Ryan Ofsky)
29bdd743bb test: Support BITCOIN_CMD environment variable (Ryan Ofsky)
9c8c68891b multiprocess: Add bitcoin wrapper executable (Ryan Ofsky)
5076d20fdb util: Add cross-platform ExecVp and GetExePath functions (Ryan Ofsky)
Pull request description:
Intended to make bitcoin command line features more discoverable and allow installing new multiprocess binaries in libexec/ instead of bin/ so they don't cause confusion.
Idea and implementation of this were discussed in https://github.com/bitcoin/bitcoin/issues/30983.
---
Initial implementation of this feature is deliberately minimal so the UX can evolve in response to feedback and there are not too many details to debate and discuss in a single PR. But many improvements are possible or planned:
- Adding manpage and bash completions.
- Showing nicer error messages that detect if an executable isn't installed and suggest how to fix [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2073194474)
- Showing wrapper command lines in subcommand in help output [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2077800405). This could be done conditionally as suggested in the comment or be unconditional.
- Showing wrapper command lines in subcommand error output. There is a bitcoin-cli error pointed out in [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2091152243) that is needlessly confusing.
- Integrating help so `bitcoin help subcommand` invokes `bitcoin subcommand -h`. `bitcoin -h subcommand` should also be supported and be equivalent [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2093116725)
- Adding support for `bitcoin-util` subcommands. Ideal interface would probably be more like `bitcoin grind` not `bitcoin util grind` but this has been punted for now. Supporting subcommands directly would require some ArgsManager modifications
- Adding a dedicated python functional test for the wrapper. Right now there is some CI coverage by setting the `BITCOIN_CMD` variable, but this doesn't cover things like the help output and version output, and support for different directory layouts.
- Better `--multiprocess` (`-m`) / `--monolithic` (`-M`) default selection. Right now, default is monolithic but it probably makes sense to chose more intelligently depending on whether -ipc options are enabled and what binaries are available.
- Maybe parsing `bitcoin.conf` and supporting options to control wrapper behavior like custom locations or preferences or aliases.
- Better command command line usability. Allow combining short options like (`-ah`). Allow fuzzy matching of subcommands or suggestions if you misspell. (suggested by stickies in review club)
- Not directly related to this PR but `bitcoin-cli named` implementation used by the wrapper should do a better job disambiguating named arguments from base64 arguments ending in = as pointed out in [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2091886628)
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722). A review club meeting for it took place in https://bitcoincore.reviews/31375
ACKs for top commit:
Sjors:
utACK a5ac43d98d
achow101:
ACK a5ac43d98d
vasild:
ACK a5ac43d98d
theStack:
ACK a5ac43d98d
ismaelsadeeq:
fwiw my last review implied an ACK a5ac43d98d
hodlinator:
ACK a5ac43d98d
Tree-SHA512: 570e6a4ff8bd79ef6554da3d01f36c0a7c6d2dd7dace8f8732eca98f4a8bc2284474a9beadeba783114fe2f3dd08b2041b3da7753bae0b7f881ec50668cb821f
f66b14d2ec test: fix pushdata scripts (Greg Sanders)
Pull request description:
The original scripts were done incorrectly,
so they are changed to represent two
different 2-byte pushes.
Fixes https://github.com/bitcoin/bitcoin/pull/32114#discussion_r2034051063
ACKs for top commit:
ajtowns:
ACK f66b14d2ec
TheCharlatan:
Re-ACK f66b14d2ec
Tree-SHA512: 0956124ee0d2e8b6a594f9feeb47c1f598c68e24d277e874f81a093268113e9da2c75a02863dbaab68b962063f7d910bfd10abe3ad33ec182bc21d72908f06e6
Instead of allowing users to load a legacy wallet from the "Open Wallet"
menu, show the legacy wallet greyed out with a message that the wallet
needs to be migrated.
Eliminate one SHA‑256 double‑hash computation of the header per block read by reusing the hash for:
* proof‑of‑work verification;
* (optional) integrity check against the supplied hash.
e5cbea416b rpc: doc: remove redundant "descriptors" parameter in `createwallet` examples (Sebastian Falbesoner)
7a05f941bb rpc: doc: drop descriptor wallet mentions in fast wallet rescan related RPCs (Sebastian Falbesoner)
db465a50e2 wallet, rpc: remove obsolete "keypoololdest" result field/code (Sebastian Falbesoner)
Pull request description:
This PR contains a few smaller wallet RPC cleanups based on that we only ever operate on descriptor wallets now:
* remove the now obsolete "keypoololdest" field from the `getwalletinfo` RPC and the corresponding CWallet/ScriptPubKeyMan methods
* in RPCs where potential fast wallet rescan is documented, remove the "descriptor wallet" mentions (back then introduced in commit ca48a4694f, PR #25957)
* for the `createwallet` RPC examples, remove the "descriptors" parameters that always have to be true now (proposed in https://github.com/bitcoin/bitcoin/pull/31250#discussion_r2042020967; corresponds to 86de8c1668, PR #32544 which did the same for functional tests)
ACKs for top commit:
achow101:
ACK e5cbea416b
1440000bytes:
ACK e5cbea416b
rkrux:
ACK e5cbea416b
Tree-SHA512: d785f621af3f3987b258e5d7fb8309344fb13c2cf41855f8adf99ff89f581142db36e3ba59919d6abf82662caa3f7e4a2bd38eba1be9e23665e6a4a23ee52acd
Although WalletBatch::LoadWallet performs the descriptor cache upgrade,
because new wallets do not have the descriptor flag set yet, the upgrade
does not run and set the flag.
Since new wallets will always being using the upgraded cache, there's no
reason to wait to set the flag, so set it when the wallet flags are
being initialized for new wallets.
This is the RPC example counterpart to commit
86de8c1668 (PR #32544).
Since the recent legacy wallet removal this parameter *must* be
true, so providing it in the examples doesn't contain valuable
information anymore and it seems best to remove them.
Now that we only ever operate on descriptor wallets, mentioning
that a faster rescan is only available for them is redundant and
can be removed.
These texts were originally introduced in commit
ca48a4694f (PR #25957).
This `getwalletinfo()` result field was only ever returned for
legacy wallets and is hence not relevant anymore, so we can
delete it and the corresponding CWallet/ScriptPubKeyMan code
behind it.
fa079538e3 ci: Downgrade DEBUG=1 to -D_GLIBCXX_ASSERTIONS in centos task (MarcoFalke)
Pull request description:
to work around https://github.com/bitcoin/bitcoin/issues/32524#issuecomment-2900903169closes#32524
ACKs for top commit:
laanwj:
ACK fa079538e3
fanquake:
ACK fa079538e3 - we can followup
Tree-SHA512: 2d8b914c7390bbf22d8b2eb906bd2a363f92e1954646677a010b15721fca0887d5987a0af932fd0013f5c1b35c0a80c67579004a0cf635916954331c80c7bef0
fd290730f5 validation: clean up and clarify CheckInputScripts logic (Cory Fields)
1a37507895 validation: use a lock for CCheckQueueControl (Cory Fields)
c3b0e6c7f4 validation: make CCheckQueueControl's CCheckQueue non-optional (Cory Fields)
4c8c90b556 validation: only create a CCheckQueueControl if it's actually going to be used (Cory Fields)
11fed833b3 threading: add LOCK_ARGS macro (Cory Fields)
Pull request description:
As part of an effort to cleanup our threading primitives and add safe `SharedMutex`/`SharedLock` impls, I'd like to get rid of the last of our legacy `ENTER_CRITICAL_SECTION`/`LEAVE_CRITICAL_SECTION` usage. This, along with a follow-up [after fixing REVERSE_LOCK](https://github.com/bitcoin/bitcoin/pull/32465) will allow us to do that.
This replaces the old macros with an RAII lock, while simplifying `CCheckQueueControl`. It now requires a `CCheckQueue`, and optionality is handled externally. In the case of validation, it is wrapped in a `std::optional`.
It also adds an `LOCK_ARGS` macro for `UniqueLock` initialization which may be helpful elsewhere.
ACKs for top commit:
fjahr:
re-ACK fd290730f5
ryanofsky:
Code review ACK fd290730f5, just removing assert since last review. Thanks for considering all the comments and feedback!
TheCharlatan:
Re-ACK fd290730f5
Tree-SHA512: 54b9db604ee1bda7d11bce1653a88d3dcbc4f525eed6a85abdd4d6409138674af4bb8b00afa4e0d3d29dadd045a3a39de253a45f0ef9c05f56cba1aac5b59303
7bc64a8859 test: properly check for per-tx sigops limit (Sebastian Falbesoner)
Pull request description:
Currently the per-tx sigops limit standardness check (bounded by `MAX_STANDARD_TX_SIGOPS_COST`, throwing "bad-txns-too-many-sigops" if exceeded):
3f83c744ac/src/validation.cpp (L925-L927)
is only indirectly tested with the much higher per-block consensus limit (`MAX_BLOCK_SIGOPS_COST`):
3f83c744ac/test/functional/data/invalid_txs.py (L236-L242)
I.e. an increase in the per-tx limit up to the per-block one would still pass all of our tests. Refine that by splitting up the invalid tx template `TooManySigops` in a per-block and a per-tx template.
The involved functional tests taking use of these templates are `feature_block.py` and `p2p_invalid_txs.py`. Can be tested by applying e.g.
```diff
diff --git a/src/policy/policy.h b/src/policy/policy.h
index 2151ec13dd..e5766d2a55 100644
--- a/src/policy/policy.h
+++ b/src/policy/policy.h
@@ -37,7 +37,7 @@ static constexpr unsigned int MIN_STANDARD_TX_NONWITNESS_SIZE{65};
/** Maximum number of signature check operations in an IsStandard() P2SH script */
static constexpr unsigned int MAX_P2SH_SIGOPS{15};
/** The maximum number of sigops we're willing to relay/mine in a single tx */
-static constexpr unsigned int MAX_STANDARD_TX_SIGOPS_COST{MAX_BLOCK_SIGOPS_COST/5};
+static constexpr unsigned int MAX_STANDARD_TX_SIGOPS_COST{MAX_BLOCK_SIGOPS_COST/5 + 4};
/** Default for -incrementalrelayfee, which sets the minimum feerate increase for mempool limiting or replacement **/
static constexpr unsigned int DEFAULT_INCREMENTAL_RELAY_FEE{1000};
/** Default for -bytespersigop */
diff --git a/test/functional/mempool_accept.py b/test/functional/mempool_accept.py
```
where the tests succeed on master, but fail on this PR.
(Found by diving deeper into the jungle of current sig-ops limit, as preparation for reviewing the [BIP 54](https://github.com/bitcoin/bips/blob/master/bip-0054.md) draft and related preparatory PRs like #32521).
ACKs for top commit:
fjahr:
tACK 7bc64a8859
tapcrafter:
tACK 7bc64a8859
darosior:
ACK 7bc64a8859
instagibbs:
crACK 7bc64a8859
Tree-SHA512: 1365409349664a76a1d46b2fa358c0d0609fb17fffdd549423d22b61749481282c928be3c2fb428725735c82d319b4279f703bde01e94e4aec14bab206abb8cf
800b7cc42c cmake: Add missed `SSE41_CXXFLAGS` (Hennadii Stepanov)
028476e71f cmake: Remove `ENABLE_ARM_SHANI` from `bitcoin-build-config.h` (Hennadii Stepanov)
1e900528d2 cmake: Remove `ENABLE_X86_SHANI` from `bitcoin-build-config.h` (Hennadii Stepanov)
8689628e2e cmake: Remove `ENABLE_AVX2` from `bitcoin-build-config.h` (Hennadii Stepanov)
a8e2342dca cmake: Remove `ENABLE_SSE41` from `bitcoin-build-config.h` (Hennadii Stepanov)
Pull request description:
`ENABLE_{SSE41,AVX2,X86_SHANI,ARM_SHANI}` are already conditionally defined for the [`bitcoin_crypto`](https://github.com/bitcoin/bitcoin/blob/master/src/crypto/CMakeLists.txt) target, and they are not used by any other targets. Defining them globally in `bitcoin-build-config.h` is therefore redundant.
Additionally, the previously missing `SSE41_CXXFLAGS` variable has been [added](https://github.com/bitcoin/bitcoin/pull/32550#issuecomment-2890918551).
ACKs for top commit:
fanquake:
ACK 800b7cc42c
Tree-SHA512: da792a0b780c67b432b09c9288ca98d62545315c721fed13510d1c11f8bb0cddd9a4ed7a009b4d052471dda19d0641bbc1eae4805fc306d23bf9b4ef510089c8
On Linux systems, `_POSIX_C_SOURCE` will default to `200809L` (since
glibc 2.10). There's currently no reason for us to undefine it, and then
set it to an earlier value. Also tested with musl libc.
I think if anything, the project should be settings macros like
`_POSIX_C_SOURCE`, globally.
Covers:
- http pipelining
- rpcservertimeout
Testing this requires adding an option to TestNode to force
the test framework to establish a new HTTP connection for
every RPC. Otherwise, attempting to reuse a persistent connection
would cause framework RPCs during startup and shutdown to fail.
- "chunked" Transfer-Encoding
This is already checked by test/lint/lint-files.py
There is no need to reword all linters into the dev notes.
Also, allow scripts in Rust (there are already some).
785e1407b0 wallet: Use util::Error throughout AddWalletDescriptor (Ava Chow)
Pull request description:
#32023 changed `AddWalletDescriptor` to return `util::Error`, but did not change all of the failure cases to do so. This may result in some callers continuing when there was actually an error. Unify all of the failure cases to use `util::Error` so that all callers handle `AddWalletDescriptor` errors in the same way.
The encapsulated return type is changed from `ScriptPubKeyMan*` to `std::reference_wrapper<DescriptorScriptPubKeyMan>`. This avoids having a value that can be interpreted as a bool, and also removes the need to constantly dynamic_cast the returned value. The only kind of `ScriptPubKeyMan` that can come out of `AddWalletDescriptor` is a `DescriptorScriptPubKeyMan` anyways.
ACKs for top commit:
Sjors:
utACK 785e1407b0
ryanofsky:
Code review ACK 785e1407b0
furszy:
Code review ACK 785e1407b0
Tree-SHA512: 52a48263c8d4161a8c0419b7289c25b0986f8e3bcd10b639eeeb0b6862d08b6c5e70998d20070ab26b39ecd90ab83dc8b71c65d85f70626282cf8cc6abff50e7
fab97f583f ci: Avoid && dropping errors (MarcoFalke)
Pull request description:
In bash, `&&` will ignore errexit. This can lead to silently ignoring errors. Compare the output of:
```
$ bash -c 'set -xe; false && false ; true; echo $?'
+ false
+ true
+ echo 0
0
```
In theory this could be fixed by using a subshell:
```
$ bash -c 'set -xe; ( false && false ) ; true; echo $?'
+ false
```
However, it is easier to just remove the `&&`.
This was introduced in commit faa807bdf8
ACKs for top commit:
janb84:
Code review ACK fab97f583f
hebasto:
ACK fab97f583f.
laanwj:
ACK fab97f583f
Tree-SHA512: 9d034829e03ef3aefdaad82c3cab59bf3fe18529762271c1ad3c838357e337e94bd403b77e30c0cf69715254b65addff6d12f2fb497d7a0e2cdcbcbf78858d47
e8661aac75 wallet: drop watch-only things from interface (Sjors Provoost)
e99188e7da qt: drop unused watch-only functionality (Sjors Provoost)
Pull request description:
The watch-only functionality in the GUI was only used for legacy wallets. Watch-only descriptor wallets do not use this.
The only visible changes of this PR should be:
- dropped "Spendable:" label from the overview tab
- column width cache is reset
This PR also removes some unused variables from the interface.
ACKs for top commit:
davidgumberg:
Review ACK e8661aac75.
hebasto:
ACK e8661aac75, I have reviewed the code and it looks OK. The `src/qt/forms/overviewpage.ui` form was reviewed in Qt Designer.
Tree-SHA512: d7edb0f167e0b934075398a76eddca69890bb36848a918c932b1c2cea85ee87285e876cbfdf1f6dec7adf26b9f405fb558c70bec0c84585c0a9df33c2af78393
ee045b61ef rpc, psbt: Require sighashes match for descriptorprocesspsbt (Ava Chow)
2b7682c372 psbt: use sighash type field to determine whether to remove non-witness utxos (Ava Chow)
28781b5f06 psbt: Add sighash types to PSBT when not DEFAULT or ALL (Ava Chow)
15ce1bd73f psbt: Enforce sighash type of signatures matches psbt (Ava Chow)
1f71cd337a wallet: Remove sighash type enforcement from FillPSBT (Ava Chow)
4c7d767e49 psbt: Check sighash types in SignPSBTInput and take sighash as optional (Ava Chow)
a118256948 script: Add IsPayToTaproot() (Ava Chow)
d6001dcd4a wallet: change FillPSBT to take sighash as optional (Ava Chow)
e58b680923 psbt: Return PSBTError from SignPSBTInput (Ava Chow)
2adfd81532 tests: Test PSBT sighash type mismatch (Ava Chow)
5a5d26d612 psbt: Require ECDSA signatures to be validly encoded (Ava Chow)
Pull request description:
Currently, we do not add the sighash field to PSBTs at all, even when we have signed with a non-default sighash. This PR changes the behavior such that when we (attempt to) sign with a sighash other than DEFAULT or ALL, the sighash type field will be added to the PSBT to inform the later signers that a different sighash type was used by a signer. Notably, this is necessary for MuSig2 support as all signers must sign using the same sighash type, but the sighash is not provided in partial signatures.
Furthermore, because the sighash type can also be provided on the command line, we require that if both a command line sighash type and the sighash field is present, they must specify the same sighash type. However, this was being checked by the wallet, rather than the signing code, so the `descriptorprocesspsbt` RPC was not enforcing this restriction at all, and in fact ignored the sighash field entirely. This PR refactors the checking code so that the underlying PSBT signing function `SignPSBTInput` does the check.
ACKs for top commit:
theStack:
re-ACK ee045b61ef
rkrux:
re-ACK ee045b61ef
fjahr:
Code review ACK ee045b61ef
Tree-SHA512: 4ead5be1ef6756251b827f594beba868a145d75bf7f4ef6f15ad21f0ae4b8d71b38c83494e5a6b75f37fadd097178cddd93d614b962a2c72fc134f00ba2f74ae
97d383af6d Test updating non-ranged descriptor with [0,0] range succeeds (Novo)
2ae1788dd4 Skip range verification for non-ranged desc (Novo)
Pull request description:
Closes https://github.com/bitcoin/bitcoin/issues/31728
This PR updates the `DescriptorScriptPubKeyMan` to skip range checks for non-ranged descriptors, which previously caused errors when updating a non-ranged descriptor with the range [0,0]
#### Testing
A unit test was added to test the new behaviour
ACKs for top commit:
achow101:
ACK 97d383af6d
rkrux:
ACK 97d383a
Tree-SHA512: 6dbd058376d9e57d26477d9d6d89646e80a32e3ffcc9f4e30eeda273575d12583ce520cc0032cc67c12ea0b3ad344fbd3945d9fc5e389b6a6bce1ea7ad5d6e59
6f7052a7b9 threading: semaphore: move CountingSemaphoreGrant to its own header (Cory Fields)
fd15469892 threading: semaphore: remove temporary convenience types (Cory Fields)
1f89e2a49a scripted-diff: threading: semaphore: use direct types rather than the temporary convenience ones (Cory Fields)
f21365c4fc threading: replace CountingSemaphore with std::counting_semaphore (Cory Fields)
1acacfbad7 threading: make CountingSemaphore/CountingSemaphoreGrant template types (Cory Fields)
e6ce5f9e78 scripted-diff: rename CSemaphore and CSemaphoreGrant (Cory Fields)
793166d381 wallet: change the write semaphore to a BinarySemaphore (Cory Fields)
6790ad27f1 scripted-diff: rename CSemaphoreGrant and CSemaphore for net (Cory Fields)
d870bc9451 threading: add temporary semaphore aliases (Cory Fields)
7b816c4e00 threading: rename CSemaphore methods to match std::semaphore (Cory Fields)
Pull request description:
This is relatively simple, but done in a bunch of commits to enable scripted diffs.
I wanted to add a semaphore in a branch I've been working on, but it was unclear if I should use `std::counting_semaphore` or stick with our old `CSemaphore`. I couldn't decide, so I just decided to remove all doubt and get rid of ours :)
This replaces our old `CSemaphore` with `std::counting_semaphore` everywhere we used it. `CSemaphoreGrant` is still there as an RAII wrapper, but is now called `CountingSemaphoreGrant` and `BinarySemaphoreGrant` to match. Those have been moved out of `sync.h` to their own file.
ACKs for top commit:
purpleKarrot:
ACK 6f7052a7b9
achow101:
ACK 6f7052a7b9
TheCharlatan:
ACK 6f7052a7b9
hebasto:
ACK 6f7052a7b9, I have reviewed the code and it looks OK.
Tree-SHA512: 5975d13aa21739174e3a22c544620ae3f36345f172b51612346d3b7baf0a07c39ef6fd54f647c87878c21a67951b347a5d4a5f90e897f3f6c0db360a3779d0df
135a0f0aa7 doc: Add missing top-level description to pruneblockchain RPC (nervana21)
Pull request description:
Previously, the `pruneblockchain` RPC help output included only the method signature and arguments, with no top-level description explaining its purpose or constraints.
This PR adds a top-level description, improving documentation consistency and alerting users to the potential impacts of using the command.
ACKs for top commit:
maflcko:
lgtm ACK 135a0f0aa7
yancyribbens:
cr ACK 135a0f0aa7
achow101:
ACK 135a0f0aa7
janb84:
re ACK [135a0f0](135a0f0aa7)
Tree-SHA512: e51475238e779555315668b7389ed312a5d2c4ad1c0b251f2314895ac473092fa458b6f931f70385e14047adb7e340e44fe2198643603da9e129f1c874578a28
CheckInputScripts behaves differently depending on whether or not it was called
with a vector for checks. Make this difference clear by calling it differently
depending on whether or not control exists. Though more verbose, it should be
more straightforward to understand what's happening this way.
Also remove parallel_script_checks, as `if(control)` is a better test.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
faf55fc80b doc: Remove ParseInt mentions in documentation (MarcoFalke)
3333282933 refactor: Remove unused Parse(U)Int* (MarcoFalke)
fa84e6c36c bitcoin-tx: Reject + sign in MutateTxDel* (MarcoFalke)
face2519fa bitcoin-tx: Reject + sign in vout parsing (MarcoFalke)
fa8acaf0b9 bitcoin-tx: Reject + sign in replaceable parsing (MarcoFalke)
faff25a558 bitcoin-tx: Reject + sign in locktime (MarcoFalke)
dddd9e5fe3 bitcoin-tx: Reject + sign in nversion parsing (MarcoFalke)
fab06ac037 rest: Use SAFE_CHARS_URI in SanitizeString error msg (MarcoFalke)
8888bb499d rest: Reject + sign in /blockhashbyheight/ (MarcoFalke)
fafd43c691 test: Reject + sign when parsing regtest deployment params (MarcoFalke)
fa123afa0e Reject + sign when checking -ipcfd (MarcoFalke)
fa479857ed Reject + sign in SplitHostPort (MarcoFalke)
fab4c2967d net: Reject + sign when parsing subnet mask (MarcoFalke)
fa89652e68 init: Reject + sign in -*port parsing (MarcoFalke)
fa9c45577d cli: Reject + sign in -netinfo level parsing (MarcoFalke)
fa98041325 refactor: Use ToIntegral in CreateFromDump (MarcoFalke)
fa23ed7fc2 refactor: Use ToIntegral in ParseHDKeypath (MarcoFalke)
Pull request description:
The legacy int parsing is problematic, because it accepts the `+` sign for unsigned integers. In all cases this is either:
* Useless, because the `+` sign was already rejected.
* Erroneous and inconsistent, when third party parsers reject it. (C.f. https://github.com/bitcoin/bitcoin/pull/32365)
* Confusing, because the `+` sign is neither documented, nor can it be assumed to be present.
Fix all issues by removing the legacy int parsing.
ACKs for top commit:
stickies-v:
re-ACK faf55fc80b
brunoerg:
code review ACK faf55fc80b
Tree-SHA512: a311ab6a58fe02a37741c1800feb3dcfad92377b4bfb61b433b2393f52ba89ef45d00940972b2767b213a3dd7b59e5e35d5b659c586eacdfe4e565a77b12b19f
4b2cd0b41f test: check that creating a wallet does not log version info (Ava Chow)
39a483c8e9 test: Check that the correct versions are logged on wallet load (Ava Chow)
359ecd3704 walletdb: Log the wallet version after it has been read from disk (Ava Chow)
Pull request description:
The wallet's version (in the minversion record) needs to be logged only after we have read it from disk. Otherwise, we always log the lowest version number of 10500 which is incorrect. Furthermore, it doesn't make sense to log the last client version number if the record didn't exist. This is a regression caused by #26021.
The wallet file version logging is moved inside of `LoadMinVersion` so that it is logged after the record is read. It will also log unconditionally if a version is read so that the version number is reported even when there is an error. The last client logging is split into its own log line that will only occur if a last client record is read. The only situation where we expect no version numbers to be logged is when a wallet is being created.
A test is added in the second commit to check that the version number is correctly logged on loading. This commit can be cherrypicked to master to verify that it fails there. The last commit adds an additional check that creating a new wallet does not log any version info at all.
ACKs for top commit:
laanwj:
Code review ACK 4b2cd0b41f
janb84:
ACK 4b2cd0b41f
furszy:
ACK 4b2cd0b41f
rkrux:
ACK 4b2cd0b41f
Tree-SHA512: b30c76f414d87be6c14b42d2d3c8794a91a7e8601501f4c24641d51ff2b5c5144776563baf41ca1c38415844740b760b19a3e5791f78013b39984dfedd3b1de7
The windows code adds an unnecessary extra space to the command line.
This can cause subtle issues, so avoid it.
Github-Pull: arun11299/cpp-subprocess#119
Rebased-From: 777cfa77d1f84bb08b3e445d5f7fc6c87282223b
fa58d6cdab ci: Move DEBUG=1 to centos task (MarcoFalke)
Pull request description:
The glibcxx debug mode has many bugs in prior gcc releases:
* https://github.com/bitcoin/bitcoin/issues/32524#issuecomment-2890411766
* https://github.com/bitcoin/bitcoin/issues/31436#issuecomment-2530717875
* ...
Instead of working around all of them, just use the existing `ci_native_centos` task with gcc-14 to have it enabled. This also follows the logic of other sanitizers (tsan, asan, ubsan, msan, valgrind, ...) to generally prefer the latest version of the sanitizer for the latests features and bugfixes.
Fixes#32524.
Also, while touching the `ci_native_previous_releases`, increase g0 to g2, so that it is easier for developers to use gdb inside the CI without having to re-compile
ACKs for top commit:
hebasto:
ACK fa58d6cdab.
fanquake:
ACK fa58d6cdab
Tree-SHA512: 26f151887bc39e88353b4ba1f91e913b830d24eac258b7caa0027aa592595150d5e713ededa3ed15a1b84165a94b14b0bfe3eb2fb7314d261d972b63ce01af43
7193245cd6 doc: remove For ... comments (fanquake)
1b9cdc933f net: drop win32 ifdef (fanquake)
19ba499b1f init: cerrno is used on all platforms (fanquake)
Pull request description:
We don't add or maintain these, and they are of little value, as
well as having the effect of polluting diffs, if changed.
They are also wrong, i.e `DEFAULT_SCRIPTCHECK_THREADS` is not in
`validation.h`.
ACKs for top commit:
stickies-v:
re-ACK 7193245cd6
fjahr:
ACK 7193245cd6
willcl-ark:
reACK 7193245cd6
Tree-SHA512: 6b5f83cd1df699356e1cbb78949f8d456b13ce288f0064138118cfb45b4c77e2d1945babe91598dffe9823ab07dfae36f4c3b61c586cf98baf16890bdf322b08
fa9198af55 lint: Check for missing trailing newline (MarcoFalke)
fa2b2aa27c lint: Add archived notes to default excludes (MarcoFalke)
Pull request description:
A missing trailing newline is harmless, but a bit problematic:
* `git` shows a warning by default
* After another line is appended, the diff will be verbose and `git blame` will be wrong for the "untouched" line.
Fix the problems by just requiring what is already the default, see also 663a9cabf8/.editorconfig (L9) and 663a9cabf8/test/lint/test_runner/src/main.rs (L327)
ACKs for top commit:
l0rinc:
utACK fa9198af55
fanquake:
ACK fa9198af55
Tree-SHA512: d144eebdeee68fc3404aa4a66ecd5c130f907ed4b869bd300f6e9ed74d125561d1f4cdd6dd20d9e969471a7d007399f928f072d1c1f626275ca31f32bc23fdbc
In the dev notes, remove the whole section, because:
* ParseDouble was removed in commit
fa9d72a794
* The locale-dependent atoi is already checked by
test/lint/lint-locale-dependence.py
Co-authored-by: Fabian Jahr <fjahr@protonmail.com>
32023 changed AddWalletDescriptor to return util::Error, but did not
change all of the failure cases to do so. This may result in some
callers continuing when there was actually an error. Unify all of the
failure cases to use util::Error so that all callers handle
AddWalletDescriptor errors in the same way.
The encapsulated return type is changed from ScriptPubKeyMan* to
std::reference_wrapper<DescriptorScriptPubKeyMan>. This avoids having a
value that can be interpreted as a bool, and also removes the need to
constantly dynamic_cast the returned value. The only kind of
ScriptPubKeyMan that can come out of AddWalletDescriptor is a
DescriptorScriptPubKeyMan anyways.
32-bit architecture is limited to 4GiB, so it doesn't make sense to set a too
high value. Since this setting is performance critical, pick an arbitrary value
higher than for -maxmempool but still reasonable.
32-bit architecture is limited to 4GiB, so it doesn't make sense to set a too high value. 500 MB is
chosen as an arbitrary maximum value that seems reasonable.
30a94b1ab9 test, wallet: Remove concurrent writes test (Ava Chow)
b44b7c03fe wallet: Write best block record on unload (Ava Chow)
876a2585a8 wallet: Remove unnecessary database Close step on shutdown (Ava Chow)
98a1a5275c wallet: Remove chainStateFlushed (Ava Chow)
7fd3e1cf0c wallet, bench: Write a bestblock record in WalletMigration (Ava Chow)
6d3a8b195a wallet: Replace chainStateFlushed in loading with SetLastBlockProcessed (Ava Chow)
7bacabb204 wallet: Update best block record after block dis/connect (Ava Chow)
Pull request description:
Implements the idea discussed in https://github.com/bitcoin/bitcoin/pull/29652#issuecomment-2010579484
Currently, `m_last_block_processed` and `m_last_block_processed_height` are not guaranteed to match the block locator stored in the wallet, nor do either of those fields actually represent the last block that the wallet is synced up to. This is confusing and unintuitive.
This PR changes those last block fields to be updated whenever the wallet makes a change to the db for new transaction state found in new blocks. Whenever a block is received that contains a transaction relevant to the wallet, the last block locator will now be written to disk. Furthermore, every block disconnection will now write an updated locator.
To ensure that the locator is relatively recent and loading rescans are fairly quick in the event of unplanned shutdown, it is also now written every 144 blocks (~1 day). Additionally it is now written when the wallet is unloaded so that it is accurate when the wallet is loaded again.
Lastly, the `chainstateFlushed` notification in the wallet is changed to be a no-op. The best block locator record is no longer written when `chainstateFlushed` is received from the node since it should already be mostly up to date.
ACKs for top commit:
rkrux:
ACK 30a94b1ab9
mzumsande:
Code Review ACK 30a94b1ab9
ryanofsky:
Code review ACK 30a94b1ab9. Only changes since last review are using WriteBestBlock method more places and updating comments.
Tree-SHA512: 46117541f8aaf13dde57430e813b4bbbd5e146e2632769675803c8e65a82f149a7cc6026489a127d32684b90124bd2b7c28216dbcfa6a47447300e8f3814e029
fa982f1425 Use WitnessV0KeyHash in TestAddAddressesToSendBook (MarcoFalke)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/32558
Fixes https://github.com/bitcoin-core/gui/issues/874
This fixes a bug introduced in commit fafee85358, which changed the type of the dummy address from `WitnessV0KeyHash` to `PKHash`. It was expected that this is fine, given that this is just a dummy address. However, the base58 characters can include the substring "io", leading to test failures later on.
Fix it by just using `WitnessV0KeyHash` again.
For reference, a passing test could look like:
```
Model contains 2 rows and 2 columns.
--- Model Data ---
Row 0 : "io - new A\tmxgkqJWAwfUwbgzZUsWrG1stKWV6fDn8YH"
Row 1 : "io - new B\tmhsxP2yrYDQiEncT8HzKxQSFSFJmUsudsP"
------------------
```
A failing test could look like:
```
Model contains 3 rows and 2 columns.
--- Model Data ---
Row 0 : "already here (s)\tmyDFZSKDQdPMMoSQgzkDtq2yioo8DA8qCX"
Row 1 : "io - new A\tmsAqQKjMrbxYRDhGXBBJ3yUEQxj5Bf5Njz"
Row 2 : "io - new B\tmtALQiit8dw33kznVfHDgE38ohfgz2Pchc"
------------------
FAIL! : AddressBookTests::addressBookTests() Compared values are not the same
Actual (table_view->model()->rowCount()): 3
Expected (2) : 2
Loc: [qt/test/addressbooktests.cpp(219)]
```
ACKs for top commit:
achow101:
ACK fa982f1425
hebasto:
ACK fa982f1425, I have reviewed the code along with the related changes from https://github.com/bitcoin/bitcoin/pull/32511.
Tree-SHA512: f55d7fe4193a0706e1a3ca1a2c0fbf2f04dc5b177699add00013ec56d64218ac85b80dad6e99f9fde26f4c9fca79f99e68ded057c5862364064404ac06b77176
e49a7274a2 rpc: Avoid join-split roundtrip for user:pass for auth credentials (Vasil Dimov)
98ff38a6f1 rpc: Perform HTTP user:pass split once in `RPCAuthorized` (laanwj)
879a17bcb1 rpc: Store all credentials hashed in memory (laanwj)
4ab9bedee9 rpc: Undeprecate rpcuser/rpcpassword, change message to security warning (laanwj)
Pull request description:
This PR does two things:
### Undeprecate rpcuser/rpcpassword, change message to security warning
Back in 2015, in https://github.com/bitcoin/bitcoin/pull/7044, we added configuration option `rpcauth` for multiple RPC users. At the same time the old settings for single-user configuration `rpcuser` and `rpcpassword` were "soon" to be deprecated.
The main reason for this deprecation is that while `rpcpassword` stores the password in plain text, `rpcauth` stores a hash, so it doesn't appear in the configuration in plain text.
As the options are still in active use, actually removing them is expected to be a hassle to many, and it's not clear that is worth it. As for the security risk, in many kinds of setups (no wallet, containerized, single-user-single-application, local-only, etc) it is an unlikely point of escalation.
In the end, it is good to encourage secure practices, but it is the responsibility of the user. Log a clear warning but remove the deprecation notice (this is also the only place where the options appear as deprecated, they were never marked as such in the -help output).
<hr>
### Store all credentials hashed in memory
This gets rid of the special-casing of `strRPCUserColonPass` by hashing cookies as well as manually provided `-rpcuser`/`-rpcpassword` with a random salt before storing them.
Also take the opportunity to modernize the surrounding code a bit. There should be no end-user visible differences in behavior.
<hr>
Closes#29240.
ACKs for top commit:
1440000bytes:
utACK e49a7274a2
janb84:
reACK e49a7274a2
vasild:
ACK e49a7274a2
Tree-SHA512: 7162848ada4545bc07b5843d1ab6fb7e31fb26de8d6385464b7c166491cd122eac2ec5e70887c414fc136600482df8277dc0cc0541d7b7cf62c4f72e25bb6145
We don't add or maintain these, and they are of little value, as
well as having the effect of polluting diffs.
They are also wrong, i.e DEFAULT_SCRIPTCHECK_THREADS is not in
validation.h.
fa330a5e38 doc: Adjust stale MSVC bug url (MarcoFalke)
Pull request description:
The old url is stale, so use the current one. See https://github.com/bitcoin/bitcoin/pull/32552#issuecomment-2889188342
ACKs for top commit:
hebasto:
ACK fa330a5e38.
Tree-SHA512: eb7813edb85f4bac06807bf2a35caf106a0faeffcbd0a4732c57edac07db00171f720888dbb7c35fae3fd515408ea7cb8e5dcbd3cf393caec23904cb800b8907
86de8c1668 scripted-diff: test: remove 'descriptors=True' argument for `createwallet` calls (Sebastian Falbesoner)
Pull request description:
Descriptor wallets are already created by default [since v23.0](7710a31f0c/doc/release-notes/release-notes-23.0.md (L171)), but since the recent legacy wallet removal the `descriptors` parameter *must* be True for the `createwallet` RPC (see commit 9f04e02ffa), i.e. still passing it wouldn't contain any information for test readers anymore. So simply drop them in the functional tests in order to reduce code bloat. The only exception is calls to older versions, which happens in `wallet_backwards_compatibility.py` and is explicitly excluded in the scripted diff.
ACKs for top commit:
Sjors:
ACK 86de8c1668
maflcko:
lgtm ACK 86de8c1668
Tree-SHA512: 1acfae27bd960aeef9e1cf6e3f042752164a4d6869773c42df4c22c03dde0922993a3220fa14d52e75a0ff1f48c5194932b74a21427efbd496b0aaad7a2eafb2
The watch-only functionality in the GUI was only used for legacy wallets.
Watch-only descriptor wallets do not use this.
The only visible changes of this commit are:
- dropped "Spendable:" label from the overview tab
- column width cache is reset
Logging the wallet version before anything has been read from disk results
in the wrong version being logged.
Also split the last client version logging as it may not always be
present to be logged.
Descriptor wallets are already created by default since v23.0, but
since the recent legacy wallet removal this parameter *must* be True
(see commit 9f04e02ffa), i.e. still
passing it wouldn't contain any information for test readers
anymore. So simply drop them in the functional tests in order to
reduce code bloat.
-BEGIN VERIFY SCRIPT-
sed -i 's/, descriptors=True//g' $(git ls-files -- 'test/functional' ':(exclude)test/functional/wallet_backwards_compatibility.py')
sed -i '/descriptors=True,/d' ./test/functional/mempool_persist.py
-END VERIFY SCRIPT-
b104d44227 test: Remove RPCOverloadWrapper (Ava Chow)
4d32c19516 test: Replace importpubkey (Ava Chow)
fe838dd391 test: Replace usage of addmultisigaddress (Ava Chow)
d314207779 test: Replace usage of importaddress (Ava Chow)
fcc457573f test: Replace importprivkey with wallet_importprivkey (Ava Chow)
94c87bbbd0 test: Remove unnecessary importprivkey from wallet_createwallet (Ava Chow)
Pull request description:
`RPCOverloadWrapper` implemented overloads for legacy wallet only RPCs so that the same function call could be used within tests for both legacy wallets and descriptor wallets. With legacy wallets now removed, there is no need to continue to have these overloads.
For `importaddress`, `addmultisigaddress`, and `importpubkey`, the uses of these are converted to `importdescriptors`.
For `importprivkey`, a new helper function `wallet_imporprivkey` is introduced that does what the overload did. This is mainly to reduce verbosity as `importprivkey` was more widely used throughout the tests.
Some tests that used these RPCs are now also no longer relevant and have been removed.
ACKs for top commit:
Sjors:
ACK b104d44227
pablomartin4btc:
cr ACK b104d44227
rkrux:
ACK b104d44227
w0xlt:
ACK b104d44227
Tree-SHA512: ded2f73829e2ce28466d4a9738eb382783ad990daee5d1859dbc4d354e6f8eec0c483ed5ecb1287fe0dd24ac332065b733a30d71b126b841bd7cd49e9a094b6d
fa1f10a49e doc: Fix minor typos in rpc help (MarcoFalke)
fae840e94b rpc: Reject beginning newline in RPC docs (MarcoFalke)
fa414eda08 scripted-diff: Remove unused leading newline in RPC docs (MarcoFalke)
Pull request description:
It is harmless, but newlines in the beginning read a bit odd ("nReturns"). So just require them to not be present.
The diff is large, but a trivial scripted-diff.
ACKs for top commit:
fanquake:
ACK fa1f10a49e
w0xlt:
ACK fa1f10a49e
Tree-SHA512: 5d2f9632f42ec1c02814d050f223941f436e5b0df426d7d6eb93fdd0ff118d57185af07b271dd73af63735dd17231125826c0c9ce0aad36bc8829c5b050a628c
7015052eba build: remove Wsuggest-override suppression from leveldb build (fanquake)
e2c84b896f Squashed 'src/leveldb/' changes from 4188247086..113db4962b (fanquake)
Pull request description:
Pulls in
* https://github.com/bitcoin-core/leveldb-subtree/pull/51
Remove the related warning suppression.
ACKs for top commit:
l0rinc:
utACK 7015052eba
hebasto:
ACK 7015052eba, I've updated the `leveldb` subtree locally and got zero diff with this branch.
Tree-SHA512: 1ac7c8ecc9025086b429e12c22fc25f654eaf68fc9500b95341fb635cea12e7f80d69298cff120e8557a4f2f5809956a3b158cdb4db745cfa605c0df6f346423
When dealing with URI parts, it seems more consistent to use
corresponding SAFE_CHARS_URI mode in error messages.
Co-Authored-By: stickies-v <stickies-v@protonmail.com>
We need to determine if CJDNS is reachable before we parse any IPv6
addresses (for example, by the -rpcallowip setting) or an RFC4193
address might get flipped to CJDNS, which can not be used with subnets
fa2be605fe ci: Enable feature_init and wallet_reorgsrestore in valgrind task (MarcoFalke)
Pull request description:
The `fork()` isn't needed and in fact makes the forked process not react to signals (like kill or terminate), so just avoid it and run the valgrind process directly in the CI task.
Can be tested with something like:
`env -i HOME="$HOME" PATH="$PATH" USER="$USER" MAKEJOBS="-j$(nproc)" FILE_ENV="./ci/test/00_setup_env_native_valgrind.sh" ./ci/test_run_all.sh`
ACKs for top commit:
fanquake:
ACK fa2be605fe - x86_64, aarch64
Tree-SHA512: 6293447d501191598c08f0cb9fcb4ed91e4cfec11255e702a926346ef8011d6ebc0ca12e9a1c14fa53541318c4e05dee5c96dfe965dcf4cf833c9392158ab883
Currently the per-tx sigops limit standardness check (bounded by
`MAX_STANDARD_TX_SIGOPS_COST`, throwing "bad-txns-too-many-sigops"
if exceeded) is only indirectly tested with the much higher per-block
consensus limit (`MAX_BLOCK_SIGOPS_COST`), i.e. an increase in the
limit would still pass all tests. Refine that by splitting up the invalid
tx template `TooManySigops` in a per-block and a per-tx one.
The involved functional tests taking use of these templates are
`feature_block.py` and `p2p_invalid_txs.py`.
fab5a3c803 test: Remove unused verify_flags suppression (MarcoFalke)
Pull request description:
`static bool verify_flags(unsigned)` was removed in commit 80f8b92f4f
ACKs for top commit:
fanquake:
ACK fab5a3c803
hebasto:
ACK fab5a3c803, I have reviewed the code and it looks OK.
Tree-SHA512: da0cfc6ee253419c0aef316cd9c8366b744231261b755a95618ca0e777c1d95cecba8199db5486fd35079ded89c64c1a9f5b056f01dada4176b815b0d97261b7
8f4ba90b8f build: document why we check for std::system (fanquake)
Pull request description:
It's probably debatable if we support targets like iOS, but for now, document why we are checking for this standard library feature.
Trying to use `std::system` for a `aarch64-darwin-ios` target results in:
```bash
test.cpp:7:10: error: 'system' is unavailable: not available on iOS
7 | std::system("some_command");
| ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdlib.h:203:6: note: 'system' has been explicitly marked unavailable here
203 | int system(const char *) __DARWIN_ALIAS_C(system);
| ^
1 error generated.
```
ACKs for top commit:
Sjors:
ACK 8f4ba90b8f
Tree-SHA512: 219cac205b36004c607194f6956c2ce6153f192bd4349e505b52c4e511e403e195ce0f462ae10c515e67f1e95d4b1636d526c8e4376004044853b574a84df427
516f0689b5 refactor: re-enable UBSan implicit-sign-change in serialize.h (Lőrinc)
5827e93507 refactor: use consistent size type for serialization template parameters (Lőrinc)
Pull request description:
Inspired by https://github.com/bitcoin/bitcoin/pull/32154, the main goal of this PR is to reenable sanitizer checks for `serialize.h` since it's modified in a few other PRs.
ACKs for top commit:
maflcko:
review ACK 516f0689b5🎈
stickies-v:
ACK 516f0689b5, nice cleanup!
Tree-SHA512: 63da9bf1988a5b68e3c053b0ed786b8735f8f75b05763511522d1601b728b55798006e063137447615c266582622642d3226318fa83e488bd363f1756f8811e8
486bc91790 depends: bump to latest config.sub (Sebastian Falbesoner)
6880383427 depends: bump to latest config.guess (Sebastian Falbesoner)
Pull request description:
Noticed that these files were last updated from [upstream](https://git.savannah.gnu.org/gitweb/?p=config.git) quite a while ago (in 2023, see #28781), so bump them again.
Can be verified via e.g.
```
$ git clone https://git.savannah.gnu.org/git/config.git /tmp/config.git
$ diff /tmp/config.git/config.guess ./depends/config.guess
$ diff /tmp/config.git/config.sub ./depends/config.sub
```
ACKs for top commit:
fanquake:
ACK 486bc91790
Tree-SHA512: cbfd21a351a2404e5821610b6ef84d1050ea1a8045da8bfb535ef1ed49b5ad3f4140e8332d1eed545332f96d3117adc531d73aa83e19e7154fe382d041102c93
301993ebf7 init: drop -upnp (fanquake)
Pull request description:
This was slated for removal in `30.0`, so remove it.
ACKs for top commit:
i-am-yuvi:
ACK 301993ebf7
maflcko:
review ACK 301993ebf7
darosior:
tACK 301993ebf7
Tree-SHA512: 635e374c013fa08c4cda7caadc465c89bb376d3ee2c66f67a27e3ed9031844674d3e996169aaffb9b65a67b0d44d92aaec000aaf69abe3dd10fce2f4138f3e27
8f4fed7ec7 symbol-check: Add check for application manifest in Windows binaries (Hennadii Stepanov)
2bb6ab8f1b ci: Add "Get bitcoind manifest" steps to Windows CI jobs (Hennadii Stepanov)
282b4913c7 cmake: Add application manifests when cross-compiling for Windows (Hennadii Stepanov)
Pull request description:
Windows [application manifests ](https://learn.microsoft.com/en-us/windows/win32/sbscs/application-manifests) provide several benefits—such as enhanced security settings, and the ability to set a process-wide code page (required for https://github.com/bitcoin/bitcoin/pull/32380), as well as granular control over supported Windows versions. Most of these benefits lie beyond the scope of this PR and will be evaluated separately.
On the current master branch @ fc6346dbc8, the linker generates and embeds a manifest only when building with MSVC:
```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
<security>
<requestedPrivileges>
<requestedExecutionLevel level="asInvoker" uiAccess="false"></requestedExecutionLevel>
</requestedPrivileges>
</security>
</trustInfo>
</assembly>
```
However, this manifest fails validation:
```
> mt.exe -nologo -inputresource:build\bin\Release\bitcoind.exe -validate_manifest
mt.exe : general error 10100ba: The manifest is missing the definition identity.
```
This PR unifies manifest embedding for both native and cross-compilation builds.
Here is the change in the manifest on Windows:
```diff
--- bitcoind-master.manifest
+++ bitcoind-pr.manifest
@@ -1,5 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
+ <assemblyIdentity type="win32" name="org.bitcoincore.bitcoind" version="29.99.0.0"></assemblyIdentity>
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
<security>
<requestedPrivileges>
```
which effectively resolves the "missing the definition identity" error.
Finally, “Get bitcoind manifest” steps have been added to the Windows CI jobs to ensure the manifest is embedded and validated.
ACKs for top commit:
sipsorcery:
re-tACK 8f4fed7ec7.
hodlinator:
re-ACK 8f4fed7ec7
davidgumberg:
Reviewed and tested ACK 8f4fed7ec7
Tree-SHA512: 6e2dbdc77083eafdc242410eb89a6678e37b11efd786505dcd7844f0bac8f44d68625e62924a03b26549bdb4aaec5330dc608e6b4d66789f0255092e23aef6cb
0671d66a8e wallet, refactor: Convert uint256 to Txid in wallet (marcofleon)
c8ed51e62b wallet, refactor: Convert uint256 to Txid in wallet interfaces (marcofleon)
b3214cefe6 qt, refactor: Convert uint256 to Txid in the GUI (marcofleon)
Pull request description:
This is part of https://github.com/bitcoin/bitcoin/pull/32189.
Converts all instances of transactions from `uint256` to `Txid` in the wallet, GUI, and related interfaces.
ACKs for top commit:
stickies-v:
re-ACK 0671d66a8e, no changes since 65fcfbb2b38bef20a58daa6c828c51890180611d except rebase.
achow101:
ACK 0671d66a8e
furszy:
Code review ACK 0671d66a8e
Tree-SHA512: 9fd4675db63195c4eed2d14c25015a1821fb597f51404674e4879a44a9cf18f475021a97c5f62f3926b7783ade5a38567386f663acba9f5861f1f59c1309ed60
fa2c662362 build: Revert "Temporarily disable compiling `fuzz/utxo_snapshot.cpp` with MSVC" (MarcoFalke)
Pull request description:
Now that GitHub Actions has a fixed version and the Windows developers have updated their compiler, the workaround is no longer needed.
ACKs for top commit:
davidgumberg:
reACK fa2c662362
hodlinator:
ACK fa2c662362
Tree-SHA512: 952b36c917c91d78d82b5013e1df338b23f860fad7be43327150581f403050e61f748fc75557ec96fb2b115a2cc0246a506bc2ddc25e05f5a41339bd466c4b1a
Would otherwise cause an OverflowError in feature_framework_startup_failures.py when calling subprocess.run() with 60 * factor.
Fixes#32506
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
This reverts commit b2d5361002.
Also, adjust the doc to reflect the new minimum version. Versions 17.6
or 17.11 (or anything in between) may still work on a best-effor basis,
but it is not checked by CI or by developers.
It is better to reject it with an error. For example,
$ bitcoin-cli -rpcconnect=127.0.0.1:+23501 -getinfo
error: Invalid port provided in -rpcconnect: 127.0.0.1:+23501
It does not make sense and it is rejected by other parsers as well:
>>> ipaddress.ip_network("1.2.3.0/+24")
ValueError: '1.2.3.0/+24' does not appear to be an IPv4 or IPv6 network
Instead of failing during initialization when encountering a legacy wallet, skip
loading the wallet and notify the user accordingly.
This allows users to access migration functionalities without needing to manually
remove the wallet from settings.json or resort to using the bitcoin-wallet utility.
This means that GUI users will be able to use the migration button, and bitcoin-cli
users will be able to call the migratewallet RPC directly after init.
e62423d6f1 doc: Improve dependencies.md documentation (Nicola Leonardo Susca)
a3520f9d56 doc: Add dependency self-compilation info (Nicola Leonardo Susca)
d1fdc84c54 doc: Remove Linux Kernel from dep. table (Nicola Leonardo Susca)
Pull request description:
Small improvements to the `dependencies.md` documentation as a follow-up for #31634.
**Linux Kernel** does not need to be in the dependencies as it is not required for cross-compiling from other systems, and users building on Linux should not expect they can build using any EOL kernel, see: https://github.com/bitcoin/bitcoin/pull/31634#discussion_r1957123270
**Runtime dependencies** can be in a separate table to improve readability. See: https://github.com/bitcoin/bitcoin/pull/31634#issuecomment-2589412550
**Version used** is redundant as the depends package definition is already linked in the table and can thus be removed, see: https://github.com/bitcoin/bitcoin/pull/31895#discussion_r2063356972
ACKs for top commit:
maflcko:
lgtm ACK e62423d6f1🛄
hebasto:
ACK e62423d6f1.
jonatack:
ACK e62423d6f1
Tree-SHA512: 586c450aec7ece5d543bcb12796a2bb7ff459e15c8813a7b5104a38d09fc51e7e902363ff023be48273ae2b1a1b0807a439c8523b4ea2e398b76b7c9a48d0dfb
fa981b90f5 ci: Exclude failing wallet_reorgsrestore.py from valgrind task for now (MarcoFalke)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/32493
For some reason terminate or kill do not work inside the CI system under valgrind.
So disable the test for now, until a solution is found.
ACKs for top commit:
fanquake:
ACK fa981b90f5
mzumsande:
utACK fa981b90f5
Tree-SHA512: ce591fa7ffffbf757e2c15744e36a9e57300edf743400938e49fd02291f3977c551a3af1635bc7a6ccc1900d5ea150a64ee2ace46c1d765019ab11bd51035139
- Remove the "Version used" column from the dependencies tables as the
depends package definition which defines the version used is already
linked. In case a developer is interested in which PR introduced this
file/version they can use `git blame` on the package definition as
usual. This removes doc. maintenance overhead and eliminates the risk
of stale information about the "Version used", see comment:
https://github.com/bitcoin/bitcoin/pull/31895#discussion_r2063356972
- Separate dependency tables into build-time and run-time tables for
easier distinction of the two and to avoid repeating the same
information ("No"/"Yes") for better readability.
- Order dependencies alphabetically
The `dependencies.md` should mention that it is possible to self-compile
the dependencies and reference `depends/README.md` for instructions.
Also mention full path to `/doc/build-*.md` for clarity.
Previously, the `pruneblockchain` RPC help output included only the method signature and arguments, with no top-level description explaining its purpose or constraints.
This PR adds a concise top-level description, improving documentation consistency and alerting users to the potential impacts of using the command.
It is only used in test. There it is problematic, because it sometimes
relies on m_default_address_type. If the default were changed to
BECH32M, those tests would fail the assert(false).
So just use PKHash{} in all tests and remove GetDestinationForKey.
Windows application manifests provide several benefits. However, on the
master branch, the linker generates and embeds manifests only when
building with MSVC.
This change unifies manifest embedding for both native and
cross-compilation.
We would only modify the parent process' first --tmpdir arg.
Now we tack on an additional --tmpdir after the parent's arguments. Also simplifies the code.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
75a185ea3d test: add skip_if_running_under_valgrind() (fanquake)
Pull request description:
Enable it in the USDT tests. The context (from 0xB10C):
> every time the tracepoint is reached a SIGTRAP is fired.
> No matter the tracepoint contents, even with an empty one.
> Valgrind intercepts SIGTRAP and aborts.
See discussion in #32374.
ACKs for top commit:
maflcko:
lgtm ACK 75a185ea3d
willcl-ark:
ACK 75a185ea3d
Tree-SHA512: 7f45c3049ab39cc514024067bd6ac26598e99202c114b48459834c26c2e1273fa58af693878298e628a10c561b954850e49e76b39567b771bb0c0534a063a524
3b824169c7 doc: remove Carls substitute server from Guix docs (fanquake)
Pull request description:
This no-longer exists. Use one of the other Guix servers in the example.
ACKs for top commit:
achow101:
ACK 3b824169c7
hebasto:
ACK 3b824169c7, the listed substitute servers are the same as in https://guix.gnu.org/manual/en/html_node/Official-Substitute-Servers.html.
Tree-SHA512: dc3a362ccaa9ce8039d3c02158de9cd71082eb4dd790368bfb11c2942a5aae57e67779b5ff3108b532c4fb765811bd9e145eedb390fc48b52b43d334d5864865
a0eed55398 run_command: Enable close_fds option to avoid lingering fds (Luke Dashjr)
c7c356a448 cpp-subprocess: Iterate through /proc/self/fd for close_fds option on Linux (Luke Dashjr)
4f5e04da13 Revert "remove unneeded close_fds option from cpp-subprocess" (Luke Dashjr)
Pull request description:
Picks up stale #30756, while addressing my fallback comment (https://github.com/bitcoin/bitcoin/pull/30756#discussion_r2030844440).
> Currently, RunCommandParseJSON runs its target with whatever fds happen to be open inherited on POSIX platforms. I don't think there's any practical scenario where this is a problem right now, but there's a lot of potential for weird problems (eg, if a process manages to outlive bitcoind - perhaps it's hanging - the listening port(s) won't get released and starting bitcoind again will fail). It's also a potential security issue if a child process is intended to be sandboxed at some point. Not to mention plain ugly :)
>
> cpp-subprocess has a feature to address this called close_fds. Not sure why it was removed in https://github.com/bitcoin/bitcoin/pull/29961 rather than fixing this during the migration, but this PR restores it, enables it for RunCommandParseJSON, and optimises it by iterating over /proc/self/fd/ like most other libraries do these days ([eg, glib]> (487b1fd20c/glib/gspawn.c (L1094))) since iterating all possible fd numbers [has been found to be problematic](https://bugzilla.redhat.com/show_bug.cgi?id=1537564).
>
> (Equivalent to https://github.com/bitcoin/bitcoin/pull/22417 was for boost::process)
ACKs for top commit:
achow101:
ACK a0eed55398
hebasto:
ACK a0eed55398, tested on Ubuntu 25.04:
vasild:
ACK a0eed55398
Tree-SHA512: 7dc1cb6cc1f45ff7c4f53512e400baad1a033b4ebf14ba6f6ffa38588314932d6d01ef67b197f081e8202bb802659ac6a87998277797721d6d7b20efde8e9a6b
5bf91ba880 wallet: Drop unused fFromMe from CWalletTx (David Gumberg)
Pull request description:
This has been unused since commit fe52346, this is a re-opening of #9351.
ACKs for top commit:
maflcko:
lgtm ACK 5bf91ba880
achow101:
ACK 5bf91ba880
Tree-SHA512: b9a84f27b6cfe7796dcf629be6a8e01a97d931ea81ef088951d54d6691ffe79d22138baacc632375093cf3176a22c265e30a80f1f63c3bc620d08bf16f6a488f
faf9082a5f test: Fix whitespace in prevector_tests.cpp (MarcoFalke)
fa7f04c8a7 refactor: Remove UB in prevector reverse iterators (MarcoFalke)
Pull request description:
`rend()` creates a pointer with offset `-1`. This is UB, according to the C++ standard: https://eel.is/c++draft/expr.add#4:
When an expression J that has integral type is added to [...] an
expression P of pointer type, the result has the type of P.
... if P points to a (possibly-hypothetical) array element i of an
array object x with n elements [...] the expressions P + J and J + P
(where J has the value j) point to the (possibly-hypothetical) array
element i+j of x if 0≤i+j≤n [...]
Otherwise, the behavior is undefined.
Also, it is unclear why the functions exist at all, when stdlib utils such as `std::reverse_iterator{it}` or `std::views::reverse` can be used out of the box.
So remove them, along with the ubsan suppressions, that are no longer used.
I've tagged this a refactor, because the code was always dead (unused outside of tests). And since commit 2925bd537c it was completely dead. Also, I could not find a sanitizer that detects this type of UB.
ACKs for top commit:
l0rinc:
tested ACK faf9082a5f
achow101:
ACK faf9082a5f
stickies-v:
ACK faf9082a5f, nice find.
theuni:
utACK faf9082a5f
Tree-SHA512: 31511d520a1c0fdd65c2e5f1a8ef6fd17464303b6bff88a5d9d9577adfee849d431deb510882b6f4e15e8fb7168861bc0d26fca3bed4278f57a9d6e7b1235dce
Since the sighash type field is written for atypical sighash types, we
can look at that field to figure out whether the psbt contains
unnecessary transactions.
Instead of having the caller have to figure out the correct sane default
to provide to FillPSBT, have FillPSBT do that by having it take the
sighash type as an optional. This further allows it to distinguish
between an explicit sighash type being provided and expecting the
default value to be used.
SignPSBTInput will need to report the specific things that caused an
error to callers, so change it to return a PSBTError. Additionally some
callers will now check the return value and report an error to the user.
Currently, this should not change any behavior as the things that
SignPBSTInput will error on are all first checked by its callers.
4b24186756 test: add test for decoding PSBT with MuSig2 PSBT key types (BIP 373) (Sebastian Falbesoner)
8ba245cb83 test: add constants for MuSig2 PSBT key types (BIP 373) (Sebastian Falbesoner)
Pull request description:
This PR is a follow-up to #31247 (see https://github.com/bitcoin/bitcoin/pull/31247#pullrequestreview-2427834909) and adds a functional test for decoding PSBTs (using the `decodepsbt` RPC) with MuSig2 per-input and per-output types. The first commit adds the new MuSig2 key types to the test frameworks and extends the PSBT serialization to cope with lists of bytestrings.
ACKs for top commit:
achow101:
ACK 4b24186756
rkrux:
re-ACK 4b24186
Tree-SHA512: f12919f71b3fff74df1d7ddaa8db455b1b139f7abd51d7f3fa5d750fc7dd613454b438c4e0dedad679476d414fa1da43ef1121e486b0bdfd97d5ef8bdf37f060
62fc42d475 interfaces: refactor: move `waitTipChanged` implementation to miner (ismaelsadeeq)
c39ca9d4f7 interfaces: move getTip implementation to miner (Sjors Provoost)
720f201e65 interfaces: refactor: move `waitNext` implementation to miner (ismaelsadeeq)
e6c2f4ce7a interfaces: refactor: move `submitSolution` implementation to miner (ismaelsadeeq)
02d4bc776b interfaces: remove redundant coinbase fee check in `waitNext` (ismaelsadeeq)
Pull request description:
#### Motivation
In [Internal interface guidelines](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#internal-interface-guidelines)
It's stated that
> Interface method definitions should wrap existing functionality instead of implementing new functionality. Any substantial new node or wallet functionality should be implemented in [src/node/](https://github.com/bitcoin/bitcoin/blob/master/src/node) or [src/wallet/](https://github.com/bitcoin/bitcoin/blob/master/src/wallet) and just exposed in [src/interfaces/](https://github.com/bitcoin/bitcoin/blob/master/src/interfaces) instead of being implemented there, so it can be more modular and accessible to unit tests.
However the some methods in the newly added `BlockTemplateImpl` and `MinerImpl` classes partially enforces this guideline, as the implementations of the `submitSolution`, `waitNext`, and `waitTipChanged` methods reside within the class itself.
#### What the PR Does
This PR introduces a simple refactor by moving certain method implementations from `BlockTemplateImpl` into the miner module. It introduces three new functions:
1. Remove rundundant coinbase fee check in `waitNext`
2. **`AddMerkleRootAndCoinbase`**: Computes the block's Merkle root, inserts the coinbase transaction, and sets the Merkle root in the block. This function is called by `submitSolution` before the block is submitted for processing.
3. **`WaitAndCreateNewBlock`**: Returns a new block template either when transaction fees reach a certain threshold or when a new tip is detected. If a timeout is reached, it returns `nullptr`. The `waitNext` method in `BlockTemplateImpl` now simply wraps this function.
4. Move `GetTip` implementation to miner.
5. **`WaitTipChanged`**: Returns the tip when the chain it changes, or `nullopt` if a timeout or interrupt occurs. The `waitTipChanged` method in `MinerImpl` now calls `GetTip` after invoking `ChainTipChanged`, and returns the tip.
#### Behavior Change
- We now only `Assert` for a valid chainman and notifications pointer once.
ACKs for top commit:
achow101:
ACK 62fc42d475
Sjors:
ACK 62fc42d475
ryanofsky:
Code review ACK 62fc42d475. Lots of suggest suggest changes made since last review, altering function names and signatures and also adding new commit to drop negative fee handling. I like the idea of making the wait function return a BlockRef, that is clearer than what I suggested. Left some comments below but they are not important and this looks good as-is
Tree-SHA512: 502632f94ced81f576b2c43cf015f1527e2c259e6ca253f670f5a6889171e2246372b4e709575701afa3f01d488d6633557fef54f48fe83bbaf1836ac5326c4f
Since CWallet::chainStateFlushed is now no-op, this test no longer tests
the concurrent writes scenario. There are no other cases where multiple
DatabaseBatches are open at the same time.
StopWallets, which was being called prior to UnloadWallets, performs an
unnecessary database closing step. This causes issues in UnloadWallets
which does additional database cleanups. Since the database closing step
is unnecessary, StopWallets is removed, and UnloadWallets is now called
from WalletLoaderImpl::stop.
chainStateFlushed is no longer needed since the best block is updated
after a block is scanned. Since the chainstate being flushed does not
necessarily coincide with the wallet having processed said block, it
does not entirely make sense for the wallet to be recording that block
as its best block, and this can cause race conditions where some blocks
are not processed. Thus, remove this notification.
Migrating a wallet requires having a bestblock record. This is always
the case in normal operation as such a record is always written on
wallet loading if it didn't already exist. However, within the unit
tests and benchmarks, this is not guaranteed. Since migration requires
the record, WalletMigration needs to also add this record before the
benchmark.
The only reason to call chainStateFlushed during wallet loading is to
ensure that the best block is written. Do these writes explicitly to
prepare for removing chainStateFlushed, while also ensuring that the
wallet's in memory state tracking is written to disk.
Additionally, after rescanning on wallet loading, instead of writing
the locator for the current chain tip, write the locator for the last
block that the rescan had scanned. This ensures that the stored best
block record matches the wallet's current state.
Any blocks dis/connected during the rescan are processed after the
rescan and the last block processed will be updated accordingly.
When a block is connected, if the new block had anything relevant to the
wallet, update the best block record on disk. If not, also sync the best
block record to disk every 144 blocks.
Also reuse the new WriteBestBlock method in BackupWallet.
a04f17a188 doc: warn that CheckBlock() underestimates sigops (Sjors Provoost)
Pull request description:
Counting sigops in the witness requires context that `CheckBlock()` does not have, so it only counts sigops for non-segwit transactions.
It's useful to document, but it should not be a problem.
The commit message contains some historical context.
ACKs for top commit:
ismaelsadeeq:
ACK a04f17a188
ryanofsky:
Code review ACK a04f17a188
Tree-SHA512: 26528367a7f3cfa8540ef0b90f7aa912c8f0bc057428f20a1fd1d4e232dac77747bc20044f0fcb0ffab8a2e1fb3dbe3dab46be749553a917744ddc7a829025cb
Made every signed/unsigned conversion in the serialization helpers explicit so the UBSan `implicit-sign-change` check passes and the `serialize.h` suppression can be dropped.
For consistency, a few other simple changes were also applied to the serialization helpers:
* remove redundant `inline` on function templates;
* unify formatting to make the differences between similar methods obvious.
- This commit creates a function `WaitTipChanged` that waits for the connected
tip to change until timeout elapsed.
- This function is now used by `waitTipChanged`
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
e7ad86e1ca test: fix another intermittent failure in wallet_basic.py (Martin Zumsande)
07350e204d test: Fix intermittent failure in wallet_basic.py (Martin Zumsande)
Pull request description:
Fixes two rare failures that happened in the CI:
#27249:
There could be a race with outstanding TxAddedToMempool notifications being applied to the soon-to-be created wallet:
1. importdescriptors during rescan sets status to `TxStateConfirmed`
2. old `transactionAddedToMempool` notification changes status back to `TxStateInMempool`
3. If the listunspent call happens here the test will fail
4. blockConnected notification will change the status back to `TxStateConfirmed` (so it's not a persistent failure)
I could reproduce this by adding a 100 microsecond sleep to `AddToWallet()`, the fix is to add a sync, so `transactionAddedToMempool` notifications won't affect the new wallet anymore.
#32456:
During init, the test framework will start using rpc after the mempool was loaded.
It will not wait for `start()` / `postInitProcess` or outstanding `transactionAddedToMempool` notifications (which would both set the status to `TxStateInMempool`), leading to
a possible race, in which `listunspent` can be called while the tx is still in `Inactive` status.
Can be reproduced by adding two sleeps: To init before calling `start()` for the chain clients, plus to `transactionAddedToMempool` in `wallet.cpp`.
Prevent this by processing outstanding notifications.
Fixes#27249Fixes#32456
ACKs for top commit:
maflcko:
review ACK e7ad86e1ca🎩
Tree-SHA512: 1f1a11e5c8e1c6d3c39a49401c2c5122befdbbec25c0451953f5bfe8dfb53221ada552a68006e266570addda12bb16c1b9b1e49ad2198c33d91c4b96b764d73e
fa427ffcee fuzz: Properly setup wallet in wallet_fees target (MarcoFalke)
Pull request description:
`g_wallet_ptr` is destructed after the `testing_setup`. This is not supported and will lead to issues such as https://github.com/bitcoin/bitcoin/pull/30221#issuecomment-2863875857 or https://github.com/bitcoin/bitcoin/pull/32409#issuecomment-2855259932.
This could be fixed by fixing the initialization order.
However, the global wallet is also modified in the fuzz target, which is bad fuzzing practise.
So instead fix it by constructing a fresh wallet for each fuzz iteration.
ACKs for top commit:
brunoerg:
code review ACK fa427ffcee
hebasto:
ACK fa427ffcee, this change fixes the issue when building the "Debug" configuration with MSVC on Windows.
marcofleon:
Code review ACK fa427ffcee
Tree-SHA512: 161b93fc39a609cb16d9ffea7366c5e339bd01712577f0782aedff46c00f79edd2a907807ac83f9fcec687b4bbbe0fd6e6f75e32169639a310e4e7b771078b3b
Enable it in the USDT tests. The context (from 0xB10C):
> every time the tracepoint is reached a SIGTRAP is fired.
> No matter the tracepoint contents, even with an empty one.
> Valgrind intercepts SIGTRAP and aborts.
See discussion in #32374.
It's probably debatable if we support targets like iOS, but for now,
document why we are checking for this standard library feature.
Trying to use `std::system` for a `aarch64-darwin-ios` target results in
```bash
test.cpp:7:10: error: 'system' is unavailable: not available on iOS
7 | std::system("some_command");
| ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdlib.h:203:6: note: 'system' has been explicitly marked unavailable here
203 | int system(const char *) __DARWIN_ALIAS_C(system);
| ^
1 error generated.
```
rend() creates a pointer with offset -1. This is UB, according to the
C++ standard: https://eel.is/c++draft/expr.add#4:
When an expression J that has integral type is added to [...] an
expression P of pointer type, the result has the type of P.
... if P points to a (possibly-hypothetical) array element i of an
array object x with n elements [...] the expressions P + J and J + P
(where J has the value j) point to the (possibly-hypothetical) array
element i+j of x if 0≤i+j≤n [...]
Otherwise, the behavior is undefined.
Also, it is unclear why the functions exist at all, when stdlib utils
such as std::reverse_iterator{it} or std::views::reverse can be used out
of the box.
So remove them, along with the ubsan suppressions, that are no longer
used.
1e0de7a6ba fees: document non-monotonic estimation edge case (willcl-ark)
Pull request description:
Closes: https://github.com/bitcoin/bitcoin/issues/11800
In scenarios where data is available for higher targets but not for lower ones, this method *may* return lower fee rates for higher confirmation targets. This could occur if `estimateCombinedFee` returns no valid data (`-1`) for some estimates for a low target, but **does** return valid data for a higher target.
Users of this function should be aware of this potential, if unlikely, inconsistency in behaviour in data-sparse scenarios.
ACKs for top commit:
adamandrews1:
Code review ACK 1e0de7a
ismaelsadeeq:
Code review ACK 1e0de7a6ba
glozow:
ACK 1e0de7a6ba
Tree-SHA512: 161e5dafdd131570853a89491753ae39a7b725d1a86cab5a7294c2a5939da1a9a5f2c4aca0900e9ad810e828b6e0e636f256384e3d1fda6dd552da189bbbe747
0750249289 mining: document gbt_rule_value helper (Sjors Provoost)
5e87c3ec09 scripted-diff: rename gbt_force and gbt_force_name (Sjors Provoost)
Pull request description:
The term "force" is ambiguous and not used in [BIP9](https://github.com/bitcoin/bips/blob/master/bip-0009.mediawiki#getblocktemplate-changes) where there ! rule prefix is introduced.
E.g. this code is hard to read:
```cpp
if (!gbt_force) {
s.insert(s.begin(), '!');
```
Additionally, #29039 renamed `gbt_vb_name` to `gbt_force_name` which, at least for me, further increased the confusion.
This is a pure (variable rename) refactor (plus documentation) and does not change behavior.
Reminder of how to verify a scripted diff:
```sh
test/lint/commit-script-check.sh origin/master..HEAD
```
ACKs for top commit:
achow101:
ACK 0750249289
janb84:
ACK [0750249](0750249289)
musaHaruna:
ACK [0750249](0750249289)
glozow:
ACK 0750249289, seems sensible
Tree-SHA512: 8c88a273a3b36040f6c641843bd20579d0065b051aad4b39fc14f0d2af2808690dff6772bd8b1a4d9699b72279a700d2661012651bc315433a123dcc8996adaa
2aa63d511a test: Use uninvolved pruned node in feature_pruning undo test (enoch)
772ba7f9ce test: Fix nTimes typo in feature_pruning test (enoch)
Pull request description:
This PR contains two commits:
1. Fixes a typo in feature_pruning.py where 'nTimes' was incorrectly
used instead of 'nTime'. This typo caused the test to always reset
mine_large_blocks.nTime to 0, rather than only on the first run.
2. Fixes the test failure exposed by the typo fix. The
test_pruneheight_undo_presence test was failing because it was using
node 2, which is involved in reorg testing and could be on a
different chain than other nodes. The solution switches to using
node 5, which is also a pruned node but isn't involved in reorg
testing.
Testing:
- Ran test/functional/feature_pruning.py multiple times to verify
consistent passing
- Verified that the test now passes with the correct nTime variable name
- Confirmed the test behavior matches the intended functionality of
verifying pruned block availability
- Ran the full test suite to ensure the changes did not introduce any
regressions or affect other tests
Thanks to fjahr for his assistance in diagnosing the issue and
suggesting the solution.
This fixes the test failure reported in #32249
ACKs for top commit:
fjahr:
tACK 2aa63d511a
maflcko:
lgtm ACK 2aa63d511a
naiyoma:
tACK 2aa63d511a
stratospher:
tested ACK 2aa63d5. verified that `nTime` is being incremented now.
Tree-SHA512: a543528fd4eeb30e978c0b43cfa109768252edaf1f94679dbbc7fe684122c00da34224e2cc1abd2a265af1b267eef1cd34246207946cf7d8e93d2c0f11aa56d8
8673e8f019 txgraph: Special-case singletons in chunk index (optimization) (Pieter Wuille)
abdd9d35a3 txgraph: Skipping end of cluster has no impact (optimization) (Pieter Wuille)
604acc2c28 txgraph: Reuse discarded chunkindex entries (optimization) (Pieter Wuille)
c734081454 txgraph: Introduce TxGraph::GetWorstMainChunk (feature) (Pieter Wuille)
394dbe2142 txgraph: Introduce BlockBuilder interface (feature) (Pieter Wuille)
883df3648e txgraph: Generalize GetClusterRefs to support subsections (preparation) (Pieter Wuille)
c28a602e00 txgraph: Introduce TxGraphImpl observer tracking (preparation) (Pieter Wuille)
9095d8ac1c txgraph: Maintain chunk index (preparation) (Pieter Wuille)
87e74e1242 txgraph: abstract out transaction ordering (refactor) (Pieter Wuille)
2614fea17f txgraph: Add GetMainStagingDiagrams function (feature) (Pieter Wuille)
Pull request description:
Part of cluster mempool: #30289.
This adds more functionality to the txgraph module, specifically:
* `TxGraph::GetMainStagingDiagrams()`, a function to obtain feerate diagrams for both the main graph and the staged changes to it, including only the clusters that differ between the two.
* `TxGraph::GetBlockBuilder()`, a function to obtain an object which can efficiently iterate the chunks of the (main) graph from high to low chunk feerate, allowing each to be skipped or included.
* `TxGraph::GetWorstMainChunk()`, a function to obtain the last chunk that would be returned by `GetBlockBuilder()`'s returned object, intended for eviction.
ACKs for top commit:
monlovesmango:
reACK 8673e8f019
instagibbs:
reACK 8673e8f019
glozow:
reACK 8673e8f019
Tree-SHA512: 5c98c54919c44eb2f9545dfc130e54dfc25b5b54d43cf5ca9bcf46e019b9fd405a572fcd70e71e2a7c5b4b096cfd540a4d09ef1f52ba188504418682f1dfc4af
During init, the test framework will start using rpc after the
mempool was loaded. It will not wait for postInitProcess or
outstanding transactionAddedToMempool notifications, leading to
a possible race, in which listunspent is being called while the
tx is still in Inactive status. Prevent this by processing
outstanding notifications.
There could be a race with outstanding TxAddedToMempool notifications
being applied to the soon-to-be created wallet.
Fixes an intermittent timeout reproducable by adding a sleep to
AddToWallet.
- Create a new function `AddMerkleRootAndCoinbase` that compute the
block's merkle root, insert the coinbase transaction and the merkle
root into the block.
`-proxy=addr:port` specifies the proxy for all networks (except I2P).
Previously only the Tor proxy could have been specified separately via
`-onion=addr:port`.
Make it possible to specify separately the proxy for IPv4, IPv6, Tor and
CJDNS by e.g. `-proxy=addr:port=ipv6`. Or remove the proxy for a given
network, e.g. `-proxy=0=cjdns`.
Resolves: https://github.com/bitcoin/bitcoin/issues/24450
This interface lets one iterate efficiently over the chunks of the main
graph in a TxGraph, in the same order as CompareMainOrder. Each chunk
can be marked as "included" or "skipped" (and in the latter case,
dependent chunks will be skipped).
This is preparation for a next commit which will introduce a class whose
objects hold references to internals in TxGraphImpl, which disallows
modifications to the graph while such objects exist.
415650cea9 guix: move *-check.py scripts under contrib/guix (fanquake)
Pull request description:
These scripts are not meant for general developer usage. They are for use on the release binaries, which have been compiled in an environment that makes various assumptions in regards to c library, compiler options, hardening options, dependency patching etc.
Anyone is free to run these scripts against self-compiled binaries, but this isn't something we want to modify/generalize the scripts to support.
ACKs for top commit:
laanwj:
Code review ACK 415650cea9
TheCharlatan:
ACK 415650cea9
Tree-SHA512: 469cc5da9df014b4a9dc39080f31c3283641691209497cdb7cc5dc4767681a7f4b67425768e81838153ce39a26547ed94cd3be4dc27352d33960765bfac9d0e5
Support new BITCOIN_CMD environment variable in functional test to be able to
test the new bitcoin wrapper executable and run other commands through it
instead of calling them directly.
Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
Intended to make bitcoin command line features more discoverable and allow
installing new multiprocess binaries in libexec/ instead of bin/ so they don't
cause confusion.
Idea and implementation of this were discussed in
https://github.com/bitcoin/bitcoin/issues/30983
Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
These functions are just meant to serve the needs of the bitcoin wrapper
executable, and are intentionally not very general purpose so they can be
simple.
d847e17c96 doc: Fix typo (Hennadii Stepanov)
Pull request description:
A translator on Transifex noticed:
> This is the only label which has two dots: ..
> Usually we see the elipsis (…)
This PR addresses this issue.
ACKs for top commit:
janb84:
good find, ACK d847e17c96
Tree-SHA512: cbdc5f283fe30655a923c34813c97406bba85657554984c26acbfed585ce3a652bcb21348d7e0f6fd9c82ca04111c412ff62c7d4b046869d13785b713e69ab6d
5b7ed460c7 cmake: Allow `WITH_DBUS` on all Unix-like systems (Hennadii Stepanov)
Pull request description:
This PR makes the `WITH_DBUS` option available on all Unix-like systems, not just Linux, thereby fixing a regression that was overlooked during the migration from Autotools.
Note: Enabling D-Bus support on macOS still makes no sense, since the `Notificator` class uses the User Notification Center regardless:746ab19d5a/src/qt/notificator.cpp (L43-L56)
Fixes https://github.com/bitcoin/bitcoin/issues/32464.
ACKs for top commit:
laanwj:
ACK 5b7ed460c7
theStack:
tACK 5b7ed460c7
Tree-SHA512: 01f5a365a8109c273489e2d46b08d8b64b82251c4814a486561221b55d8a47cb1effc9140dbd6b2ab5f12e2966d0ea0d07a4cbf67d310001ee681b0f5b63f300
3bbdbc0a5e qt, docs: Unify term "clipboard" (Hennadii Stepanov)
Pull request description:
A translator on Transifex noticed:
> The term "system clipboard" appears twice. The term "clipboard" appears 10 times. Perhaps we could standardize on just saying "clipboard"?
This PR addresses this issue.
ACKs for top commit:
davidgumberg:
ACK 3bbdbc0a5e
pablomartin4btc:
ACK 3bbdbc0a5e
Tree-SHA512: 61a100f60890d81122a4b8ce3e2cb7d355c7fb643de3196573f7f9107c6f52fa0b3e7a4f743ce2833e8c67b9cdad3568b761d730fef5c9781f5e1c45252888c4
002b792b9a gui: decouple WalletModel from RPCExecutor (furszy)
Pull request description:
A more comprehensive fix for the issue described in #837.
Since the `WalletModel` class is unavailable when compiling without wallet support
`(-DENABLE_WALLET=0)`, the RPC executor class should not be coupled to it.
This decoupling ensures GUI compatibility with builds that omit wallet support.
This also drops an extra `#ifdef ENABLE_WALLET` block which is always good.
ACKs for top commit:
w0xlt:
Code Review ACK 002b792b9a
pablomartin4btc:
tACK 002b792b9a
BrandonOdiwuor:
tACK 002b792b9a
hebasto:
ACK 002b792b9a, I have reviewed the code and it looks OK.
Tree-SHA512: a8e6b7e9d88dd8e0ff5e2d0de91be2f85fd0559265267d3bf6cae5a37606cf1ab6bc7415d5817a11006008de362f2ca3557ba772b4e1bd9fbef5f564be3b53bb
This change makes the `WITH_DBUS` option available on all Unix-like
systems, not just Linux, thereby fixing a regression that was
overlooked during the migration from Autotools.
Note: Enabling D-Bus support on macOS still makes no sense, since the
`Notificator` class uses the User Notification Center regardless.
ab878a7e74 build: simplify *ifaddr handling (fanquake)
Pull request description:
We really just want to skip this when building for Windows. So do that,
and remove the two header checks (we also already use both of these
headers, unguarded, in the !windows part of the codebase).
Squash the two *iffaddrs defines into one, as I haven't seen an
`iffaddrs.h` that implements one, but not the other.
ACKs for top commit:
hebasto:
ACK ab878a7e74. Only addressed my [comment](https://github.com/bitcoin/bitcoin/pull/32446#discussion_r2079994126) and rebased since my recent [review](https://github.com/bitcoin/bitcoin/pull/32446#pullrequestreview-2825606189).
TheCharlatan:
ACK ab878a7e74
Tree-SHA512: 7667305df9fef4728526c7217f85b51e739ec63b38e808da51d6ae65cb6f2696afa5ba82e5a72ed4a7a9b79ffa2402640448af4392587253027122eab7618e30
a58cb3b1c1 qa: sanity check mined block have their coinbase timelocked to height (Antoine Poinsot)
8f2078af6a miner: timelock coinbase transactions (Antoine Poinsot)
788aeebf34 qa: use prev height as nLockTime for coinbase txs created in unit tests (Antoine Poinsot)
c76dbe9b8b qa: timelock coinbase transactions created in fuzz targets (Antoine Poinsot)
9c94069d8b contrib: timelock coinbase transactions in signet miner (Antoine Poinsot)
a5f52cfcc4 qa: timelock coinbase transactions created in functional tests (Antoine Poinsot)
Pull request description:
The Consensus Cleanup soft fork proposal includes enforcing that coinbase transactions set their
nLockTime field to the block height minus 1, as well as their nSequence such as to not disable the
timelock. If such a fork were to be activated by Bitcoin users, miners need to be ready to produce
compliant blocks at the risk of losing substantial amounts mining would-be invalid blocks. As miners
are unfamously slow to upgrade, it's good to make this change as early as possible.
Although Bitcoin Core's GBT implementation does not provide the `coinbasetxn` field, and mining
pool software crafts the coinbase on its own, updating the Bitcoin Core mining code is a first step
toward convincing pools to update their (often closed source) code. A possible followup is also to
introduce new fields to GBT. In addition, this first step also makes it possible to test future
Consensus Cleanup changes.
The commit making the change also updates a bunch of seemingly-unrelated tests. This is because those tests were asserting error messages based on the txid of transactions involved, and changing the coinbase transaction structure necessarily changes the txid of all tests' transactions.
ACKs for top commit:
Sjors:
Code review ACK a58cb3b1c1
achow101:
ACK a58cb3b1c1
TheCharlatan:
Re-ACK a58cb3b1c1
Tree-SHA512: a2aae009a187eb760d34435f518a895ee76c6b02a667eb030ddf6bd584da6e8eae2737d974dbf81a928d60c07bcb4820f055adc067e18d8819640db0240bb513
1ee698fde2 test: refactor: negate signature-s using libsecp256k1 (Sebastian Falbesoner)
Pull request description:
This small PR gets rid of manual mod-n inversion of the ECDSA signature-s part in unit tests (introduced a long time ago in #5256, triggered by https://github.com/bitcoin-core/secp256k1/pull/69) by using secp256k1 instead. The function wasn't available at that time, but was introduced about three years later, see https://github.com/bitcoin-core/secp256k1/pull/408. Note that as the name suggests, `secp256k1_ec_seckey_negate` is meant to be used for secret keys, but it obviously works in general for scalars modulo the group order.
ACKs for top commit:
achow101:
ACK 1ee698fde2
laanwj:
Code review ACK 1ee698fde2
w0xlt:
ACK 1ee698fde2
rkrux:
tACK 1ee698fde2
Tree-SHA512: dc36ea1572b538d11ae34e1871f310a1cda8083ffb753e93e7ee9d56e91ebd8ec78d35758dfb700254720914b734ef7a071eeef71b6239f19e1e2fb289fb5435
importprivkey was a legacy wallet only RPC which had a helper for
descriptor wallets in tests. Add wallet_importprivkey helper and use it
wherever importprivkey is used (other than backward compatibility tests)
This test was testing importprivkey behavior in a legacy wallet without
private keys. As legacy wallets no longer exist, this test case is no
longer relevant.
This gets rid of the special-casing of `strRPCUserColonPass` by hashing
cookies as well as manually provided `-rpcuser`/`-rpcpassword` with a
random salt before storing them.
Also take the opportunity to modernize the surrounding code a bit. There
should be no end-user visible differences in behavior.
Back in 2015, in #7044, we added configuration option `rpcauth` for
multiple RPC users. At the same time the old settings for single-user
configuration `rpcuser` and `rpcpassword` were "soon" to be deprecated.
The main reason for this deprecation is that while `-rpcpassword` stores
the password in plain text, `-rpcauth` stores a hash, so it doesn't
appear in the configuration in plain text.
As the options are still in active use, actually removing them is
expected to be a hassle to many, and it's not clear that is worth it. As
for the security risk, in many kinds of setups (no wallet,
containerized, single-user-single-application, local-only, etc) it is an
unlikely point of escalation.
In the end, it is good to encourage secure practices, but it is the
responsibility of the user. Log a clear warning but remove the
deprecation notice.
Closes#29240.
It is confusing that the chain client flush happens between
StopHTTPServer and StopMapPort. Also, it is unused code. Seems best to
just add it back properly when it is needed again.
These scripts are not meant for general developer usage. They are for
use on the release binaries, which have been compiled in an environment
that makes various assumptions in regards to c library, compiler
options, hardening options, patching etc.
31c5ebc400 tracing: fix invalid argument in mempool_monitor (William Casarin)
Pull request description:
The mempool_monitor tracing tool is incorrectly reading the reason as the first argument. Fix this!
Noticed this during the bitcoin++ mempool hackathon 😅
cc 0xB10C
ACKs for top commit:
0xB10C:
Code Review ACK 31c5ebc400
Tree-SHA512: 6f3d64f0f75a44e1fdcad71af8e737ce948833498cd3879ef74cbabf53e3649145b83febceca19b1662de55346c199bf4259e17f5b28cf0352aefa730e07ea63
10845cd7cc qa: Add feature_framework_startup_failures.py (Hodlinator)
28e282ef9a qa: assert_raises_message() - Stop assuming certain structure for exceptions (Hodlinator)
1f639efca5 qa: Work around Python socket timeout issue (Hodlinator)
9b24a403fa qa: Only allow calling TestNode.stop() after connecting (Hodlinator)
6ad21b4c01 qa: Include ignored errors in RPC connection timeout (Hodlinator)
879243e81f qa refactor: wait_for_rpc_connection - Treat OSErrors the same (Hodlinator)
Pull request description:
Improves handling of startup errors in functional tests and puts tests in place to ensure knock-on errors don't creep in.
- `wait_for_rpc_connection()` now appends specific failures leading up to the `Unable to connect to bitcoind` error to that error message:
`[node 0] Unable to connect to bitcoind after 60s (ignored errors: {'missing_credentials': 1, 'OSError.ECONNREFUSED': 239}, latest error: ConnectionRefusedError(111, 'Connection refused'))`
- Fixes Windows Python issue where `socket.timeout` exceptions end up with unset `errno`-fields.
- Also adds comments, refactors code, improves logging.
The underlying purpose is to ensure developer efficiency in finding root causes of test failures.
Prior iterations of the PR partially focused on fixing the same issue as #31620.
Originally inspired by #30390.
### Testing
Can be tested by reverting either faf2f2c654 or fae3bf6b87 from #31620, or the "qa: Avoid calling stop-RPC if not connected" from this PR, and running *feature_framework_startup_failures.py*.
ACKs for top commit:
l0rinc:
ACK 10845cd7cc
ryanofsky:
Code review ACK 10845cd7cc. Only changes since last review were adding a new commit tweaking assert_raises_message(), extending the new test to have a self-check, and to pass through all options to child tests instead of a hardcoded list of options. I left some cleanup suggestions below but they are not important.
Tree-SHA512: f0235c5cbb6d1bb85d8dc5de492a08a34f6edc83499cbf0a5f9a3824809ff84635888c62c9c01101e3cc9ef9f1cdee2c9ab6537fea6feeb005b29f428caf8b22
No warning is currently emitted because our reverse_lock does not enforce our
thread-safety annotations. Once it is fixed, the unlock would cause a warning.
Clang added new "capability"-based thread-safety attributes years ago, but the
old ones remain supported for backwards-compatibility.
However, while adding annotations for our reverse_lock, I noticed that there
is a difference between the unlock_function and release_capability attributes.
unlock_function actually maps to release_generic_capability, which does not
work properly when implementing a scoped unlocker.
To be consistent, the other capability-based attributes are updated here as
well. To avoid having to update our macro usage throughout the codebase, I
reused our existing ones.
Additionally, SHARED_UNLOCK_FUNCTION is added here, as a subsequent PR will
introduce annotations for shared_mutex and shared_lock.
f9dfe8d5e0 contrib: remove bdb exception from FORTIFY check (fanquake)
Pull request description:
BDB has been removed (#28710), so we no-longer need to ignore functions from BDB in this check.
Guix building this branch, and looking for `*_chk` functions across all binaries produces:
```
# nm -C * | grep -i _chk | sort | uniq
U __fdelt_chk@GLIBC_2.15
U __fprintf_chk@GLIBC_2.3.4
U __fread_chk@GLIBC_2.7
U __longjmp_chk@GLIBC_2.11
U __memcpy_chk@GLIBC_2.3.4
U __printf_chk@GLIBC_2.3.4
U __snprintf_chk@GLIBC_2.3.4
U __sprintf_chk@GLIBC_2.3.4
U __stack_chk_fail@GLIBC_2.4
U __vsnprintf_chk@GLIBC_2.3.4
```
ACKs for top commit:
achow101:
ACK f9dfe8d5e0
theuni:
utACK f9dfe8d5e0
laanwj:
Code review ACK f9dfe8d5e0
Tree-SHA512: e9491c8b348a0d777c3f7186cab48b478548654712f8b85e7bde2f8b94f3a8b52bc7be8fb1b4a486954359d3109cfb74e3485ccfff67c6546f0efcabf2eda0e0
Counting sigops in the witness and for p2sh requires
context that CheckBlock() does not have, so it only
counts a subset of sigops.
The check here was introduced by Satoshi as a "cleanup" in
f1e1fb4bde. With the attempted
introduction of OP_EVAL, it was replaced by the check in
ConnectBlock(). Commit e679ec969c
marked this code as a placeholder for backward compatibility.
Then when P2SH replaced OP_EVAL in 922e8e2929
the phrase "compatibility-breaking" was replaced by a simple
observation that before v0.6 this is how sigops were counted.
It's unclear why the check was kept and there were no review comments
about it.
We really just want to skip this when building for Windows. So do that,
and remove the two header checks (we also already use both of these
headers, unguarded, in the !windows part of the codebase).
Squash the two *iffaddrs defines into one, as I haven't seen an
iffaddrs.h that implements one, but not the other.
edde96376a cmake: Respect user-provided configuration-specific flags (Hennadii Stepanov)
Pull request description:
This PR addresses [this](https://github.com/bitcoin/bitcoin/issues/31491#issuecomment-2542140874) comment:
> I suppose that should only happen if the `-O3` isn't coming from an explicitly set `CMAKE_CXX_FLAGS_RELEASE`.
With this PR:
```
$ cmake -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS_RELEASE="-O3"
<snip>
C++ compiler flags .................... -O3 -std=c++20 -fPIC -fno-extended-identifiers -fdebug-prefix-map=/home/hebasto/dev/bitcoin/src=. -fmacro-prefix-map=/home/hebasto/dev/bitcoin/src=. -fstack-reuse=none -Wall -Wextra -Wformat -Wformat-security -Wvla -Wredundant-decls -Wdate-time -Wduplicated-branches -Wduplicated-cond -Wlogical-op -Woverloaded-virtual -Wsuggest-override -Wimplicit-fallthrough -Wunreachable-code -Wbidi-chars=any -Wundef -Wno-unused-parameter -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -Wstack-protector -fstack-protector-all -fcf-protection=full -fstack-clash-protection
Linker flags .......................... -O3 -fstack-reuse=none -fstack-protector-all -fcf-protection=full -fstack-clash-protection -Wl,-z,relro -Wl,-z,now -Wl,-z,separate-code -fPIE -pie
```
and
```
$ cmake -B build -DCMAKE_BUILD_TYPE=Release
<snip>
C++ compiler flags .................... -O2 -std=c++20 -fPIC -fno-extended-identifiers -fdebug-prefix-map=/home/hebasto/dev/bitcoin/src=. -fmacro-prefix-map=/home/hebasto/dev/bitcoin/src=. -fstack-reuse=none -Wall -Wextra -Wformat -Wformat-security -Wvla -Wredundant-decls -Wdate-time -Wduplicated-branches -Wduplicated-cond -Wlogical-op -Woverloaded-virtual -Wsuggest-override -Wimplicit-fallthrough -Wunreachable-code -Wbidi-chars=any -Wundef -Wno-unused-parameter -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -Wstack-protector -fstack-protector-all -fcf-protection=full -fstack-clash-protection
Linker flags .......................... -O2 -fstack-reuse=none -fstack-protector-all -fcf-protection=full -fstack-clash-protection -Wl,-z,relro -Wl,-z,now -Wl,-z,separate-code -fPIE -pie
```
When calling `cmake` repeatedly using the same build directory, each newly provided `CMAKE_CXX_FLAGS_RELEASE` value will be accommodated. In such a scenario, if the user wishes to revert to the build system defaults, they should unset the `CMAKE_CXX_FLAGS_RELEASE` variable by passing `-UCMAKE_CXX_FLAGS_RELEASE` to `cmake`.
---
This PR does not aim to resolve _all_ issues mentioned in https://github.com/bitcoin/bitcoin/issues/31491.
ACKs for top commit:
purpleKarrot:
ACK edde96376a
janb84:
ACK [edde963](edde96376a)
ryanofsky:
Code review ACK edde96376a
Tree-SHA512: 1fbc879bd02cf0be726ced490f65985e728f0686ccb3a32cd38787b56377aa666e1965448e5069515abc814df49a0083c8000bc3f6f322f5f395695638168fb6
1372eb09c5 doc: swap "Docker image" for "container image" (fanquake)
Pull request description:
I haven't used Docker for some time (now Podman), and the images are generic, so just use "container image". I'll be pushing some changes to https://github.com/fanquake/core-review/tree/master/guix, to reflect this.
ACKs for top commit:
janb84:
ACK 1372eb09c5
laanwj:
ACK 1372eb09c5
hebasto:
ACK 1372eb09c5.
Tree-SHA512: 45bb74d25a0faf7e5c3666d6897fb6b999144308c43cdf8a290d3a4210285b1e95286d27bb3d90bc50be4784c2242ad3f93794086f4634439a46a48ff68c7343
fa24fdcb7f lint: Remove string exclusion from locale check (MarcoFalke)
Pull request description:
The exclusion isn't needed. In fact, it prevents detection of `"bla" + wrong()`.
For example, the following is not detected:
```diff
diff --git a/src/wallet/rpc/addresses.cpp b/src/wallet/rpc/addresses.cpp
index 1c2951deee..c1209013e5 100644
--- a/src/wallet/rpc/addresses.cpp
+++ b/src/wallet/rpc/addresses.cpp
@@ -336,7 +336,8 @@ RPCHelpMan addmultisigaddress()
RPCHelpMan keypoolrefill()
{
return RPCHelpMan{"keypoolrefill",
- "\nFills the keypool."+
+ "\nRefills each descriptor keypool in the wallet up to the specified number of new keys.\n"
+ "By default, descriptor wallets have 4 active ranged descriptors (\"legacy\", \"p2sh-segwit\", \"bech32\", and \"bech32m\"), each with " + std::to_string(DEFAULT_KEYPOOL_SIZE) + " entries.\n" +
HELP_REQUIRING_PASSPHRASE,
{
{"newsize", RPCArg::Type::NUM, RPCArg::DefaultHint{strprintf("%u, or as set by -keypool", DEFAULT_KEYPOOL_SIZE)}, "The new keypool size"},
```
Fix the script by detecting it.
ACKs for top commit:
laanwj:
Code review ACK fa24fdcb7f.
rkrux:
ACK fa24fdcb7f
w0xlt:
ACK fa24fdcb7f
Tree-SHA512: cb7e6ed9fec5d2089e94031329ebf26b83a1814ffbbbca94f7527c127bc759d13c0f4ea79b71ff7f5f009d071dcf01958c8921163d6dc5e1ae6256cc40b57eea
4e8ab5e00f crypto: disable ASan for sha256_sse4 with Clang (fanquake)
Pull request description:
This also fails to compile when optimisations are being used, see: https://github.com/bitcoin/bitcoin/issues/31913.
So just disable ASan under any optimisation level.
Closes#31913.
ACKs for top commit:
maflcko:
lgtm ACK 4e8ab5e00f
davidgumberg:
Tested ACK 4e8ab5e00f
laanwj:
Code review ACK 4e8ab5e00f
Tree-SHA512: 680fb424f43b35730e03e0c7443c80445a2cf423d4f9161414ea22fea0b955f49197f8a96d1241896d981c6c13814d3eb7b5e4d8c9138813fb69e437ac4768ea
ff35a4b021 docs: Improve `keypoolrefill` RPC docs (w0xlt)
Pull request description:
Update `keypoolrefill` RPC docs to make it clear that descriptor wallets have 4 ScriptPubKeyManagers by default and each of them is updated in this command, as suggested https://github.com/bitcoin/bitcoin/issues/29924#issuecomment-2849321859
Closes https://github.com/bitcoin/bitcoin/issues/29924
ACKs for top commit:
achow101:
ACK ff35a4b021
brunoerg:
code review ACK ff35a4b021
Tree-SHA512: b6b9abe3fecebf9551b4ce9280794292c6ac44ccaeb2b9d60eeb4b2c177fe8372d0fe103f99c9cc0baeb2559ec019d1c495c233f24a600531153a38eeacb9670
Also, support serialization of lists of byte-strings as PSBTMap values,
which will be simply concatenated without any compact-size prefixes
(neither for the individual items nor for the size of the list).
Using helper variables has two issues:
1. They contaminate the global namespace of the main build script.
2. They can be used as `set(var)`, effectively exposing a cache variable
`var`, which makes the toolchain file susceptible to the build
environment.
In most cases throughout the wallet, the implicit conversion from `Txid` to
`const uint256&` works. However, `commitBumpTransaction` requires a `uint256&`
out parameter, so `bumped_txid` in `feebumper::CommitTransaction` is also
updated here to use `Txid`.
de054df6dc contrib: Remove legacy wallet RPCs from bash completions (Ava Chow)
5dff04a1bb legacy spkm: Make IsMine() and CanProvide() private and migration only (Ava Chow)
c0f3f3264f wallet: Remove unused db functions (Ava Chow)
83af1a3cca wallet: Delete LegacySPKM (Ava Chow)
8ede6dea0c wallet, rpc: Remove legacy wallet only RPCs (Ava Chow)
4de3cec28d test: rpcs disabled for descriptor wallets will be removed (Ava Chow)
84f671b01d test: Run multisig script limit test (Ava Chow)
810476f31e test: Remove unused options and variables, correct comments (Ava Chow)
04a7a7a28c build, wallet, doc: Remove BDB (Ava Chow)
Pull request description:
The final step of #20160.
A bare minimum of legacy wallet code is kept in order to perform wallet migration. Migration of legacy wallets uses the independent BDB parser and a minimal `LegacyDataSPKM` that allows the legacy data to be loaded so that the migration can be completed.
BDB has been removed as a dependency and documentation have been updated to reflect that.
ACKs for top commit:
Sjors:
re-ACK de054df6dc
maflcko:
re-ACK de054df6dc🔗
w0xlt:
reACK de054df6dc
rkrux:
Concept ACK de054df6dc
Tree-SHA512: 16a6c265bc1ada5e7a5ef9b95f0ff65015672ca46d9a43b7e10d60e9e085052e9bbfe01ac3e494cc606afb652a1b476b10e434d13e9877b67d2cb0196a9bd190
The string exclusion would fail to detect `"bla" + wrong()`.
Also, remove /* */ comment exclusion, which would fail to detect stuff
like `/* bla */ wrong()`.
Instead, require the function to be called by adding \\( to the regex.
Finally, also remove the section in the dev notes, because:
* It was outdated and missing some functions such as std::to_string in
the list.
* The maintenance overhead of having to update two places is fragile and
questionable.
* Many other linters are also not mentioned in the dev notes, even
though they are important.
* A dev (and CI) is more likely to run the linters than to read the dev
notes.
* The dev notes are more than 1000 lines of dense information. It would
be easier to digest if they focused on the important stuff that is not
checked by automated tools.
fa4804009c fuzz: Remove unused TimeoutExpired catch in fuzz runner (MarcoFalke)
Pull request description:
Currently, the way to check for libFuzzer is to search the stderr of the fuzz executable when passed `-help=1` for the string `libFuzzer`. See also 14b8dfb2bd/contrib/devtools/deterministic-fuzz-coverage/src/main.rs (L90-L101)
The python test runner additionally includes a timeout catch, which was needed before the plain `read_file` fallback was implemented, see 14b8dfb2bd/src/test/fuzz/fuzz.cpp (L251).
However, it is no longer needed and the printed error message would be wrong, so remove it.
(side-note: On Windows the fuzz executable seems to time out when an assert is hit in a debug build, see https://github.com/bitcoin/bitcoin/issues/32341#issuecomment-2842716175. However, no one is running fuzz debug on Windows. Also, the newly added debug logging is a preferable replacement in this case anyway.)
ACKs for top commit:
kevkevinpal:
crACK [fa48040](fa4804009c)
Crypt-iQ:
crACK fa4804009c
marcofleon:
crACK fa4804009c
brunoerg:
code review ACK fa4804009c
Tree-SHA512: 64f5e3862fece9ab2b6592615b72b81e9c087dcd394b1d062a96df0d88d8b5999674f0faa1165a5998c05289c1874e29311d7b24d84fee9bc6c46d1662d29e4d
b5f580c580 scripted-diff: adapt script error constant names in feature_taproot.py (Sebastian Falbesoner)
Pull request description:
While reviewing #31622 I noticed that the constant name `(SCRIPT_)ERR_SIG_HASHTYPE` is used for two different script verification error codes, namely one for legacy and one for Schnorr signatures:
eba5f9c4b6/src/script/script_error.cpp (L56-L57)eba5f9c4b6/test/functional/feature_taproot.py (L600)
In order to resolve this confusion, this PR adapts all script error constant names in the functional tests (currently only in feature_taproot.py) to the ones used in our C++ codebase (see [script_error.cpp](https://github.com/bitcoin/bitcoin/blob/master/src/script/script_error.cpp)) with a scripted diff. This also makes checking whether we have test coverage for a certain script error easier.
ACKs for top commit:
jamesob:
crACK b5f580c580
achow101:
ACK b5f580c580
rkrux:
tACK b5f580c580
stratospher:
ACK b5f580c. liked the consistency in script error names.
Tree-SHA512: bc0ccec70bc3cb6ce51ce8e27a5e54770d1bb93c1db5a9c815caa25f3d96ebb382104bd9b51626f501d4f5b95148db8d20c806a27153e9bb9cf823a20d3046c0
85368aafa0 test: Run simple tests at various feerates (Murch)
d610951c15 test: Recreate BnB iteration exhaustion test (Murch)
2a1b2754f1 test: Remove redundant repeated test (Murch)
4781f5c8be test: Recreate simple BnB failure tests (Murch)
a94030ae98 test: Recreate BnB clone skipping test (Murch)
7db6f012c0 test: Move BnB feerate sensitivity tests (Murch)
2bafc46261 test: Recreate simple BnB success tests (Murch)
Pull request description:
This PR is splitting off some of the improvements made in #28985 and starts addressing the issues raised in #27754.
I aim to completely replace `coinselector_tests` with `coinselection_tests`. The goal is to generally use coins created per a nominal _effective value_ so we can get away from testing with `CoinSelectionParams` that are non-representative and effectuate counterintuitive behavior such as `feerate = 0` or `cost_of_change = 0`
ACKs for top commit:
achow101:
ACK 85368aafa0
monlovesmango:
ACK 85368aafa0
w0xlt:
ACK 85368aafa0
Tree-SHA512: 1a984837b4efddc0d8abe11668898fb207fb539e784bf911d4038211274b82e0fe1f8fffe7e5a19e0e013ccb7dc40e3f62d853a2a729980d0d935e66f12b9156
84de8c93e7 ci: Add `deploy` target for native macOS CI job (Hennadii Stepanov)
fad57e9e0f build: Fix `macdeployqtplus` after switching to Qt 6 (Hennadii Stepanov)
938208d91a build: Resolve `@rpath` in `macdeployqtplus` (Hennadii Stepanov)
Pull request description:
Homebrew's Qt 6 package — namely `qt` or `qt@6` — introduces a few differences that must be properly handled by the `macdeployqtplus` script:
1. Use of `@rpath` references:
```
% objdump --macho --dylibs-used $(brew --prefix qt@5)/Frameworks/QtGui.framework/QtGui
/usr/local/opt/qt@5/Frameworks/QtGui.framework/QtGui:
/usr/local/opt/qt@5/lib/QtGui.framework/Versions/5/QtGui (compatibility version 5.15.0, current version 5.15.16)
/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 2575.30.19)
/System/Library/Frameworks/Metal.framework/Versions/A/Metal (compatibility version 1.0.0, current version 367.6.0)
/usr/local/Cellar/qt@5/5.15.16_1/lib/QtCore.framework/Versions/5/QtCore (compatibility version 5.15.0, current version 5.15.16)
/System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration (compatibility version 1.0.0, current version 1.0.0)
/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
/System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics (compatibility version 64.0.0, current version 1889.2.7)
/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 3208.0.0)
/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
/System/Library/Frameworks/AGL.framework/Versions/A/AGL (compatibility version 1.0.0, current version 1.0.0)
/usr/local/opt/libpng/lib/libpng16.16.dylib (compatibility version 64.0.0, current version 64.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.12)
/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 65.0.0)
/usr/local/opt/md4c/lib/libmd4c.0.dylib (compatibility version 0.0.0, current version 0.5.2)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1800.105.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1351.0.0)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 3208.0.0)
/System/Library/Frameworks/CoreText.framework/Versions/A/CoreText (compatibility version 1.0.0, current version 844.2.0)
/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
% objdump --macho --dylibs-used $(brew --prefix qt@6)/Frameworks/QtGui.framework/QtGui
/usr/local/opt/qt/Frameworks/QtGui.framework/QtGui:
/usr/local/opt/qt/lib/QtGui.framework/Versions/A/QtGui (compatibility version 6.0.0, current version 6.9.0)
/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 2575.30.19)
/System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 170.0.0)
/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
/System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO (compatibility version 1.0.0, current version 1.0.0)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 3208.0.0)
/System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics (compatibility version 64.0.0, current version 1889.2.7)
/System/Library/Frameworks/CoreText.framework/Versions/A/CoreText (compatibility version 1.0.0, current version 844.2.0)
/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 3208.0.0)
/System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore (compatibility version 1.2.0, current version 1.11.0)
/System/Library/Frameworks/Metal.framework/Versions/A/Metal (compatibility version 1.0.0, current version 367.6.0)
/usr/local/opt/glib/lib/libglib-2.0.0.dylib (compatibility version 8401.0.0, current version 8401.0.0)
@rpath/QtDBus.framework/Versions/A/QtDBus (compatibility version 6.0.0, current version 6.9.0)
/System/Library/Frameworks/AGL.framework/Versions/A/AGL (compatibility version 1.0.0, current version 1.0.0)
/usr/local/opt/libpng/lib/libpng16.16.dylib (compatibility version 64.0.0, current version 64.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.12)
/usr/local/opt/harfbuzz/lib/libharfbuzz.0.dylib (compatibility version 61100.0.0, current version 61100.0.0)
/usr/local/opt/md4c/lib/libmd4c.0.dylib (compatibility version 0.0.0, current version 0.5.2)
/usr/local/opt/freetype/lib/libfreetype.6.dylib (compatibility version 27.0.0, current version 27.2.0)
/usr/local/opt/glib/lib/libgthread-2.0.0.dylib (compatibility version 8401.0.0, current version 8401.0.0)
@rpath/QtCore.framework/Versions/A/QtCore (compatibility version 6.0.0, current version 6.9.0)
/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
/System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration (compatibility version 1.0.0, current version 1.0.0)
/System/Library/Frameworks/UniformTypeIdentifiers.framework/Versions/A/UniformTypeIdentifiers (compatibility version 1.0.0, current version 709.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1800.105.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1351.0.0)
/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 1226.0.0)
/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
```
2. Different directory layout:
```
% ls -l $(brew --prefix qt@5)/
total 544
drwxr-xr-x 79 hebasto admin 2528 13 Nov 06:22 Frameworks
-rw-r--r-- 1 hebasto admin 7533 16 Apr 09:09 INSTALL_RECEIPT.json
-rw-r--r-- 1 hebasto admin 22961 13 Nov 06:22 LICENSE.FDL
-rw-r--r-- 1 hebasto admin 36363 13 Nov 06:22 LICENSE.GPL3-EXCEPT
-rw-r--r-- 1 hebasto admin 15351 13 Nov 06:22 LICENSE.GPLv2
-rw-r--r-- 1 hebasto admin 35641 13 Nov 06:22 LICENSE.GPLv3
-rw-r--r-- 1 hebasto admin 26828 13 Nov 06:22 LICENSE.LGPLv21
-rw-r--r-- 1 hebasto admin 8174 13 Nov 06:22 LICENSE.LGPLv3
-rw-r--r-- 1 hebasto admin 106262 13 Nov 06:22 LICENSE.QT-LICENSE-AGREEMENT
-rw-r--r-- 1 hebasto admin 3842 13 Nov 06:22 README
drwxr-xr-x 57 hebasto admin 1824 16 Apr 09:09 bin
drwxr-xr-x 4 hebasto admin 128 13 Nov 06:22 doc
drwxr-xr-x 95 hebasto admin 3040 13 Nov 06:22 include
drwxr-xr-x 119 hebasto admin 3808 16 Apr 09:09 lib
drwxr-xr-x 8 hebasto admin 256 13 Nov 06:22 libexec
drwxr-xr-x 79 hebasto admin 2528 16 Apr 09:09 mkspecs
drwxr-xr-x 15 hebasto admin 480 13 Nov 06:22 phrasebooks
drwxr-xr-x 31 hebasto admin 992 13 Nov 06:22 plugins
drwxr-xr-x 28 hebasto admin 896 13 Nov 06:22 qml
-rw-r--r-- 1 hebasto admin 6952 16 Apr 09:09 sbom.spdx.json
drwxr-xr-x 3 hebasto admin 96 13 Nov 06:22 share
drwxr-xr-x 347 hebasto admin 11104 13 Nov 06:22 translations
% ls -l $(brew --prefix qt@6)/share/qt/
total 0
drwxr-xr-x 4 hebasto admin 128 30 Mar 09:49 doc
drwxr-xr-x 35 hebasto admin 1120 16 Apr 09:16 libexec
drwxr-xr-x 167 hebasto admin 5344 30 Mar 09:49 metatypes
drwxr-xr-x 70 hebasto admin 2240 16 Apr 09:16 mkspecs
drwxr-xr-x 178 hebasto admin 5696 30 Mar 09:49 modules
drwxr-xr-x 15 hebasto admin 480 30 Mar 09:49 phrasebooks
drwxr-xr-x 31 hebasto admin 992 30 Mar 09:49 plugins
drwxr-xr-x 34 hebasto admin 1088 30 Mar 09:49 qml
drwxr-xr-x 45 hebasto admin 1440 30 Mar 09:49 sbom
drwxr-xr-x 285 hebasto admin 9120 30 Mar 09:49 translations
```
This PR addresses both issues and additionally adds a `deploy` target to the native macOS CI job to prevent any similar recessions in the future.
Fixes https://github.com/bitcoin/bitcoin/issues/32267.
ACKs for top commit:
fanquake:
ACK 84de8c93e7
Tree-SHA512: 27a0eff3cd9317647529ff4571bd79c5dd8f007775b19415c8c27ca4912a60d85074c840cf0443be314d9a404f78bb015029d46dab18e292462249a5d90c6c47
22cff32319 doc: recommend gmake for FreeBSD (Sjors Provoost)
b645c52071 doc: recommend modern make for macOS depends (Sjors Provoost)
99e6490dc5 doc: shuffle depends instructions (Sjors Provoost)
Pull request description:
macOS ships with GNU Make 3.81 from 2006. This has caused
difficult to debug issues, e.g. #32070 and #30978.
Tell users / developers who use the depends system to install a modern version of `make`.
This PR does not change the non-depends build.
Although Homebrew allows overriding the system `make`, we instead just instruct users to build with `gmake`. This way there should be no impact on other projects they wish to compile.
To increase the likeliness of anyone actually seeing and following this instruction, the first commit moves things around in `depends/README.md`. It now starts with instructions for a local build and moves cross-compilation to the end. For each platform it shows what to install (`apt install`, `brew install`, etc) and what command to run (`make` or `gmake`).
There previously was no macOS specific section, so this is added. It points to the general `build-osx.md` for how to install the Xcode Command Line Tools and Homebrew Package Manager.
I didn't test on an empty system.
Preview: https://github.com/Sjors/bitcoin/tree/2025/03/mc-make/depends#depends-build
ACKs for top commit:
maflcko:
review ACK 22cff32319🏣
hebasto:
re-ACK 22cff32319.
willcl-ark:
ACK 22cff32319
Tree-SHA512: 11648ae73f3b70bc2df771e4eddca37221cd88b88bea4139a183e3f67f24a4c3e5aadf61a713ed73f3fc206511dfcf8670e4c4143c49dd4e56e501030be9c7ba
GetAll() is non-const, preventing CheckBlockIndex() from being
const. Rather than add a const GetAll() method, just iterate over
the chainstates directly.
Slight behaviour change by also subjecting non-`IsUsable()`
chainstates to consistency checks.
Native compilation is explained before cross-compilation. Move
install and (g)make steps up.
In the Configuring section, use Linux native compilation as the
example instead of Windows cross-compile.
Closes: #11800
Note: In certain rare edge cases, monotonically increasing estimates may
not be guaranteed. Specifically, given two targets N and M, where M > N,
if a sub-estimate for target N fails to return a valid fee rate, while
target M has valid fee rate for that sub-estimate, target M may result
in a higher fee rate estimate than target N.
See: https://github.com/bitcoin/bitcoin/issues/11800#issuecomment-349697807
In order to remove potential confusion, this commit adapts all script
error constant names in the functional tests (currently only in
feature_taproot.py) to the ones used in our C++ codebase. This also
makes checking whether we have test coverage for a certain script error
easier.
-BEGIN VERIFY SCRIPT-
ren() { sed -i "s|$1|$2|g" $( git grep -l "$1" -- "./test" ) ; }
ren ERR_SIG_SIZE ERR_SCHNORR_SIG_SIZE
ren ERR_SIG_HASHTYPE ERR_SCHNORR_SIG_HASHTYPE
ren ERR_SIG_SCHNORR ERR_SCHNORR_SIG
ren ERR_CONTROLBLOCK_SIZE ERR_TAPROOT_WRONG_CONTROL_SIZE
ren ERR_PUSH_LIMIT ERR_PUSH_SIZE
ren ERR_MINIMALIF ERR_TAPSCRIPT_MINIMALIF
ren ERR_UNKNOWN_PUBKEY ERR_PUBKEYTYPE
ren ERR_STACK_EMPTY ERR_INVALID_STACK_OPERATION
ren ERR_SIGOPS_RATIO ERR_TAPSCRIPT_VALIDATION_WEIGHT
ren ERR_UNDECODABLE ERR_BAD_OPCODE
ren ERR_NO_SUCCESS ERR_EVAL_FALSE
ren ERR_EMPTY_WITNESS ERR_WITNESS_PROGRAM_WITNESS_EMPTY
-END VERIFY SCRIPT-
3add6ab9ad test: remove Boost SIGCHLD workaround. (fanquake)
Pull request description:
The related code was removed from Boost in 2e3bd1025d.
ACKs for top commit:
achow101:
ACK 3add6ab9ad
laanwj:
ACK 3add6ab9ad
hebasto:
ACK 3add6ab9ad, I have reviewed the code and it looks OK.
mabu44:
ACK 3add6ab9ad
Tree-SHA512: a0db2bb4e6a476c920a97183bd807e800d935114ff28f8802373a08b5330df42a9be953e7ea6e3c09f2ed45175f60c26c33bb4e25010269e6e491f12867ba008
Add missing error check for fcntl(fd, F_GETFD, 0) in set_clo_on_exec.
Raise OSError on failure to align with existing FD_SETFD behavior.
This improves robustness in subprocess setup and error visibility.
Github-Pull: arun11299/cpp-subprocess#117
Rebased-From: 9974ff69cdd5fc1a2722cb63f006df9308628b35
This commit makes sure:
1. WaitForSingleObject returns with expected
code before proceeding.
2. Process handle is properly closed.
Github-Pull: arun11299/cpp-subprocess#116
Rebased-From: 625a8775791e62736f20f3fa3e6cc4f1b24aa89a
* refactor: Guard `util::quote_argument()` with `#ifdef __USING_WINDOWS__`
The `util::quote_argument()` function is specific to Windows and is used
in code already guarded by `#ifdef __USING_WINDOWS__`.
* Do not escape double quotes for command line arguments on Windows
This change fixes the handling of double quotes and aligns the behavior
with Python's `Popen` class. For example:
```
>py -3
>>> import subprocess
>>> p = subprocess.Popen("cmd.exe /c dir \"C:\\Program Files\"", stdout=subprocess.PIPE, text=True)
>>> print(f"Captured stdout:\n{stdout}")
```
Currently, the same command line processed by the `quote_argument()`
function looks like `cmd.exe /c dir "\"C:\Program" "Files\""`, which is
broken.
With this change, it looks correct: `cmd.exe /c dir "C:\Program Files"`.
Github-Pull: arun11299/cpp-subprocess#113
Rebased-From: ed313971c04ac10dc006104aba07d016ffc6542a
This suppresses the following warning caused by clang-20.
```
error: definition of implicit copy constructor for 'Streams' is deprecated because it has a user-declared copy assignment operator [-Werror,-Wdeprecated-copy]
```
Copy constructor or move constructor is called when std::vector re-allocates
memory. In this case, move constructor should be called, because copying
Streams instances breaks file-descriptor management.
Communication class is modified as well, since it's instance is a member of
Streams class.
Github-Pull: arun11299/cpp-subprocess#107
Rebased-From: 38d98d9d20be50c7187b98ac9bc9a6e66920f6ef
The commit a32c0f3df4b6bcd1d2e93f19e8f380bb890cd507 introduced code to
silence MSVC's "warning C4996: The POSIX name for this item is
deprecated."
However, it exhibits several issues:
1. The aliases may leak into code outside the `subprocess.hpp` header.
2. They are unnecessarily applied when using the MinGW-w64 toolchain.
3. The fix is incomplete: downstream projects still see C4996 warnings.
4. The implementation lacks documentation.
This change addresses all of the above shortcomings.
Github-Pull: arun11299/cpp-subprocess#112
Rebased-From: 778543b2f2ca7f5d1c4f0241b635bbb265d750dd
Co-authored-by: Luke Dashjr <luke-jr+git@utopios.org>
Currently, wait() returns 0 on windows regardless
of the actual return code of processes.
Github-Pull: arun11299/cpp-subprocess#109
Rebased-From: 04b015a8e52ead4d8bb5f0eb486419c77e418a17
When passing in a rvalue reference, compiler
considers it ambiguous between std::string and
std::string&&. Making one of them take a lvalue
reference makes compilers correctly pick the right
one depending on whether the passed in value binds
to a rvalue or lvalue reference.
Github-Pull: arun11299/cpp-subprocess#110
Rebased-From: 2d8a8eebb03e509840e2c3c755d1abf32d930f33
I encountered this issue while running my code with Valgrind today.
Below is part of the Valgrind error message:
```
==1578139== 472 bytes in 1 blocks are still reachable in loss record 1 of 1
==1578139== at 0x4848899: malloc (...)
==1578139== by 0x4B3AF62: fdopen@@GLIBC_2.2.5 (...)
==1578139== by 0x118B09: subprocess::Popen::execute_process() (...)
```
I noticed that a similar fix had been proposed by another contributor
previously. I did not mean to scoop their work, but merely hoping to fix
it sooner so other people don't get confused by it just as I did today.
Github-Pull: arun11299/cpp-subprocess#106
Rebased-From: 3afe581c1f22f106d59cf54b9b65251e6c554671
e976bd3045 validation: add randomness to periodic write interval (Andrew Toth)
2e2f410681 refactor: replace m_last_write with m_next_write (Andrew Toth)
b557fa7a17 refactor: rename fDoFullFlush to should_write (Andrew Toth)
d73bd9fbe4 validation: write chainstate to disk every hour (Andrew Toth)
0ad7d7abdb test: chainstate write test for periodic chainstate flush (Andrew Toth)
Pull request description:
Since #28233, periodically writing the chainstate to disk every 24 hours does not clear the dbcache. Since #28280, periodically writing the chainstate to disk is proportional only to the amount of dirty entries in the cache. Due to these changes, it is no longer beneficial to only write the chainstate to disk every 24 hours. The periodic flush interval was necessary because every write of the chainstate would clear the dbcache. Now, we can get rid of the periodic flush interval and simply write the chainstate along with blocks and block index at least every hour.
Three benefits of doing this:
1. For IBD or reindex-chainstate with a combination of large dbcache setting, slow CPU, slow internet speed/unreliable peers, it could be up to 24 hours until the chainstate is persisted to disk. A power outage or crash could potentially lose up to 24 hours of progress. If there is a very large amount of dirty cache entries, writing to disk when a flush finally does occur will take a very long time. Crashing during this window of writing can cause https://github.com/bitcoin/bitcoin/issues/11600. By syncing every hour in unison with the block index we avoid this problem. Only a maximum of one hour of progress can be lost, and the window for crashing during writing is much smaller. For IBD with lower dbcache settings, faster CPU, or better internet speed/reliable peers, chainstate writes are already triggered more often than every hour so this change will have no effect on IBD.
2. Based on discussion in #28280, writing only once every 24 hours during long running operation of a node causes IO spikes. Writing smaller chainstate changes every hour like we do with blocks and block index will reduce IO spikes.
3. Faster shutdown speeds. All dirty chainstate entries must be persisted to disk on shutdown. If we have a lot of dirty entries, such as when close to 24 hours or if we sync with a large dbcache, it can take a long time to shutdown. By keeping the chainstate clean we avoid this problem.
Inspired by [this comment](https://github.com/bitcoin/bitcoin/pull/28280#issuecomment-2121088705).
Resolves https://github.com/bitcoin/bitcoin/issues/11600
ACKs for top commit:
achow101:
ACK e976bd3045
davidgumberg:
utACK e976bd3045
sipa:
utACK e976bd3045
l0rinc:
ACK e976bd3045
Tree-SHA512: 5bccd8f1dea47f9820a3fd32fe3bb6841c0167b3d6870cc8f3f7e2368f124af1a914bca6acb06889cd7183638a8dbdbace54d3237c3683f2b567eb7355e015ee
6cbc28b8dd doc: Fix test_bitcoin path (monlovesmango)
Pull request description:
This commit fixes a couple command paths for interacting with the test_bitcoin binary within the Unit Test documentation.
If the commands are run as is a `command not found` error is returned.
```bash
❯ test_bitcoin --list_content
bash: test_bitcoin: command not found
```
```bash
❯ test_bitcoin --help
bash: test_bitcoin: command not found
```
ACKs for top commit:
davidgumberg:
ACK 6cbc28b8dd
Tree-SHA512: 0b10bc3aead360fa499beef7c9715f95a9acacdda44cbfac15566428594a7a8bdece24114a42618355959e20754bedc7a903bdddbf21b819c7b75375bdc80a93
We do not need to repeat the same test multiple times because BnB is
deterministic and will therefore always have the same outcome.
Additionally, this test was redundant because it repeats the "Smallest
combination too big" test.
Originally these tests verified that at a SelectCoins level that a
solution with fewer inputs gets preferred at high feerates, and a
solution with more inputs gets preferred at low feerates. This outcome
relies on the behavior of BnB, so we move these tests under the umbrella
of BnB tests.
Originally these tests relied on SFFO to work.
Recreates the tests in a new test suite coinselection_tests.cpp that is
based on UTXOs being created per their effective values rather than
nominal values and uses transactions with non-zero feerates.
97eaadc3bf util: Remove `fsbridge::get_filesystem_error_message()` (Hennadii Stepanov)
Pull request description:
The `fsbridge::get_filesystem_error_message()` function exhibits several drawbacks:
1. It was introduced in https://github.com/bitcoin/bitcoin/pull/14192 to account for platform-specific variations in
`boost::filesystem::filesystem_error::what()`. Since [migrating](https://github.com/bitcoin/bitcoin/pull/20744) to `std::filesystem`, those discrepancies no longer exist.
2. It fails to display UTF-8 paths correctly on Windows:
```
> build\bin\Release\bitcoind.exe -datadir="C:\Users\hebasto\dd_₿_🏃" -regtest
...
2025-04-30T00:17:48Z DeleteAuthCookie: Unable to remove random auth cookie file: remove: Access is denied.: "C:\Users\hebasto\dd_?_??\regtest\.cookie"
...
```
3. It relies on `std::wstring_convert`, which was deprecated in C++17 and removed in C++26 (also see https://github.com/bitcoin/bitcoin/issues/32361).
This PR removes the obsolete `fsbridge::get_filesystem_error_message()` function, thereby resolving all of the above issues.
ACKs for top commit:
maflcko:
lgtm re-ACK 97eaadc3bf
davidgumberg:
untested crACK 97eaadc3bf
achow101:
ACK 97eaadc3bf
laanwj:
Code review ACK 97eaadc3bf
Tree-SHA512: 3c7378a9b143ac2a71add967318a13c346ae3bccbec6e9879d7873083f3fa469b3eef529b2c9c142b2489ba9563e4e12f685745c09a8a219d58b384f7ecf1be1
The term "force" is ambiguous and not used in BIP9 where the ! rule
prefix is introduced.
Additionally, #29039 renamed gbt_vb_name to gbt_force_name which
might increase the confusion.
-BEGIN VERIFY SCRIPT-
sed -i s/gbt_force_name/gbt_rule_value/g ./src/rpc/mining.cpp
sed -i s/gbt_force/gbt_optional_rule/g $(git grep -l gbt_force)
-END VERIFY SCRIPT-
The `fsbridge::get_filesystem_error_message()` function exhibits several
drawbacks:
1. It was introduced in https://github.com/bitcoin/bitcoin/pull/14192 to
account for platform-specific variations in
`boost::filesystem::filesystem_error::what()`. Since migrating to
`std::filesystem`, those discrepancies no longer exist.
2. It fails to display UTF-8 paths correctly on Windows.
3. It relies on `std::wstring_convert`, which was deprecated in C++17
and removed in C++26.
This change removes the `fsbridge::get_filesystem_error_message()`
function, thereby resolving all of the above issues.
Additionally, filesystem error messages now use the "Warning" log level.
a8333fc9ff scripted-diff: wallet: rename plain and encrypted master key variables (Sebastian Falbesoner)
5a92077fd5 wallet: refactor: dedup master key decryption (Sebastian Falbesoner)
846545947c wallet: refactor: dedup master key encryption / derivation rounds setting (Sebastian Falbesoner)
a6d9b415aa wallet: refactor: introduce `CMasterKey::DEFAULT_DERIVE_ITERATIONS` constant (Sebastian Falbesoner)
62c209f50d wallet: doc: remove mentions of unavailable scrypt derivation method (Sebastian Falbesoner)
Pull request description:
This PR contains various cleanups around the wallet's master key encryption logic. The default/minimum key derivation rounds magic number of 25000 is hoisted into a constant (member of `CMasterKey`) and two new functions `EncryptMasterKey`/`DecryptMasterKey` are introduced in order to deduplicate code for the derivation round determination and master key en/decryption. Also, mentions of the never-implemented derivation method `scrypt` are removed from the wallet crypter header and both plain and encrypted master key instances are renamed to adapt to moderning coding style (hopefully improving readability).
ACKs for top commit:
davidgumberg:
ACK a8333fc9ff
achow101:
ACK a8333fc9ff
Tree-SHA512: 5a66d3b26f481347d0b5b4f742dd237803a35aad6e3480ed15fd38b7fa3700650bd5f67f4c30ed88f5fad45d6cd4c893fe4f1657e36e563b4294fd3596187724
524f981bb8 Bugfix: Miner: Don't reuse block_reserved_weight for "block is full enough to give up" weight delta (Luke Dashjr)
Pull request description:
PR #30356 incorrectly changed a constant of `4000` to `m_options.coinbase_max_additional_weight` in the check for when to give up finding another transaction to fill the block:
```diff
if (nConsecutiveFailed > MAX_CONSECUTIVE_FAILURES && nBlockWeight >
- m_options.nBlockMaxWeight - 4000) {
+ m_options.nBlockMaxWeight - m_options.block_reserved_weight) {
// Give up if we're close to full and haven't succeeded in a while
break;
}
```
But this constant did not deal with the reserved weight at all. It was in fact simply checking if the block was close to full, and if so, giving up finding another transaction to pad it with after `MAX_CONSECUTIVE_FAILURES` failed attempts.
It doesn't seem very logical to reuse the reserve weight for this purpose, and it would be overcomplicated to add yet another setting, so this PR changes it to a new constexpr.
ACKs for top commit:
achow101:
ACK 524f981bb8
darosior:
utACK 524f981bb8
ismaelsadeeq:
ACK 524f981bb8
Tree-SHA512: c066debc34a021380424bd21b40444071b736325e41779a41590c2c8a6822ceeaf910fe067817c1dba108210b24c574977b0350b29520502e7af79d3b405928b
7e8ef959d0 refactor: Fix Sonar rule `cpp:S4998` - avoid unique_ptr const& as parameter (Lőrinc)
e400ac5352 refactor: simplify repeated comparisons in `FindChallenges` (Lőrinc)
f670836112 test: remove old recursive `FindChallenges_recursive` implementation (Lőrinc)
b80d0bdee4 test: avoid stack overflow in `FindChallenges` via manual iteration (Lőrinc)
Pull request description:
`FindChallenges` explores the `Miniscript` node tree by going deep into the first child's subtree, then the second, and so on - effectively performing a pre-order Traversal (Depth-First Search) recursively, using the call stack which can result in stack overflows on Windows debug builds.
This change replaces the recursive implementation with an iterative version using an explicit stack. The new implementation also performs a pre-order depth-first traversal, though it processes children in right-to-left order (rather than left-to-right) due to the LIFO nature of the stack. Since both versions store results in a `std::set`, which automatically sorts and deduplicates elements, the exact traversal order doesn't affect the final result.
It is an alternative to increasing the Windows stack size, as proposed in #32349, and addresses the issue raised in #32341 by avoiding deep recursion altogether.
The change is done in two commits:
* add a new iterative `FindChallenges` method and rename the old method to `*_recursive` (to simplify the next commit where we remove it), asserting that its result matches the original;
* remove the original recursive implementation.
This approach avoids ignoring the `misc-no-recursion` warning as well.
I tried modifying the new method to store results in a vector instead, but it demonstrated that the deduplication provided by `std::set` was necessary. One example showing the need for deduplication:
Recursive (using set):
```
(6, 9070746)
(6, 19532513)
(6, 3343376967)
```
Iterative (using vector attempt):
```
(6, 19532513)
(6, 9070746)
(6, 3343376967)
(6, 9070746) // Duplicate entry
```
The performance of the test is the same as before, with the recursive method.
Fixes https://github.com/bitcoin/bitcoin/issues/32341
ACKs for top commit:
achow101:
ACK 7e8ef959d0
sipa:
utACK 7e8ef959d0
hodlinator:
re-ACK 7e8ef959d0
Tree-SHA512: 9e52eff82a7d76f5d37e3b74c508f08e5fced5386dad504bed111b27ed2b529008a6dd12a5116f009609a94c7ee7ebe3e80a759dda55dd1cb3ae52078f65ec71
b9d4d5f66a net: Use GetAdaptersAddresses to get local addresses on Windows (laanwj)
Pull request description:
Instead of a `gethostname` hack, which is not guaranteed to return all addresses, use the official way of calling `GetAdaptersAddresses` to get local network addresses on Windows.
Do the same checks as the UNIX path: interface is up, interface is not loopback.
Suggested by Ava Chow.
Addiional changes:
- Cleanup: move out `FromSockAddr` in `netif.cpp` from MacOS and use it everywhere appropriate. This avoids code duplication.
ACKs for top commit:
davidgumberg:
utreACK b9d4d5f66a
achow101:
ACK b9d4d5f66a
Tree-SHA512: e9f0a7ec0c46f21c0377d5174e054a6569f858630727f94dac00c0cb7c241c56892d0b902706d6dd53880cc3b5ae1f2dba9caa1fec40e64cd4cf0d34493a49c1
abe43dfadd doc: release note for #27826 (Sjors Provoost)
f9fa28788e Use LogBlockHeader for compact blocks (Sjors Provoost)
bad7c91479 Log which peer sent us a header (Sjors Provoost)
9d3e39c29c Log block header in net_processing (Sjors Provoost)
Pull request description:
Fixes#27744
Since #27278 we log received headers. For compact blocks we also log which peer sent it (e5ce857634), but not for regular headers. That required an additional refactor, which this PR provides.
Move the logging from validation to net_processing.
This also reduces the number of log entries (under default configuration) per compact block header from 3 to 2: one for the header and one for the connected tip.
The PR introduces a new helper method `LogBlockHeader`.
When receiving a _compact block_ we call `LogBlockHeader` from the exact same place as where we previously logged. So that log message doesn't change. What does change is that we no longer _also_ log from `AcceptBlockHeader`.
When receiving a regular header(s) message, _we only log the last one_. This is a change in behaviour because it was simpler to implement, but it's probably better anyway. It does mean that if a peer sends of a bunch of headers of which _any_ is invalid, we won't log it (here).
Lastly I expanded the code comment explaining why we log this. It initially only covered selfish mining, but we also care about peers sending us headers but not following up (see e.g. #27626).
Example log:
```
2023-06-05T13:12:21Z Saw new header hash=000000000000000000045910263ef84b575ae3af151865238f1e5c619e69c330 height=792964 peer=0
2023-06-05T13:12:23Z UpdateTip: new best=000000000000000000045910263ef84b575ae3af151865238f1e5c619e69c330 height=792964 version=0x20000000 log2_work=94.223098 tx=848176824 date='2023-06-05T13:11:49Z' progress=1.000000 cache=6.4MiB(54615txo)
2023-06-05T13:14:05Z Saw new cmpctblock header hash=00000000000000000003c6fd4ef2e1246a3f9e1fffab7247344f94cadb9de979 height=792965 peer=0
2023-06-05T13:14:05Z UpdateTip: new best=00000000000000000003c6fd4ef2e1246a3f9e1fffab7247344f94cadb9de979 height=792965 version=0x20000000 log2_work=94.223112 tx=848179461 date='2023-06-05T13:13:58Z' progress=1.000000 cache=7.2MiB(61275txo)
2023-06-05T13:14:41Z Saw new header hash=000000000000000000048e6d69c8399992782d08cb57f5d6cbc81a9f996c3f43 height=792966 peer=8
2023-06-05T13:14:42Z UpdateTip: new best=000000000000000000048e6d69c8399992782d08cb57f5d6cbc81a9f996c3f43 height=792966 version=0x2db3c000 log2_work=94.223126 tx=848182944 date='2023-06-05T13:14:35Z' progress=1.000000 cache=8.0MiB(69837txo)
```
ACKs for top commit:
danielabrozzoni:
tACK abe43dfadd
achow101:
ACK abe43dfadd
vasild:
ACK abe43dfadd
Tree-SHA512: 081e0de62cbd8a0b35cf54daaa09e3e6991d0cc9f706ef3eb50908752fe7815de69b367f7313381c90cd8d5de0ae5f532d1cd54948c5c1133b1832f266d9c232
f1b142856a test: Same addr, diff port is already connected (David Gumberg)
94e85a82a7 net: remove unnecessary check from AlreadyConnectedToAddress() (Vasil Dimov)
Pull request description:
`CConnman::AlreadyConnectedToAddress()` searches the existent nodes by address or by address-and-port:
```cpp
FindNode(static_cast<CNetAddr>(addr)) || FindNode(addr.ToStringAddrPort())
```
but:
* if there is a match by just the address, then the address-and-port search will not be evaluated and the whole condition will be `true`
* if the there is no node with the same address, then the second search by address-and-port will not find a match either.
The search by address-and-port is comparing against `CNode::m_addr_name` which could be a hostname, e.g. `"node.foobar.com:8333"`, but `addr.ToStringAddrPort()` is always going to be numeric.
---
In other words: let `A` be "CNetAddr equals" and `B` be "addr:port string matches", then:
* If `A` (is `true`), then `B` is irrelevant, so the condition `A || B` is equivalent to `A` is `true`.
* Observation in this PR: if `!A` (`A` is `false`), then `!B` for sure, thus the condition `A || B` is equivalent to `A` is `false`.
So, simplify `A || B` to `A`.
https://en.wikipedia.org/wiki/Modus_tollens `!A => !B` is equivalent to `B => A`. So the added fuzz test asserts that if `B` is `true`, then `A` is `true`.
ACKs for top commit:
davidgumberg:
crACK f1b142856a
achow101:
ACK f1b142856a
theuni:
utACK f1b142856a
mzumsande:
Code Review ACK f1b142856a
Tree-SHA512: d744b60e9bace121faa3a746463f6b6e0e6ef08eac0e7879326cbd5f4721e47e6e10f6203dfd3870a2057c4ddd1860692c070ef048a76d773b84e6c2f840cc86
e3014017ba test: add IsActiveAfter tests for versionbits (Anthony Towns)
60950f77c3 versionbits: docstrings for BIP9Info (Anthony Towns)
7565563bc7 tests: refactor versionbits fuzz test (Anthony Towns)
2e4e9b9608 tests: refactor versionbits unit test (Anthony Towns)
525c00f91b versionbits: Expose VersionBitsConditionChecker via impl header (Anthony Towns)
e74a7049b4 versionbits: Expose StateName function (Anthony Towns)
d00d1ed52c versionbits: Split out internal details into impl header (Anthony Towns)
37b9b67a39 versionbits: Simplify VersionBitsCache API (Anthony Towns)
1198e7d2fd versionbits: Move BIP9 status logic for getblocktemplate to versionbits (Anthony Towns)
b1e967c3ec versionbits: Move getdeploymentinfo logic to versionbits (Anthony Towns)
3bd32c2055 versionbits: Move WarningBits logic from validation to versionbits (Anthony Towns)
5da119e5d0 versionbits: Change BIP9Stats to uint32_t types (Anthony Towns)
a679040ec1 consensus/params: Move version bits period/threshold to bip9 param (Anthony Towns)
e9d617095d versionbits: Remove params from AbstractThresholdConditionChecker (Anthony Towns)
9bc41f1b48 versionbits: Use std::array instead of C-style arrays (Anthony Towns)
Pull request description:
Increases the encapsulation/modularity of the versionbits code, moving more of the logic into the versionbits module rather than having it scattered across validation and rpc code. Updates unit/fuzz tests to test the actual code used rather than just a close approximation of it.
ACKs for top commit:
achow101:
ACK e3014017ba
TheCharlatan:
Re-ACK e3014017ba
darosior:
ACK e3014017ba
Tree-SHA512: 2978db5038354b56fa1dd6aafd511099e9c16504d6a88daeac2ff2702c87bcf3e55a32e2f0a7697e3de76963b68b9d5ede7976ee007e45862fa306911194496d
fa655da159 test: [refactor] Use ToIntegral in CheckInferDescriptor (MarcoFalke)
fa55dd01df descriptors: Reject + sign when parsing multi threshold (MarcoFalke)
fa6f77ed3c descriptors: Reject + sign in ParseKeyPathNum (MarcoFalke)
Pull request description:
As a follow-up to https://github.com/bitcoin/bitcoin/pull/30577, reject `+` for unsigned values in key-path parsing and multi threshold parsing as well.
Both of those are using unsigned, and Bitcoin Core would never serialize a descriptor string with a stray `+`. Accepting stray `+` signs could lead to checksum mismatches, or other incompatibilities later on.
Just like https://github.com/bitcoin/bitcoin/pull/30577, both changes are breaking changes on the RPC interface, but hopefully no one should be relying on this behavior in production. Similarly, both changes should be fine for the wallet, because it normalizes the strings on import, see https://github.com/bitcoin/bitcoin/pull/30577#pullrequestreview-2218436014.
ACKs for top commit:
achow101:
ACK fa655da159
brunoerg:
code review ACK fa655da159
janb84:
tACK [fa655da](fa655da159)
Tree-SHA512: d0c7262a167f7ba98b44ed8bf49ff4c15a1eb647cbac39a59b83c7cc379903c24dae3996e5f557497eb08e16d7121417916147058d97bdf168cd6eada57dceef
32d55e28af test: Use the correct node for doubled keypath test (Ava Chow)
Pull request description:
#29124 had a silent merge conflict with #32350 which resulted in it using the wrong node. Fix the test to use the correct v22 node.
ACKs for top commit:
maflcko:
lgtm ACK 32d55e28af
rkrux:
ACK 32d55e28af
BrandonOdiwuor:
Code Review ACK 32d55e28af
Tree-SHA512: 1e0231985beb382b16e1d608c874750423d0502388db0c8ad450b22d17f9d96f5e16a6b44948ebda5efc750f62b60d0de8dd20131f449427426a36caf374af92
fadf12a56c test: Add missing check for empty stderr in util tester (MarcoFalke)
Pull request description:
Now that wine support was removed from the CI in 25b56fd9b4, it can probably be removed from the util tester as well.
If someone really needs this, they can comment the new check out, or submit a patch to add an option/env var to silence the new check.
ACKs for top commit:
achow101:
ACK fadf12a56c
i-am-yuvi:
tACK fadf12a56c
BrandonOdiwuor:
Code Review ACK fadf12a56c
ismaelsadeeq:
Tested ACK fadf12a56c
Tree-SHA512: d9e4d7a7f724e114391070ea7f17b585a7e4c4f3221c3bf510eeb11df6ccd089b881ab5654adfef8d3a1f8fa7ec6bf5e3a3feeb0cdfe724a8f3e5a146c388e66
c7e2b9e264 tests: Test migration cleans up bad inactive chain derivation path (Ava Chow)
Pull request description:
A bug in 0.21.x and 22.x resulted in some wallets having invalid derivation paths that are the concatenation of two derivation paths. These appear only when inactive hd chains are topped up.
Since key metadata is a legacy wallet only record, migrating legacy wallets to descriptor wallets will fix this issue as all key metadata records are deleted. The derivation path information is derived on-the-fly from the descriptor that is produced for the inactive hd chain.
Thus we only need a test to verify that the derivation paths are good, and that all key metadata records are deleted from the migrated wallet.
ACKs for top commit:
murchandamus:
re-ACK c7e2b9e264 via range-diff:
rkrux:
re-ACK c7e2b9e264
furszy:
utACK c7e2b9e264
Tree-SHA512: 3117c4a43798972109fe2d3539341a8b69db70c6457fcabdd019e6044834dc4b17212abbc006d7b8008f560dce4b7856142b057981b9404f406d58fa0955cbd9
fa58f40b89 test: Slim down previous releases bdb check (MarcoFalke)
Pull request description:
The check iterates over several previous BDB-only releases to check that descriptor wallets are considered "corrupt" when loading. It is unclear why this needs to be done for more than one release.
Avoid the confusion by removing the unused releases from the test and from the download script.
ACKs for top commit:
achow101:
ACK fa58f40b89
rkrux:
ACK fa58f40b89
Tree-SHA512: 8084392481bfe1fba9b80bb865ffbdfa454e9e6e14e02c39fa3f61c1a596b1def2c531c5da1c7566e5fddb77ac7e56f19feabaaf9b5af043fa6c230d9e2370b5
fa48be3ba4 test: Force named args for RPCOverloadWrapper optional args (MarcoFalke)
aaaa45399c test: Remove unused createwallet_passthrough (MarcoFalke)
cccc1f4e91 test: Remove unused RPCOverloadWrapper is_cli field (MarcoFalke)
Pull request description:
This can avoid bugs and makes the test code easier to read, because the
order of positional args does not have to be known or assumed.
Also, contains two commits to remove dead code.
ACKs for top commit:
achow101:
ACK fa48be3ba4
rkrux:
tACK fa48be3ba4
janb84:
tACK [fa48be3](fa48be3ba4)
Tree-SHA512: d938fbc18be5035ad0d0e1ad2bf7297b2b66ede3bb2d3f10b8d27aa2a19d27a897b024a5f5a2a1cceca467837890729c26054928cb06acbe282b9e9eea94ae69
35e57fbe33 depends: Fix cross-compiling `qt` package from macOS to Windows (Hennadii Stepanov)
Pull request description:
Native packages cannot be used during cross-compiling. However, Qt still unconditionally tries to find them, which causes issues in some cases, such as when [cross-compiling from macOS to Windows](https://github.com/bitcoin/bitcoin/issues/32346).
This PR explicitly disables this unnecessary Qt behaviour.
Fixes https://github.com/bitcoin/bitcoin/issues/32346.
Here is a full workflow on my macOS Sequoia 15.4.1 (Intel):
```
% brew install make cmake ninja mingw-w64 nsis
% gmake -C depends -j 10 HOST=x86_64-w64-mingw32
% cmake -B build --toolchain depends/x86_64-w64-mingw32/toolchain.cmake
% cmake --build build -j 10 -t deploy
```
ACKs for top commit:
shahsb:
ACK 35e57fbe33
fanquake:
ACK 35e57fbe33
Tree-SHA512: 2822fb49bc84dd094dbd189d8a9ca0f023e1e48127db7beaefb9db92de53df63bb0f399c9c430c33941f9a9ee6976b9161d80467d889f7717385b9d1ea9fee43
The original recursive `FindChallenges` explores the Miniscript node tree using depth-first search. Specifically, it performs a pre-order traversal (processing the node's data, then recursively visiting children from left-to-right). This recursion uses the call stack, which can lead to stack overflows on platforms with limited stack space, particularly noticeable in Windows debug builds.
This change replaces the recursive implementation with an iterative version using an explicit stack. The iterative version also performs a depth-first search and processes the node's data before exploring children (preserving pre-order characteristics), although the children are explored in right-to-left order due to the LIFO nature of the explicit stack.
Critically, both versions collect challenges into a `std::set`, which automatically deduplicates and sorts elements. This ensures that not only the final result, but the actual state of the set at any equivalent point in traversal remains identical, despite the difference in insertion order.
This iterative approach is an alternative to increasing the default stack size (as proposed in #32349) and directly addresses the stack overflow issue reported in #32341 by avoiding deep recursion.
The change is done in two commits:
* add a new iterative `FindChallenges` method and rename the old method to `*_recursive` (to simplify removal in the next commit), asserting that its result matches the original;
* Remove the original recursive implementation.
This approach avoids needing to suppress `misc-no-recursion` warnings and provides a portable, low-risk fix.
Using a `std::set` is necessary for deduplication, matching the original function's behavior. An experiment using an `std::vector` showed duplicate challenges being added, confirming the need for the set:
Example failure with vector:
Recursive (set):
(6, 9070746)
(6, 19532513)
(6, 3343376967)
Iterative (vector attempt):
(6, 19532513)
(6, 9070746)
(6, 3343376967)
(6, 9070746) // Duplicate
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
77e553ab6a build: refactor: hardening flags -> core_interface (David Gumberg)
00ba3ba303 build: Drop option for disabling hardening (David Gumberg)
f57db75e91 build: Use `-z noseparate-code` on NetBSD < 11.0 (David Gumberg)
Pull request description:
Follow up to #32038 which dropped `NO_HARDEN` from depends builds, this PR drops the `ENABLE_HARDENING` build option since disabling hardening of binaries should not be a supported or maintained use case. With this change, hardening flags are always enabled.
Individual hardening flags and options can still be disabled by appending flags, e.g.:
```bash
cmake -B build \
-DAPPEND_CPPFLAGS='-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -fno-stack-protector -fcf-protection=none -fno-stack-clash-protection' \
-DAPPEND_LDFLAGS='-Wl,-z,lazy -Wl,-z,norelro -Wl,-z,noseparate-code'
```
There is an issue with NetBSD 10.0's dynamic linker that makes one of the hardening linker flags, `-z separate-code`, [problematic](https://github.com/bitcoin/bitcoin/pull/28724#issuecomment-2589347934), so this PR also introduces a check to prevent the use of this flag in NetBSD versions < 11.0, (where this issue is [fixed](acf7fb3abf)). The fix for this [might be backported](https://mail-index.netbsd.org/tech-userlevel/2023/01/05/msg013670.html) to NetBSD 10.0.
I suggest reviewing the diff with whitespace changes hidden (`git diff -w` or using github's hide whitespace option)
ACKs for top commit:
hebasto:
re-ACK 77e553ab6a.
laanwj:
re-ACK 77e553ab6a
janb84:
ACK [77e553a](77e553ab6a)
vasild:
ACK 77e553ab6a
musaHaruna:
tested ACK [77e553](77e553ab6a)
Tree-SHA512: b149fb0371d12312c140255bf674c2bdc9f5272a5750a5b9ec5f192323364bb2ea8e164af13b9ab981ab3aa7ceb91b7a64785081e7458470e81c2f5228abf7b1
61f238e84a doc: Fix fuzz test_runner.py path (monlovesmango)
Pull request description:
This commit fixes the path listed in the documentation for the fuzz testing test_runner.py. Previously the --help option worked but running fuzz tests from the documented path did not.
ACKs for top commit:
kevkevinpal:
ACK [61f238e](61f238e84a)
maflcko:
lgtm ACK 61f238e84a
mabu44:
Tested ACK 61f238e84a
hebasto:
ACK 61f238e84a.
Tree-SHA512: e8770f38e49a428e0e7f0450db193ec90cc1e66c05bcde307763c065ac7051f3f05923bb3e0eca7a337da9c14cfd17512ff0d01ffa330796159d4f3552103b7f
71656bdfaa gui: crash fix, disconnect numBlocksChanged() signal during shutdown (furszy)
Pull request description:
Aiming to fixbitcoin-core/gui#862.
The crash stems from the order of the shutdown procedure:
We first unset the client model, then destroy the wallet controller—but we leave
the internal wallet models (`m_wallets`) untouched for a brief period. As a result,
there’s a point in time where views still have connected signals and access to
wallet models that are not connected to any wallet controller.
Now.. since the `clientModel` is only replaced with nullptr locally and not destroyed
yet, signals like `numBlocksChanged` can still emit. Thus, when wallet views receive
them, they see a non-null wallet model ptr, and proceed to call backend functions
from a model that is being torn down.
As the shutdown procedure begins by unsetting `clientModel` from all views. It’s safe
to ignore events when `clientModel` is nullptr.
ACKs for top commit:
maflcko:
lgtm ACK 71656bdfaa
pablomartin4btc:
re-ACK 71656bdfaa
hebasto:
ACK 71656bdfaa, I have reviewed the code and it looks OK.
Tree-SHA512: e6a369c40aad8a5a3da64e92daa10250006f60c53feef353a5580e1bdb17fe8e1ad102abf5419ddeff1caa703b69ab634265ef3b9cfef87e9304f97bfdd2c4aa
PR #30356 incorrectly changed a constant of `4000` to `m_options.coinbase_max_additional_weight` in the check for when to give up finding another transaction to fill the block:
```diff
if (nConsecutiveFailed > MAX_CONSECUTIVE_FAILURES && nBlockWeight >
- m_options.nBlockMaxWeight - 4000) {
+ m_options.nBlockMaxWeight - m_options.block_reserved_weight) {
// Give up if we're close to full and haven't succeeded in a while
break;
}
```
But this constant did not deal with the reserved weight at all. It was in fact simply checking if the block was close to full, and if so, giving up finding another transaction to pad it with after `MAX_CONSECUTIVE_FAILURES` failed attempts.
It doesn't seem very logical to reuse the reserve weight for this purpose, and it would be overcomplicated to add yet another setting, so this PR changes it to a new constexpr.
Homebrew’s `qt@6` package places the `translations` and `plugins`
directories in the `share/qt` subdirectory.
This change updates the `macdeployqtplus` script accordingly.
edd46566bd qt: Replace stray tfm::format to cerr with qWarning (laanwj)
Pull request description:
GUI warnings should go to the log, not to the console (which may not be connected at all).
ACKs for top commit:
hebasto:
ACK edd46566bd, I have reviewed the code and it looks OK.
Tree-SHA512: 32944e00dae0c62bb23e3d7abd486b63e445702483ca03c74c3057ef942f06e771d4d3d3a58fd728582889d6b638fae11ecc536a25febfd89a28522b7d6d08ba
The Consensus Cleanup soft fork proposal includes enforcing that coinbase transactions set their
locktime field to the block height, minus 1 (as well as their nSequence such as to not disable the
timelock). If such a fork were to be activated by Bitcoin users, miners need to be ready to produce
compliant blocks at the risk of losing substantial amounts mining would-be invalid blocks. As miners
are unfamously slow to upgrade, it's good to make this change as early as possible.
Although Bitcoin Core's GBT implementation does not provide the "coinbasetxn" field, and mining
pool software crafts the coinbase on its own, updating the Bitcoin Core mining code is a first step
toward convincing pools to update their (often closed source) code. A possible followup is also to
introduce new fields to GBT. In addition, this first step also makes it possible to test future
Consensus Cleanup changes.
The changes to the seemingly-unrelated RBF tests is because these tests assert an error message
which may vary depending on the txid of the transactions used in the test. This commit changes the
coinbase transaction structure and therefore impact the txid of transactions in all tests.
The change to the "Bad snapshot" error message in the assumeutxo functional test is because this
specific test case reads into the txid of the next transaction in the snapshot and asserts the error
message based it gets on deserializing this txid as a coin for the previous transaction. As this
commit changes this txid it impacts the deserialization error raised.
This commit fixes the path listed in the documentation for the fuzz
testing test_runner.py. Previously the --help option worked but running
fuzz tests from the documented path did not.
A bug in 0.21.x and 22.x resulted in some wallets having invalid
derivation paths that are the concatenation of two derivation paths.
These appear only when inactive hd chains are topped up.
Since key metadata is a legacy wallet only record, migrating legacy
wallets to descriptor wallets will fix this issue as all key metadata
records are deleted. The derivation path information is derived
on-the-fly from the descriptor that is produced for the inactive hd
chain.
Thus we only need a test to verify that the derivation paths are good,
and that all key metadata records are deleted from the migrated wallet.
`CConnman::AlreadyConnectedToAddress()` searches the existent nodes by
address or by address-and-port:
```cpp
FindNode(static_cast<CNetAddr>(addr)) || FindNode(addr.ToStringAddrPort())
```
but:
* if there is a match by just the address, then the address-and-port
search will not be evaluated and the whole condition will be `true`
* if the there is no node with the same address, then the second search
by address-and-port will not find a match either.
The search by address-and-port is comparing against `CNode::m_addr_name`
which could be a hostname, e.g. `"node.foobar.com:8333"`, but
`addr.ToStringAddrPort()` is always going to be numeric.
Non-range desc are always added to the wallet with the range [0,0]. After the descriptor is added, the wallet will TopUp the keypool. For non-range descriptors, this process updates the desc range to [0,1].
Any attempts to update this non-range descriptor with a [0,0] range will result in an error because the range checks rejects new ranges not included in the old range.
Since this is a non-range desc, the range information should be disregarded and AddWalletDescriptor should always succeed regardless of provided range information
3dbd50a576 Fix failing util_time_GetTime test on Windows (VolodymyrBg)
Pull request description:
Remove unreliable steady clock time checking from the test that was causing CI failures primarily on Windows. The test previously tried to verify that steady_clock time increases after a 1ms sleep, but this approach is not reliable on all platforms where such a short sleep interval may not consistently result in observable clock changes.
This addresses issue #32197 where the test was reporting failures in the cross-built Windows CI environment. As noted in the discussion, the test is not critical to the functionality of Bitcoin Core, and removing the unreliable part is the most straightforward solution.
ACKs for top commit:
maflcko:
lgtm ACK 3dbd50a576
achow101:
ACK 3dbd50a576
laanwj:
re-ACK 3dbd50a576
Tree-SHA512: 25c80558d9587c7845d3c14464e8d263c8bd9838a510faf44926e5cda5178aee10b03a52464246604e5d27544011d936442ecfa1e4cdaacb66d32c35f7213902
The crash stems from the order of the shutdown procedure:
We first unset the client model, then destroy the wallet controller—but we leave
the internal wallet models ('m_wallets') untouched for a brief period. As a result,
there’s a point in time where views still have connected signals and access to
wallet models that are not connected to any wallet controller.
Now.. since the clientModel is only replaced with nullptr locally and not destroyed
yet, signals like numBlocksChanged can still emit. Thus, when wallet views receive
them, they see a non-null wallet model ptr, and proceed to call backend functions
from a model that is being torn down.
As the shutdown procedure begins by unsetting clientModel from all views. It’s safe
to ignore events when clientModel is nullptr.
Remove unreliable steady clock time checking from the test that was causing
CI failures primarily on Windows. The test previously tried to verify that
steady_clock time increases after a 1ms sleep, but this approach is not reliable
on all platforms where such a short sleep interval may not consistently result
in observable clock changes.
This addresses issue #32197 where the test was reporting failures in the
cross-built Windows CI environment. As noted in the discussion, the test is not
critical to the functionality of Bitcoin Core, and removing the unreliable part
is the most straightforward solution.
Rename and refocus util_time_GetTime test to util_mocktime
Co-Authored-By: maflcko <6399679+maflcko@users.noreply.github.com>
facb9b327b scripted-diff: Use bpf_cflags (MarcoFalke)
fa0c1baaf8 test: Add imports for util bpf_cflags (MarcoFalke)
Pull request description:
On some Linux kernel versions, the bpf compiler invoked in the functional tests will issue a `-Wduplicate-decl-specifier` warning.
This seems harmless and should be fixed upstream in the Linux kernel.
Here, simply suppress it for now. Fixes https://github.com/bitcoin/bitcoin/issues/32322
ACKs for top commit:
laanwj:
Code review ACK facb9b327b
hebasto:
ACK facb9b327b, I have reviewed the code and it looks OK.
Tree-SHA512: 53387127e3c2a2dbfe05281b2d2e61efbd3c3adcc3b4bf2f11540042f86e1e8c06637f80d246310bc44ca0612318472f25545c1e1ca3636fda97d04381f9e905
3c3548a70e validation: clarify final |= BLOCK_FAILED_VALID in InvalidateBlock (Matt Corallo)
aac5488909 validation: correctly update BlockStatus for invalid block descendants (stratospher)
9e29653b42 test: check BlockStatus when InvalidateBlock is used (stratospher)
c99667583d validation: fix traversal condition to mark BLOCK_FAILED_CHILD (stratospher)
Pull request description:
This PR addresses 3 issues related to how `BLOCK_FAILED_CHILD` is set:
1. In `InvalidateBlock()`
- Previously, `BLOCK_FAILED_CHILD` was not being set when it should have been.
- This was due to an incorrect traversal condition, which is fixed in this PR.
2. In `SetBlockFailure()`
- `BLOCK_FAILED_VALID` is now cleared before setting `BLOCK_FAILED_CHILD`.
3. In `InvalidateBlock()`
- if block is already marked as `BLOCK_FAILED_CHILD`, don't mark it as `BLOCK_FAILED_VALID` again.
Also adds a unit test to check `BLOCK_FAILED_VALID` and `BLOCK_FAILED_CHILD` status in `InvalidateBlock()`.
<details>
<summary><h3>looking for feedback on an alternate approach</h3></summary>
<br>
An alternate approach could be removing `BLOCK_FAILED_CHILD` since even though we have a distinction between
`BLOCK_FAILED_VALID` and `BLOCK_FAILED_CHILD` in the codebase, we don't use it for anything. Whenever we check for BlockStatus, we use `BLOCK_FAILED_MASK` which encompasses both of them. See similar discussion in https://github.com/bitcoin/bitcoin/pull/16856.
I have a branch with this approach in https://github.com/stratospher/bitcoin/commits/2025_02_remove_block_failed_child/.
Compared to the version in #16856, it also resets `BLOCK_FAILED_CHILD` already on disk to `BLOCK_FAILED_VALID` when loading from disk so that we won't be in a dirty state in a no-`BLOCK_FAILED_CHILD`-world.
I'm not sure if it's a good idea to remove `BLOCK_FAILED_CHILD` though. would be curious to hear what others think of this approach.
thanks @ mzumsande for helpful discussion regarding this PR!
</details>
ACKs for top commit:
achow101:
ACK 3c3548a70e
TheCharlatan:
Re-ACK 3c3548a70e
mzumsande:
re-ACK 3c3548a70e
Tree-SHA512: 83e0d29dea95b97519d4868135c965b86f6f43be50b15c0bd8f998b3476388fc7cc22b49c0c54ec532ae8222e57dfc436438f0c8e98f54757b384f220488b6a6
55b931934a removed duplicate calling of GetDescriptorScriptPubKeyMan (Saikiran)
Pull request description:
Removed duplicate call to GetDescriptorScriptPubKeyMan and
Instead of checking linearly I have used find method so time complexity reduced significantly for GetDescriptorScriptPubKeyMan
after this fix improved performance of importdescriptor part refs https://github.com/bitcoin/bitcoin/issues/32013.
**Steps to reproduce in testnet environment**
**Input size:** 2 million address in the wallet
**Step1:** call importaddresdescriptor rpc method
observe the time it has taken.
**With the provided fix:**
Do the same steps again
observe the time it has taken.
There is a huge improvement in the performance. (previously it may take 5 to 6 seconds now it will take 1 seconds or less)
main changes i've made during this pr:
1. remove duplicate call to GetDescriptorScriptPubKeyMan method
2. And inside GetDescriptorScriptPubKeyMan method previously we checking **each address linearly** so each time it is calling HasWallet method which has aquired lock.
3. Now i've modified this logic call **find method on the map (O(logn)**) time it is taking, so only once we calling HasWallet method.
**Note:** Smaller inputs in the wallet you may not see the issue but huge wallet size it will definitely impact the performance.
ACKs for top commit:
achow101:
ACK 55b931934a
w0xlt:
ACK 55b931934a
Tree-SHA512: 4a7fdbcbb4e55bd034e9cf28ab4e7ee3fb1745fc8847adb388c98a19c952a1fb66d7b54f0f28b4c2a75a42473923742b4a99fb26771577183a98e0bcbf87a8ca
Legacy wallets do not have the descriptors flag set. Don't load wallets
without the descriptors flag.
At the same time, we will no longer load BDB databases since they are
only used for legacy wallets.
Removes all legacy wallet specific functional tests.
Also removes the --descriptor and --legacy-wallet options as these are
no longer necessary with the legacy wallet removed.
faca46b042 test: Run all benchmarks in the sanity check (MarcoFalke)
Pull request description:
It is unclear why not all benchmarks are run, given that:
* they only run as a sanity check (fastest version)
* no one otherwise runs them, not even CI
* issues have been missed due to this
ACKs for top commit:
l0rinc:
ACK faca46b042
BrandonOdiwuor:
Code Review ACK faca46b042
Tree-SHA512: 866f1ccff0313017dd313d5a218d7ee088b823601a129b9ed4c5819b0d57fd808d78e3ea28ca00714ae6b209df5312b7b9dea091b2b028821ff46b8ba263c48a
3669ecd4cc doc: Document fuzz build options (Anthony Towns)
c1d01f59ac fuzz: enable running fuzz test cases in Debug mode (Anthony Towns)
Pull request description:
When building with
BUILD_FOR_FUZZING=OFF
BUILD_FUZZ_BINARY=ON
CMAKE_BUILD_TYPE=Debug
allow the fuzz binary to execute given test cases (without actual fuzzing) to make it easier to reproduce fuzz test failures in a more normal debug build.
In Debug builds, deterministic fuzz behaviour is controlled via a runtime variable, which is normally false, but set to true automatically in the fuzz binary, unless the FUZZ_NONDETERMINISM environment variable is set.
ACKs for top commit:
maflcko:
re-ACK 3669ecd4cc🏉
marcofleon:
re ACK 3669ecd4cc
ryanofsky:
Code review ACK 3669ecd4cc with just variable renamed and documentation added since last review
Tree-SHA512: 5da5736462f98437d0aa1bd01aeacb9d46a9cc446a748080291067f7a27854c89f560f3a6481b760b9a0ea15a8d3ad90cd329ee2a008e5e347a101ed2516449e
cad39f86fb bench: ensure wallet migration benchmark runs exactly once (Lőrinc)
c1f458aaa0 ci: re-enable all benchmark runs (Lőrinc)
1da11dbc44 bench: clean up migrated descriptor wallets via loader teardown (Lőrinc)
Pull request description:
The low-priority `WalletMigration` benchmark existed for some time but was never run automatically in our CI.
Although the failure first surfaced on Windows as a hang during temporary directory cleanup, it could also be reproduced on Linux and macOS when forcing multiple iterations (e.g. via a long `--min-time`).
### Root causes
1. **Leaked open wallets on Windows**
`MigrateLegacyToDescriptor` produces two new descriptor wallets (the primary spendable wallet and a companion watch‑only wallet). Without unloading them, their database files remained open in the `WalletContext`, blocking directory removal and hanging the test harness.
<details><summary>Details</summary>
```bash
what(): filesystem error: cannot remove all: The process cannot access the file because it is being used by another process [C:\Users\RUNNER\~1\AppData\Local\Temp\test_common bitcoin\WalletMigration\d8ffd89a7700ce01c31f] [C:\Users\RUNNER~1\AppData\Local\Temp\test_common bitcoin\WalletMigration\d8ffd89a7700ce01c31f\regtest\wallet.dat]
```
</details>
2. **Undefined behavior on repeated runs**
The benchmark body calls `std::move(wallet)`, invalidating the local `wallet` pointer. Running more than one iteration causes a use-after-move by the sanitizers.
<details><summary>Details</summary>
```bash
error: bench_bitcoin 0x00067927: DW_TAG_member '_M_local_buf' refers to type 0x00000000000b3ba7 which extends beyond the bounds of 0x0006791d
* thread #1, name = 'b-test', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0xc8)
* frame #0: 0x00005555556a3f33 bench_bitcoin`... basic_string<char>::length(this=<unavailable>) const at basic_string.h:1079:16
```
</details>
### Fixes
- **Automatic wallet teardown**
Wrap the benchmark in a `MakeWalletLoader` (owning a `WalletContext`), so that both migrated wallets are unloaded when the loader goes out of scope, eliminating any lingering open files.
- **Re-enable benchmarks in CI**
Drop the temporary filter in GitHub Actions. The `-sanity-check` run already executes each benchmark once, so `WalletMigration` now runs automatically without hangs or crashes.
- **Single iteration**
Configure the microbenchmark with `.epochs(1).epochIterations(1)`, ensuring the migration code runs exactly once and avoiding use-after-move.
No measurable change in benchmark performance.
ACKs for top commit:
maflcko:
review ACK cad39f86fb🍥
furszy:
utACK cad39f86fb
hebasto:
ACK cad39f86fb, tested on Ubuntu 25.04.
Tree-SHA512: 10343ce7ab9b63ba4f51a7673018215577ea7ec188e41d535a66d69d73b85bca6ba301c33f6920c02f8f7d686c75c65c4a4e9bdafb04b60be85d66aa743cfa20
fa653cb416 ci: Add missing -Wno-error=array-bounds to valgrind fuzz (MarcoFalke)
Pull request description:
Due to an upstream GCC issue, any debug/fuzz build which aborts on failed assumes will print a false positive array-bounds warning in `src/test/fuzz/txgraph.cpp`.
This also affects one CI task.
Fix the CI task by ignoring the error for now.
Fixes https://github.com/bitcoin/bitcoin/issues/32276
ACKs for top commit:
fanquake:
ACK fa653cb416 - checked native fuzz
Tree-SHA512: 0f6c5ec8d96e0bf96cd008e2de5db59e528086a67dcb77f3e59a0d83225d880a59e960d65c5bc8b5ae3de9d5d301bfc7737d95c282aa1bcc740a42561f610ca8
7f5a35cf4b doc: Add deps install notes for multiprocess (TheCharlatan)
Pull request description:
These just mirror the content in src/ipc/libmultiprocess/doc/install.md
ACKs for top commit:
Sjors:
re-ACK 7f5a35cf4b
ryanofsky:
Code review ACK 7f5a35cf4b just dropping dependencies.md update since last review
Tree-SHA512: f9bf4f54542323aa4a0600db874640e575e40355f08515331a27fb139e6e47ee58aa0c6635206f978696e3da7b5aa93efb45b181b02e99e308537fcb90bd6751
e34f12bdd4 ci: Drop no longer necessary `-Wno-error=array-bounds` (Hennadii Stepanov)
Pull request description:
The build log of the "Linux->Windows cross" CI job no longer shows any `-Warray-bounds` compiler warnings. Therefore, there's no need to suppress them with `-Wno-error=array-bounds`.
I likely overlooked this when reviewing https://github.com/bitcoin/bitcoin/pull/29881, as I can run that CI job locally without such warnings even at commit 785649f397.
ACKs for top commit:
TheCharlatan:
ACK e34f12bdd4
Tree-SHA512: ac66160866097538af6f196c0cb22d370427c59c071b0ddcb1a6717e233bbd3dfed4e090d266221c55ae0ddd3d5dffb0ca7ae01582eda07f25fb886a775b6ac5
The migration benchmark crashes if run more than once, because of `std::move(wallet)` and leaves subsequent iterations in an undefined state - avoiding `UndefinedBehaviorSanitizer` null‑dereference error.
`MigrateLegacyToDescriptor` returns both a spendable descriptor wallet and a watch‑only wallet.
If these remain attached, their files stay open and on Windows this can hang CI when removing the test directory.
By constructing them via `MakeWalletLoader` (which owns the `WalletContext`), both wallets are automatically unloaded when the loader is destroyed at the end.
This ensures no lingering handles or resource leaks when running the benchmark on CI with `-sanity-check`.
Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
When building with
BUILD_FOR_FUZZING=OFF
BUILD_FUZZ_BINARY=ON
CMAKE_BUILD_TYPE=Debug
allow the fuzz binary to execute given test cases (without actual
fuzzing) to make it easier to reproduce fuzz test failures in a more
normal debug build.
In Debug builds, deterministic fuzz behaviour is controlled via a runtime
variable, which is normally false, but set to true automatically in the
fuzz binary, unless the FUZZ_NONDETERMINISM environment variable is set.
fa86190e6e rpc: Allow fullrbf fee bump (MarcoFalke)
Pull request description:
The RPCs (psbt)bumpfee, and the GUI, reject fee bumps when BIP 125 signalling is absent in the transaction even when the mempool and other RPCs allow them. Fix the confusion by allowing the fee bump.
This is done after fullrbf is always on (https://github.com/bitcoin/bitcoin/pull/30592)
ACKs for top commit:
1440000bytes:
reACK fa86190e6e
achow101:
ACK fa86190e6e
w0xlt:
ACK fa86190e6e
rkrux:
reACK fa86190e6e
glozow:
ACK fa86190e6e
Tree-SHA512: b2ffe8dcadbe71e9be767a16cf8aa0bf383c2de7aa1aee9438d125f444e24f3f7e4f02ddb28981bd3b8b645b6a24a407b4ad6bb0b21946ae637e78f6386e05bf
acee5c59e6 descriptors: Have GetPrivKey fill keys directly (Ava Chow)
4b0303197e descriptors: Move FlatSigningProvider pubkey filling to GetPubKey (Ava Chow)
25a3b9b0f5 descriptors: Have GetPubKey fill origins directly (Ava Chow)
6268bde0af descriptor: Remove unused parent_info from BIP32PUbKeyProvider::GetPubKey (Ava Chow)
0ff072caa1 wallet, rpc: Only allow keypool import from single key descriptors (Ava Chow)
Pull request description:
Instead of having `MakeScripts` infer what pubkeys need to go into the output `FlatSigningProvider`, have each of the `PubkeyProviders` that have `GetPubKey` and `GetPrivKey` called fill it directly with relevant keys and origins.
This allows for keys and origins to be added that won't directly appear in the output, which is necessary for `musig()` descriptors.
Split from #29675
ACKs for top commit:
fjahr:
Code review ACK acee5c59e6
theStack:
re-ACK acee5c59e6
rkrux:
ACK acee5c5
Tree-SHA512: c1841359bcb08cdd433122deef96579236928660785f3357a3eb584e47d290cd1c60ebe8f7fba50f178ba45c9a90773124e0f509e36c5a0df97c1a4890e03e5c
e3d7533ac9 test: improves tapscript unit tests (Ethan Heilman)
3e167085ba test: Ensures test fails if witness is not hex (Ethan Heilman)
Pull request description:
This commit creates new test utilities for future Taproot script tests within script_tests.json. The key features of this commit are the addition of three new tags: `#SCRIPT#`, `#CONTROLBLOCK#`, and `#TAPROOTOUTPUT#`. These tags streamline the test creation process by eliminating the need to manually generate these components outside the test suite.
* `#SCRIPT#`: Parses Tapscript and outputs a byte string of opcodes.
* `#CONTROLBLOCK#`: Automatically generates the control block for a given Taproot output.
* `#TAPROOTOUTPUT#`: Generates the final Taproot scriptPubKey.
This code was originally part of the OP_CAT PR https://github.com/bitcoin/bitcoin/pull/29247 but was pulled out into a separate PR to reduce the rebase treadmill for the OP_CAT PR.
Additionally this PR adds a check to ensure that if the witness data can not be parsed as hex the test fails. Prior to this PR, the test code would fail silently and set the values it couldn't parse as empty stack elements. This fix was suggested by @instagibbs.
## Rationale
While writing JSON script tests (script_tests.json) for https://github.com/bitcoin/bitcoin/pull/29247 we ran into the following problem. The JSON script tests are simple and easy to write for pre-Tapscript scripts, but adding or changing a Tapscript test requires substantial work per test. Consider the following pre-tapscript test:
```
["'aa' 'bb'", "CAT 0x4c 0x02 0xaabb EQUAL", "P2SH,STRICTENC", "DISABLED_OPCODE", "CAT disabled"]
````
whereas a Tapscript test for the same script (annotated with comments for better readability) would look like:
```
[
[
"aa",
"bb",
"7e4c02aabb87", // output script
"c0d6889cb081036e0faefa3a35157ad71086b123b2b144b649798b494c300a961d", // control block
0.00000001
],
"",
"0x51 0x20 0x15048ed3a65748549c27b671936987093cf73a4c9cb18522a74fb9553060ca99", // Tapscript output
"P2SH,WITNESS,TAPROOT",
"OK",
"TAPSCRIPT CATs aa and bb together and checks if EQUAL to aabb"
]
```
Computing the Tapscript output, such as `0x51 0x20 0x15048ed3a65748549c27b671936987093cf73a4c9cb18522a74fb9553060ca99`, requires writing custom code and running it for each test. The same is true for the Tapscript control block, such as `c0d6889cb081036e0faefa3a35157ad71086b123b2b144b649798b494c300a961d`. If a test is changed or updated new outputs and control blocks must be computed. The complexity of doing this is likely the reason that no one has added any Tapscript tests to JSON script tests until this PR.
In this PR we address this issue by adding the following improvements to JSON script tests:
Adding simple macros ("#SCRIPT# and #CONTROLBLOCK#) that allow the script test parser to automatically generate and inject a valid Tapscript output and control block to be computed automatically from the JSON script.
Allowing Tapscript scripts to use the human readable strings like pre-script scripts by marking the location of the script in the witness stack using #SCRIPT#. This transforms the unreadable script 7e4c02aabb87 into #SCRIPT# CAT 0x4c 0x02 0xaabb EQUAL.
This results in the following JSON script test which is far easier to write and easier to read.
```
[
[
"aa",
"bb",
"#SCRIPT# CAT",
"#CONTROLBLOCK#",
0.00000001
],
"",
"0x51 0x20 #TAPROOTOUTPUT#",
"P2SH,WITNESS,TAPROOT,OP_CAT",
"OK",
"TAPSCRIPT Test of OP_CAT flag by calling CAT on two elements. TAPSCRIPT_OP_CAT flag is set so CAT is executed."
],
```
ACKs for top commit:
instagibbs:
reACK e3d7533ac9
sipa:
utACK e3d7533ac9
janb84:
Re ACK [e3d7533](e3d7533ac9)
Tree-SHA512: 948c3ec28a4b2b222c2d77e48918ed19d298b51d64662fc20959073edd9978fc796516a392da9755a7e173f556e3021816dc6ce8eb3ed16bbe0fa6ebc574fd48
Observed on local machine running Windows / Python v3.13.1 when overriding rpc_timeout to small values (5- seconds). Next commit performs such overrides.
This commit creates new test utilities for future Taproot script
tests within script_tests.json. The key features of this commit are the
addition of three new tags: `#SCRIPT#`, `#CONTROLBLOCK#`, and
`#TAPROOTOUTPUT#`. These tags streamline the test creation process by
eliminating the need to manually generate these components outside the
test suite.
* `#SCRIPT#`: Parses Tapscript and outputs a byte string of opcodes.
* `#CONTROLBLOCK#`: Automatically generates the control block for a given
Taproot output.
* `#TAPROOTOUTPUT#`: Generates the final Taproot scriptPubKey.
Update src/test/script_tests.cpp
Co-authored-by: Jan B <608446+janb84@users.noreply.github.com>
After fixing the nTime variable name, the test_pruneheight_undo_presence
test began failing because node 2, which is involved in reorg testing,
could be on a different chain than other nodes. This caused failures
when trying to fetch blocks from other nodes that didn't recognize
node 2's chain.
Switch to using node 5 instead, which is also a pruned node but isn't
involved in reorg testing, ensuring it stays on the same chain as the
other nodes. This allows the block fetching to work as intended in the
test.
Fix incorrect variable name in comment (nTimes -> nTime) in
feature_pruning.py. This typo caused the test to always reset
mine_large_blocks.nTime to 0, rather than only on the first run
as intended.
(Still tolerate calling it on a no longer (self.)running node, as in a node that has been queried for is_node_stopped() and modified state before returning True).
Tests should not attempt to use the non-functioning RPC interface to call stop() unless wait_for_connections() has succeeded.
No longer log and suppress http.client.CannotSendRequest as a consequence of stop()-RPC, as error conditions causing this knock-on issue are now guarded against before the call.
When an RPC connection attempt with bitcoind times out, include which ignored errors occurred in the exception message.
May provide clues of what has gone wrong.
ConnectionResetError is an OSError as well (ECONNRESET), no reason to have a separate except-block for it.
Also improves comments for other exceptions and make condition above more Pythonic.
18a035145d ci: Temporarily disable `WalletMigration` benchmark (Hennadii Stepanov)
Pull request description:
The `WalletMigration` benchmark is currently failing on CI.
This PR temporarily disables it until the issue is resolved.
An alternative to https://github.com/bitcoin/bitcoin/pull/32302.
ACKs for top commit:
maflcko:
lgtm ACK 18a035145d
TheCharlatan:
ACK 18a035145d
Tree-SHA512: bb1451fd0743a2955216a6d06916e411420a76bfed8b69ffcfadf99d0996d8f3b89ed72f855f25269f943ca4c3b4422065fde2374a1bf76c8bb64f14ab883092
d85895e5a7 build, msvc: Build only required `qtbase` features (Hennadii Stepanov)
fe5a6dcc53 build, msvc: Update vcpkg manifest baseline (Hennadii Stepanov)
Pull request description:
This PR:
1. Updates the vcpkg manifest baseline from the [2024.09.30 Release](https://github.com/microsoft/vcpkg/releases/tag/2024.09.30) to the [2025.03.19 Release](https://github.com/microsoft/vcpkg/releases/tag/2025.03.19), with the following package changes:
- boost: 1.85.0#1,2 --> 1.87.0
- qtbase: 6.7.2#3 -> 6.8.2#1
- qttools: 6.7.2#1 -> 6.8.2
- sqlite3: 3.46.1 --> 3.49.1
The previous update was made in https://github.com/bitcoin/bitcoin/pull/31186.
3. Explicitly specifies required features for the `qtbase` package, which makes vcpkg skip unused features such as `dnslookup`, `openssl`, etc.
ACKs for top commit:
hodlinator:
ACK d85895e5a7
Tree-SHA512: e05fb50825c6ba2e0caf8ded1dfb55fa42f28f06dcd28597f748c1843a1c4cc888e62ad87b6481c66481a576fa224dd4943fe76e286ff579c6d4dc2ec6a78a71
5cb1241814 feefrac: avoid integer overflow in temporary (Pieter Wuille)
Pull request description:
In `FeeFrac::Div(__int128 n, int32_t d, bool round_down)` in src/util/feefrac.h, the following line computes the result:
```c++
return quot + (mod > 0) - (mod && round_down);
```
The function can only be called under conditions where the result is in range, and thus doesn't involve any integer overflow. However, the intermediary result computed by just `quot + (mod > 0)` may still overflow if it's going to be corrected by the `- (mod && round_down)` that follows.
Fix this by balancing the two correction steps with each other first:
```c++
return quot + ((mod > 0) - (mod && round_down));
```
Fixes#32294.
ACKs for top commit:
l0rinc:
Tested ACK 5cb1241814
maflcko:
lgtm ACK 5cb1241814
achow101:
ACK 5cb1241814
Tree-SHA512: 9daaccdf9acd7652d53b52cad2dc12872558265e863acdde2d6015f885cb87c0505f9bd5be5499fc0a0eded29bec719643f6af1fbc3604518143985094226c95
e261eb8d50 tests: Add BIP 373 test vectors (Ava Chow)
26370c68d0 rpc: Include MuSig2 fields in decodepsbt (Ava Chow)
ff3d460898 psbt: Implement un/ser of musig2 fields (Ava Chow)
Pull request description:
Implements un/serialization of MuSig2 PSBT fields and prepares PSBT to be able to sign for MuSig2 inputs.
Split from #29675
ACKs for top commit:
fjahr:
re-ACK e261eb8d50
theStack:
re-ACK e261eb8d50
rkrux:
tACK e261eb8d50
Tree-SHA512: bb852ad074978847ac4dc656332025e2d4d1025d4283537b89618c7cadd61a8ecd2eff24779b8a014bc8d7b431125060449768192fa05ad0577f29e3c64b2374
2835216ec0 txgraph: make GroupClusters use partition numbers directly (optimization) (Pieter Wuille)
c72c8d5d45 txgraph: compare sequence numbers instead of Cluster* (bugfix) (Pieter Wuille)
Pull request description:
Part of cluster mempool: #30289
The implicit transaction ordering for transactions in a TxGraphImpl is defined by:
1. higher chunk feerate first
2. lower Cluster* object pointer first
3. lower position within cluster linearization first.
Number (2) is not deterministic, as it intricately depends on the heap allocation algorithm. Fix this by giving each Cluster a unique `uint64_t m_sequence` value, and sorting by those instead.
The second commit then uses this new approach to optimize GroupClusters a bit more, avoiding some repeated checks and dereferences, by making a local copy of the involved sequence numbers.
Thanks to @dergoegge for pointing this out.
ACKs for top commit:
instagibbs:
reACK 2835216ec0
marcofleon:
ACK 2835216ec0
glozow:
utACK 2835216ec0
Tree-SHA512: d772a55b9ed620159b934a42a39fca7f900d4aa89c099a280a0c61ea0bd7c4fc39b388281ffc775064ea77b0b17263871b4c9763aa71c710a79287d5eb2cd4b4
fa6a007b8e fuzz: Avoid integer sanitizer warnings in policy_estimator target (MarcoFalke)
Pull request description:
It seems odd to write a fuzz target to trigger integer sanitizer warnings in `CBlockPolicyEstimator::processBlockTx` and then suppress them. If the scenario can happen in reality, the code should be properly fixed to handle the cases. If not, it seems better to fix the fuzz target to not trigger meaningless traces.
Do that here by keeping track of the current height and limiting mempool entries to at most this entry height.
ACKs for top commit:
brunoerg:
ACK fa6a007b8e
dergoegge:
utACK fa6a007b8e
Tree-SHA512: 2092017dc309fb095fe5d43cfb76efb691795f303d567ee919be2b5cac19a944293636229903dc4d1e8b9fe5daf9dc3058544321eff1735f91f804c3baa36cd0
Also, fix the incorrect documention of the 'replaceable' RPC argument
with respect to sequence number handling. The docs were incorrect
before, so the fix could be extracted, but it seems fine to include here
as well.
9ccee9cd02 doc: Document WITH_EXTERNAL_LIBMULTIPROCESS build option better (Ryan Ofsky)
Pull request description:
This includes a cmake documentation change suggested https://github.com/bitcoin/bitcoin/pull/31741#discussion_r2039716492 and another change to mention the option in markdown documentation
ACKs for top commit:
hebasto:
ACK 9ccee9cd02, changes look good.
TheCharlatan:
ACK 9ccee9cd02
Tree-SHA512: c9103b001b970ac57afedc6dc384091f5661975d569573e93003cbd7df1891c54cefb06d7296eac5b9a5c57251803dcab2bd3b26c9d81aa476c62f211dcb3d6e
27f11217ca ci: drop -priority-level from bench in win cross CI (fanquake)
Pull request description:
So there's at least one CI sanity checking all benchmarks.
Related to #32277.
ACKs for top commit:
l0rinc:
utACK 27f11217ca
hebasto:
ACK 27f11217ca.
mabu44:
utACK 27f11217ca
Tree-SHA512: 4853584bf9db418f6e31aa0f558d08bc45479d672b193e1d25a25907f82fb225bc4388321f8f23286cd9fd9168c7546c713829607eb0cf5e3c62b98e88f8e68b
faeb1babe2 ci: refactor: Use version id over version codename consistently (MarcoFalke)
fae322a43a ci: Slim down lint image (MarcoFalke)
3333273a8f ci: Bump lint imagefile FROM base (MarcoFalke)
Pull request description:
Currently, the lint_test_runner is built and installed into the lint CI image. This is problematic, because it triggers a full image build on every change to its source code. Doing a build of the lint test_runner on every run is easier and faster.
ACKs for top commit:
l0rinc:
ACK faeb1babe2
janb84:
Re- ACK [faeb1ba](faeb1babe2)
Tree-SHA512: 39103e61ec2587096213bc1ce55b80087f6f03775592827d8c96a366453b798570d912690bf96fde4685798e5fc8ee2695ce851f473b4c8782d1a4c50c65a594
7912cd4125 bench: Fix WalletMigration benchmark (pablomartin4btc)
Pull request description:
The keys and scripts created for the Legacy Wallet needed to be persisted in order for the migration to work properly.
Fixes#32277.
ACKs for top commit:
achow101:
ACK 7912cd4125
davidgumberg:
Tested ACK 7912cd4125
furszy:
utACK 7912cd4125
Tree-SHA512: fe7b8e0a80d4d030ad3fd6446717ee09a260ab2bd6140bc817bdca52d233e3af8a8fed2d754743ca2ba022f7d2c8615a36b5070991d12942c13835e8f72e359f
7a93544cdc doc: Fix and clarify description of ZMQ message format (Jiri Jakes)
Pull request description:
This change stresses that all ZMQ messages share the same structure and that they differ only in the format of the bodies. Previously this was not clear.
Further it removes the notion of endianness of 32-byte hashes, as it was misleading, and replaces it with the term 'reversed byte order' (as opposed to natural or normal byte order produced by hashing functions).
Additionally, it states that ZMQ 32-byte hashes are in the same format as in RPC. Previously it incorrectly stated that the two were in different formats.
[Rendered](https://github.com/jirijakes/bitcoin/blob/zmq-doc/doc/zmq.md).
Fixes#31856.
ACKs for top commit:
w0xlt:
Code review ACK 7a93544cdc
achow101:
ACK 7a93544cdc
ryanofsky:
Code review ACK 7a93544cdc. Nice changes. Documentation seems less repetitive and easier to understand now
Tree-SHA512: 8c5ab047c5fd9b5b6910d691b725886d7743dfd01510735b46e43d01c2d0d25ec52d79d71ec75dbeb142e96a88ad503d69ee14b971e3cdaeb8fd85e5292a8c21
a4041c77f0 test: Handle empty string returned by CLI as None in RPC tests (Brandon Odiwuor)
Pull request description:
Partially Fixes https://github.com/bitcoin/bitcoin/issues/32264
Some tests are failing when `bitcoin-cli` returns an empty string. This change treats an empty response as `None`. See https://github.com/bitcoin/bitcoin/issues/32264#issuecomment-2807616694
This fixes the error for:
- feature_bip68_sequence.py
- feature_nulldummy.py
- feature_signet.py
- mining_mainnet.py
- rpc_scanblocks.py
- rpc_scantxoutset.py
- wallet_descriptor.py --descriptors
ACKs for top commit:
maflcko:
lgtm ACK a4041c77f0
achow101:
ACK a4041c77f0
pablomartin4btc:
ACK a4041c77f0
mzumsande:
ACK a4041c77f0
Tree-SHA512: 2f1a416a18e0b3eebdb014c2e2e8dadf1d46b15c231cb61f577d47f5e551994ab0e2aeb7c179c01be7c1f07ebc03476236d29cf2d04c358ffb1fae985aa385c9
32dcec269b rpc: update RPC help of `createpsbt` (rkrux)
931117a46f rpc: update the doc for `data` field in `outputs` argument (rkrux)
8134a6b5d4 rpc: add cli example for `walletcreatefundedpsbt` RPC (rkrux)
Pull request description:
### add cli example for `walletcreatefundedpsbt` and `createpsbt` RPCs
The only example present earlier was one that creates an OP_RETURN output. This
lack of examples has discouraged me earlier to use this RPC. Adding an example
that creates PSBT sending bitcoin to address, a scenario that is much more common.
### rpc: update the doc for `data` field in `outputs` argument
It was not evident to me that this field creates an `OP_RETURN` output until
I read the code and tried it out. Thus, making the doc explicitly mention it.
This affects docs of the following RPCs:
`bumpfee`, `psbtbumpfee`, `send`, `walletcreatefundedpsbt`, `createpsbt`,
and `createrawtransaction`
ACKs for top commit:
sipa:
utACK 32dcec269b
1440000bytes:
utACK 32dcec269b
achow101:
ACK 32dcec269b
ryanofsky:
Concept ACK 32dcec269b. These seem like helpful clarifications, but I did not look into the details
Tree-SHA512: f994488ba7d52d00960fc52064bb419cf548e29822fe23d6ee0452fdf514dd93f089145eddb32b8086a7918cf8cf33a4c3f16bfcb7948f3c9d5afd95e8d3a1cb
7749d929a0 Remove support for RNDR/RNDRRS for aarch64 on Linux (laanwj)
Pull request description:
This hardware feature is
- Rarely supported on SoCs (and broken on like half of the chips that support it in the first place) (#31817). It is not clear if, or how, the brokenness will be worked around in the kernel, but working around it in user space seems the wrong thing to do, this is not the place to maintain special workarounds for specific hardware (which despite that, was attempted in #31826, but had to be reverted in #31908 due to other problems).
- Apparently not compiled into the release binary anymore (https://github.com/bitcoin/bitcoin/issues/31817#issuecomment-2795885962). Did check this at the time, but a build system change must have caused this, and went undetected.
- Hard to test in CI (as well as manually), due to unavailability of hardware.
Better to remove it.
This reverts commit aee5404e02 from #26839.
Closes#31817.
ACKs for top commit:
sipa:
utACK 7749d929a0
davidgumberg:
utACK 7749d929a0
achow101:
ACK 7749d929a0
w0xlt:
utACK 7749d929a0
Tree-SHA512: d243ad7f745fb46f711f24b6983d9ea1d94e5d8ee60959229bafdba5caa210a60801a1c2cb5b558a0e72f365371b32285aee9a8d0cd24a60589adc7b03dd6a44
Extend `getFrameworks()` with an optional `rpath` parameter to replace
`@rpath` in dependency paths. This fixes resolution of framework
dependencies when using Homebrew's `qt@6` package.
Previously ChainstateManager::AcceptBlockHeader would log when it
saw a new header. This commit moves logging to the call site(s) in
net_processing. The next commits will then log which peer sent it
and whether it was part of a compact block.
This commit changes behavior:
- when multiple headers are received in a single message, only the
last one is logged
- if any of the headers are invalid, the valid ones are not logged
This happens because net_processing calls ProcessNewBlockHeaders
with multiple headers, which then calls AcceptBlockHeader one
header at a time.
Additionally:
- when the header is received via a compact block, there's no more
duplicate log (a later commit also unifies logging code paths)
Instead of GetPrivKey returning a key and having the caller fill the
FlatSigningProvider, have GetPrivKey take the FlatSigningProvider and
fill it by itself. This will be necessary for descriptors such as
musig() where there are private keys that need to be added to the
FlatSigningProvider but do not directly appear in any resulting scripts.
GetPrivKey is now changed to void as the caller no longer cares whether
it succeeds or fails.
Instead of having ExpandHelper fill in the origins in the
FlatSigningProvider output, have GetPubKey do it by itself. This reduces
the extra variables needed in order to track and set origins in
ExpandHelper.
Also changes GetPubKey to return a std::optional<CPubKey> rather than
using a bool and output parameters.
Legacy wallets should only import keys to the keypool if they came in a
single key descriptor. Instead of relying on assumptions about the
descriptor based on how many pubkeys show up after expanding the
descriptor, explicitly mark descriptors as being single key type and use
that for the check.
05117e6e17 rpc: clarify longpoll behavior (Sjors Provoost)
5315278e7c Have createNewBlock() wait for a tip (Sjors Provoost)
64a2795fd4 rpc: handle shutdown during long poll and wait methods (Sjors Provoost)
a3bf43343f rpc: drop unneeded IsRPCRunning() guards (Sjors Provoost)
f9cf8bd0ab Handle negative timeout for waitTipChanged() (Sjors Provoost)
Pull request description:
This PR prevents Mining interface methods from sometimes crashing when called during startup before a tip is connected. It also makes other improvements like making more RPC methods usable from the GUI. Specifically this PR:
- Adds an `Assume` check to disallow passing negative timeout values to `Mining::waitTipChanged`
- Makes `waitfornewblock`, `waitforblock` and `waitforblockheight` RPC methods usable from the GUI when `-server=1` is not set.
- Changes `Mining::waitTipChanged` to return `optional<BlockRef>` instead of `BlockRef` and return `nullopt` instead of crashing if there is a timeout or if the node is shut down before a tip is connected.
- Changes `Mining::waitTipChanged` to not time out before a tip is connected, so it is convenient and safe to call during startup, and only returns `nullopt` on early shutdowns.
- Changes `Mining::createNewBlock` to block and wait for a tip to be connected if it is called on startup instead of crashing. Also documents that it will return null on early shutdowns.
This allows `waitNext()` (added in https://github.com/bitcoin/bitcoin/pull/31283) to safely assume `TipBlock()` isn't `null`, not even during a scenario of early shutdown.
Finally this PR clarifies long poll behaviour, mostly by adding code comments, but also through an early `break`.
ACKs for top commit:
achow101:
ACK 05117e6e17
ryanofsky:
Code review ACK 05117e6e17, just updated a commit message since last review
TheCharlatan:
ACK 05117e6e17
vasild:
ACK 05117e6e17
Tree-SHA512: 277c285a6e73dfff88fd379298190b264254996f98b93c91c062986ab35c2aa5e1fbfec4cd71d7b29dc2d68e33f252b5cfc501345f54939d6bd78599b71fec04
fa21f83d29 ci: Use G++ in valgrind tasks (MarcoFalke)
fabd05bf65 refactor: Fix net_processing iwyu includes (MarcoFalke)
fa1622db20 refactor: Make node_id a const& in RemoveBlockRequest (MarcoFalke)
Pull request description:
Currently, `valgrind` is not usable on a default build with GCC. Specifically, `p2p_compactblocks.py --valgrind` gives a false-positive in `RemoveBlockRequest` when comparing `node_id` with `from_peer`. According to the upstream bug report, this happens because both symbols are on the stack and the compiler can more aggressively optimize the compare (order). See https://bugs.kde.org/show_bug.cgi?id=472329#c7
It is possible to work around this bug by pulling at least one value from the stack. For example, by making `from_peer` a `const` reference. Alternatively, by replacing `auto [node_id, list_it]` with `const auto& [node_id, list_it]`, which is done here.
I think this workaround is acceptable, because it does not look like valgrind can trivially fix this. The alternative would be to add a (temporary?) suppression.
Fixes https://github.com/bitcoin/bitcoin/issues/27741
Also, fix iwyu includes, while touching this module.
Also, switch the CI valgrind scripts to use G++.
ACKs for top commit:
achow101:
ACK fa21f83d29
TheCharlatan:
ACK fa21f83d29
darosior:
utACK fa21f83d29
ryanofsky:
Code review ACK fa21f83d29. Code changes all look good but I'm a little confused about purpose of the third commit, so left a question about that
Tree-SHA512: 7b92cdafd525a5ac53ae2c1a7a92e599bc9b5fd5d315a694b493cd5079ac323d884393b57aa18581b7789247a588c9a27d47698de25b340bc76fc9f1dd1850b4
The obfuscation (XOR) operations are currently done byte-by-byte during serialization. Buffering the reads will enable batching the obfuscation operations later.
Different operating systems handle file caching differently, so reading larger batches (and processing them from memory) is measurably faster, likely because of fewer native fread calls and reduced lock contention.
Note that `ReadRawBlock` doesn't need buffering since it already reads the whole block directly.
Unlike `ReadBlockUndo`, the new `ReadBlock` implementation delegates to `ReadRawBlock`, which uses more memory than a buffered alternative but results in slightly simpler code and a small performance increase (~0.4%). This approach also clearly documents that `ReadRawBlock` is a logical subset of `ReadBlock` functionality.
The current implementation, which iterates over a fixed-size buffer, provides a more general alternative to Cory Fields' solution of reading the entire block size in advance.
Buffer sizes were selected based on benchmarking to ensure the buffered reader produces performance similar to reading the whole block into memory. Smaller buffers were slower, while larger ones showed diminishing returns.
------
> macOS Sequoia 15.3.1
> C++ compiler .......................... Clang 19.1.7
> cmake -B build -DBUILD_BENCH=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ && cmake --build build -j$(nproc) && build/bin/bench_bitcoin -filter='ReadBlockBench' -min-time=10000
Before:
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 2,271,441.67 | 440.25 | 0.1% | 11.00 | `ReadBlockBench`
After:
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 1,738,971.29 | 575.05 | 0.2% | 10.97 | `ReadBlockBench`
------
> Ubuntu 24.04.2 LTS
> C++ compiler .......................... GNU 13.3.0
> cmake -B build -DBUILD_BENCH=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ && cmake --build build -j$(nproc) && build/bin/bench_bitcoin -filter='ReadBlockBench' -min-time=20000
Before:
| ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
| 6,895,987.11 | 145.01 | 0.0% | 71,055,269.86 | 23,977,374.37 | 2.963 | 5,074,828.78 | 0.4% | 22.00 | `ReadBlockBench`
After:
| ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
| 5,771,882.71 | 173.25 | 0.0% | 65,741,889.82 | 20,453,232.33 | 3.214 | 3,971,321.75 | 0.3% | 22.01 | `ReadBlockBench`
Co-authored-by: maflcko <6399679+maflcko@users.noreply.github.com>
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: Martin Leitner-Ankerl <martin.ankerl@gmail.com>
Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
Made every OpenBlockFile#fReadOnly value explicit.
Replaced hard-coded values in ReadRawBlock with STORAGE_HEADER_BYTES.
Changed `STORAGE_HEADER_BYTES` and `UNDO_DATA_DISK_OVERHEAD` to `uint32_t` to avoid casts.
Also added `LIFETIMEBOUND` to the `AutoFile` parameter of `BufferedFile`, which stores a reference to the underlying `AutoFile`, allowing Clang to emit warnings if the referenced `AutoFile` might be destroyed while `BufferedFile` still exists.
Without this attribute, code with lifetime violations wouldn't trigger compiler warnings.
Co-authored-by: maflcko <6399679+maflcko@users.noreply.github.com>
Renames the constant to be less verbose and better reflect its purpose:
it represents the size of the storage header that precedes serialized block data on disk,
not to be confused with a block's own header.
-BEGIN VERIFY SCRIPT-
git grep -q "STORAGE_HEADER_BYTES" $(git ls-files) && echo "Error: Target name STORAGE_HEADER_BYTES already exists in the codebase" && exit 1
sed -i 's/BLOCK_SERIALIZATION_HEADER_SIZE/STORAGE_HEADER_BYTES/g' $(git grep -l 'BLOCK_SERIALIZATION_HEADER_SIZE')
-END VERIFY SCRIPT-
Reorganized error handling in block-related operations by grouping related operations together within the same scope.
In `ReadBlockUndo()` and `ReadBlock()`, moved all deserialization operations, comments and checksum verification inside a single try/catch block for cleaner error handling.
In `WriteBlockUndo()`, consolidated hash calculation and data writing operations within a common block to better express their logical relationship.
8fe001d597 doc: Updates how to reproduce fuzz CI failure locally (Sergi Delgado Segura)
Pull request description:
The current version of the doc does not explain how to reproduce a recent fuzzing CI failure (not yet part of the corpora). Add instructions on how to manually create a crash file based on a report.
ACKs for top commit:
maflcko:
lgtm ACK 8fe001d597
glozow:
ACK 8fe001d597
Tree-SHA512: 7436d71a30bbbffc34770027f1deeacca2de528d8d1b333431d6070c2ba779ecfcdaf25dc791d2154ba4dd37824d06aed2695a8412d7ca1f29e5bd1796d42aeb
Currently, the lint_test_runner is built and installed into the lint CI
image. This is problematic, because it triggers a full image build on
every change to its source code. Doing a build of the lint test_runner
on every run is easier and faster.
babb9f5db6 depends: remove non-native libmultiprocess build (Cory Fields)
5d105fb8c3 depends: Switch libmultiprocess packages to use local git subtree (Ryan Ofsky)
9b35518d2f depends, moveonly: split up int_get_build_id function (Ryan Ofsky)
2d373e2707 lint: Add exclusions for libmultiprocess subtree (Ryan Ofsky)
e88ab394c1 doc: Update documentation to explain libmultiprocess subtree (Ryan Ofsky)
d4bc563982 cmake: Fix clang-tidy "no input files" errors (Ryan Ofsky)
abdf3cb645 cmake: Fix warnings from boost headers (Ryan Ofsky)
8532fcb1c3 cmake: Fix ctest mptest "Unable to find executable" errors (Ryan Ofsky)
d597ab1dee cmake: Support building with libmultiprocess subtree (Ryan Ofsky)
69f0d4adb7 scripted-diff: s/WITH_MULTIPROCESS/ENABLE_IPC/ in cmake (Ryan Ofsky)
a2f28e4be9 Squashed 'src/ipc/libmultiprocess/' content from commit 35944ffd23fa (Ryan Ofsky)
d6244f85c5 depends: Update libmultiprocess library to simplify cmake subtree build (Ryan Ofsky)
Pull request description:
This adds the [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library and code generator as a subtree in `src/ipc/libmultiprocess` and allows it to be built with the cmake `-DENABLE_IPC` option, which is disabled by default.
This PR does not entirely remove the depends system [libmultiprocess package](https://github.com/bitcoin/bitcoin/blob/master/depends/packages/native_libmultiprocess.mk) because the package is useful when cross compiling. (A cross-compiling cmake build cannot easily build and run a native code generation tool.) However, it does update the depends package to build from the new git subtree, instead of being downloaded separately from github, so the same sources are used to build both the runtime library and the code generator.
This PR includes the following manual changes (not created automatically with `git subtree add`) which just update the build system and documentation:
- [`d6244f85c509` depends: Update libmultiprocess library to simplify cmake subtree build](d6244f85c5)
- [`69f0d4adb72c` scripted-diff: s/WITH_MULTIPROCESS/ENABLE_IPC/ in cmake](69f0d4adb7)
- [`d597ab1dee6b` cmake: Support building with libmultiprocess subtree](d597ab1dee)
- [`8532fcb1c30d` cmake: Fix ctest mptest "Unable to find executable" errors](8532fcb1c3)
- [`abdf3cb6456f` cmake: Fix warnings from boost headers](abdf3cb645)
- [`d4bc5639829f` cmake: Fix clang-tidy "no input files" errors](d4bc563982)
- [`e88ab394c163` doc: Update documentation to explain libmultiprocess subtree](e88ab394c1)
- [`2d373e27071f` lint: Add exclusions for libmultiprocess subtree](2d373e2707)
- [`9b35518d2f3f` depends, moveonly: split up int_get_build_id function](9b35518d2f)
- [`5d105fb8c3ff` depends: Switch libmultiprocess packages to use local git subtree](5d105fb8c3)
- [`babb9f5db641` depends: remove non-native libmultiprocess build](babb9f5db6)
---
Previous minisketch subtree PR #23114 may be useful for comparison
Instructions for subtree verification can be found:
- https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#subtrees
- https://github.com/bitcoin/bitcoin/tree/master/test/lint#git-subtree-checksh
TL&DR:
```sh
git remote add --fetch libmultiprocess https://github.com/chaincodelabs/libmultiprocess.git
test/lint/git-subtree-check.sh -r src/ipc/libmultiprocess
```
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
Sjors:
re-ACK babb9f5db6
TheCharlatan:
tACK babb9f5db6
vasild:
ACK babb9f5db6
Tree-SHA512: 43d4eecca5aab63e55c613de935965666eaced327f9fe859a0e9c9b85f7685dc16c5c8d6e03e09ca998628c5d468633f4f743529930b037049abe8e0101e0143
This hardware feature is
- rarely supported on SoCs (and broken on like half of the chips that support it in the first place) (#31817)
- apparently not compiled into the release binary (https://github.com/bitcoin/bitcoin/issues/31817#issuecomment-2795885962)
- hard to test in CI, due to unavailable of hardware
Better to remove it.
This reverts commit aee5404e02.
Closes#31817.
ff0194a7ce miniscript: convert non-critical asserts to CHECK_NONFATAL (Antoine Poinsot)
Pull request description:
The Miniscript code contains assertions to prevent ending up in an insane state or prevent UB, but also to enforce logical invariants. For the latter it is not necessary to crash the program if they are broken. Raising an exception suffices, especially as this code is often called through the RPC interface which can in turn handle the exception and the user can report it to developers.
This revives #28678 from Pieter Wuille.
ACKs for top commit:
hodlinator:
ACK ff0194a7ce
TheCharlatan:
ACK ff0194a7ce
brunoerg:
code review ACK ff0194a7ce
Tree-SHA512: 8ed8f7b494e46ecf7cdebe75120cd0ffe543b6bc289bf882dac631fe2ec2cae590d5f7bc2316e52db085791694b136dffbc71c40c1e16886fa53ab00bd8cabd0
ec81a72b36 net: Add randomized prefix to Tor stream isolation credentials (laanwj)
c47f81e8ac net: Rename `_randomize_credentials` Proxy parameter to `tor_stream_isolation` (laanwj)
Pull request description:
Add a class TorsStreamIsolationCredentialsGenerator that generates unique credentials based on a randomly generated session prefix and an atomic counter. Use this in `ConnectThroughProxy` instead of a simple atomic int counter.
This makes sure that different launches of the application won't share the same credentials, and thus circuits, even in edge cases.
Example with `-debug=proxy`:
```
2025-03-31T16:30:27Z [proxy] SOCKS5 sending proxy authentication 0afb2da441f5c105-0:0afb2da441f5c105-0
2025-03-31T16:30:31Z [proxy] SOCKS5 sending proxy authentication 0afb2da441f5c105-1:0afb2da441f5c105-1
```
Thanks to hodlinator in https://github.com/bitcoin/bitcoin/pull/32166#discussion_r2020973352 for the idea.
ACKs for top commit:
hodlinator:
re-ACK ec81a72b36
jonatack:
ACK ec81a72b36
danielabrozzoni:
tACK ec81a72b36
Tree-SHA512: 195f5885fade77545977b91bdc41394234ae575679cb61631341df443fd8482cd74650104e323c7dbfff7826b10ad61692cca1284d6810f84500a3488f46597a
The current version of the doc does not explain how to reproduce a recent fuzzing CI failure
(not yet part of the corpora). Add instructions on how to manually create a crash file based
on a report.
faa3ce3199 fuzz: Avoid influence on the global RNG from peerman m_rng (MarcoFalke)
faf4c1b6fc fuzz: Disable unused validation interface and scheduler in p2p_headers_presync (MarcoFalke)
fafaca6cbc fuzz: Avoid setting the mock-time twice (MarcoFalke)
fad22149f4 refactor: Use MockableSteadyClock in ReportHeadersPresync (MarcoFalke)
fa9c38794e test: Introduce MockableSteadyClock::mock_time_point and ElapseSteady helper (MarcoFalke)
faf2d512c5 fuzz: Move global node id counter along with other global state (MarcoFalke)
fa98455e4b fuzz: Set ignore_incoming_txs in p2p_headers_presync (MarcoFalke)
faf2e238fb fuzz: Shuffle files before testing them (MarcoFalke)
Pull request description:
This should make the `p2p_headers_presync` fuzz target more deterministic.
Tracking issue: https://github.com/bitcoin/bitcoin/issues/29018.
The first commits adds an `ElapseSteady` helper and type aliases. The second commit uses those helpers in `ReportHeadersPresync` and in the fuzz target to increase determinism.
### Testing
It can be tested via (setting 32 parallel threads):
```
cargo run --manifest-path ./contrib/devtools/deterministic-fuzz-coverage/Cargo.toml -- $PWD/bld-cmake/ $PWD/../b-c-qa-assets/fuzz_corpora/ p2p_headers_presync 32
```
The failing diff is contained in the commit messages, if applicable.
ACKs for top commit:
Crypt-iQ:
tACK faa3ce3199
janb84:
Re-ACK [faa3ce3](faa3ce3199)
marcofleon:
ACK faa3ce3199
Tree-SHA512: 7e2e0ddf3b4e818300373d6906384df57a87f1eeb507fa43de1ba88cf03c8e6752a26b6e91bfb3ee26a21efcaf1d0d9eaf70d311d1637b671965ef4cb96e6b59
868816d962 refactor: Remove SetHexDeprecated (marcofleon)
6b63218ec2 qt: Update SetHexDeprecated to FromHex (marcofleon)
Pull request description:
This is part of https://github.com/bitcoin/bitcoin/pull/32189. I'm separating this out because it's not immediately obvious that it's just a refactor. `SetHexDeprecated()` doesn't do any correctness checks on the input, while `FromHex()` does, so it's theoretically possible that there's a behavior change.
Replaces `uint256::SetHexDeprecated()` calls with `Txid::FromHex()` in four locations:
- `TransactionTableModel::updateTransaction`
- `TransactionView::contextualMenu`
- `TransactionView::abandonTx`
- `TransactionView::bumpFee`
The input strings in these cases aren't user input, so they should only be valid hex strings from `GetHex()` (through `TransactionRecord::getTxHash()`). These conversions should be safe without additional checks.
ACKs for top commit:
laanwj:
Code review ACK 868816d962
w0xlt:
Code review ACK 868816d962
BrandonOdiwuor:
Code Review ACK 868816d962
TheCharlatan:
ACK 868816d962
hebasto:
ACK 868816d962, I have reviewed the code and it looks OK.
Tree-SHA512: 121f149dcc7358231d0327cb3212ec96486a88410174d3c74ab8cbd61bad35185bc0a9740d534492b714811f72a6736bc7ac6eeae590c0ea1365c61cc791da37
a2bc330da8 feefrac test: avoid integer overflow (bugfix) (Pieter Wuille)
Pull request description:
The `feefrac_mul_div` fuzz test fails after #30535 with the following (base64) input: `Nb6Fc/97AACAAAD/ewAAgAAAAIAAAACAAAAAoA==` (see https://cirrus-ci.com/task/5240029192126464?logs=ci#L3353).
This is caused by an internal multiplication inside `CFeeRate` that *just* exceeds the limit of the `int64_t` type. Fix that by tightening the bounds slightly further.
ACKs for top commit:
sr-gi:
utACK a2bc330da8
instagibbs:
ACK a2bc330da8
glozow:
ACK a2bc330da8, was able to reproduce + verify this fix
Tree-SHA512: cfbcdc8becfd518f4349ddc00c9af3ed0a23bb9534af71cc21df167d7038e5967127e5d97c4b3e8aeff6bf071c4f630c32ffaf81d8ec227954d21fdcbe205333
This should avoid the remaining non-determistic code coverage paths.
Without this patch, the tool would report a diff (only when running
without libFuzzer):
cargo run --manifest-path ./contrib/devtools/deterministic-fuzz-coverage/Cargo.toml -- $PWD/bld-cmake/ $PWD/../qa-assets/fuzz_corpora/ p2p_headers_presync 32
It should be sufficient to set it once. Especially, if the dynamic value
is only used by ResetAndInitialize.
This also avoids non-determistic code paths, when ResetAndInitialize may
re-initialize m_next_inv_to_inbounds.
Without this patch, the tool would report a diff:
cargo run --manifest-path ./contrib/devtools/deterministic-fuzz-coverage/Cargo.toml -- $PWD/bld-cmake/ $PWD/../qa-assets/fuzz_corpora/ p2p_headers_presync 32
...
- 1126| 3| m_next_inv_to_inbounds = now + m_rng.rand_exp_duration(average_interval);
- 1127| 3| }
+ 1126| 10| m_next_inv_to_inbounds = now + m_rng.rand_exp_duration(average_interval);
+ 1127| 10| }
1128| 491| return m_next_inv_to_inbounds;
...
This allows the clock to be mockable in tests. Also, replace cs_main
with GetMutex() while touching this function.
Also, use the ElapseSteady test helper in the p2p_headers_presync fuzz
target to make it more deterministic.
The m_last_presync_update variable is a global that is not reset in
ResetAndInitialize. However, it is only used for logging, so completely
disable it for now.
Without this patch, the tool would report a diff:
cargo run --manifest-path ./contrib/devtools/deterministic-fuzz-coverage/Cargo.toml -- $PWD/bld-cmake/ $PWD/../qa-assets/fuzz_corpora/ p2p_headers_presync 32
...
4468| 81| auto now = std::chrono::steady_clock::now();
4469| 81| if (now < m_last_presync_update + std::chrono::milliseconds{250}) return;
- ^80
+ ^79
...
This refactor clarifies that the MockableSteadyClock::mock_time_point
has millisecond precision by defining a type an using it.
Moreover, a ElapseSteady helper is added which can be re-used easily.
The global m_headers_presync_stats is not reset in ResetAndInitialize.
This may lead to non-determinism.
Fix it by incrementing the global node id counter instead.
Without this patch, the tool would report a diff:
cargo run --manifest-path ./contrib/devtools/deterministic-fuzz-coverage/Cargo.toml -- $PWD/bld-cmake/ $PWD/../qa-assets/fuzz_corpora/ p2p_headers_presync 32
...
2587| 3.73k| if (best_it == m_headers_presync_stats.end()) {
------------------
- | Branch (2587:17): [True: 80, False: 3.65k]
+ | Branch (2587:17): [True: 73, False: 3.66k]
------------------
...
When iterating over all fuzz input files in a folder, the order should
not matter.
However, shuffling may be useful to detect non-determinism.
Thus, shuffle in fuzz.cpp, when using neither libFuzzer, nor AFL.
Also, shuffle in the deterministic-fuzz-coverage tool, when using
libFuzzer.
Replace `uint256::SetHexDeprecated()` calls with `Txid::FromHex()`
in four locations:
- TransactionTableModel::updateTransaction
- TransactionView::contextualMenu
- TransactionView::abandonTx
- TransactionView::bumpFee
The input strings are generally expected to be valid hex strings
from `GetHex()`. However, due to the potentially unpredictable return
value of `.data(TransactionTableModel::TxHashRole)`, check the
`Txid::FromHex` result in `contextualMenu` and return early if the
transaction hash is invalid. The other two functions, `abandonTx`
and `bumpFee` will only be called if the context menu is enabled.
924f25f6fc bench: Match ConnectBlock tx output counts (monlovesmango)
Pull request description:
There turned out to be a mismatch in the tx output counts which caused 'ConnectBlockMixedEcdsaSchnorr' benchmark to run slower than 'ConnectBlockAllEcdsa' and 'ConnectBlockAllSchnorr'. This commit makes the tx output counts uniform across all benchmarks.
This commit also renames the 'taproot_tx' variable to 'tx' to reflect that this variable represents a general tx and not just a taproot tx.
ACKs for top commit:
davidgumberg:
Tested ACK 924f25f6fc
Prabhat1308:
reACK [`924f25f`](924f25f6fc)
janb84:
re ACK [924f25f](924f25f6fc)
josibake:
ACK 924f25f6fc
Tree-SHA512: bbf33e0c31b0c46571fd5d6ecd32426e7e823f9e156fd3d39a975bd5f0c1b6cd3dda55fa869cb0954c68dcf28cf4d0a0af40a72e440c1c78380b5b98e1eb6615
fac978fb21 test: Remove fragile and ancient release 0.17 wallet test (MarcoFalke)
Pull request description:
The test checks that the 0.17 wallet rejects wallet files created in "the future".
This is nice, and good to know. However,
* The 0.17 release is ancient and should be unused outside of tests, especially to load future wallets.
* The test intermittently fails, due to ancient RPC server bugs, that were fixed in the meantime. [1]
* Albeit they are not identical, the 0.18 release is still checked in this test, so any theoretical bug that would be caught by 0.17 is hopefully still caught by 0.18 as well.
So fix all issues by removing the test case.
[1] For example from https://api.cirrus-ci.com/v1/task/6161588714995712/logs/ci.log:
```
190/321 - [1mwallet_backwards_compatibility.py --descriptors[0m failed, Duration: 23 s
[17:21:40.700]
[17:21:40.700] [1mstdout:
[17:21:40.700] [0m2025-04-02T21:21:16.575000Z TestFramework (INFO): PRNG seed is: 5772716217847090743
[17:21:40.700] 2025-04-02T21:21:16.580000Z TestFramework (INFO): Initializing test directory /ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20250402_210134/wallet_backwards_compatibility_134
[17:21:40.700] 2025-04-02T21:21:26.378000Z TestFramework (INFO): Test wallet backwards compatibility...
[17:21:40.700] 2025-04-02T21:21:33.191000Z TestFramework (INFO): Testing 0.19 addmultisigaddress case (#18075)
[17:21:40.700] 2025-04-02T21:21:33.637000Z TestFramework (INFO): Test that a wallet made on master can be opened on:
[17:21:40.700] 2025-04-02T21:21:33.637000Z TestFramework (INFO): - 250000
[17:21:40.700] 2025-04-02T21:21:34.055000Z TestFramework (INFO): - 240001
[17:21:40.700] 2025-04-02T21:21:34.435000Z TestFramework (INFO): - 230000
[17:21:40.700] 2025-04-02T21:21:34.858000Z TestFramework (INFO): - 220000
[17:21:40.700] 2025-04-02T21:21:35.614000Z TestFramework (INFO): - 210000
[17:21:40.700] 2025-04-02T21:21:35.707000Z TestFramework (INFO): Test descriptor wallet incompatibility on:
[17:21:40.700] 2025-04-02T21:21:35.707000Z TestFramework (INFO): - 200100
[17:21:40.700] 2025-04-02T21:21:35.878000Z TestFramework (INFO): - 190100
[17:21:40.700] 2025-04-02T21:21:36.021000Z TestFramework (INFO): - 180100
[17:21:40.700] 2025-04-02T21:21:36.319000Z TestFramework (INFO): Test descriptor wallet incompatibility with 0.17
[17:21:40.700] 2025-04-02T21:21:37.328000Z TestFramework (INFO): Test that 0.21 cannot open wallet containing tr() descriptors
[17:21:40.700] 2025-04-02T21:21:37.356000Z TestFramework (INFO): Test that a wallet can upgrade to and downgrade from master, from:
[17:21:40.700] 2025-04-02T21:21:37.361000Z TestFramework (INFO): - 250000
[17:21:40.700] 2025-04-02T21:21:37.665000Z TestFramework (INFO): - 240001
[17:21:40.700] 2025-04-02T21:21:37.970000Z TestFramework (INFO): - 230000
[17:21:40.700] 2025-04-02T21:21:38.439000Z TestFramework (INFO): - 220000
[17:21:40.700] 2025-04-02T21:21:38.793000Z TestFramework (INFO): - 210000
[17:21:40.700] 2025-04-02T21:21:39.470000Z TestFramework (INFO): Stopping nodes
[17:21:40.700]
[17:21:40.700]
[17:21:40.700] [1mstderr:
[17:21:40.700] [0mTraceback (most recent call last):
[17:21:40.700] File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/functional/wallet_backwards_compatibility.py", line 389, in <module>
[17:21:40.700] BackwardsCompatibilityTest(__file__).main()
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/test_framework.py", line 206, in main
[17:21:40.700] exit_code = self.shutdown()
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/test_framework.py", line 379, in shutdown
[17:21:40.700] self.stop_nodes()
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/test_framework.py", line 643, in stop_nodes
[17:21:40.700] node.stop_node(wait=wait, wait_until_stopped=False)
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/test_node.py", line 397, in stop_node
[17:21:40.700] self.stop()
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/coverage.py", line 50, in __call__
[17:21:40.700] return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/authproxy.py", line 132, in __call__
[17:21:40.700] response, status = self._request('POST', self.__url.path, postdata.encode('utf-8'))
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/authproxy.py", line 106, in _request
[17:21:40.700] return self._get_response()
[17:21:40.700] File "/ci_container_base/test/functional/test_framework/authproxy.py", line 169, in _get_response
[17:21:40.700] http_response = self.__conn.getresponse()
[17:21:40.700] File "/usr/lib/python3.10/http/client.py", line 1375, in getresponse
[17:21:40.700] response.begin()
[17:21:40.700] File "/usr/lib/python3.10/http/client.py", line 318, in begin
[17:21:40.700] version, status, reason = self._read_status()
[17:21:40.700] File "/usr/lib/python3.10/http/client.py", line 287, in _read_status
[17:21:40.700] raise RemoteDisconnected("Remote end closed connection without"
[17:21:40.700] http.client.RemoteDisconnected: Remote end closed connection without response
[17:21:40.700] [node 10] Cleaning up leftover process
[17:21:40.700] [node 9] Cleaning up leftover process
[17:21:40.700] [node 8] Cleaning up leftover process
[17:21:40.700] [node 7] Cleaning up leftover process
[17:21:40.700] [node 6] Cleaning up leftover process
[17:21:40.700] [node 5] Cleaning up leftover process
[17:21:40.700] [node 4] Cleaning up leftover process
[17:21:40.700] [node 3] Cleaning up leftover process
[17:21:40.700] [node 2] Cleaning up leftover process
[17:21:40.700] [node 1] Cleaning up leftover process
[17:21:40.700] [node 0] Cleaning up leftover process
ACKs for top commit:
laanwj:
Code review ACK fac978fb21
janb84:
Re ACK [fac978f](fac978fb21)
pablomartin4btc:
re ACK fac978fb21
BrandonOdiwuor:
Code Review ACK fac978fb21
Tree-SHA512: 13acdfc6be4293a0ff45ae20b26ba60636e130097da380b7b51716faaa950320462399bef55e74b3cedc82944586dcc1bfd078babb96edb03c4efdb8f40af5a4
b639417b39 net: Add Tor extended SOCKS5 error codes (laanwj)
Pull request description:
Add support for reporting Tor extended SOCKS5 error codes as defined here:
- https://spec.torproject.org/socks-extensions.html#extended-error-codes
- https://gitlab.torproject.org/tpo/core/arti/-/blob/main/crates/tor-socksproto/src/msg.rs?ref_type=heads#L183
These give a more direct indication of the problem in case of errors connecting to hidden services, for example:
```
2025-04-02T10:34:13Z [net] Socks5() connect to [elided].onion:8333 failed: onion service descriptor can not be found
```
In the C Tor implementation, to get these one should set the "ExtendedErrors" flag on the "SocksPort" definition, introduced in version 0.4.3.1.
In Arti, extended error codes are always enabled.
Also, report the raw error code in case of unknown reply values.
ACKs for top commit:
1440000bytes:
utACK b639417b39
w0xlt:
utACK b639417b39
pablomartin4btc:
utACK b639417b39
Tree-SHA512: b30e65cb0f5c9183701373b0ee64cdec40680a3de1a1a365b006538c4d0b7ca8a047d7c6f81a7f5b8a36bae3a20b47a4c2a9850423c7034866e3837fa8fdbfe2
e419b0e17f refactor: Remove manual CDBBatch size estimation (Lőrinc)
8b5e19d8b5 refactor: Delegate to LevelDB for CDBBatch size estimation (Lőrinc)
751077c6e2 Coins: Add `kHeader` to `CDBBatch::size_estimate` (Lőrinc)
Pull request description:
### Summary
The manual batch size estimation of `CDBBatch` serialized size was [added](e66dbde6d1) when LevelDB [didn't expose this functionality yet](https://github.com/google/leveldb/commit/69e2bd2).
The PR refactors the logic to use the native `leveldb::WriteBatch::ApproximateSize()` function, structured in 3 focused commits to incrementally replace the old behavior safely.
### Context
The previous manual size calculation initialized the estimate to 0, instead of LevelDB's header size (containing an 8-byte sequence number followed by a 4-byte count).
This PR corrects that and transitions to the now-available native LevelDB function for improved accuracy and maintainability.
### Approach
The fix and refactor follow a strangle pattern over three commits:
* correct the initialization bug in the existing manual calculation, isolating the fix and ensuring the subsequent assertions use the corrected logic;
* introduce the native `ApproximateSize()` method alongside the corrected manual one, adding assertions to verify their equivalence at runtime;
* remove the verified manual calculation logic and assertions, leaving only the native method.
ACKs for top commit:
sipa:
utACK e419b0e17f
TheCharlatan:
ACK e419b0e17f
laanwj:
Code review ACK e419b0e17f
Tree-SHA512: a12b973dd480d4ffec4ec89a119bf0b6f73bde4e634329d6e4cc3454b867f2faf3742b78ec4a3b6d98ac4fb28fb2174f44ede42d6c701ed871987a7274560691
459807d566 test: remove strict restrictions on rpc_deprecated (Pol Espinasa)
Pull request description:
Removed the wallet restrictions for `rpc_deprecated.py` and added specific test case for the current deprecated rpc.
`skip_test_if_missing_module` will skip the whole test when the wallet is missing, even if a part of the test is non-wallet related. This PR ensures that other tests not related to wallet can be ran and only this specific test will be skipped if there's no wallet
For more context check https://github.com/bitcoin/bitcoin/pull/31278#discussion_r2011661090
ACKs for top commit:
maflcko:
lgtm ACK 459807d566
rkrux:
ACK 459807d
Tree-SHA512: 922b0fafe8fb5bd88a677ce8be5c3fe2fdd4d0aadcd32cc11738a714cd6f765f07e7e7158c829f8338db0d46a15c030437a1ea09a3187c072bebebb4ca53ad85
f974359e21 test: Add encodable PUSHDATA1 examples to feature_taproot (Greg Sanders)
Pull request description:
Inspired by discussion in https://github.com/bitcoin/bitcoin/pull/31640#issuecomment-2743492906 I made an example adding coverage I think is missing, with some extra commentary that might help future contributors (including myself when I forget how it works again).
Open for suggestions how we can make it more welcoming beyond this.
cc darosior EthanHeilman sipa
ACKs for top commit:
janb84:
Re-ACK [f974359](f974359e21)
rkrux:
ACK f974359e21
Tree-SHA512: 7544d41c39c13d245a8a33522e53f22b4dd7593c069631978303e5a349cd12cf9d45bed648c391618c4732831232c4b82b8de2bf6cba5bf5e1232501db926122
58914ab459 fuzz: assert min diff between FeeFrac and CFeeRate (Pieter Wuille)
0c6bcfd8f7 feefrac: support both rounding up and down for Evaluate (Pieter Wuille)
ecf956ec9d feefrac: add support for evaluating at given size (Pieter Wuille)
7963aecead feefrac: add helper functions for 96-bit division (Pieter Wuille)
800c0dea9a feefrac: rework comments around Mul/MulFallback (Pieter Wuille)
fcfe008db2 feefrac fuzz: use arith_uint256 instead of ad-hoc multiply (Pieter Wuille)
46ff4220bf arith_uint256: modernize comparison operators (Pieter Wuille)
Pull request description:
The `FeeFrac` type represents a fraction, intended to be used for sats/vbyte or sats/WU. This PR adds functionality to evaluate that feerate for a given size, in order to obtain the fee it corresponds with (rounding down, or rounding up).
The motivation here is being able to do accurate feerate evaluations in cluster mempool block building heuristics (where rounding down is needed), but in principle this makes it possible to use `FeeFrac` as a more accurate replacement for `CFeeRate` (where for feerate estimation rounding up is desirable). Because of this, both rounding modes are implemented.
Unit tests are included for known-correct values, plus a fuzz test that verifies the result using `arith_uint256`.
ACKs for top commit:
l0rinc:
ACK 58914ab459
ismaelsadeeq:
reACK 58914ab459
glozow:
light code review ACK 58914ab459
Tree-SHA512: 362b88454bf355cae1f12d6430b1bb9ab66824140e12b27db7c48385f1e8db936da7d0694fb5aad2a00eb9e5fe3083a3a2c0cc40b2a68e2d37e07b3481d4eeae
Rather than use an ad-hoc reimplementation of wide multiplication inside the
fuzz test, reuse arith_uint256, which already has this. It's larger than what we
need here, but performance isn't a concern in this test, and it does what we need.
Since C++20, operator!= is implicitly defaulted using operator==, and
operator<, operator<=, operator>, and operator>= are defaulted using
operator<=>, so it suffices to just provide these two.
Remove the manual batch size estimation logic (`SizeEstimate()` method and `size_estimate` member) from `CDBBatch`.
Size is now determined solely by the `ApproximateSize()` method introduced in the previous commit, which delegates to the native LevelDB function.
The manual calculation is no longer necessary as LevelDB now provides this functionality directly, and the previous commit verified that the native function's results matched the manual estimation.
Assertions comparing the two methods are removed from `txdb.cpp`.
Co-authored-by: Wladimir J. van der Laan <laanwj@protonmail.com>
Serialized batch size can be queried via the underlying LevelDB implementation calling the native `leveldb::WriteBatch::ApproximateSize()`.
The previous manual calculation was added in e66dbde6d1 as part of https://github.com/bitcoin/bitcoin/pull/10195. At that time (April 2017), the version of LevelDB used by Bitcoin Core (and even the latest source) lacked a native function for this. LevelDB added this capability in 69e2bd224b, merged later that year.
The old manual estimation method (`SizeEstimate()`) is kept temporarily in this commit, and assertions are added in `txdb.cpp` to verify its results against `ApproximateSize()` during batch writes. This ensures the native function behaves as expected before removing the manual calculation in the subsequent commit.
The initialization of the manual `size_estimate` in `CDBBatch::Clear()` is corrected from `0` to `kHeader` (LevelDB's fixed batch header size).
This aligns the manual estimate with LevelDB's actual size immediately after clearing, fixing discrepancies that would otherwise be caught by tests in the next commit (e.g., `coins_tests`, `validation_chainstatemanager_tests`).
fadf8f078e test: Remove confusing and failing system time test (MarcoFalke)
Pull request description:
This was just added as a sanity check in fa013664ae by myself.
However, the test uses system time, so it may obviously (albeit rarely) fail.
Fix it by removing it.
Can be tested by running two bash loops at the same time:
`while ( ./bld-cmake/bin/test_bitcoin -t util_tests/util_time_GetTime ) ; do true ; done`
`while ( date -s "$(date -d 'now + 0.015 seconds' '+%Y-%m-%d %H:%M:%S.%3N')" && date -s "$(date -d 'now - 0.015 seconds' '+%Y-%m-%d %H:%M:%S.%3N')" ) ; do true ; done`
Eventually, it will fail:
```
test/util_tests.cpp(595): error: in "util_tests/util_time_GetTime": check ms_0 < GetTime<std::chrono::milliseconds>() has failed
test/util_tests.cpp(596): error: in "util_tests/util_time_GetTime": check us_0 < GetTime<std::chrono::microseconds>() has failed
*** 2 failures are detected in the test module "Bitcoin Core Test Suite"
ACKs for top commit:
janb84:
ACK [fadf8f0](fadf8f078e)
mabu44:
Tested ACK fadf8f078e
hebasto:
ACK fadf8f078e, tested on Ubuntu 24.10.
Tree-SHA512: fc468546f46a12804802df4f0e64d2898aca3db4df69602e5919ac31646c2fcb1e75b614fc2d1a3959c3db10fb0e315da5886d348b41589dba7cb43e618444a1
fa10a1ded5 ci: Use GITHUB_BASE_REF over hard-coded master (MarcoFalke)
fa0d0be05c ci: Merge master in test-each-commit task (take 2) (MarcoFalke)
Pull request description:
Calling the script `.github/ci-test-each-commit-exec.sh`, which merges `master`, obviously doesn't work, if the script itself is missing.
Fix it by a move-only to first merge `master` and then call the script.
ACKs for top commit:
l0rinc:
Code review ACK fa10a1ded5
sipa:
ACK fa10a1ded5, this fixed the CI issue in #31444.
Tree-SHA512: bcab2b03cb46d456e29f8d4237312a4525b9acd819578b26b4d5670ca14e075cf473b77b235b3063e06422325b627587f12dec7b4fbba134086d162c67dc81b3
* Run git config earlier and only once
* Run git merge in the yaml, before calling the bash script
* Run git reset in the yaml as well, for symmetry
* Replace "git merge --abort" with "git reset --hard", because it does
not fail when already up to date and no merge was started.
c5a7ffd1e8 preserve llvm profile env (Prabhat Verma)
Pull request description:
While generating `profraw` for fuzz tests using steps in [PR 32206](https://github.com/bitcoin/bitcoin/pull/32206) , the profraw was not being built at the desired location and only one `default.profraw` was being created which was being overwritten for multiple fuzz targets. This PR fixes that.
ACKs for top commit:
maflcko:
lgtm ACK c5a7ffd1e8
mabu44:
ACK c5a7ffd1e8
Tree-SHA512: 11f74caa8cba6f841aa899a5e294f658aed1b6a3d4cf68992609ea99fadb4a092b2350ffacea5c2d5eb377eb10082de018f27a1d6486a72460cb3905aaa15664
faa807bdf8 ci: Merge master in test-each-commit task (MarcoFalke)
Pull request description:
The `test-each-commit` task will often fail, when the CI config yaml is updated along with code changes.
This is because, GitHub seems to be merging the CI config on a fresh pull with the current target branch (`master`). However, the code changes are not.
A tedious workaround would be for every developer to rebase on every intermittent (https://github.com/bitcoin/bitcoin/issues/31946#issuecomment-2740911853) and non-intermittent CI issue.
However, fix this instead by merging with `master`.
ACKs for top commit:
laanwj:
ACK faa807bdf8
hebasto:
ACK faa807bdf8.
Tree-SHA512: 4849bd558dc6cdc7d86b95164ccee32ab7c08c9b7d31cf8ec5c8e9a2251fc819630f8fa9b929ed39e8e033c67bb006f0beb33e0de216e1224680be88c5fa0161
There turned out to be a mismatch in the tx output counts which caused
'ConnectBlockMixedEcdsaSchnorr' benchmark to run slower than
'ConnectBlockAllEcdsa' and 'ConnectBlockAllSchnorr'. This commit makes
the tx output counts uniform across all benchmarks.
This commit also renames the 'taproot_tx' variable to 'tx' to reflect
that this variable represents a general tx and not just a taproot tx.
The new helper will be used to fix a crash in the
wallet migration process (watch-only, non-blank,
private keys disabled, empty wallet - no scripts
or addresses imported).
Co-authored-by: Matias Furszyfer <mfurszy@protonmail.com>
2e751f559a doc: Amend Qt 6 dependency packages for Ubuntu (Hennadii Stepanov)
Pull request description:
On older systems, such as Ubuntu 22.04, `qt6-tools-dev-tools` and `libgl-dev` are not treated as dependencies of `qt6-tools-dev` and `qt6-base-dev`, respectively. This PR explicitly lists them in the installation documentation.
Fixes https://github.com/bitcoin/bitcoin/issues/32210.
ACKs for top commit:
maflcko:
lgtm ACK 2e751f559a
laanwj:
Code review ACK 2e751f559a
Tree-SHA512: a6997c74c83789cb5fe5b97a719b8ff6e2180d5f6ae5502ccccfce3a22394d25eef05204ecda0a6deb368de77975e2a1da89b5749eff01a979f2f60843efebff
Add a class TorsStreamIsolationCredentialsGenerator that generates
unique credentials based on a randomly generated session prefix
and an atomic counter.
This makes sure that different launches of the application won't share
the same credentials, and thus circuits, even in edge cases.
Example with `-debug=proxy`:
```
2025-03-31T16:30:27Z [proxy] SOCKS5 sending proxy authentication 0afb2da441f5c105-0:0afb2da441f5c105-0
2025-03-31T16:30:31Z [proxy] SOCKS5 sending proxy authentication 0afb2da441f5c105-1:0afb2da441f5c105-1
```
Thanks to hodlinator for the idea.
This change updates the vcpkg manifest baseline from the "2024.09.30
Release" to the "2025.03.19 Release", with the following package
changes:
- boost: 1.85.0#1,2 --> 1.87.0
- qtbase: 6.7.2#3 -> 6.8.2#1
- qttools: 6.7.2#1 -> 6.8.2
- sqlite3: 3.46.1 --> 3.49.1
On older systems, such as Ubuntu 22.04, `qt6-tools-dev-tools` and
`libgl-dev` are not treated as dependencies of `qt6-tools-dev` and
`qt6-base-dev`, respectively. This change explicitly lists them in the
installation documentation.
a40bd374aa Get*Union: disallow nulltpr Refs (Greg Sanders)
57433502e6 CountDistinctClusters: nullptrs disallowed (Greg Sanders)
8bca0d325a TxGraphImpl::Compact: m_main_clusterset.m_removed is always empty (Greg Sanders)
2c5cf987e9 TxGraphImpl::PullIn: only allowed when staging exists (Greg Sanders)
Pull request description:
Was looking at my local coverage report, and noticed a few spots that will not or cannot be hit.
CountDistinctClusters, GetAncestorsUnion, and GetDescendantsUnion accept nullptrs, but the test harness never employs them. Disallow them.
We never call PullIn whenever there isn't staging, so just enforce that invariant via assertion.
Remaining places that are not covered:
1) Relinearize: Currently we seem to always start with a cold (not known to be optimal) cluster, and after one attempt at linearization result into something optimal. This means we never shortcircuit, nor run PostLinearization, nor store the quality as ACCEPTABLE. Reducing iterations causes these lines to be hit. sipa says he will take this on as varying the amount of iterations was meant to be done eventually anyways.
2) We never do a move assignment operator when the lvalue already has a `m_graph` (so we never call UnlinkRef) 3358b1d105/src/txgraph.cpp (L2097)
3) We never use the move constructor: 3358b1d105/src/txgraph.cpp (L2108)
ACKs for top commit:
sipa:
utACK a40bd374aa
glozow:
utACK a40bd374aa
Tree-SHA512: ca88297222e80e0d590889698899f892b9335cfa587a76a6c6ca62c8d846f208b6b0b9a9b1829bafabdb929a1a0c3a75f23edf7dd2b4f5e2dad0235e5bc68ba3
With newly introduced libmultiprocess subtree, there's no need for depends
system to download and track changes to the upstream repository.
Note that adding the libmultiprocess subtree does not allow dropping
libmultiprocess packages from the depends build, because libmultiprocess
includes a code generation tool called mpgen, and in cross-compiled builds,
bitcoin core's cmake build system doesn't have access to a native toolchain and
can't build mpgen itself, so the depends system (or the native environment if
not using depends) needs to supply it.
Move parts of the int_get_build_id into a new int_get_build_properties
function. There is no change in behavior. This just organizes assignments
better so some build properties can be used to help compute build ids in the
next commit.
Without this change linter produces errors about:
- Use of std::filesystem the libmultiprocess example program.
- Use of locale-dependent functions in example program, in the build time code
generator, and in the runtime library for debug logging.
- Include guards not beginning with BITCOIN_
When ENABLE_IPC option is on, build with libmultiprocess subtree and
`add_subdirectory(src/ipc/libmultiprocess)` instead of external package
and `find_package(Libmultiprocess)` by default.
Behavior can be toggled with `WITH_EXTERNAL_LIBMULTIPROCESS` option. Using a
subtree should be more convenient for most bitcoin developers, but using an
external package is more convenient for developing in the libmultiprocess
repository.
The `WITH_EXTERNAL_LIBMULTIPROCESS` option is also used to avoid needing to
changing the depends build here. But in later commits, the depends build is
switched to use the add_subdirectory build as well.
Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
Rename WITH_MULTIPROCESS to ENABLE_IPC, because ENABLE_IPC is a more accurate
name for the feature. It controls whether the src/ipc/ directory is built and
whether IPC features like -ipcbind, -ipcconnect, and -ipcfd are available. It
does NOT currently enable multiprocess features which are implemented in #10102
building on top of the IPC features. It will also no longer (as of the next
commit), control whether a find_package call is made so the "WITH_" prefix is
also inappropriate.
-BEGIN VERIFY SCRIPT-
git grep -l WITH_MULTIPROCESS | xargs sed -i s/WITH_MULTIPROCESS/ENABLE_IPC/g
-END VERIFY SCRIPT-
Add support for reporting Tor extended SOCKS5 error codes as defined
here:
- https://spec.torproject.org/socks-extensions.html#extended-error-codes
- https://gitlab.torproject.org/tpo/core/arti/-/blob/main/crates/tor-socksproto/src/msg.rs?ref_type=heads#L183
These give a more direct indication of the problem in case of errors
connecting to hidden services, for example:
```
2025-04-02T10:34:13Z [net] Socks5() connect to [elided].onion:8333 failed: onion service descriptor can not be found
```
In the C Tor implementation, to get these one should set the
"ExtendedErrors" flag on the "SocksPort" definition, introduced in
version 0.4.3.1.
In Arti, extended error codes are always enabled.
Also, report the raw error code in case of unknown reply values.
Filter netlink responses to only consider default routes by checking the
destination prefix length (rtm_dst_len == 0).
Previously, we selected the first route with an RTA_GATEWAY attribute,
which for IPv6 often resulted in choosing a non-default route instead of
the actual default.
This caused occasional PCP port mapping failures because a gateway for a
non-default route was selected.
1. Do not set `C{PLUS}_INCLUDE_PATH` variables
The build system for Qt 6 differs entirely from that of Qt 5. Building a
set of native Qt 6 tools now forms a separate step when cross-compiling.
Under these new circumstances, the `C{PLUS}_INCLUDE_PATH` environment
variables may alter the default include directories for both native and
cross compilers.
Previously, we explicitly unset these variables when invoking clang for
cross-compiling; however, that approach proved suboptimal (see #30451).
This change sets the native toolchain for dependencies explicitly,
rather than relying on the `C{PLUS}_INCLUDE_PATH` environment variables.
Additionally, it facilitates the transition towards using clang for
building native tools when cross-compiling for macOS.
2. Add `ninja` package.
3. Adjust allowed symbol lists.
b96f1a696a add clang/llvm based coverage report generation (Prabhat Verma)
Pull request description:
Followed up from the [comment](https://github.com/bitcoin/bitcoin/issues/31927#issuecomment-2674522975) on the issue [#31927](https://github.com/bitcoin/bitcoin/issues/31927) , issues have been observed building coverage reports with `gcov` in MacOs and NixOs. This PR adds the steps to generate a coverage report based on the default llvm/clang tooling.
ACKs for top commit:
Crypt-iQ:
tACK b96f1a696a
hodlinator:
re-ACK b96f1a696a
janb84:
Re ACK [b96f1a6](b96f1a696a)
Tree-SHA512: bc54f170e84bb76b3eba7285bd49f051c0b99b784d583a550d8e51511497bcc4df8964bbe3991777648d2f829809db8eabb0cbf0d25f9da5e49e1cfc62f6d8d0
fa51310121 contrib: Warn about using libFuzzer for coverage check (MarcoFalke)
fa17cdb191 test: Avoid script check worker threads while fuzzing (MarcoFalke)
fa900bb2dc contrib: Only print fuzz output on failure (MarcoFalke)
fa82fe2c73 contrib: Use -Xdemangler=llvm-cxxfilt in deterministic-*-coverage (MarcoFalke)
fa7e931130 contrib: Add optional parallelism to deterministic-fuzz-coverage (MarcoFalke)
Pull request description:
This should make the `partially_downloaded_block` fuzz target even more deterministic.
Follow-up to https://github.com/bitcoin/bitcoin/pull/31841. Tracking issue: https://github.com/bitcoin/bitcoin/issues/29018.
This bundles several changes:
* First, speed up the `deterministic-fuzz-coverage` helper by introducing parallelism.
* Then, a fix to remove spawned test threads or spawn them deterministically. (While testing this, high parallelism and thread contention may be needed)
### Testing
It can be tested via (setting 32 parallel threads):
```
cargo run --manifest-path ./contrib/devtools/deterministic-fuzz-coverage/Cargo.toml -- $PWD/bld-cmake/ $PWD/../b-c-qa-assets/fuzz_corpora/ partially_downloaded_block 32
```
Locally, on a failure, the output would look like:
```diff
....
- 150| 0| m_worker_threads.emplace_back([this, n]() {
- 151| 0| util::ThreadRename(strprintf("scriptch.%i", n));
+ 150| 1| m_worker_threads.emplace_back([this, n]() {
+ 151| 1| util::ThreadRename(strprintf("scriptch.%i", n));
...
```
This excerpt likely indicates that the script threads were started after the fuzz init function returned.
Similarly, for the scheduler thread, it would look like:
```diff
...
227| 0| m_node.scheduler = std::make_unique<CScheduler>();
- 228| 1| m_node.scheduler->m_service_thread = std::thread(util::TraceThread, "scheduler", [&] { m_node.scheduler->serviceQueue(); });
+ 228| 0| m_node.scheduler->m_service_thread = std::thread(util::TraceThread, "scheduler", [&] { m_node.scheduler->serviceQueue(); });
229| 0| m_node.validation_signals =
...
```
ACKs for top commit:
Prabhat1308:
re-ACK [`fa51310`](fa51310121)
hodlinator:
re-ACK fa51310121
janb84:
Re-ACK [fa51310](fa51310121)
Tree-SHA512: 1a935eb19da98c7c3810b8bcc5287e5649ffb55bf50ab78c414a424fef8e703839291bb24040a552c49274a4a0292910a00359bdff72fa29a4f53ad36d7a8720
28dc118001 fuzz: wallet: fix crypter target (brunoerg)
Pull request description:
The crypter target has an issue, it's calling `DecryptKey` with a random secret and a random public key that will unlikely be related to the key used to encrypt, so it won't have any effect. This PR changes fixes it and also removes the `DecryptSecret` call since this function is already (and only) called within `DecryptKey`.
ACKs for top commit:
maflcko:
lgtm ACK 28dc118001🥊
Tree-SHA512: e96b7d33879bf06eeec0726e74e8e0d7020997659bf97dfca5d7c1a7ba65c4d93c78e666b97eebde110564cef2eefc7209d3e3586e4658145827b14d1b01dfc9
fa69c42fdf refactor: Remove spurious virtual from final ~CZMQNotificationInterface (MarcoFalke)
Pull request description:
`virtual` does not make sense here, because:
* The class is `final`, thus the destructor isn't overridden in a derived class
* The destructor also isn't overriding the destructor of the base, clarified in commit 2b3ea39de4
* Clang 21 may warn about this
```
src/zmq/zmqnotificationinterface.h:25:13: error: virtual method '~CZMQNotificationInterface' is inside a 'final' class and can never be overridden [-Werror,-Wunnecessary-virtual-specifier]
25 | virtual ~CZMQNotificationInterface();
| ^
```
Fix all issues by removing it.
ACKs for top commit:
davidgumberg:
crACK fa69c42fdf
janb84:
ACK [fa69c42](fa69c42fdf)
TheCharlatan:
ACK fa69c42fdf
Tree-SHA512: 26ea977f31fe24c116d68dea6c583de7c6fc480877e1baefcde11db4ac191e352027d492ee6ad69a60fe4ff537e0841c638b3a3e81356d9e00c60030845fc96e
4774a0c923 test: fix spelling in Python code comment (John Bampton)
Pull request description:
Fixed a couple of typos
Top commit has no ACKs.
Tree-SHA512: 5334995672b2c7d4a9cb916f71dff6a2ce13dc7ced6bbc30ddb0fe8e0ae0b4094b675b3dfced1ffc1b92e3a33ee22df07af3032b8c2928f27051b6376dca3361
4a679936bb ci, windows: Do not exclude `wallet_migration.py` in command line (Hennadii Stepanov)
Pull request description:
This PR amends the recently merged https://github.com/bitcoin/bitcoin/pull/31176 to resolve a silent merge conflict with the previously merged https://github.com/bitcoin/bitcoin/pull/31248.
Since https://github.com/bitcoin/bitcoin/pull/31248, it is no longer necessary to use `--exclude wallet_migration.py`, as the test is skipped due to not using previous releases.
The `wallet_migration.py` test itself still needs to be fixed for Windows by someone who will work on https://github.com/bitcoin/bitcoin/issues/32192.
ACKs for top commit:
davidgumberg:
crACK 4a679936bb
Tree-SHA512: f42428016958cdaccb509cc49341e726eaf1314d85989a7b49888f3862dc4ea0c2988a4792ae62dd925302d0073906397801c8dd2fb06c23381d7cad38730249
Removed the wallet restrictions for rpc_deprecated.py and added specific test case for the current deprecated rpc.
skip_test_if_missing_module will skip the whole test when the wallet is missing, even if a part of the test is non-wallet related.
Rename the `_randomize_credentials` parameter to Proxy's constructor to
`tor_stream_isolation` to make it more clear, and more specific what its
purpose is.
Also change all call sites to use a named parameter.
7bb83f6718 test: create assert_not_equal util and add to where imports are needed (kevkevin)
Pull request description:
In the functional tests there are lots of cases where we assert != which we now swap with assert_not_equal to be more readable
This is motivated/uses logic from this PR which was closed https://github.com/bitcoin/bitcoin/pull/28528
This partially helps https://github.com/bitcoin/bitcoin/issues/23119
I've broken it up to just `assert_not_equal` to keep the PR smaller as suggested in https://github.com/bitcoin/bitcoin/pull/28528#issuecomment-1959945805
I can create follow up PR's if this is wanted
ACKs for top commit:
hodlinator:
re-ACK 7bb83f6718
ryanofsky:
Code review ACK 7bb83f6718. Only change since last review is fixing error message formatting and passing it as a keyword argument
janb84:
Re-ACK [7bb83f6](7bb83f6718)
Tree-SHA512: de09f41a690033a5b61e6f861d3bd69a32b889d6655a28fbc0d5cfac9f7ec9c642432967d33913970882b4cfdd47bdd377d0ddc44e25976cbaa49f7f9d8f7b10
8e4a0ddd50 torcontrol: Add comment explaining Proxy credential randomization for Tor privacy (Eval EXEC)
ec5c0b26ce torcontrol: Define tor reply code as const to improve maintainability (Eval EXEC)
Pull request description:
This PR want to:
1. replace tor repy code with const to improve out maintainability.
2. cherry-picked https://github.com/bitcoin/bitcoin/pull/31973 , add comment to explain Proxy credential randomization for Tor privacy
ACKs for top commit:
hodlinator:
re-ACK 8e4a0ddd50
laanwj:
re-ACK 8e4a0ddd50
Tree-SHA512: 038daa6508ca88fceed5c8e155430614cb56976f36d1f8baee5114bca1141122cf94f51814a869848b3442691ee765cbf609cf946b2b35d5135015a9b749d917
6afffba34e contrib: (asmap) add docs about encode and decode commands (jurraca)
67d5cc2a06 contrib: (asmap) add documentation on diff and diff-addrs commands (jurraca)
e047b1deca contrib: (asmap) add diff-addrs example to README (jurraca)
Pull request description:
This README was a little sparse in my opinion, and was missing a mention of the `diff-addrs` command.
The README updates add background and examples for each command, split in two sections (encode/decode and diff/diff-addrs). This is intended to help people know how and when to run the commands available in the `asmap-tool.py` script.
However, I could use some confirmation on the behavior of the `--fill` flag. It's true that files generated with this flag set cannot be used to diff files after the fact, but i don't quite follow what the fill flag does to make that true. sipa could you maybe provide some insight?
ACKs for top commit:
fjahr:
re-ACK 6afffba34e
brunoerg:
reACK 6afffba34e
laanwj:
re-ACK 6afffba34e
Tree-SHA512: 073e8d7255f7270aa2f5a070332872f5fa6fbe6532eee1f7e3e4158ac0125a49c155f4933bf00655ff3a89f666f3f3bea521e70c516ab09a448845016d2b880a
0ff66b1c4a fuzz: coinselection: cover `SetBumpFeeDiscount` (brunoerg)
Pull request description:
`SetBumpFeeDiscount` sets the bump fee discount which is used to calculate the waste. We currently have no fuzz coverage for this function, so this PR adds it by calling `SetBumpFeeDiscount` before `RecalculateWaste`.
ACKs for top commit:
marcofleon:
ACK 0ff66b1c4a
Tree-SHA512: d5c1d97daaeb7f9b096bf9bdf6374b8a674a75f464e2b9bb3e1e1774a5805b22840ca1f31bae63f106640d9ce27a99432c3034524340be91c235f6ec3b185cff
8284229a28 refactor: deduplicate anchor witness program bytes (`0x4e,0x73`) (Sebastian Falbesoner)
41f2f058d0 test: add missing segwitv1 test cases to `script_standard_tests` (Sebastian Falbesoner)
Pull request description:
Currently we have two segwitv1 output script types that are considered standard:
- `TxoutType::WITNESS_V1_TAPROOT` (P2TR): witness program has size 32 (introduced with taproot soft-fork)
- `TxoutType::ANCHOR` (P2A): witness program is {0x4e, 0x7e} (introduced with #30352)
This PR adds them to the script standardness unit tests where missing, i.e. for using them with the `ExtractDestination` and `GetScriptForDestination` functions.
ACKs for top commit:
rkrux:
ACK 8284229a28
instagibbs:
reACK 8284229a28
hodlinator:
Code Review ACK 8284229a28
Tree-SHA512: d4a3b47fd31ba33f62d4367811e72a7f442c01b046b0a7217a66be0b9dea5c9041eebfe812c31839ec0f0b14c56948c7c016d3d2de79283583ad8e32c192c6ff
aa7a898c23 doc: use testnet4 in developer docs (Sjors Provoost)
6c217d22fd test: use testnet4 in argsman test (Sjors Provoost)
7c200ece80 test: use testnet4 in key_io_valid.json (Sjors Provoost)
d424bd5941 test: drop unused testnet3 magic bytes (Sjors Provoost)
8cfc09fafe test: cover testnet4 magic in assumeutxo.py (Sjors Provoost)
4281e3603a zmq: use testnet4 in zmq_sub.py example (Sjors Provoost)
Pull request description:
In preparation for dropping testnet3 entirely in #31974 this PR migrates a few things to testnet4:
* the ZMQ examples
* developer docs
* various unit tests
* the snapshot magic byte check in `feature_assumeutxo.py`
It drops `testnet3` from `MAGIC_BYTES` in the test framework, since no test uses it.
ACKs for top commit:
fjahr:
re-ACK aa7a898c23
maflcko:
lgtm ACK aa7a898c23🔊
hodlinator:
re-ACK aa7a898c23
Tree-SHA512: 235f74273234e8fb2aedf0017dea5c16bb9813ec7a1f89a51abe85691f09830a5ead834115d7db0936e12e55a40bc81888856a8002fe507c1474407e77f8b9fb
In the functional tests there are lots of cases where we assert != which
this new util will replace, we also are adding the imports and the new assertion
Threads may execute their function any time after they are spawned, so
coverage could be non-deterministic.
Fix this,
* for the script check worker threads by disabling them while fuzzing.
* for the scheduler thread by waiting for it to fully start and run the
service queue.
This makes it humanly possible to track progress as only "[N/M]"-lines are printed as long as we succeed.
Also, use char (a, b) to indicate run_id instead of u8 (0, 1).
Also, use emojis to indicate final success or error.
Co-Authored-By: Hodlinator <172445034+hodlinator@users.noreply.github.com>
25b56fd9b4 ci: Test cross-built Windows executables on Windows natively (Hennadii Stepanov)
3501bca8c7 ci: Move "Windows cross" job from Cirrus CI to GHA CI (Hennadii Stepanov)
f8619196ce ci: Use `bash` by default for all platforms (Hennadii Stepanov)
Pull request description:
This PR enables on the CI tests of cross-compiled Windows binaries on Windows.
It is important to have such tests in CI because the release binaries for Windows are also cross-compiled.
Two functional tests, `wallet_migration.py` and `wallet_multiwallet.py`, are temporarily disabled. They require fixes, such as https://github.com/bitcoin/bitcoin/pull/31410, and adjustments for error message handling. Re-enabling these tests will be addressed in follow-up PRs.
Resolves https://github.com/bitcoin/bitcoin/issues/31071.
ACKs for top commit:
davidgumberg:
tested reACK 25b56fd9b4
hodlinator:
re-ACK 25b56fd9b4
willcl-ark:
utACK 25b56fd9b4
maflcko:
review-only ACK 25b56fd9b4🍎
Tree-SHA512: fb9150807b7ebb248e8f4fe7b16e5179251e7be9336459287787f27e542583d73d937e6969667fd836378b676bb9be7f66756dc1abca8a01364bc9ee3e3720a5
This has no functional affect, as the any CBlockIndex*s which
to_mark_failed is set to will already have been marked failed.
Also prevents a situation where block already marked as
BLOCK_FAILED_CHILD is again unconditionally marked as
BLOCK_FAILED_VALID in the final |= BLOCK_FAILED_VALID.
invalid_block ----------> block_index
- before this commit, only if block_index is not invalid, it will mark
block_index as BLOCK_FAILED_CHILD
- it's possible that block_index encountered is invalid and was marked
as BLOCK_FAILED_VALID previously
- in this case, correctly update BlockStatus of block_index by
clearing BLOCK_FAILED_VALID and then setting it to BLOCK_FAILED_CHILD
when a block is invalidated using InvalidateBlock, check that:
1. it's status is BLOCK_FAILED_VALID
2. it's children's status is BLOCK_FAILED_CHILD
and not BLOCK_FAILED_VALID
3. it's ancestors are valid
this block of code is not reached on master since other than
initialisation, all other iterations have invalid_walk_tip
and to_mark_failed pointers in some form of this layout
where 1, 2, 3 and 4 are block heights.
invalid_walk_tip
↓
1 <- 2 <- 3 <- 4
↑
to_mark_failed
fix it so that blocks are correctly marked as BLOCK_FAILED_CHILD
if it's a descendant of BLOCK_FAILED_VALID block.
This change stresses that all ZMQ messages share the same structure
and that they differ only in the format of the bodies. Previously this
was not clear.
Further it removes the notion of endianness of 32-byte hashes,
as it was misleading, and replaces it with the term 'reversed byte
order' (as opposed to natural or normal byte order produced by hashing
functions).
Additionally, it states that ZMQ 32-byte hashes are in the same format
as in RPC. Previously it incorrectly stated that the two were in
different formats.
* Range-for avoids ++i/i++ debate and decreases linecount.
* seen_multipath is only used if multipath_segment_index hasn't already been set. Rename it to seen_substitutes to better describe what it does, now that the context implies its involved in multipath.
57d8b1f1b3 cmake: Avoid fuzzer "multiple definition of `main'" errors (Ryan Ofsky)
Pull request description:
This change builds libraries with `-fsanitize=fuzzer-no-link` instead of `-fsanitize=fuzzer` when the cmake `-DSANITIZERS=fuzzer` option is specified. This is necessary to make fuzzing and IPC cmake options compatible with each other and avoid CI failures in #30975 which enables IPC in the fuzzer CI build:
https://cirrus-ci.com/task/5366255504326656?logs=ci#L2817https://cirrus-ci.com/task/5233064575500288?logs=ci#L2384
The failures can also be reproduced by checking out #31741 and building with `cmake -B build -DBUILD_FOR_FUZZING=ON -DSANITIZERS=fuzzer -DENABLE_IPC=ON` with this fix reverted.
The fix updates the cmake build so when `-DSANITIZERS=fuzzer` is specified, the fuzz test binary is built with `-fsanitize=fuzzer` (so it can use libFuzzer's main function), and libraries are built with `-fsanitize=fuzzer-no-link` (so they can be linked into other executables with their own main functions).
Previously when `-DSANITIZERS=fuzzer` was specified, `-fsanitize=fuzzer` was applied to ALL libraries and executables. This was inappropriate because it made it impossible to build any executables other than the fuzz test executable without triggering link errors:
- `` multiple definition of `main' ``
- `` "undefined reference to `LLVMFuzzerTestOneInput' ``
if they depended on any libraries instrumented for fuzzing.
This was especially a problem when the `ENABLE_IPC` option was set because it made building the `mpgen` code generator impossible so nothing else that depended on generated sources, including the fuzz test binary, could be built either.
This commit was previously part of https://github.com/bitcoin/bitcoin/pull/31741 and had some discussion there starting in https://github.com/bitcoin/bitcoin/pull/31741#pullrequestreview-2619682385
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
hebasto:
ACK 57d8b1f1b3, tested on Ubuntu 24.04.
Tree-SHA512: 4011adbc0b08742e83cf7c0560d3d5b5694a863358e6ac9a21239626b4a8fedceca66db34b5a46136a7b26849bb1d8710c894689322ae97e1c407687c3f57d50
ae6b6ea296 wallet: remove redundant `Assert` call when block is disconnected (rkrux)
Pull request description:
It was highlighted in a PR discussion previously that the recently moved `Assert` macro call inside the block disconnected loop had been redundant for quite a while because of the presence of the `assert` macro call at the start of the function. Therefore, it is removed now.
refs #https://github.com/bitcoin/bitcoin/pull/31757#discussion_r1995416821
ACKs for top commit:
fjahr:
utACK ae6b6ea296
l0rinc:
crACK ae6b6ea296
hodlinator:
Code Review ACK ae6b6ea296
Prabhat1308:
Code Review ACK [`ae6b6ea`](ae6b6ea296)
Tree-SHA512: 6bbced88f4b39afcacefb7babe97c180a397d9cd55f18c4c2875bd594547dcdccb2059ac32495e0e8d4e7263b4c1349ca80b2f0fbd46b4450d1d847ba5abd903
d065208f0f test: get rid of redundant TODO tag (Chandra Pratap)
Pull request description:
The `FEE` parameter in `test/functional/feature_dbcrash.py::generate_small_transaction()` is not a fee rate, but an absolute fee. Hence, it doesn't make sense to replace it with node relay based fee calculation. Get rid of the TODO comment suggesting otherwise.
ACKs for top commit:
maflcko:
lgtm ACK d065208f0f
Tree-SHA512: f2b7f51ffb23de8e14ca071edd731410176a20750115a65db0ae67714389e03ffe1593ce88368e96d211329bd93c772f665de7c3a59b932681bc5b80db908d9f
This abstracts out the finding of the connected component that includes
a given element from FindConnectedComponent (which just finds any connected
component).
Use this in the txgraph fuzz test, which was effectively reimplementing this
logic. At the same time, improve its performance by replacing a vector with a
set.
ca55613fd1 test: Add functional test for bitcoin-chainstate (TheCharlatan)
3f9c716e7f test: Fix docstring for cmake migration (TheCharlatan)
Pull request description:
While the `bitcoin-chainstate` utility is not shipped in a release, it is the only current utility directly using the bitcoin kernel library. Adding a simple test for it is useful for checking that the library is actually usable. The test is also useful in future to demonstrate that the `bitcoin-chainstate` binary using the API for the kernel library introduced in #30595 actually works and offers similar features.
ACKs for top commit:
laanwj:
Code review ACK ca55613fd1
maflcko:
ACK ca55613fd1🎭
kevkevinpal:
ACK ca55613fd1
Tree-SHA512: 282627f5fac868a84aab9962ef2cbd3a8d3941d9f9dc2a3f26db1e7706ffa8051637ab5f8372676800e426e077ca40449a9e3e42a003048472339d81ed81bca8
a82829f37e test: simplify (w)txid checks by avoiding .calc_sha256 calls (Sebastian Falbesoner)
346a099fc1 test: avoid unneeded hash -> uint256 -> hash roundtrips (Sebastian Falbesoner)
Pull request description:
In the functional test framework we currently have a strong tendency to treat and store identifiers that result from hash functions (e.g. (w)txids, block hashes) as integers, which seems an unnatural and confusing choice. Hashes are just pseudo-random sequences of bytes, and there is usually no need to apply integer operations on them; the only exceptions I could think of is PoW-verification of block hashes with the less-than (`<`) operator, or interpreting the byte-string as scalar in the EC-context for e.g. key derivation.
I'd hence argue that most uses of `ser_uint256`/`uint256_from_str` and txid conversions via `int(txid/blockhash, 16)` are potential code smells and should be reduced to a minimum long-term if possible. This PR is a first step into this direction, intentionally kept small with (what I think) uncontroversial changes for demonstration purposes, to check out if other contributors are interested in this. A next step could be to change the classes of primitives (CTransaction, CBlock etc.) and network messages (msg_) to store hash results as actual bytes (maybe in a class wrapping the bytes that offers conversion from/to human-readable strings [1], for easier interaction with RPC calls and debug outputs) rather than ints. But that would of course need larger, potentially more controversial changes, and its questionable if its really worth the effort.
[1] unfortunately, txids and block hashes are shown to user in reverse byte order, so e.g. a txid_bytes->txid_str conversion is not just a simple `txid_bytes.hex()`, but a `txid_bytes[::-1].hex()`
ACKs for top commit:
maflcko:
review ACK a82829f37e🐘
rkrux:
Concept and utACK a82829f37e
ryanofsky:
Code review ACK a82829f37e. Nice changes, and sorry about the false bug report
Tree-SHA512: bb0465802d743a495207800f922b65f49ed0d20552f95bb0bee764944664092aad74812e29df6e01ef40bcb8f9bc6c84c7e9cbbe6f008ee1a14d94ed88e698b4
It was highlighted in a PR discussion previously that the recently
moved `Assert` macro call inside the block disconnected loop had
been redundant for quite a while because of the presence of the
`assert` macro call at the start of the function. Therefore, it
is removed now.
refs #https://github.com/bitcoin/bitcoin/pull/31757#discussion_r1995416821
329a0dcdaf doc: clarify the documentation of `Assume` (ismaelsadeeq)
Pull request description:
An Expression inside `Assume` may be optimized away in production builds when the compiler proves they are side-effect-free.
This use case is demonstrated in #31363 and is suggested to be documented in https://github.com/bitcoin/bitcoin/pull/31363#issuecomment-2736410023.
ACKs for top commit:
l0rinc:
ACK 329a0dcdaf
hodlinator:
re-ACK 329a0dcdaf
jonatack:
ACK 329a0dcdaf
rkrux:
re-ACK 329a0dcdaf
Tree-SHA512: 4bbb807a1e632694863c1a1fa2e93cc5a756b19f8d78f0642ebe7ffafb01835765fa66c76a680dc6f3c412a5abb0c4a33fb7212c26b4b2d80b6b3b7ee8284b2e
248fdd88dc test: accept unordered tracepoints in... (willcl-ark)
Pull request description:
We have encountered an instance where the tracepoints were not collected in the same order they were fired (#31951).
Tracepoint ordering is not guaranteed in userspace for a number of reasons.
As this test does not require a strict collection/processing order collect `expected` and `actual` events into dicts and compare them.
This will gracefully handle both the number of events, and out-of-order events should they reoccur in the future.
Fixes: #31951
ACKs for top commit:
0xB10C:
re-ACK 248fdd88dc
laanwj:
Code review ACK 248fdd88dc
Tree-SHA512: 78d1aa936194d386d919ed26133aac3af5fc6d3d0b1fe1e767288d9e6226e2c701d640e71e994a63ccd48344bd2a0db508cb353cdd5ce1f644cd6f7313654623
52ede28a8a doc: Update comments for AreInputsStandard to match code (Anthony Towns)
Pull request description:
The comment about extra data stuffed in scriptSigs was introduced in #4365 which introduced `ScriptSigArgsExpected()`, and became incorrect after #7387 / #7453 (checks are now performed by `SCRIPT_VERIFY_CLEANSTACK` during script validation and `IsPushOnly()` in `IsStandardTx()`). Drops the details on what a p2sh with many checksigs would look like, which was already done in #4365, but only for main.cpp not the duplicated comment in main.h, which was merged into policy/policy.cpp in #6335 and later moved to the right place in #10682.
ACKs for top commit:
instagibbs:
ACK 52ede28a8a
darosior:
ACK 52ede28a8a
Tree-SHA512: 5ee9a775c81d4c23aca2f8f938ab8bfa7605af489ddb78788613195be8744c7fb7a37bae271093f67f572577452651d4958706b55346e99cf8d32ac0fc34df03
52ac17757e cmake: Add `NO_CACHE_IF_FAILED` option for checking linker flags (Hennadii Stepanov)
Pull request description:
Use it for checking `-fsanitize`.
This change improves the user experience when the configuration step fails due to a missing library. Now, there is no need to manually clean the CMake cache after installing the required library.
Addresses [this](https://github.com/bitcoin/bitcoin/issues/31942#issuecomment-2703801270) comment from https://github.com/bitcoin/bitcoin/issues/31942.
ACKs for top commit:
fanquake:
ACK 52ac17757e
Tree-SHA512: 4004110585413792faa01551cf5a5b3b0de7f213c7a1dd333647107741f84abf626fd0ed067fc17e4c5a523de549432738d3752facf25d1e3dab240be8d13d03
9f35d4d070 test: fix intermittent timeout in p2p_ibd_stalling.py (Martin Zumsande)
Pull request description:
After sending the headers message add a sync, so that we wait until the header message from the previous peer has been received before connecting additional peers.
In the failed NetBSD run linked in #32090, the second node managed to complete the handshake and send its own headers message before the message from the first node was received.
Fixes#32090
ACKs for top commit:
maflcko:
lgtm ACK 9f35d4d070
Tree-SHA512: 30a98c6ec04f819c892ab5ce76a309df81b3a4644be021f938eefddbd00ec4141e055c3cf735c9e04ce2fe9e950470a99d54cb6be43230110348a802a6a6c252
b1de59e896 fuzz: extract unsequenced operations with side-effects (Lőrinc)
Pull request description:
https://github.com/bitcoin/bitcoin/pull/30746#discussion_r1817851827 introduced unsequenced operations with side-effects - which is undefined behavior, i.e. the right hand side can be evaluated before the left hand side, which happens to mutate it.
<details>
<summary>Tried to find other occurrences</summary>
```bash
clang++ --analyze -std=c++20 -I./src -I./src/test -I./src/test/fuzz src/test/fuzz/base_encode_decode.cpp src/psbt.cpp
```
but it didn't warn about UB.
Grepped for similar ones, but could find any other one in the codebase:
```bash
> grep -rnE --include='*.cpp' --include='*.h' '\b(\w+)\(([^)]*\b(\w+)\b[^)]*)\)\s*==\s*\3\.' .
./src/test/arith_uint256_tests.cpp:373: BOOST_CHECK(R1L.GetHex() == R1L.ToString());
./src/test/arith_uint256_tests.cpp:374: BOOST_CHECK(R2L.GetHex() == R2L.ToString());
./src/test/arith_uint256_tests.cpp:375: BOOST_CHECK(OneL.GetHex() == OneL.ToString());
./src/test/arith_uint256_tests.cpp:376: BOOST_CHECK(MaxL.GetHex() == MaxL.ToString());
./src/test/fuzz/cluster_linearize.cpp:565: assert(depgraph.FeeRate(best_anc.transactions) == best_anc.feerate);
./src/test/fuzz/cluster_linearize.cpp:646: assert(depgraph.FeeRate(found.transactions) == found.feerate);
./src/test/fuzz/cluster_linearize.cpp:765: assert(depgraph.FeeRate(chunk_info.transactions) == chunk_info.feerate);
./src/test/fuzz/base_encode_decode.cpp:95: assert(DecodeBase64PSBT(psbt, random_string, error) == error.empty());
./src/test/fuzz/key.cpp:102: assert(pubkey.data() == pubkey.begin());
./src/test/skiplist_tests.cpp:42: BOOST_CHECK(vIndex[from].GetAncestor(0) == vIndex.data());
./src/script/signingprovider.cpp:535: ComputeTapbranchHash(node.sub[1]->hash, node.sub[1]->hash) == node.hash) {
./src/pubkey.h:78: return vch.size() > 0 && GetLen(vch[0]) == vch.size();
./src/cluster_linearize.h:881: Assume(elem.inc.feerate.IsEmpty() == elem.pot_feerate.IsEmpty());
```
</details>
Hodlinator deduced the UB on Windows in https://github.com/bitcoin/bitcoin/issues/32135#issuecomment-2751723855Fixes#32135
ACKs for top commit:
maflcko:
lgtm ACK b1de59e896
hodlinator:
ACK b1de59e896
marcofleon:
Nice, ACK b1de59e896
brunoerg:
code review ACK b1de59e896
Tree-SHA512: d66524424c7f749eba870f5bd6038da79666ac638047b31dd8ff15a77d927facb54b4735e8afb7984648fdc9e2dd59ea213996c352301fa05978f041511361d4
963355037f depends: set CMAKE_*_COMPILER_TARGET in toolchain (fanquake)
Pull request description:
According to the CMake docs, this is the correct way to setup a toolchain file for cross-compilation using Clang. See https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html#cross-compiling-using-clang
Internally it looks like CMake will only take this variable into account if it detects the compiler to be Clang, so this shouldn't effect other builds, but in the case of our Apple cross builds, we'd end up with a duplicated `--target=$ARCH-apple-darwin` on the compiler line, given we are already setting `--target` for Darwin builds.
Would fix#31748.
ACKs for top commit:
hebasto:
ACK 963355037f, tested on Ubuntu 24.10.
Tree-SHA512: 1aa0c5d9cb069ce277e53b5551baf5249c449331b0a160edb9a8ceb56209f886a9e2051e2ba63e0874904f652ace0280b7483dd5d81bfff9e993eb18abb961ad
0000fb3fd9 doc: Remove outdated and stale todo comment (MarcoFalke)
fa2b529f92 refactor: Remove redundant call to IsArgSet (MarcoFalke)
fa29842c1f refactor: Remove IsArgSet guard when fallback value is provided (MarcoFalke)
Pull request description:
`IsArgSet` is problematic:
* It returns whether an arg has been set, even if it has been negated. `IsArgSet` is sometimes used to check for a truthy value, which is wrong, but usually harmless. Cleanup of those cases may or may not be done in a follow-up.
* In most other cases, calling it is redundant, because the immediately following `Get*Arg` calls can already return an `std::optional` nullopt value to indicate an unset arg.
So relieve both issues by removing all `IsArgSet` that are redundant.
ACKs for top commit:
pablomartin4btc:
re-ACK 0000fb3fd9
ryanofsky:
Code review ACK 0000fb3fd9. No changes since last review other than rebase.
Tree-SHA512: d142d71d136b2dbd5fd005667875099777704176f5e08fdeb38f05d6afce40b435a257c5bb6a1f545459fe4f81f967cee3083ab666cb0befdef3f6234f1e3d32
fb2b05b125 build: Remove bitness suffix from Windows installer (Hennadii Stepanov)
Pull request description:
Since support for 32-bit Windows has been dropped, the suffix is no longer necessary.
ACKs for top commit:
l0rinc:
utACK fb2b05b125
hodlinator:
ACK fb2b05b125
laanwj:
ACK fb2b05b125
Tree-SHA512: cef18ddbc21bb8b57fd1f6b26d0c8bdee4aa47a20552c1f02ac7fcc084ab9887dcb2632c9e0915fbce156d843625aaad01a3ad5e11fbed56548e404719cc9a52
e3ce2bd982 Remove needless borrow to fix Clippy warning (dennsikl)
Pull request description:
Pull Request Description
**Summary**
Removes a needless borrow in `test/lint/test_runner/src/main.rs` that triggered a
Clippy warning (`needless_borrows_for_generic_args`). This minor refactoring
makes the code cleaner without changing functionality.
**Rationale**
- Eliminates a Clippy warning when running:
```bash
cargo clippy --manifest-path test/lint/test_runner/Cargo.toml -- -D warnings
ACKs for top commit:
maflcko:
lgtm ACK e3ce2bd982
kevkevinpal:
ACK [e3ce2bd](e3ce2bd982)
TheCharlatan:
ACK e3ce2bd982
Tree-SHA512: 9f3e07b45df0af6ad4bf87216b257108cc9b50b8e6bc591cac58b5cf6f78ebaeff27181cb0e8a6bc401626e1c707b925315f2e5ebd8dd5216e04c95d70237f85
b2ea365648 txgraph: Add Get{Ancestors,Descendants}Union functions (feature) (Pieter Wuille)
54bceddd3a txgraph: Multiple inputs to Get{Ancestors,Descendant}Refs (preparation) (Pieter Wuille)
aded047019 txgraph: Add CountDistinctClusters function (feature) (Pieter Wuille)
b685d322c9 txgraph: Add DoWork function (feature) (Pieter Wuille)
295a1ca8bb txgraph: Expose ability to compare transactions (feature) (Pieter Wuille)
22c68cd153 txgraph: Allow Refs to outlive the TxGraph (feature) (Pieter Wuille)
82fa3573e1 txgraph: Destroying Ref means removing transaction (feature) (Pieter Wuille)
6b037ceddf txgraph: Cache oversizedness of graphs (optimization) (Pieter Wuille)
8c70688965 txgraph: Add staging support (feature) (Pieter Wuille)
c99c7300b4 txgraph: Abstract out ClearLocator (refactor) (Pieter Wuille)
34aa3da5ad txgraph: Group per-graph data in ClusterSet (refactor) (Pieter Wuille)
36dd5edca5 txgraph: Special-case removal of tail of cluster (Optimization) (Pieter Wuille)
5801e0fb2b txgraph: Delay chunking while sub-acceptable (optimization) (Pieter Wuille)
57f5499882 txgraph: Avoid looking up the same child cluster repeatedly (optimization) (Pieter Wuille)
1171953ac6 txgraph: Avoid representative lookup for each dependency (optimization) (Pieter Wuille)
64f69ec8c3 txgraph: Make max cluster count configurable and "oversize" state (feature) (Pieter Wuille)
1d27b74c8e txgraph: Add GetChunkFeerate function (feature) (Pieter Wuille)
c80aecc24d txgraph: Avoid per-group vectors for clusters & dependencies (optimization) (Pieter Wuille)
ee57e93099 txgraph: Add internal sanity check function (tests) (Pieter Wuille)
05abf336f9 txgraph: Add simulation fuzz test (tests) (Pieter Wuille)
8ad3ed2681 txgraph: Add initial version (feature) (Pieter Wuille)
6eab3b2d73 feefrac: Introduce tagged wrappers to distinguish vsize/WU rates (Pieter Wuille)
d449773899 scripted-diff: (refactor) ClusterIndex -> DepGraphIndex (Pieter Wuille)
bfeb69f6e0 clusterlin: Make IsAcyclic() a DepGraph member function (Pieter Wuille)
0aa874a357 clusterlin: Add FixLinearization function + fuzz test (Pieter Wuille)
Pull request description:
Part of cluster mempool: #30289.
### 1. Overview
This introduces the `TxGraph` class, which encapsulates knowledge about the (effective) fees, sizes, and dependencies between all mempool transactions, but nothing else. In particular, it lacks knowledge about `CTransaction`, inputs, outputs, txids, wtxids, prioritization, validatity, policy rules, and a lot more. Being restricted to just those aspects of the mempool makes the behavior very easy to fully specify (ignoring the actual linearizations produced), and write simulation-based tests for (which are included in this PR).
### 2. Interface
The interface can be largely categorized into:
* Mutation functions:
* `AddTransaction` (add a new transaction with specified feerate, and get a `Ref` object back to identify it).
* `RemoveTransaction` (given a `Ref` object, remove the transaction).
* `AddDependency` (given two `Ref` objects, add a dependency between them).
* `SetTransactionFee` (modify the fee associated with a Ref object).
* Inspector functions:
* `GetAncestors` (get the ancestor set in the form of `Ref*` pointers)
* `GetAncestorsUnion` (like above, but for the union of ancestors of multiple `Ref*` pointers)
* `GetDescendants` (get the descendant set in the form of `Ref*` pointers)
* `GetDescendantsUnion` (like above, but for the union of ancestors of multiple `Ref*` pointers)
* `GetCluster` (get the connected component set in the form of `Ref*` pointers, in the order they would be mined).
* `GetIndividualFeerate` (get the feerate of a transaction)
* `GetChunkFeerate` (get the mining score of a transaction)
* `CountDistinctClusters` (count the number of distinct clusters a list of `Ref`s belong to)
* Staging functions:
* `StartStaging` (make all future mutations operate on a proposed transaction graph)
* `CommitStaging` (apply all the changes that are staged)
* `AbortStaging` (discard all the changes that are staged)
* Miscellaneous functions:
* `DoWork` (do queued-up computations now, so that future operations are fast)
This `TxGraph::Ref` type used as a "handle" on transactions in the graph can be inherited from, and the idea is that in the full cluster mempool implementation (#28676, after it is rebased on this), `CTxMempoolEntry` will inherit from it, and all actually used Ref objects will be `CTxMempoolEntry`s. With that, the mempool code can just cast any `Ref*` returned by txgraph to `CTxMempoolEntry*`.
### 3. Implementation
Internally the graph data is kept in clustered form (partitioned into connected components), for which linearizations are maintained and updated as needed using the `cluster_linearize.h` algorithms under the hood, but this is hidden from the users of this class. Implementation-wise, mutations are generally applied lazily, appending to queues of to-be-removed transactions and to-be-added dependencies, so they can be batched for higher performance. Inspectors will generally only evaluate as much as is needed to answer queries, with roughly 5 levels of processing to go to fully instantiated and acceptable cluster linearizations, in order:
1. `ApplyRemovals` (take batches of to-be-removed transactions and translate them to "holes" in the corresponding Clusters/DepGraphs).
2. `SplitAll` (creating holes in Clusters may cause them to break apart into smaller connected components, so make turn them into separate Clusters/linearizations).
3. `GroupClusters` (figure out which Clusters will need to be combined in order to add requested to-be-added dependencies, as these may span clusters).
4. `ApplyDependencies` (actually merge Clusters as precomputed by `GroupClusters`, and add the dependencies between them).
5. `MakeAcceptable` (perform the LIMO linearization algorithm on Clusters to make sure their linearizations are acceptable).
### 4. Future work
This is only an initial version of TxGraph, and some functionality is missing before #28676 can be rebased on top of it:
* The ability to get comparative feerate diagrams before/after for the set of staged changes (to evaluate RBF incentive-compatibility).
* Mining interface (ability to iterate transactions quickly in mining score order) (see #31444).
* Eviction interface (reverse of mining order, plus memory usage accounting) (see #31444).
* Ability to fix oversizedness of clusters (before or after committing) - this is needed for reorgs where aborting/rejecting the change just is not an option (see #31553).
* Interface for controlling how much effort is spent on LIMO. In this PR it is hardcoded.
Then there are further improvements possible which would not block other work:
* Making Cluster a virtual class with different implementations based on transaction count (which could dramatically reduce memory usage, as most Clusters are just a single transaction, for which the current implementation is overkill).
* The ability to have background thread(s) for improving cluster linearizations.
ACKs for top commit:
instagibbs:
reACK b2ea365648
ajtowns:
reACK b2ea365648
ismaelsadeeq:
reACK b2ea365648🚀
glozow:
ACK b2ea365648
Tree-SHA512: 0f86f73d37651fe47d469db1384503bbd1237b4556e5d50b1d0a3dd27754792d6fc3481f77a201cf2ed36c6ca76e0e44c30e175d112aacb53dfdb9e11d8abc6b
The 'FEE' parameter in test/functional/feature_dbcrash.py::
generate_small_transaction() is not a fee rate, but an
absolute fee. Hence, it doesn't make sense to replace it
with node relay based fee calculation. Get rid of the TODO
comment suggesting otherwise.
...interface_usdt_utxocache.py
We have encountered an instance where the tracepoints were not collected
in the same order they were fired (#31951).
Tracepoint ordering is not guaranteed in userspace for a number of
reasons.
As this test does not require a strict collection/processing order
collect `expected` and `actual` events into dicts and compare them.
This will gracefully handle both the number of events, and out-of-order
events should they reoccur in the future.
Update the example wherein the PSBT sends bitcoin to an address instead
of creating an OP_RETURN output. Also, update the RPC description to
reflect the fact that the created transaction is unsigned.
This affects docs of the following RPCs:
`bumpfee`, `psbtbumpfee`, `send`, `walletcreatefundedpsbt`, `createpsbt`,
and `createrawtransaction`
It was not evident to me that this field creates an `OP_RETURN` output until
I read the code and tried it out. Thus, making the doc explicitly mention it.
fa5674c264 fuzz: Fix off-by-one in package_rbf target (MarcoFalke)
Pull request description:
Running the while loop up to `NUM_ITERS` times may set `iter` to `g_outpoints.size()`, which will then lead to an out-of-bounds read.
There was an assert, which I guess tried to catch this, but the condition in the assert was wrong as well.
Fix all issues by replacing the broken assert with the internal and correct check inside `std::vector::at` and by limiting `iter` to `NUM_ITERS` in the while loop.
Fixes https://github.com/bitcoin/bitcoin/issues/32121
ACKs for top commit:
glozow:
ACK fa5674c264
brunoerg:
code review ACK fa5674c264
Tree-SHA512: 91b849ce969fd25c0ff8c90c2908d3096c77607d8e5fd81201ef33d88a57760199618174b8a6fd634cb5ef2a9068e94703b5c963ca473bd96a14d4bf9ec835cb
https://github.com/bitcoin/bitcoin/pull/30746#discussion_r1817851827 introduced an unsequenced operations with side-effects - which is undefined behavior, i.e. the right hand side can be evaluated before the left hand side, which happens to mutate it.
Tried:
```
clang++ --analyze -std=c++20 -I./src -I./src/test -I./src/test/fuzz src/test/fuzz/base_encode_decode.cpp src/psbt.cpp
```
but it didn't warn about UB.
Grepped for similar ones, but could find any other one in the codebase:
> grep -rnE --include='*.cpp' --include='*.h' '\b(\w+)\(([^)]*\b(\w+)\b[^)]*)\)\s*==\s*\3\.' .
```
./src/test/arith_uint256_tests.cpp:373: BOOST_CHECK(R1L.GetHex() == R1L.ToString());
./src/test/arith_uint256_tests.cpp:374: BOOST_CHECK(R2L.GetHex() == R2L.ToString());
./src/test/arith_uint256_tests.cpp:375: BOOST_CHECK(OneL.GetHex() == OneL.ToString());
./src/test/arith_uint256_tests.cpp:376: BOOST_CHECK(MaxL.GetHex() == MaxL.ToString());
./src/test/fuzz/cluster_linearize.cpp:565: assert(depgraph.FeeRate(best_anc.transactions) == best_anc.feerate);
./src/test/fuzz/cluster_linearize.cpp:646: assert(depgraph.FeeRate(found.transactions) == found.feerate);
./src/test/fuzz/cluster_linearize.cpp:765: assert(depgraph.FeeRate(chunk_info.transactions) == chunk_info.feerate);
./src/test/fuzz/base_encode_decode.cpp:95: assert(DecodeBase64PSBT(psbt, random_string, error) == error.empty());
./src/test/fuzz/key.cpp:102: assert(pubkey.data() == pubkey.begin());
./src/test/skiplist_tests.cpp:42: BOOST_CHECK(vIndex[from].GetAncestor(0) == vIndex.data());
./src/script/signingprovider.cpp:535: ComputeTapbranchHash(node.sub[1]->hash, node.sub[1]->hash) == node.hash) {
./src/pubkey.h:78: return vch.size() > 0 && GetLen(vch[0]) == vch.size();
./src/cluster_linearize.h:881: Assume(elem.inc.feerate.IsEmpty() == elem.pot_feerate.IsEmpty());
```
Hodlinator deduced the UB on Windows in https://github.com/bitcoin/bitcoin/issues/32135#issuecomment-2751723855
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
226d81f8b7 mining: drop unused -nFees and sigops from CBlockTemplate (Sjors Provoost)
53ad845fb9 test: check fees and sigops in getblocktemplate (Sjors Provoost)
Pull request description:
For the coinbase `vTxFees` used a dummy value of -nFees.
Similarly the first `vTxSigOpsCost` entry was calculated from
the dummy coinbase transaction.
This was introduced in #2115, but the values were never returned by the RPC or used in a test.
Drop 'm and add code comments to prevent confusion.
This PR also adds test coverage for the `fees` and `sigops` fields in `getblocktemplate`, so it closes#32053.
ACKs for top commit:
ismaelsadeeq:
re-ACK 226d81f8b7
ryanofsky:
Code review ACK 226d81f8b7. New test was added since last review, which seems very cleanly written and fixes some missing coverage.
glozow:
ACK 226d81f8b7
Tree-SHA512: 79c534e6bc4810d29114b04dd6db798877732cb473e773bf3cc28f83d14ee3982392587bd0baa39857bd53a79eae3b730d7a7029b08a9b6c3b5c51f86657ca5d
Replace test_rpcwhitelistdefault_0_no_permissions and
test_rpcwhitelistdefault_1_no_permissions with a single
test_rpcwhitelistdefault_permissions function.
Checking for IsArgSet before calling GetArg while providing an arbitrary
default value as fallback is both confusing and fragile.
It is confusing, because the provided fallback is dead code. So it would
be better to just call GetArg without a fallback.
Even better would be to provide the true fallback value and sanitize it
as if it were user-input, but this can be done in a follow-up.
Removing the redundant call to IsArgSet will have to be done either way,
so do it now.
Checking for IsArgSet before calling GetArg while providing the args
default value as fallback is both confusing and fragile.
It is confusing, because the provided fallback is dead code. So it would
be better to just call GetArg without a fallback.
However, ignoring the fallback value is fragile, because it would not be
sanitized.
Fix all issues by sanitizing the fallback value.
288481aabd doc: clarify that testnet min-difficulty is not optional (Sjors Provoost)
Pull request description:
When 20 minutes have gone by on testnet3 or testnet4, the next block `MUST` have difficulty 1. I've seen people be confused about this several times now in recent months. It doesn't help that the code comment is wrong. So fixing that.
The reason is that `nBits` must match exactly:
e568c1dd13/src/validation.cpp (L4212-L4215)
ACKs for top commit:
fjahr:
ACK 288481aabd
kevkevinpal:
ACK [288481a](288481aabd)
Tree-SHA512: 17d426301f386fa5810cceedfdb20a3523ab3ac2f17257ca7a525edd869fa409b150eff4cc258b27adecd0ded1c18ff48a9998fc9caed2faa461e410d4c5a884
6869fb4170 net: Block v2->v1 transport downgrade if !CConnman::fNetworkActive (Hodlinator)
Pull request description:
We might have just set `CNode::fDisconnect` in the first loop because of `!CConnman::fNetworkActive`.
Attempting to reconnect using v1 transport just because `fNetworkActive` was set to `false` at the "right" stage in the v2 handshake does not make sense.
Issue [discovered](https://github.com/bitcoin/bitcoin/pull/31633#discussion_r1930908304) by davidgumberg.
ACKs for top commit:
davidgumberg:
Tested and Reviewed ACK 6869fb4170
mabu44:
ACK 6869fb4170
stratospher:
ACK 6869fb4. I've reviewed the code but don't have strong preference for this branch vs master since only functional change is just a single log not being printed in a low probability scenario (we happen to be attempting v2 connection when P2P network activity is being turned off).
vasild:
ACK 6869fb4170
Tree-SHA512: 54f596e54c5a6546f2c3fec2609aa8d10dec3adcf1001ca16666d8b374b8d79d64397f46c90d9b3915b4e91a5041b6ced3044fd2a5b4fb4aa7282eb51f61296a
Building unhardened executables is not a supported use case that should
be maintained and those that want unhardened executables can still
override them by appending disable flags.
For example:
cmake -B build -DAPPEND_CPPFLAGS='-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0 -fno-stack-protector -fcf-protection=none -fno-stack-clash-protection' -DAPPEND_LDFLAGS='-Wl,-z,lazy -Wl,-z,norelro -Wl,-z,noseparate-code'
a015b7e13d test: Add expected result assertions (yancy)
Pull request description:
~This is a trivial addition to the test suit, however it shouldn't be required to add debug statements and manually run the tests if someone needs to know the results of this test.~
Add an assertion for the values returned. The goal of the test is to show that a minimal weight selection of UTXOs is returned by coin-grinder. Since there are multiple possible solutions, the added assertion shows that coin-grinder finds the solution with the lowest weight. Without this assertion, it's ambiguous whether or not coin-grinder is returning the solution with the lowest weight.
Remove the check that a result is returned since the expected result assertion implies a result.
ACKs for top commit:
janb84:
re ACK [a015b7e](a015b7e13d)
murchandamus:
ACK a015b7e13d
Tree-SHA512: ee3c2688b4a4a07ab209f7655c3956e62a1084419df5e87c27d751a38ff64d4c3457df2317f8077149a6947cdb05b249975de2b8f0e18ca8b17b41f4735fb1c6
2f2ab47bf7 Release notes (Pol Espinasa)
bf194c920c wallet, rpc: deprecate settxfee and paytxfee (Pol Espinasa)
Pull request description:
**Summary**
This PR deprecates the settxfee RPC and paytxfee setting, marking it for removal in Bitcoin Core 31.0.
**Motivation**
The PR was initially motivated by https://github.com/bitcoin/bitcoin/issues/31088. The intention was to create a new function `settxfeerate` to allow users to set a static fee rate in `sat/vB` instead of `btc/kvB`.
The `settxfee` RPC allows users to set a static fee rate for all transactions created by the wallet. However, in a dynamic fee environment, this can lead to poor fee choices, either overpaying when the mempool is empty or underpaying when congestion is high. The preferred approach is to rely on fee estimation, which is designed to adapt to network conditions, and is the one by default. Same argument apply for `paytxfee` setting.
During discussion the consensus was that static fee settings are a footgun and that users should instead specify the fee rate per transaction if they don't want to rely on the fee estimation. Given this, rather than introducing a `settxfeerate` alternative, this PR goes towards removing `settxfee` and `paytxfee` entirely.
**Key Changes**
`settxfee` and `paytxfee` is now deprecated and will be removed in Bitcoin Core 31.0.
Users should rely on fee estimation or explicitly specify a fee rate when constructing transactions.
**Impact on Users**
If users currently use settxfee or paytxfee, they should transition to specifying fees per transaction.
No immediate breakage in 30.0 (must use `-deprecatedrpc=settxfee`), but `settxfee` and `paytxfee` will be removed in 31.0.
**Alternative Approaches Considered**
A settxfeerate alternative (using sat/vB) was initially proposed but ultimately rejected in favor of deprecating static fee setting entirely.
**Notes for removal**
- When removing paytxfee we should also update txconfirmtarget startup option help text.
- Get back the comment from `rpc_deprecated.py` test. [+info](https://github.com/bitcoin/bitcoin/pull/31278#discussion_r1998876768)
ACKs for top commit:
fjahr:
re-ACK 2f2ab47bf7
ismaelsadeeq:
re-ACK 2f2ab47bf7
rkrux:
Concept and utACK 2f2ab47bf7
Tree-SHA512: 0272812cbe5a519737c5d0683acc2072e67559792b4a6472bca8b23426e5ce9e88a3a1eba987feda70a082b8b474b3126893848628d7bf11e1520357b18c8d3e
In order to make it possible for higher layers to compare transaction quality
(ordering within the implicit total ordering on the mempool), expose a comparison
function and test it.
Before this commit, if a TxGraph::Ref object is destroyed, it becomes impossible
to refer to, but the actual corresponding transaction node in the TxGraph remains,
and remains indefinitely as there is no way to remove it.
Fix this by making the destruction of TxGraph::Ref trigger immediate removal of
the corresponding transaction in TxGraph, both in main and staging if it exists.
In order to make it easy to evaluate proposed changes to a TxGraph, introduce a
"staging" mode, where mutators (AddTransaction, AddDependency, RemoveTransaction)
do not modify the actual graph, but just a staging version of it. That staging
graph can then be commited (replacing the main one with it), or aborted (discarding
the staging).
Move a number of related modifications to TxGraphImpl into a separate
function for removal of transactions. This is preparation for a later
commit where this will be useful in more than one place.
This is a preparation for a next commit where a TxGraph will start representing
potentially two distinct graphs (a main one, and a staging one with proposed
changes).
When transactions are removed from the tail of a cluster, we know the existing
linearization remains acceptable (if it already was), but may just need splitting
and postlinearization, so special case these into separate quality levels.
Chunk-based information (primarily, chunk feerates) are never accessed without
first bringing the relevant Clusters to an "acceptable" quality level. Thus,
while operations are ongoing and Clusters are not acceptable, we can omit
computing the chunkings and chunk feerates for Clusters.
Since m_deps_to_add has been sorted by child Cluster* already, all dependencies
with the same child will be processed consecutively. Take advantage of this by
remember the last partition merged with, and reusing that if applicable.
The m_deps_to_add vector is sorted by child Cluster*, which matches the
order of an_clusters. This means we can walk through m_deps_to_add while
doing the representative lookups for an_clusters, and reuse them.
Instead of leaving the responsibility on higher layers to guarantee that
no connected component within TxGraph (a barely exposed concept, except through
GetCluster()) exceeds the cluster count limit, move this responsibility to
TxGraph itself:
* TxGraph retains a cluster count limit, but it becomes configurable at construction
time (this primarily helps with testing that it is properly enforced).
* It is always allowed to perform mutators on TxGraph, even if they would cause the
cluster count limit to be exceeded. Instead, TxGraph exposes an IsOversized()
function, which queries whether it is in a special "oversize" state.
* During oversize state, many inspectors are unavailable, but mutators remain valid,
so the higher layer can "fix" the oversize state before continuing.
Instead construct a single vector with the list of all clusters in all groups,
and then store per-group offset/range in that list.
For dependencies, reuse m_deps_to_add, and store offset/range into that.
To make testing more powerful, expose a function to perform an internal sanity
check on the state of a TxGraph. This is especially important as TxGraphImpl
contains many redundantly represented pieces of information:
* graph contains clusters, which refer to entries, but the entries refer back
* graph maintains pointers to Ref objects, which point back to the graph.
This lets us make sure they are always in sync.
This adds a simulation fuzz test for txgraph, by comparing with a naive
reimplementation that models the entire graph as a single DepGraph, and
clusters in TxGraph as connected components within that DepGraph.
This adds an initial version of the txgraph module, with the TxGraph class.
It encapsulates knowledge about the fees, sizes, and dependencies between all
mempool transactions, but nothing else.
In particular, it lacks knowledge about txids, inputs, outputs, CTransactions,
... and so forth. Instead, it exposes a generic TxGraph::Ref type to reference
nodes in the TxGraph, which can be passed around and stored by layers on top.
Since cluster_linearize.h does not actually have a Cluster type anymore, it is more
appropriate to rename the index type to DepGraphIndex.
-BEGIN VERIFY SCRIPT-
sed -i 's/Data type to represent transaction indices in clusters./Data type to represent transaction indices in DepGraphs and the clusters they represent./' $(git grep -l 'using ClusterIndex')
sed -i 's|\<ClusterIndex\>|DepGraphIndex|g' $(git grep -l 'ClusterIndex')
-END VERIFY SCRIPT-
This function takes an existing ordering for transactions in a DepGraph, and
makes it a valid linearization for it (i.e., topological). Any topological
prefix of the input remains untouched.
Removed duplicate call to GetDescriptorScriptPubKeyMan and
Instead of checking linearly I have used find method so time complexity reduced significantly for GetDescriptorScriptPubKeyMan
after this fix improved performance of importdescriptor part refs #32013.
Move the comparison to hashWatchedChain inside the while loop.
Although this early return prevents the GetTransactionsUpdated()
call in cases where the tip updates, it's only done to improve
readability. The check itself is very cheap (although a more
useful check might not be).
Also add code comments.
- return null on shutdown instead of the last tip
- ignore timeout value node initialization
This allows consumers of BlockTemplate to safely
assume that a tip is connected, instead of having
to account for startup and early shutdown scenarios.
d423fd9ec8 cli, bugfix: for -getinfo, replace IsArgSet() with GetBoolArg() (Jon Atack)
e99e41b307 cli, refactor: simplify public-only classes with structs (Jon Atack)
fdbfd250fb cli, refactor: deduplicate NetworkStringToId() (Jon Atack)
be82139b2a cli, refactor: simplify DetailsRequested() (Jon Atack)
Pull request description:
These have been accumulating over the past few years.
Each is described in its commit message.
ACKs for top commit:
pablomartin4btc:
re-ACK d423fd9ec8
hodlinator:
Code review ACK d423fd9ec8
l0rinc:
ACK d423fd9ec8
ryanofsky:
Code review ACK d423fd9ec8, just running clang-format and updating commit messages since last review
Tree-SHA512: a8e5f7827cef308186d5a7c3a627d2cd8f57437f4465d181986e5d3274ff0e2b9faac252dd55d9257d66a7aa99fca62b3000cdc0988d23385df20ff1f870a046
eb0724f0de doc: banman: reference past vuln due to unbounded banlist (Antoine Poinsot)
ad616b6c01 doc: net: mention past vulnerability as rationale to limit incoming message size (Antoine Poinsot)
4489117c3f doc: txrequest: point to past censorship vulnerability in tx re-request handling (Antoine Poinsot)
68ac9542c4 doc: net_proc: reference past DoS vulnerability in orphan processing (Antoine Poinsot)
c02d9f6dd5 doc: net_proc: reference past defect regarding invalid GETDATA types (Antoine Poinsot)
5e3d9f21df doc: validation: add a reference to historical header spam vulnerability (Antoine Poinsot)
Pull request description:
It is useful when reading code to have context about why it is written or behaves the way it does. Some instances in this PR may seem obvious but i think nonetheless offer important context to anyone willing to change (or review a change to) this code.
ACKs for top commit:
ryanofsky:
Code review ACK eb0724f0de. No changes since last review other than rebase
Tree-SHA512: 271902f45b8130d44153d793bc1096cd22b6ce05494e67c665a5bc45754e3fc72573d303ec8fc7db4098d473760282ddbf0c1cf316947539501dfd8d7d5b8828
7edaf8b64c Benchmark Chainstate::ConnectBlock duration (Eunovo)
Pull request description:
Introduce benchmarks to evaluate ConnectBlock performance for:
- Blocks containing only Schnorr signatures
- Blocks containing both Schnorr and ECDSA signatures
- Blocks containing only ECDSA signatures
The benchmarks in this PR, focus on signature validation. Additional benchmarks may be added in the future to assess other aspects of ConnectBlock.
This is the first step toward implementing Batch Verification of Schnorr Signatures in Core. It provides a way to test and measure the performance improvements of batch verification on Core.
For more details on batch validation, refer to the [batch-verify module on secp](https://github.com/bitcoin-core/secp256k1/pull/1134) and [batch-verify on core](https://github.com/bitcoin/bitcoin/pull/29491).
ACKs for top commit:
josibake:
reACK 7edaf8b64c
fjahr:
utACK 7edaf8b64c
l0rinc:
ACK 7edaf8b64c
Tree-SHA512: 883c8a5e4e4de401ffb9ac9b6789b7fe0737afefbdaf02c6d7e1645392efc4f0d2d28b423ba7e34366a33608e0835793f5e7a1312b5c8063de14446319529cc7
According to the CMake docs, this is the correct way to setup a
toolchain file for cross-compilation using Clang. See
https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html#cross-compiling-using-clang
Internally it looks like CMake will only take this variable into account
if it detects the compiler to be Clang, so this shouldn't effect other
builds, but in the case of our Apple cross builds, we'd end up with a
duplicated `--target=arm64-apple-darwin` on the compiler line, given we
are already setting `--target` for Darwin builds.
Would fix#31748.
fa7a40d952 contrib: Print deterministic-coverage runs (MarcoFalke)
fa751639fb contrib: Make deterministic-coverage error messages more readable (MarcoFalke)
Pull request description:
This is almost a "refactor" to tidy up the error messages. Apart from the messages, the behavior of the tools is identical.
This was requested in https://github.com/bitcoin/bitcoin/pull/31901#discussion_r1969502508.
Previously, the tool would abort the program early on some errors. Now, the tool propagates an `std::result::Result::Err` up to `main` via an early return. Getting rid of the aborts also allows to drop the `RUST_BACKTRACE` env setting.
ACKs for top commit:
hodlinator:
re-ACK fa7a40d952
janb84:
ACK [fa7a40d](fa7a40d952)
Tree-SHA512: 6c97861306e2fececa14b2d12deafb78995fc2bcf75e4e22773cb0ab4231de78834db9f1f89b30c49d77499433b1c16c1d90b97eb4069c81855bd2a7944b554f
ba82240553 fuzz: split `coinselection` harness (brunoerg)
Pull request description:
This PR splits the `coinselection` fuzz harness into 3 targets (`coinselection_bnb`, `coinselection_knapsack`, `coinselection_srd`). The goal is to be able to fuzz each algorithm separately (to avoid performance issues) and also all of them together.
ACKs for top commit:
janb84:
Tested ACK [ba82240](ba82240553)
maflcko:
review ACK ba82240553👐
marcofleon:
reACK ba82240553
zaidmstrr:
reACK [ba82240](ba82240553)
Tree-SHA512: 277cffd524e57d286dbbbcb2aa0a9f1d720b4c56331dfb0f4425e1666246330616508e47977da23f28a72705aa142bbaf536e2cf7fe4703a2cd2e4b2fd441d9d
63b534f97e fuzz: sanity check hardcoded snapshot in utxo_snapshot target (Antoine Poinsot)
3b85eba83a test util: split up ConnectBlock from MineBlock (Antoine Poinsot)
d1527f6b88 qa: correct off-by-one in utxo snapshot fuzz target (Antoine Poinsot)
Pull request description:
The assumeutxo data for the fuzz target could change and invalidate the hash silently, preventing the fuzz target from reaching some code paths. Fix this by introducing a unit test which would break if the snapshot data the fuzz target relies on were to change.
In implementing this i noticed the height used for coins in the fuzz target is actually off-by-one (as if the first block in the created chain was the genesis but it's block `1`), so fix that too.
ACKs for top commit:
mzumsande:
Code Review ACK 63b534f97e
fjahr:
tACK 63b534f97e
Tree-SHA512: 2399b6e74db9b78aab8efba67c57a405d2d7d880ae3b7d8518a1c96cc6266f61f5e77722cd999adeac5d3e03e73d84cf9ae7bdbcc0afae198cc87049dde4012f
f708498293 torcontrol: Limit reconnect timeout to max seconds and log delay in whole seconds (Eval EXEC)
Pull request description:
I'm reviewing the Tor controller's reconnect-related code and noticed that the reconnect timeout had no limit. This could lead to excessively long delays.
This PR introduces a maximum reconnect timeout of 600 seconds (10 minutes) to prevent excessive delays in reconnection attempts. It also updates the log message to display the retry delay in whole seconds for better readability.
ACKs for top commit:
mabu44:
ACK f708498293
laanwj:
Code review ACK f708498293
luke-jr:
utACK f708498293
Tree-SHA512: 8f18c6c84da6b4e7328638fd74539fbd3dd44f46c5107638de56b72fc079487690861199ceba1197ca34421dcedf79a1ca6bacf2a918a683e71bce9ff710b5d4
fa310cc6f4 test: Fix intermittent issue in p2p_orphan_handling.py (MarcoFalke)
Pull request description:
The test may fail intermittently when the `net` thread is lagging while calling `DeleteNode`. This may result in a split `getdata`, meaning that `peer2.wait_for_parent_requests([int(parent_peekaboo_AB["txid"], 16), int(parent_missing["txid"], 16)])` fails.
Fix it by adding a sync on the `net` thread.
Fixes#31700
ACKs for top commit:
mzumsande:
Code Review ACK fa310cc6f4
Tree-SHA512: e4a58093ab5b9e280c479b845fecb5d228e65519ea3dc2111b393202225fd0feded423e8812452454b6b9348cb37a9c1b01b9d1b1802e9f4aa76b9e56b4b54ef
fa4fb6a8f1 fuzz: Use serial task runner to increase fuzz stability (MarcoFalke)
Pull request description:
Leaking a scheduler with a non-empty queue from the fuzz initialization phase into the fuzz target execution phase is problematic, because it messes with coverage data. This in turn is problematic, because it leads to:
* Decrease in fuzz target execution stability (non-determinism when running the fuzz target).
* Decrease in fuzz input merge stability (non-determinism when selecting a minimum set of fuzz input to reach maximum coverage), which leads to qa-assets bloat.
Fix one such issue. Tracking issue: https://github.com/bitcoin/bitcoin/issues/29018
Can be tested via: `RUST_BACKTRACE=1 cargo run --manifest-path ./contrib/devtools/deterministic-fuzz-coverage/Cargo.toml -- $PWD/bld-cmake $PWD/../b-c-qa-assets/fuzz_corpora/ partially_downloaded_block`.
The failure is non-deterministic (obviously) and will show coverage in validation signals such as `UpdatedBlockTip` before this change and will have this one fixed after this change.
ACKs for top commit:
marcofleon:
ACK fa4fb6a8f1
dergoegge:
Code review ACK fa4fb6a8f1
Tree-SHA512: fd1f66562c1d3c21553c7dd324399cdc16faa2fedfdb8e7544ea6a68b8b356e7c81d81815ecf70e0d334307dab6b275c1889b3b889b6f15eec514beee22c95f4
d190f0facc test, contrib: Fix signer/miner command line escaping (Ryan Ofsky)
0d2eefca8b test, refactor: Add TestNode.binaries to hold binary paths (Ryan Ofsky)
Pull request description:
Add new `TestNode.binaries` object to manage paths to bitcoin binaries.
The `binaries` object makes it possible for the test framework to exercise the bitcoin wrapper executable introduced in https://github.com/bitcoin/bitcoin/pull/31375 and also makes it easier in general to add new binaries, and new options and environment variables controlling how they are invoked, because logic for invoking them that was previously spread out is now consolidated in one place.
These changes were originally part of #31375 but made that PR harder to review because they were unrelated to the other changes there. If this PR can get merged first, python changes in #31375 will be simple, and the test framework changes here should also get a higher quality review.
ACKs for top commit:
maflcko:
re-review-ACK d190f0facc🍓
Sjors:
ACK d190f0facc
vasild:
ACK d190f0facc
Tree-SHA512: 5a6c0553cd2822585810d827ef1c1772cbf3097d3336daf733f8378dd3da79c00fc3721e50ed0f7455908fbd7a509e9739f9be33f588d6bc1aaa400b9d75c650
ffff4a293a bench: Update span-serialize comment (MarcoFalke)
fa4d6ec97b refactor: Avoid false-positive gcc warning (MarcoFalke)
fa942332b4 scripted-diff: Bump copyright headers after std::span changes (MarcoFalke)
fa0c6b7179 refactor: Remove unused Span alias (MarcoFalke)
fade0b5e5e scripted-diff: Use std::span over Span (MarcoFalke)
fadccc26c0 refactor: Make Span an alias of std::span (MarcoFalke)
fa27e36717 test: Fix broken span_tests (MarcoFalke)
fadf02ef8b refactor: Return std::span from MakeUCharSpan (MarcoFalke)
fa720b94be refactor: Return std::span from MakeByteSpan (MarcoFalke)
Pull request description:
`Span` has some issues:
* It does not support fixed-size spans, which are available through `std::span`.
* It is confusing to have it available and in use at the same time with `std::span`.
* It does not obey the standard library iterator build hardening flags. See https://github.com/bitcoin/bitcoin/issues/31272 for a discussion. For example, this allows to catch issues like the one fixed in commit fabeca3458.
Both types are type-safe and can even implicitly convert into each other in most contexts.
However, exclusively using `std::span` seems less confusing, so do it here with a scripted-diff.
ACKs for top commit:
l0rinc:
reACK ffff4a293a
theuni:
ACK ffff4a293a.
Tree-SHA512: 9cc2f1f43551e2c07cc09f38b1f27d11e57e9e9bc0c6138c8fddd0cef54b91acd8b14711205ff949be874294a121910d0aceffe0e8914c4cff07f1e0e87ad5b8
fac3d93c2b fuzz: Speed up *_package_eval fuzz targets a bit (MarcoFalke)
fa40fd043a fuzz: [refactor] Avoid confusing c-style cast (MarcoFalke)
Pull request description:
Each target is at least 10% faster for me when running over the current set of qa-assets, which seems nice.
The changes `outpoints_value` from a map to an unordered map, which is safe, because the element order is not used in the fuzz test and the map is only used for lookup.
(`mempool_outpoints` can't be changed, because the order matters here. Using unordered_set here may result in a non-deterministic fuzz target, given the same fuzz input.)
ACKs for top commit:
l0rinc:
ACK fac3d93c2b
dergoegge:
Code review ACK fac3d93c2b
Tree-SHA512: 8ae5d4e281505aff76a4003d6e9ea388dbb73860e167385bd6a0a201b3acc939db29ee212594952a9e80e85b3cc4cd726ce6dd49551f74013cb4da8a15cbdfb3
c8fab35617 ci: remove -Wno-error=deprecated-declarations from ASAN (fanquake)
a130bbd154 Squashed 'src/leveldb/' changes from 04b5790928..4188247086 (fanquake)
Pull request description:
Cherry-picks two commits from upstream (302786e211, e829478c6a), which remove the usage of `std::aligned_storage/std::aligned_union`.
Note the first cherry-pick is not clean, because due to Google tooling issues, it accidently contained a revert of the prior two commits. See https://github.com/google/leveldb/pull/1249 for more details.
Also see https://issues.chromium.org/issues/388068052, although note that they [reverted the roll to latest leveldb](https://issues.chromium.org/issues/388068052#comment9). I'm guessing due to the acidental reversion issue above.
ACKs for top commit:
l0rinc:
ACK c8fab35617
darosior:
ACK c8fab35617 -- checked it's a clean subtree pull from https://github.com/bitcoin-core/leveldb-subtree/tree/bitcoin-fork
dergoegge:
utACK c8fab35617
Tree-SHA512: 966e61b9ac88af5ae7bf71514bfd5bbdbd8c38c7af65feb6d5e4415062dcff5896dc33fe968ded3462cc599abd921d49ee8336db3e12ed3f59c91ceb949317b7
387385ba1e test: replace assert with assert_equal and assert_greater_than (Chandra Pratap)
Pull request description:
In `test/functional/interface_usdt_net.py`, `assert_equal` is already used to check for equality between objects. Replace `assert.*==` with `assert_equal` and `assert.*>` with `assert_greater_than` to further easify debugging.
Relevant issue: #23119
ACKs for top commit:
maflcko:
lgtm ACK 387385ba1e
0xB10C:
had a quick look, lgtm ACK 387385ba1e
theStack:
utACK 387385ba1e
brunoerg:
code review ACK 387385ba1e
i-am-yuvi:
Great! ACK 387385ba1e
Tree-SHA512: 741a3d98288c9999f62bcbaa3806716b0519ec9b521e1e6e17aa458392245f6eff886af6cb601c66f2147e0265ff1eae57cea3dcfd67af93bef6dff25b056935
cec14ee47d test: switch wallet_crosschain.py to signet (Sjors Provoost)
9c2951541c test: drop testnet4 from wallet_crosschain.py (Sjors Provoost)
Pull request description:
It's sufficient to check only one test network, so this PR reverts the addition of testnet4 from #29775.
Testnet3 is deprecated. Instead of moving to testnet4, which might also be deprecated in the future, use signet.
ACKs for top commit:
fjahr:
utACK cec14ee47d
maflcko:
lgtm ACK cec14ee47d🌰
Tree-SHA512: c5aad6e7d251957f090145eac906f7985fddc3e3ba82df7184d72b961f9c856d324a1065ac98323b75501d136bd7b669fcc2565b9e66b0743eb3f3906ef37570
20fe41e9e8 test: avoid disk space warning for non-regtest (Sjors Provoost)
Pull request description:
`feature_config_args.py` incorrectly assumed that its testnet4 node would not log a disk space warning.
But when #31978 increased `m_assumed_blockchain_size` on testnet4 from 1 to 11 GiB, it triggered this bug on my RAM disk, see https://github.com/bitcoin/bitcoin/tree/master/test#speed-up-test-runs-with-a-ram-disk
This PR fixes the issue by using `-prune` which prevents the warning.
ACKs for top commit:
fjahr:
ACK 20fe41e9e8
maflcko:
lgtm ACK 20fe41e9e8
rkrux:
ACK 20fe41e9e8
Tree-SHA512: f4bbb3ede307e06bf097a3cf7a4099eacc9388e33f551e1d0c4c5f53747bfa593a4b22e5d2e713ce6dd8adf91602fade36fbec9cfc2b250a6b1cf09f11bc8473
In test/functional/interface_usdt_net.py, assert_equal is already
used to check for equality between objects. Replace 'assert.*=='
with 'assert_equal' and 'assert.*>' with 'assert_greater_than'
to further easify debugging.
21e9d39a37 docs: add release notes for 31603 (brunoerg)
a8b548d75d test: `getdescriptorinfo`/`importdescriptors` with whitespace in pubkeys (brunoerg)
c7afca3d62 test: descriptor: check whitespace into keys (brunoerg)
cb722a3cea descriptor: check whitespace in ParsePubkeyInner (brunoerg)
50856695ef test: fix descriptors in `ismine_tests` (brunoerg)
Pull request description:
Currently, we successfully parse descriptors which contains spaces in the beginning or end of the public/private key within a fragment (e.g. `pk( KEY)`, `pk(KEY )` or `pk( KEY )`). I have noticed that one of the reasons is that the `DecodeBase58` function simply ignore these whitespaces.
This PR changes the `ParsePubkeyInner ` to reject pubkeys that contain a whitespace at the beginning and/or at the end. We will only check the whitespace in some RPCs (e.g. `importdescriptors`), but an already imported descriptor won't be affected by this check, especially because we store descriptors from `ToString`.
For context: https://github.com/brunoerg/bitcoinfuzz/issues/72
ACKs for top commit:
rkrux:
tACK 21e9d39a37
darosior:
re-ACK 21e9d39a37
sipa:
utACK 21e9d39a37
Tree-SHA512: 54f48a89a235517e5cdc29a46dceeb7dabbee93c7616a166288ff3f90131808eb0ece43b0797a11fe827a5f7bd51d65e3e75c16789b0a42020934cabb684cc8f
Pass bitcoin binary command lines from test framework to signet/miner utility
using shell escaping so they are unambigous and don't get mangled if they
contain spaces.
This change is not needed for tests to pass currently, but is a useful change
to avoid CI failures in followup PR
https://github.com/bitcoin/bitcoin/pull/31375 and to avoid other bugs.
Add new TestNode.binaries object to manage paths to bitcoin binaries.
Having this object makes it possible for the test framework to exercise the
bitcoin wrapper executable introduced in
https://github.com/bitcoin/bitcoin/pull/31375 and also makes it easier to add
new binaries and options and environment variables controlling how they are
invoked, because logic for invoking them that was previously spread out is now
consolidated in one place.
Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
1f9b2e150c cmake: Require `zip` only for `deploy` target (Hennadii Stepanov)
0aeff29951 cmake: Check for `makensis` tool before using it (Hennadii Stepanov)
Pull request description:
For `x86_64-w64-mingw32` and `*-apple-darwin` targets, the optional `deploy` target requires dedicated tools: `makensis` and `zip`, respectively.
This PR introduces a uniform checks for those tools when attempting to build the `deploy` target, ensuring they are not required for configuring and building any other targets.
Here is an example of workflow for `x86_64-w64-mingw32`:
```
$ # `nsis` is not installed
$ cmake -B build -G "GNU Makefiles" --toolchain depends/x86_64-w64-mingw32/toolchain.cmake
$ cmake --build build -j $(nproc)
$ cmake --build build -t deploy
Error: NSIS not found.
Please install NSIS and/or ensure that its executable is accessible to the find_program() command—
for example, by setting the MAKENSIS_EXECUTABLE variable or another relevant CMake variable.
Then re-run cmake to regenerate the build system.
Built target deploy
$ sudo apt install nsis
$ cmake -B build
$ cmake --build build -t deploy
...
[100%] Generating bitcoin-win64-setup.exe
[100%] Built target deploy
```
Fixes https://github.com/bitcoin/bitcoin/issues/32018.
ACKs for top commit:
hodlinator:
re-ACK 1f9b2e150c
fanquake:
ACK 1f9b2e150c
Tree-SHA512: 5e2bd28a13bd8fa7c4ba8cf1756d200a4651afe83c463d76ece10027cca343e124eff97012a5368028f761df60f420ab891106b4e33b50045051d57c7464ff98
This reverts the changes to test/functional/wallet_crosschain.py
in commit 74a04f9e7a.
It's sufficient to check only one test network. The next commit
will change that network away from testnet3.
54e6eacc1f test: Enable ResetCoverageCounters beyond Linux (janb84)
Pull request description:
In PR [#31901](https://github.com/bitcoin/bitcoin/pull/31901), Coverage.cpp was introduced as a separate utility file, based on existing code. However, the macro defined in Coverage.cpp was limited to Clang and Linux, which caused issues for users on macOS when using the newly introduced deterministic test tooling.
This change adds fallback functions which are used when building without code coverage on non linux env.
This adds support for macOS to ResetCoverageCounters. ResetCoverageCounters is used by the unit tests in `g_rng_temp_path_init` to support the deterministic unit test tooling. It is also used in fuzz tests to completely suppress coverage from anything init-related.
See [Readme](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/README.md) on how to test this for deterministic unit & fuzz test.
Suggestion for test files:
- for unit test: `util_string_tests`
- for fuzz test: `addition_overflow `
These files should give deterministic results
ACKs for top commit:
maflcko:
review-only ACK 54e6eacc1f
hodlinator:
re-ACK 54e6eacc1f
Tree-SHA512: dd71da6f76d4fc9e64bf521bbfe5e7483d77c2ca0380f9e692502e64b529068ea33f21b19399481feb7c6780a23d893d8b7f733cef641a2db18a13397c98deea
fa3b442715 ci: Use Cirrus dockerfile cache (MarcoFalke)
Pull request description:
The lint task is problematic, because:
* It doesn't check modifications to `ci/lint_imagefile`
* It calls a separate script that installs packages on every run (taking time)
* It uses `*_cache` instructions to cache some installed parts, but not all
Fix all issues by using `ci/lint_imagefile` (https://cirrus-ci.org/guide/docker-builder-vm/#dockerfile-as-a-ci-environment)
ACKs for top commit:
willcl-ark:
ACK fa3b442715
Tree-SHA512: 243d78219639b83721d4e5cb32d16e5c208a61c919d04646279be5825ba92d97c490b5d4d28ea103eb820b1a259904574cb3e32eaca3f11c3031810e3d87ff4a
6f9f415a4f doc: shallow clone qa-assets (Lőrinc)
Pull request description:
While reviewing https://github.com/bitcoin/bitcoin/pull/31457#pullrequestreview-2690077410 I noticed that cloning `qa-assets` takes a lot of time - shallow cloning should suffice here.
I haven't checked the other clones in this file but suggestion are welcome.
ACKs for top commit:
maflcko:
lgtm ACK 6f9f415a4f
Tree-SHA512: 21bd676c7709dbf7fd30b239d0a72f9c230453ed8f8a1b5319ac92ef9c5e67780939f095a239dd31bcb4550f8d69eaed4931a221e19cb0b957f18fac623c4a01
We might have just set CNode::fDisconnect in the first loop because of being offline.
Also caches CConnman::fNetworkActive in case it's changed concurrently with our own thread.
52482cb244 test: Check datadir cleanup after assumeutxo was successful (Fabian Jahr)
Pull request description:
I noticed that the proper datadir cleanup after a successful restart of an assumutxo node does not seem to be covered in our tests. This is added here.
ACKs for top commit:
l0rinc:
utACK 52482cb244
mabu44:
Re-ACK 52482cb244
Prabhat1308:
re-ACK [`52482cb`](52482cb244)
TheCharlatan:
Re-ACK 52482cb244
Tree-SHA512: cc941afeba250050eaccf5112255d961253fec9b2683545454a0d2fbe4d542178394b301d169a9dd79edbf6b5d478d95282727dbb0aca96ee79d4cd1ff80f19b
7ebc458a8c qt: doc: adapt outdated binary paths to CMake changes (Sebastian Falbesoner)
Pull request description:
Adapt the qt-related instances of outdated binary paths to `./build/bin/...` (see [#30454](https://github.com/bitcoin/bitcoin/pull/30454) and the more recently merged [#31161](https://github.com/bitcoin/bitcoin/pull/31161)). According to `$ git grep src/qt.*bitcoin` there should be no more left to address.
ACKs for top commit:
maflcko:
lgtm ACK 7ebc458a8c
Sjors:
utACK 7ebc458a8c
fanquake:
ACK 7ebc458a8c
hebasto:
ACK 7ebc458a8c.
Tree-SHA512: 8cd6579fdf209ec4ee3c4c9cfb94cb11d5d5115068d31613d356ca1303214dc4461580535c2d3f2773f373a4271e9a82df25596d8369eef8235822f7030d88bd
4cd95a2921 refactor: modernize remaining outdated trait patterns (Lőrinc)
ab2b67fce2 scripted-diff: modernize outdated trait patterns - values (Lőrinc)
8327889f35 scripted-diff: modernize outdated trait patterns - types (Lőrinc)
Pull request description:
The use of [`std::underlying_type_t<T>`](https://en.cppreference.com/w/cpp/types/underlying_type) or [`std::is_enum_v<T>`](https://en.cppreference.com/w/cpp/types/is_enum) (and similar ones, introduced in C++14) replace the `typename std::underlying_type<T>::type` and `std::is_enum<T>::value` constructs (available in C++11).
The `_t` and `_v` helper alias templates offer a more concise way to extract the type and value directly.
I've modified the instances I found in the codebase one-by-one (noticed them while investigating https://github.com/bitcoin/bitcoin/pull/31868), and afterwards extracted scripted diff commits to do the trivial ones automatically.
The last commit contains the values that were easier done manually.
I've excluded changes from `src/bench/nanobench.h`, `src/leveldb`, `src/minisketch`, `src/span.h` and `src/sync.h` - let me know if you think they should be included instead.
A few of the code changes can also be reproduced by clang-tidy (but not all of them):
```bash
cmake -B build -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DBUILD_BENCH=ON -DBUILD_FUZZ_BINARY=ON -DBUILD_FOR_FUZZING=ON && cmake --build build -j$(nproc)
run-clang-tidy -quiet -p build -j $(nproc) -checks='-*,modernize-type-traits' -fix $(git grep -lE '::(value|type)' ./src ':(exclude)src/bench/nanobench.h' ':(exclude)src/leveldb' ':(exclude)src/minisketch' ':(exclude)src/span.h' ':(exclude)src/sync.h')
```
ACKs for top commit:
laanwj:
Concept and code review ACK 4cd95a2921
Tree-SHA512: a4bcf0f267c0f4e02983b4d548ed6f58d464ec379ac5cd1f998b9ec0cf698b53a9f2557a05a342b661f1d94adefc9a0ce2dc8f764d49453aaea95451e2c4c581
3301d2cbe8 qa: Wait for txindex to avoid race condition (Hodlinator)
9bfb0d75ba qa: Remove unnecessary -txindex args (Hodlinator)
7ac281c19c qa: Add missing coverage of corrupt indexes (Hodlinator)
Pull request description:
- Add synchronization in 3 places where if the Transaction Index happens to be slow, we get rare test failures when querying it for transactions (one such case experienced on Windows, prompting investigation).
- Remove unnecessary TxIndex initialization in some tests.
- Add some test coverage where TxIndex aspect could be tested in feature_init.py.
ACKs for top commit:
fjahr:
re-ACK 3301d2cbe8
mzumsande:
Code Review ACK 3301d2cbe8
furszy:
Code review ACK 3301d2cbe8
Prabhat1308:
Concept ACK [`3301d2c`](3301d2cbe8)
Tree-SHA512: 7c2019e38455f344856aaf6b381faafbd88d53dc88d13309deb718c1dcfbee4ccca7c7f1b66917395503a6f94c3b216a007ad432cc8b93d0309db9805f38d602
36b0713edc test: fix intermittent failure in wallet_reorgsrestore.py (furszy)
Pull request description:
In response to #32066 intermittent failure.
Wait until the node's process has fully stopped before starting a new instance of it.
Same behavior as in the [tool_wallet.py](698f86964c/test/functional/tool_wallet.py (L540)) test.
ACKs for top commit:
maflcko:
lgtm ACK 36b0713edc
Chand-ra:
tACK [36b0713](36b0713edc)
Tree-SHA512: 8e01493ef1fb58589479f3e12d7429d02ca75a2183d5f79d3b6a2fbf13334878926274a20857f1b4729afc1d30b65789daed229ce06ba236b91d949b73f45d5a
02942056fd test: fix intermittent failure in p2p_orphan_handling.py (Martin Zumsande)
Pull request description:
If the mocktime is bumped before the node has successfully disconnected the peer, the requests for both parents could be spread over two GETDATAS: The first time `GetRequestsToSend` is invoked it would only request one tx from peer2, because the other one would only be available after peer1 was disconnected and its outstanding txrequest cleared.
So two GETDATAs would be sent, which would make the test fail.
Fixes#31700
ACKs for top commit:
maflcko:
lgtm ACK 02942056fd
instagibbs:
ACK 02942056fd
Tree-SHA512: 769200898345da197d86d673d9506f08f0a64b72a456e7e7c988ac37450d9c54ec65da1c8447c566c8578f7cfccdc5723ea680e636bfbe0b3d38265e5ef57774
Non-Linux linkers require a fallback implementation for when coverage is not enabled.
The fallbacks are marked weak to have lower precedence than built-in implementations when available, removing ambiguity from the linker.
9157d9e449 build: use make < 3.82 syntax for define directive (Sjors Provoost)
Pull request description:
From the GNU make 3.82 [release announcement](https://lists.gnu.org/archive/html/info-gnu/2010-07/msg00023.html) (2010):
> The 'define' make directive now allows a variable assignment operator
after the variable name, to allow for simple, conditional, or appending
multi-line variable assignment.
macOS ships with 3.81 (2006). This caused the multiprocess config options to be ignored.
Fixes#32068
ACKs for top commit:
ryanofsky:
Code review ACK 9157d9e449. This is a pretty unusual bug and I don't understand how it wasn't causing any errors with make 3.81, just causing the flags to be ignored.
Tree-SHA512: a07322e25ef18296264379a2704f31c654df196d3ea09fe712885c38813e54d758a2d603ee9f7a302da8011fba6d139aa30a356175ca99df728ade2572a87560
fa9cf38ab6 scripted-diff: test: Rename send_message to send_without_ping (MarcoFalke)
fa4356717d test: Prefer send_and_ping over send_message+sync_with_ping (MarcoFalke)
Pull request description:
`send_message` is problematic, because it is easy to forget a `sync_with_ping` (or other `wait_until`), leading to intermittent test failures. (Example: https://github.com/bitcoin/bitcoin/pull/31837#discussion_r1950370246)
There are more uses of `send_and_ping` in the codebase than `send_message`, so in most cases `send_and_ping` is needed anyway.
For the remaining cases, clearly document that no sync happens by renaming `send_message` to `send_without_ping`.
ACKs for top commit:
instagibbs:
ACK fa9cf38ab6
Tree-SHA512: 31caa6568d292ae3d3dda931a94aaa30cc1205ec2ef537a484393eb55687f86c212f1e751ac4a7636610bdf591502a50995dc63bf02f97be9fdc482072160b07
d5537c18a9 fuzz: make sure DecodeBase58(Check) is called with valid values more often (Lőrinc)
bad1433ef2 fuzz: Always restrict base conversion input lengths (Lőrinc)
Pull request description:
This is a follow-up to https://github.com/bitcoin/bitcoin/pull/30746, expanding coverage by:
* restricting every input for the base58 conversions, capping max sizes to `100` instead of `1000` or all available input (suggested by marcofleon in https://github.com/bitcoin/bitcoin/pull/30746#discussion_r1963718683) since most actual usage has lengths of e.g. `21`, `34`, `78`.
* providing more valid values to the decoder (suggested by maflcko in https://github.com/bitcoin/bitcoin/pull/30746#discussion_r1957847712) by randomly providing a random input or a valid encoded one; this also enables unifying the roundtrip tests to a single roundtrip per fuzz.
ACKs for top commit:
mzumsande:
Code Review / lightly tested ACK d5537c18a9
maflcko:
review ACK d5537c18a9🚛
Tree-SHA512: 50365654cdac8a38708a7475eaa43396642b7337e2ee8999374c3faafff4f05457abc1a54c701211e0ed24d36c12af77bcad17b49695699be42664f2be660659
2819c51482 test: Use rpc_deprecated only for testing deprecation (Fabian Jahr)
Pull request description:
The comment in `functional/rpc_deprecated.py` says "This test should be used to verify correct behaviour of deprecated RPC methods with and without the -deprecatedrpc flags." I think we can get rid of the "with" part since we can assume that every deprecated RPC is already tested in at least one other functional test. (I didn't look but I could verify in our coverage if someone has doubts about that.) In order for this test to continue working, the flag will need to be used there. Otherwise this seems to prescribe copy+pasting a basic test from another file and I don't see a good reason for that.
ACKs for top commit:
maflcko:
lgtm ACK 2819c51482
janb84:
re ACK [2819c51](2819c51482)
polespinasa:
reACK 2819c51482
Tree-SHA512: 96edfd07be863ad19f99feb27afbc2c3ad53560c93c93eac8de8d766ad8c46e5aa02fd013fec99af794cbe9adca8e459d5b80b454caea7e67b2388003e010bb6
feature_config_args.py incorrectly assumed that its testnet4 node
would not log a disk space warning.
0683b8ebf3 increased m_assumed_blockchain_size
on testnet4 from 1 to 11 GiB which triggers this bug on more
systems, e.g. a RAM disk.
Prevent the warning by setting -prune for these nodes.
Fix the same issue in feature_signet.py
From the GNU make 3.82 release announcement:
* The 'define' make directive now allows a variable assignment operator
after the variable name, to allow for simple, conditional, or appending
multi-line variable assignment.
macOS ships with 3.81. This caused the multiprocess config options
to be ignored.
Fixes#32068
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Wait until the node's process has fully stopped before starting a new instance.
Since the same code is used in tool_wallet.py, this consolidates the behavior
into a 'kill_process()' function.
send_message only drops the bytes in a buffer and a sync is needed to
avoid intermittent test issues. Change the name of the method to make
this more apparent during review.
-BEGIN VERIFY SCRIPT-
sed -i 's/send_message(/send_without_ping(/g' $( git grep -l 'send_message(' )
-END VERIFY SCRIPT-
36b6f36ac4 build: require sqlite when building the wallet (Sjors Provoost)
Pull request description:
Require that sqlite is available in order to compile the wallet. Removes instances of `USE_SQLITE` since it is no longer possible to not have sqlite available.
The `NO_SQLITE` option is dropped from depends.
This is another step towards dropping the legacy wallet, extracted from #31250.
ACKs for top commit:
m3dwards:
ACK 36b6f36ac4
davidgumberg:
crACK 36b6f36ac4
hebasto:
re-ACK 36b6f36ac4.
Tree-SHA512: 870a0135671c80c4f28602119eb8637a1ed43b51b1673bfe88425782fb62ec6ef0f3d6baf0d5984d6a243779b0f63423fd4c4dc324ef87bffba13d63e05ad793
80b5e7f2cb build: Remove manpages when making MacOS app (Ava Chow)
Pull request description:
When creating the MacOS app, the only file that should be in `Bitcoin-Qt.app/Contents/MacOS` is `Bitcoin-Qt`. Since #31844, there was also a `share/` containing the manpage for bitcoin-qt. This manpage is not useful to app users, and it is also causing code signing issues. Thus the directory should be removed when making the app.
Fixes https://github.com/bitcoin/bitcoin/issues/32052#issuecomment-2723007926
ACKs for top commit:
fanquake:
ACK 80b5e7f2cb
Tree-SHA512: fd15b7f99737484e40d31c583e01acc7470d038b0c584dfaefecc740811565ceee048913b6e5e37e7935b74d8100dc8323aed3f69d9a6baa5f434754009eb18c
5dfef6b9b3 depends: remove NO_HARDEN option (fanquake)
Pull request description:
This was only needed to work around a (Libtool related iirc) Windows issue, when hardening was disabled. I can no-longer recreate this failure, so it'd be good to remove this Windows carveout.
ACKs for top commit:
davidgumberg:
crACK 5dfef6b9b3
laanwj:
Code review ACK 5dfef6b9b3
Tree-SHA512: 38657f09c537ba02ecaf0676d47087a835283cabfc81ad9b2d5e68858dcd7a610b6a1df6730920d40b48be2bbc55a45d6b8aea4364884b5f1c1bd12126940f5b
If we bump the mocktime before the node has successfully disconnected
the peer, the requests for both parents could be spread over
two GETDATAS, which would make the test fail.
a24419f8be contrib: Fix `gen-bitcoin-conf.sh`. (David Gumberg)
Pull request description:
In #31118, the format of bitcoind's `--help` output changed slightly in a way that breaks `gen-bitcoin-conf.sh`, modify the script to accommodate the new format, by starting after the line that says "Options:" and stripping the `-help` options and descriptions from the script output.
Before this PR, all options above `-help` were excluded from the example bitcoin.conf.
ACKs for top commit:
mabu44:
Tested ACK a24419f8be
glozow:
ACK a24419f8be
rkrux:
tACK a24419f8be
BrandonOdiwuor:
crACK a24419f8be
Tree-SHA512: 2ef697166d0b37b61ec1a20e357b91d611c932a0e453c4669f74ab69e6310ea1776dce09c1b77e82746072265763cb0c750e6df4c8b4a7d39068fc03f97b221b
e5ff4e416e qa: use a clearer and documented amount error in malleated snapshot (Antoine Poinsot)
b34fdb5ade test: introduce output amount (de)compression routines (Sebastian Falbesoner)
a7911ed101 test: introduce VARINT (de)serialization routines (Sebastian Falbesoner)
Pull request description:
The `feature_assumeutxo.py` functional test checks various errors with malleated snapshots. Some of these cases are brittle or use confusing and undocumented values. Fix one of those by using a clear, documented and forward-compatible value.
I ran across those when working on an unrelated changeset which affected the snapshot. It took me a while to understand where the seemingly magic byte string was coming from, so i figured it was worth proposing this patch on its own for the sake of making the test more maintainable.
See commit messages for details.
ACKs for top commit:
janb84:
re ACK [e5ff4e4](e5ff4e416e)
theStack:
ACK e5ff4e416e
fjahr:
Code review ACK e5ff4e416e
i-am-yuvi:
tACK e5ff4e416e
Tree-SHA512: 60f022b7176836ce05e8f287b436329d7ca6460f3fcd95f78cd24e07a95a7d4d9cbbb68a117916a113fe451732b09a012d300fe860ff33d61823eca797ceddaf
11f8ab140f test: wallet, coverage for crash on dup block disconnection during unclean shutdown (Martin Zumsande)
9ef429b6ae wallet: fix crash on double block disconnection (furszy)
Pull request description:
The wallet crashes if it processes the same block disconnection event twice in a row due
to an incompatible coinbase transaction state.
This happens because `disconnectBlock` provides `TxStateInactive` without the "abandoned"
flag for coinbase transactions to `SyncTransaction`, while `AddToWallet()` internally modifies
it to retain the abandoned state.
The crash flow is as follows:
1) On the first disconnection, the transaction state transitions from "confirmed" to
"inactive," bypassing the state equality check since the provided state differs. Then,
`AddToWallet` internally updates the state to "inactive + abandoned"
2) On the second disconnection, as we provide only the "inactive" state
to `SyncTransaction()`, the state equality assertion fails and crashes the wallet.
Reviewers Note:
The crash can easily be replicated by cherry-picking the test commit in master.
ACKs for top commit:
mzumsande:
Code Review ACK 11f8ab140f
rkrux:
ACK 11f8ab140f
pinheadmz:
ACK 11f8ab140f
Tree-SHA512: 971069bca562f0afb06c34a2516842d01b5cbc2b18ed851392aa3caa3bb7488f4a84a5d017ea334e6361261d3c44aa597cc67a1d4fa16781f85e081f3d1f8771
de1ada079b doc: Adjust path in comment (Hennadii Stepanov)
Pull request description:
It was overlooked in bitcoin/bitcoin#31161.
ACKs for top commit:
l0rinc:
utACK de1ada079b
dergoegge:
ACK de1ada079b
Tree-SHA512: ea2b858f9c6520a71a3e14a3877ad38acfcddee6ee4aad5fa8fc1c08a0589d9d4df9b5fdbae834bd03bbb956f8b52fdd2b35144fe0bf943bc9455ed6c3fc1c80
893ca54585 contrib: Fix deterministic-unittest-coverage tool path (janb84)
Pull request description:
Fix for the tooling introduced/modified in #31901 but the tool path is broken due to silent merge conflict introduced by #31161.
The `deterministic-unittest-coverage` and `deterministic-fuzz-coverage` tools uses the `fuzz` and `test_bitcoind` binaries, for which the location was modified in #31161. This patch updates the location to align with that change.
ACKs for top commit:
maflcko:
lgtm ACK 893ca54585
hebasto:
ACK 893ca54585, I have reviewed the code and it looks OK.
Tree-SHA512: efd5a75b607a6a7889333674b9a1fccf30b6a80bb2aa941f9dcc10b2e0b6158d8c870d58aa76cef2a0280782570400730828d6a4aaf806b83959e9f4bc7313f4
The waitTipChanged() now returns nullopt if the node is shutting down.
Previously it would return the last known tip during shutdown, but
this creates an ambiguous circumstance in the scenario where the
node is started and quickly shutdown, before notifications().TipBlock()
is set.
The getblocktemplate, waitfornewblock and waitforblockheight RPC
are updated to handle this. Existing behavior is preserved.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
This was preventing the (hidden) waitfornewblock, waitforblock and
waitforblockheight methods from being used in the GUI.
The check was added in d6a5dc4a2e
when these RPC methods were first introduced.
They could have been dropped when dca923150e
refactored these methods to use waitTipChanged(), which already
checks for shutdown.
Making this change now simplifies the next commit.
For the coinbase vTxFees used a dummy value of -nFees. This
value was never returned by the RPC or used in a test.
Similarly the fist vTxSigOpsCost entry was calculated from
the dummy coinbase transaction.
Drop both and add code comments to prevent confusion.
fa21597064 ci: Revert "Temporary workaround for old CCACHE_DIR cirrus env" (MarcoFalke)
Pull request description:
Seems fine to revert this now. If this still happens it should be rare enough and trivial to fix via a new push (normal push, force-push, rebase, ...), or to just ignore the failure.
ACKs for top commit:
fanquake:
ACK fa21597064
Tree-SHA512: 125946dcd4025b04f11276e1df1e0f1b361fc2deed2e3919d680d58df97e6d82881b21c0c683142df6995226b4c12b2dde5415ad78474b8fcda115cc69fa0207
1ef22ce335 depends: patch around PlacementNew issue in capnp (fanquake)
Pull request description:
See #31772 and https://github.com/capnproto/capnproto/pull/2235.
Given there isn't agreement in #29796, pulled this out so it could be merged separately, and it's easier to run different test configurations externally.
Closes#31772.
ACKs for top commit:
ryanofsky:
Code review ACK 1ef22ce335. Confirmed patch is identical to one merged upstream. Only change since last review was tweaking the file paths and commit data in the patch.
TheCharlatan:
ACK 1ef22ce335
Tree-SHA512: 9c9ecf50c43cf74315f6659afab55aeafb436f70e83b328016ad574136dce46867220c6e1a85aefd8d3d3d027cd94cc807c79721a4983da9428de70f11224e52
Calls on the tx.calc_sha256 method can be confusing, as they return
the result (either txid or wtxid, depending on the with_witness
boolean parameter) as integer rather than as actual (w)txid. Use
.rehash() and .getwtxid() instead to improve readability and in some
cases avoid a conversion from string-txid to an integer.
In the functional test framework, we often treat hashes
as uint256 integers, which seems to be confusing and for no
good reason, as hashes are just sequences of bytes. This commit
gets rid of obvious internal instances of that where individual
functional tests are not affected. In the long-term, it might make
sense to store other hashes (mostly txids) as actual bytes to
avoid annoying conversions and improve code readability.
In #31118, the format of bitcoind's `--help` output changed slightly in
a way that breaks `gen-bitcoin-conf.sh`, modify the script to accomodate
the new format, by starting after the line that says "Options:" and
strip the `-help` option and its description from the output.
for consistency with the other CLI commands (-netinfo, -addrinfo, -generate).
This can be considered a bugfix because IsArgSet() returns whether an arg has
been set even if it has been negated. After this change, we no longer treat
-nogetinfo and -getinfo=0 the same as -getinfo and -getinfo=1, and instead as if
-getinfo was not specified.
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
18e83534ac wallet: Replace "non-0" with "non-zero" in translatable error message (Hennadii Stepanov)
Pull request description:
Transifex interprets the "-0" substring as a number in translatable strings. Since not all translations preserve "-0," this triggers a corresponding warning. While this warning could be disabled globally, it is more reasonable to adjust the original string instead.
ACKs for top commit:
davidgumberg:
ACK 18e83534ac
l0rinc:
ACK 18e83534ac
1440000bytes:
ACK 18e83534ac
BrandonOdiwuor:
Code Review ACK 18e83534ac
laanwj:
Code review ACK 18e83534ac
Tree-SHA512: 5c38cfc4b352dbbcc8de5fb907cf988a77a7ecded7a90fe0517bfb9e4cd5097bdeb1aa6edf5d9ca37de54d1d7939d5e49533ec93c403db90d9169ad7732e5124
cadbd4137d miner: have waitNext return after 20 min on testnet (Sjors Provoost)
d4020f502a Add waitNext() to BlockTemplate interface (Sjors Provoost)
Pull request description:
This PR introduces `waitNext()`. It waits for either the tip to update or for fees at the top of the mempool to rise sufficiently. It then returns a new template, with which the caller can rinse and repeat.
On testnet3 and testnet4 the difficulty drops after 20 minutes, so the second ensures that a new template is returned in that case.
Alternative approach to #31003, suggested in https://github.com/bitcoin/bitcoin/issues/31109#issuecomment-2451942362
ACKs for top commit:
ryanofsky:
Code review ACK cadbd4137d. Main change since last review is adding back a missing `m_interrupt` check in the waitNext loop. Also made various code cleanups in both commits.
ismaelsadeeq:
Code review ACK cadbd4137d
vasild:
ACK cadbd4137d
Tree-SHA512: c5a40053723c1c1674449ba1e4675718229a2022c8b0a4853b12a2c9180beb87536a1f99fde969a0ef099bca9ac69ca14ea4f399d277d2db7f556465ce47de95
GCC 14.2.1 will complain about a dangling reference after replacing Span
wiht std::span. This is a false-positive, because std::find does not
return a reference.
Remove the `&` to silence the warning. Also use ranges::find while
touching the line.
src/i2p.cpp:312:21: error: possibly dangling reference to a temporary [-Werror=dangling-reference]
312 | const auto& pos = std::find(kv.begin(), kv.end(), '=');
| ^~~
src/i2p.cpp:312:36: note: the temporary was destroyed at the end of the full expression ‘std::find<__gnu_cxx::__normal_iterator<const char*, span<const char> >, char>((& kv)->std::span<const char>::begin(), (& kv)->std::span<const char>::end(), '=')’
312 | const auto& pos = std::find(kv.begin(), kv.end(), '=');
| ~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: all warnings being treated as errors
Historically, the headers have been bumped some time after a file has
been touched. Do it now to avoid having to touch them again in the
future for that reason.
-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended 's;( 20[0-2][0-9])(-20[0-2][0-9])? The Bitcoin Core developers;\1-present The Bitcoin Core developers;g' $( git show --pretty="" --name-only HEAD~1 )
-END VERIFY SCRIPT-
This uses a macro, which can be a bit more brittle than an alias
template. However, class template argument deduction for alias templates
is only implemented in clang-19.
* The comment is wrong claiming that void* was returned when void was
returned in reality.
* The namespace is missing a name, leading to compile errors that are
suppressed with non-standard pragmas, and leading to compile errors in
future commits. Instead of using more non-standard suppressions, just
add the missing name.
* The SpanableYes/No types are missing begin/end iterators, which will
be needed when using std::span.
In theory this commit should only touch the span.h header, because
std::span can implicilty convert into Span in most places, if needed.
However, at least when using the clang compiler, there are some
false-positive lifetimebound warnings and some implicit conversions can
not be resolved.
Thus, this refactoring commit also changed the affected places to
replace Span with std::span.
Require that sqlite is available in order to compile the wallet. Removes
instances of USE_SQLITE since it is no longer possible to not have
sqlite available.
The NO_SQLITE option is dropped from depends.
Co-authored-by: Ava Chow <github@achow101.com>
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
This only existed to workaround a (iirc libtool related) windows issue
that only occured when compiling without hardening. We no-longer use
libtool, and I can no-longer create the failure.
e637dc2c01 refactor: Replace uint256 type with Wtxid in PackageMempoolAcceptResult struct (marcofleon)
a3baead7cb validation: use wtxid instead of txid in CheckEphemeralSpends (marcofleon)
Pull request description:
This PR addresses a small bug in [`AcceptMultipleTransactions`](45719390a1/src/validation.cpp (L1598)) where a txid was being inserted into a map that should only hold wtxids. `CheckEphemeralSpends` has an out parameter on failure that records that the child transaction did not spend the parent's dust. Instead of using the txid of this child, use its wtxid.
The second commit in this PR is a refactor of the `PackageMempoolAcceptResult` struct to use the `Wtxid` type instead of `uint256`. This helps to prevent errors like this in the future.
ACKs for top commit:
instagibbs:
ACK e637dc2c01
glozow:
ACK e637dc2c01, hooray for type safety
dergoegge:
Code review ACK e637dc2c01
Tree-SHA512: 17039efbb241b7741e2610be5a6d6f88f4c1cbe22d476931ec99e43f993d259a1a5e9334e1042651aff49edbdf7b9e1c1cd070a28dcba5724be6db842e4ad1e0
59c4930394 qa: Enable feature_init.py on Windows (Hodlinator)
Pull request description:
Windows has been skipped since feature_init.py was added in #23289. Possibly due to poorer support on older Python versions, or attempts to use `CTRL_C_EVENT` (which didn't work in my testing either) instead of `CTRL_BREAK_EVENT`.
ACKs for top commit:
maflcko:
lgtm ACK 59c4930394
BrandonOdiwuor:
Code Review ACK 59c4930394
hebasto:
ACK 59c4930394, I have reviewed the code and it looks OK.
Tree-SHA512: 4f3649b41bcba2e8d03b8dcb1a7a6882edafb2c456db4b0768fc86018e9e9ed7171cb3d3c99e74b4ef38a3fcf3ab5d2f1865bbd49d791f1ce0a246806634e1a7
568fcdddae scripted-diff: Adjust documentation per top-level target output location (Hennadii Stepanov)
026bb226e9 cmake: Set top-level target output locations (Hennadii Stepanov)
Pull request description:
This PR sets the target output locations to the `bin` and `lib` subdirectories within the build tree, creating a directory structure that mirrors that of the installed targets.
This approach is widely adopted by the large projects, such as [LLVM](e146c1867e/lldb/cmake/modules/LLDBStandalone.cmake (L128-L130)):
```cmake
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib${LLVM_LIBDIR_SUFFIX})
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib${LLVM_LIBDIR_SUFFIX})
```
The `libsecp256k1` project has also recently [adopted](https://github.com/bitcoin-core/secp256k1/pull/1553) this approach.
With this PR, all binaries are conveniently located. For example, run:
```
$ ./build/bin/fuzz
```
instead of:
```
$ ./build/src/test/fuzz/fuzz
```
On Windows, all required DLLs are now located in the same directory as the executables, allowing to run `bitcoin-chainstate.exe` (which loads `bitcoinkernel.dll`) without the need to copy DLLs or modify the `PATH` variable.
The idea was briefly discussed among the build team during the recent CoreDev meeting.
---
**Warning**: This PR changes build locations of newly built executables like `bitcoind` and `test_bitcoin` from `src/` to `bin/` without deleting previously built executables. A clean build is recommended to avoid accidentally running old binaries.
ACKs for top commit:
theStack:
Light re-ACK 568fcdddae
ryanofsky:
Code review ACK 568fcdddae. Only change since last review was rebasing. I'm ok with this PR in its current form if other developers are happy with it. I just personally think it is inappropriate to \*silently\* break an everyday developer workflow like `git pull; make bitcoind`. I wouldn't have a problem with this PR if it triggered an explicit error, or if the problem was limited to less common workflows like changing cmake options in an existing build.
TheCharlatan:
Re-ACK 568fcdddae
theuni:
ACK 568fcdddae
Tree-SHA512: 1aa5ecd3cd49bd82f1dcc96c8e171d2d19c58aec8dade4bc329df89311f9e50cbf6cf021d004c58a0e1016c375b0fa348ccd52761bcdd179c2d1e61c105e3b9f
fac1dd9dff test: Fix authproxy named args debug logging (MarcoFalke)
Pull request description:
In Python the meaning of `args or argsn` is that `argsn` is fully ignored when `args` is a list with at least one element. However, the RPC server accepts mixed positional and named args in the same RPC.
Fix the debug log by always printing both. Also, add a new `_json_dumps` helper to avoid bloated code.
Can be tested via `--tracerpc` on a call that uses named args mixed with positional args.
ACKs for top commit:
i-am-yuvi:
Tested ACK fac1dd9dff
rkrux:
tACK fac1dd9dff
musaHaruna:
Tested ACK [fac1dd9](fac1dd9dff)
ryanofsky:
Code review ACK fac1dd9dff. Thanks for logging fix. This change should have been included in #19762
Tree-SHA512: ff63fbc2564b2c7589e9294baacf4c7a79f10d593776813392510702ca726e3893a29db3ba261f3aee1789a59bb215d7cb10fc85ca1a02632631d3722ddcdfc5
The translations for the following languages, which appear to be the
result of a mistake or an act of vandalism, have been discarded:
- Czech (cs)
- Danish (da)
- Dutch (nl)
Changes to the Thai (th) translation have been discarded due to multiple
unsolicited pronunciation notes.
Use it for checking `-fsanitize`.
This change improves the user experience when the configuration step
fails due to a missing library. Now, there is no need to manually clean
the CMake cache after installing the required library.
c94195c077 doc: add note to windows build about stripping bin (fanquake)
Pull request description:
The Windows binaries are particularly big when they contain debug info, closing in on 500mb. Add a note to the Windows build instructions about using `--strip`.
I haven't tested this (the copying out to WSL). If we don't want to add this note, in favour of [user-presents or similar](https://github.com/bitcoin/bitcoin/issues/30593#issuecomment-2271304490), then we should just close#30593.
Fixes#30593.
ACKs for top commit:
hodlinator:
ACK c94195c077
hebasto:
ACK c94195c077.
Tree-SHA512: c55670486ef60c6bda720e65443e17747b840e220c5bf6d6c0b77590d95cd6c8f040bc0e67dfa8eb11451f4f2eac9faf25d74ea68251b881773836f4113e8595
d79dab0fa9 doc: warn against having qt6 installed on macOS (Sjors Provoost)
Pull request description:
Document #31009 in time for the v29 release.
ACKs for top commit:
achow101:
ACK d79dab0fa9
hebasto:
ACK d79dab0fa9.
Tree-SHA512: 4c6e557b6410c7fd766e1cdc356ae9f7410fbb4746732580e5bdf33ba43dca64e6f2fb66677d1e0c8fa71c19f212dc81ac73dc4277f2fd966bbd41c20d9291f8
611999e097 doc: link to benchcoin over bitcoinperf (fanquake)
Pull request description:
Seems like linking to https://github.com/bitcoin-dev-tools/benchcoin is now the best thing to do here. If not, we can just drop the other links.
ACKs for top commit:
l0rinc:
ACK 611999e097
laanwj:
ACK 611999e097
hebasto:
ACK 611999e097. I agree. I've had a great experience using it.
Tree-SHA512: 558060bec92099befaa047e9192e5172e6a0cdfc5530d1f8b4d64ac717ce999a993d39c5d108fa9df3e30b2fc089e31d720f344153381e7c53f0ed40938ae1e0
The Windows binaries are particularly big when they contain debug
info, closing in on 500mb. Add a note to the Windows build instructions
about using `--strip`.
f5d8b66a8c Squashed 'src/minisketch/' changes from eb37a9b8e7..d1e6bb8bbf (fanquake)
Pull request description:
Includes:
* https://github.com/bitcoin-core/minisketch/pull/92
ACKs for top commit:
hebasto:
ACK 4fde88bc46, I've updated the subtree locally and got zero diff with this PR.
Tree-SHA512: 0ddaa6b64ca14da244d455594bc122a059fd1d199d28a7a78f266e352811568bd0f30d3b1e5e5d859f92753d3979831c095e3f6078f0ba2c909b1566a0e74a0c
We shouldn't have | at the end of the last clause, as this will make it match
the empty string too (so effectively everything starting with Satoshi: matches).
While doing this, put the | at the beginning of every line of regex rather than
the end, to make it easier to update in the future without accidentally running
into this problem again.
d76647eb8f ci: use LLVM 20.1.0 for MSAN (fanquake)
Pull request description:
Now that the release is out, stop using rc1.
ACKs for top commit:
l0rinc:
utACK d76647eb8f
Sjors:
ACK d76647eb8f
Tree-SHA512: 2eefad118dd617c3e9554caee1549965fc4acc437dbef1252c1912faea57ccbfcb39f879911f27f4274931c1d8723b9c7f981aafa6958e70d4371f16c368c627
In Base58 fuzz the two roundtrips are merged now, the new `decode_input` switches between a completely random input and a valid encoded one, to make sure the decoding passes more often.
The `max_ret_len` can also exceed the original length now and is being validated more thoroughly.
Co-authored-by: maflcko <6399679+maflcko@users.noreply.github.com>
Co-authored-by: marcofleon <marleo23@proton.me>
They seem to cause timeouts:
> Issue 397734700: bitcoin-core:base58check_encode_decode: Timeout in base58check_encode_decode
The `encoded_string.empty()` check was corrected here to `decoded.empty()` to make sure the `(0, decoded.size() - 1)` range is always valid.
Co-authored-by: maflcko <6399679+maflcko@users.noreply.github.com>
Co-authored-by: marcofleon <marleo23@proton.me>
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
e181bda061 guix: Apply all codesignatures to Windows binaries (Ava Chow)
aafbd23fd9 guix: Apply codesignatures to all MacOS binaries (Ava Chow)
3656b828dc contrib: Sign all Windows binaries too (Ava Chow)
31d325464d contrib: Sign and notarize all MacOS binaries (Ava Chow)
710d5b5149 guix: Update signapple (Ava Chow)
e8b3c44da6 build: Include all Windows binaries for codesigning (Ava Chow)
dd4ec840ee build: Include all MacOS binaries for codesigning (Ava Chow)
4e5c9ceb9d guix: Rename Windows unsigned binaries to unsigned.zip (Ava Chow)
d9d49cd533 guix: Rename MacOS binaries to unsigned.tar.gz (Ava Chow)
c214e5268f guix: Rename unsigned.tar.gz to codesigning.tar.gz (Ava Chow)
Pull request description:
I have updated signapple to notarize MacOS app bundles without adding any additional dependencies. Further, it can also sign and apply detached signatures to standalone binaries.
As such, we can use signapple to perform the notarization and stapling steps so that MacOS will run the app bundle after it is installed. `detached-sig-create.sh` is updated to have a notarization step and to download the ticket which will be included in the detached signatures. The workflow is largely unchanged for the MacOS codesigners except for the additional requirement of having an App Store Connect API key and Team UUID, instructions for which can be found at https://github.com/achow101/signapple/blob/master/docs/notarization.md. For guix builders, the workflow is unchanged.
Additionally, the standalone binaries packaged in the MacOS `.tar.gz` and Windows `.zip` will now be codesigned. `detached-sig-create.sh` was updated to handle these, so the workflow for both MacOS and Windows codesigners remains unchanged. For guix builders, the workflow is also unchanged.
Because those binaries will how have codesigned and unsigned versions, the build command is modified to output `-unsigned.{tar.gz,zip}` archives containing the binaries. Since this happens to conflict with the tarball used for codesigning, the codesigning tarball was renamed to `-codesigning.tar.gz`. Both MacOS and Windows codesigners will need to adjust their workflows to account for the new name.
Fixes#15774 and #29749
ACKs for top commit:
Sjors:
Tested ACK e181bda061
davidgumberg:
Tested ACK e181bda061.
pinheadmz:
tested ACK e181bda061
Tree-SHA512: ce0e2bf38e1748cdaa0d13be6f61c3289cd09cfb7d071a68b0b13d2802b3936c9112eda6e4c7b29c535c0995d56b14871442589cdcea2e7707e35c1b278b9263
d1e6bb8bbf Merge bitcoin-core/minisketch#92: doc: rename from sipa to bitcoin-core
b7b7dd0ec8 doc: rename from sipa to bitcoin-core
git-subtree-dir: src/minisketch
git-subtree-split: d1e6bb8bbf8ef104b9dd002cab14a71b91061177
972b604dc4 doc: update location of minisketch repository (fanquake)
Pull request description:
This repository is now at https://github.com/bitcoin-core/minisketch.
ACKs for top commit:
hebasto:
ACK 972b604dc4.
theStack:
ACK 972b604dc4
Tree-SHA512: a164f97700e73b284429993f9639d1d4eab23cc09ded3104be392d5d259297c2906906a565ffa8848a495e8f35cbbe18ba4155fe1d16cda0406ac3c75f9d9a62
75486c8ed8 doc: update fuzz instructions when on macOS (Max Edwards)
Pull request description:
Fixes: #31049
Updates the instructions for fuzzing on macOS to use `lld` instead of `ld`.
Tested instructions on M1 Mac running 14.6.1
ACKs for top commit:
l0rinc:
ACK 75486c8ed8
brunoerg:
ACK 75486c8ed8
hebasto:
ACK 75486c8ed8, tested on macOS 15.3.1 (Apple M1) + Clang 19.1.7.
Tree-SHA512: 2c5645d78fce1644964dee55c8ca6a6549bfd4f4a9a5719bbe49264f7216f0267c27999e23402a47eecbc8502985d812b986bf6850a5d63d110bdb98769f23c2
18749efb07 scripted-diff: rename libmultiprocess repository (fanquake)
Pull request description:
For when we shift `libmultiprocess` into the `bitcoin-core` organisation.
ACKs for top commit:
Sjors:
tACK 18749efb07
hebasto:
ACK 18749efb07.
Tree-SHA512: df361e239da072dba2574e90231bbf8c4daf906786a838fe63761d38d5624510dbeeb6308567dc32321bd3bc76f1117606c8eb2c22e299aa164786ec342bd4b3
c718bffc36 build: don't use ccache with MSVC (fanquake)
Pull request description:
Set `WITH_CCACHE` to `OFF` for MSVC, so it doesn't show as `ON` in the configure summary.
Fixes#31771.
ACKs for top commit:
hebasto:
ACK c718bffc36, I have reviewed the code and it looks OK.
Tree-SHA512: cf20b36a496a446051d0586c55d05a9133dce26a958e6b39b510094c56014275b866806c88be40ffa918b4280970c082b981b4e52e4bfdb8cec1556f666898fc
This change builds libraries with -fsanitize=fuzzer-no-link instead of
-fsanitize=fuzzer when the cmake -DSANITIZERS=fuzzer option is specified. This
is necessary to make fuzzing and IPC cmake options compatible with each other
and avoid CI failures in #30975 which enables IPC in the fuzzer CI build:
https://cirrus-ci.com/task/5366255504326656?logs=ci#L2817https://cirrus-ci.com/task/5233064575500288?logs=ci#L2384
The failures can also be reproduced by checking out #31741 and building with
`cmake -B build -DBUILD_FOR_FUZZING=ON -DSANITIZERS=fuzzer -DENABLE_IPC=ON`
with this fix reverted.
The fix updates the cmake build so when -DSANITIZERS=fuzzer is specified, the
fuzz test binary is built with -fsanitize=fuzzer (so it can use libFuzzer's
main function), and libraries are built with -fsanitize=fuzzer-no-link (so they
can be linked into other executables with their own main functions).
Previously when -DSANITIZERS=fuzzer was specified, -fsanitize=fuzzer was
applied to ALL libraries and executables. This was inappropriate because it
made it impossible to build any executables other than the fuzz test executable
without triggering link errors:
- "multiple definition of `main'"
- "undefined reference to `LLVMFuzzerTestOneInput'"
if they depended on any libraries instrumented for fuzzing.
This was especially a problem when the ENABLE_IPC option was set because it
made building the mpgen code generator impossible so nothing else that depended
on generated sources, including the fuzz test binary, could be built either.
This commit was previously part of
https://github.com/bitcoin/bitcoin/pull/31741 and had some discussion there
starting in
https://github.com/bitcoin/bitcoin/pull/31741#pullrequestreview-2619682385
02fae33635 doc: add assumeutxo chainparams to release proc (willcl-ark)
Pull request description:
This should ideally be bumped every major (and perhaps even minor?) release to avoid falling too far behind, and therefore keeping this feature as useful as it can be.
Document in release-process.md to avoid forgetting to do this.
ACKs for top commit:
achow101:
ACK 02fae33635
glozow:
ACK 02fae33635
Tree-SHA512: 1c570b476a2c1369cde20965a762a4bce76fc27e7cf2704032132c9679ac1bc003d5dcc5b2bf39625f1b92b182254bec60743e52858ef89428df2b90ff4fb804
fff4f93dff doc: Bring reduce-memory.md up to date (laanwj)
Pull request description:
Update default number of RPC threads to 16 (#31215) and remove reference to very old version of bitcoin core.
Let me know if you notice other mismatches with current defaults.
ACKs for top commit:
achow101:
ACK fff4f93dff
brunoerg:
ACK fff4f93dff
TheCharlatan:
ACK fff4f93dff
vasild:
ACK fff4f93dff
Tree-SHA512: 14d91da1f86c8b460828a6e4ae9151ec430cbbaefa85d258c574b5e340cbf64244de981d5b3f37a0d97aafe872f3edb100596fc9e2b11c0df7874b1da8054a55
14f1674855 chainparams: add mainnet assumeutxo param at height 880_000 (Sjors Provoost)
Pull request description:
#31940 suggests adding a snapshot at every major release.
This snapshot should be suitable for v29. I picked the most recent multiple of 10K blocks.
You can either download this torrent:
```
magnet:?xt=urn:btih:559bd78170502971e15e97d7572e4c824f033492&dn=utxo-880000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969
```
Or generate the snapshot yourself:
```sh
bitcoin-cli -rpcclienttimeout=0 -named dumptxoutset utxo-880000.dat rollback=880000
```
The SHA-256 hash should be:
```
shasum -a 256 utxo-880000.dat
43b3b1ad6e1005ffc0ff49514d0ffcc3e3ce671cc8d02da7fa7bac5405f89de4
```
And then load it on a fresh node during IBD with:
```
bitcoin-cli -rpcclienttimeout=0 loadtxoutset utxo-880000.dat
```
Note that it's more performant to turn off networking while the snapshot is loading, see #29993:
```sh
bitcoin-cli setnetworkactive false
```
Once the snapshot is loaded:
```sh
bitcoin-cli setnetworkactive true
```
And enjoy a speedy ride to the tip.
ACKs for top commit:
achow101:
ACK 14f1674855
fjahr:
tACK 14f1674855
hodlinator:
ACK 14f1674855
rkrux:
Concept ACK 14f1674855
polespinasa:
ACK 14f1674855
Tree-SHA512: e7ed3e8ce3a247614545ecd3254a91814d7f87b3ca1be46df3b9a4c1e6353b46c82ab97d9fc9c5bed8938f28a6a61e6b70baa7c9649fe5da0f2f390b7932f15e
In the assumeutxo functional tests, the final test case with alternated UTxO data tests the error
raised when deserializing a snapshot that contains a coin with an amount not in range (<0 or
>MAX_MONEY).
The current malleation uses an undocumented byte string and offset which makes it hard to maintain.
In addition, the undocumented offset is set surprisingly high (39 bytes is well into the
serialization of the amount which starts at offset 36). Similarly the value is surprisingly small,
presumably one was adjusted for the other. But there is no comment explaining how they were chosen,
why not in a clearer manner and what they are supposed to represent.
Instead replace this seemingly magic value with a clear one, MAX_MONEY + 1, serialize the whole
value for the amount field at the correct offset, and document the whole thing for the next person
around.
Transifex interprets the "-0" substring as a number in translatable
strings. Since not all translations preserve "-0," this triggers a
corresponding warning. While this warning could be disabled globally, it
is more reasonable to adjust the original string instead.
This should be bumped every major release to avoid falling too far
behind, therefore making this feature as useful as it can be.
Document this in release-process.md to avoid forgetting to add a new
hardcoded height during release.
44041ae0ec init: Handle dropped UPnP support more gracefully (laanwj)
Pull request description:
Closesbitcoin-core/gui#843.
In that issue it was brought up that users likely don't care what kind of port forwarding is used, and that the setting is opportunistic anyway, so instead of showing an extensive warning, we can simply "upgrade" from UPNP to NAT-PMP+PCP.
- Change the logic for removed runtime setting `-upnp` to set `-natpmp` instead, and log a message.
- Also remove any lingering `upnp` from `settings.json` and replace it with `natpmp`, when it makes sense (this is important so that the UI shows the right values in the settings):
```json
{
"upnp": true
}
```
becomes
```json
{
"natpmp": true
}
```
and
```json
{
"upnp": false
}
```
becomes
```json
{
"natpmp": false
}
```
ACKs for top commit:
darosior:
tACK 44041ae0ec
davidgumberg:
lightly reviewed code, tested ACK 44041ae0ec
achow101:
ACK 44041ae0ec
ryanofsky:
Code review ACK 44041ae0ec. Code changes look good. Could potentially add test coverage for this, though I don't think it is too important.
hodlinator:
cr-ACK 44041ae0ec
Tree-SHA512: ca822f7160529e59973bab6a7cc31753ffa3caaa806887b5073b42c4ae5c918a5ea2cf93c666e5125ea70d10c6954709a535a264b04c2fd4cf916b3c59ab9964
ae92bd8e1b delete release note fragments for v29 (glozow)
Pull request description:
Delete release note fragments in preparation for 29.x branch-off.
Everything here has been copied over to the draft release notes at https://github.com/bitcoin-core/bitcoin-devwiki/wiki/29.0-Release-Notes-draft
ACKs for top commit:
laanwj:
ACK ae92bd8e1b
achow101:
ACK ae92bd8e1b
Tree-SHA512: adc4d34dfaf1cbbe4c7eddf4d220664e327c450de126040ab8565fc5f20696aaed2d58438aad76e4e343f0abf1b4f28fe1a21d09d957e606403c200858072682
ecf54a32ed cmake: Add support for builtin `codegen` target (Hennadii Stepanov)
a8c78a0574 cmake: Revamp handling of data files (Hennadii Stepanov)
Pull request description:
This PR leverages the approach from the https://github.com/chaincodelabs/libmultiprocess project and introduces a new functions `target_json_data_sources()` and `target_raw_data_sources()`, which minimize the amount of code required to assign to assign a `*.json` or `*.raw` data file to the `test_bitcoin`, `bench_bitcoin` or `unitester` targets.
As requested in https://github.com/bitcoin/bitcoin/pull/30901#issuecomment-2654622689, the `codegen` build target is now supported, if available:
```
$ cmake --version
cmake version 3.31.5
CMake suite maintained and supported by Kitware (kitware.com/cmake).
$ cmake -G "Ninja" -B build
$ cmake --build build --target codegen
```
ACKs for top commit:
fjahr:
re-ACK ecf54a32ed
Sjors:
re-tACK ecf54a32ed
theuni:
ACK ecf54a32ed
Tree-SHA512: bab92df6b81c47d9d97ba8db37470a6d7aa435d5578afe40df7154885eda55afc59f0bf20dc9db3b2fd88ceb9a0319b9678f9e9af01e7afd4851ec3a79f3f402
The only example present earlier was one that creates an OP_RETURN output. This
lack of examples has discouraged me earlier to use this RPC. Adding an example
that creates PSBT sending bitcoin to address, a scenario that is much more common.
This test returns the lowest weight solution. Other possibilities
either exceed allowed weight or result in a higher weight. Add an
assertion which shows that the lowest weight solution is found and
update the test description accordingly.
Remove the check that a result is returned since the expected result
assertion implies a result.
75d5d235a6 doc: Update translation generation instructions (pablomartin4btc)
Pull request description:
This is a follow-up of #31731.
Technically this change [fixes](https://github.com/bitcoin/bitcoin/pull/31731#discussion_r1928888001) the preset configuration execution failure as it needs multiprocess to be enabled, so we disable it using `-DWITH_MULTIPROCESS=OFF`.
This code will need to be updated by removing `-DWITH_MULTIPROCESS=OFF` in https://github.com/bitcoin/bitcoin/pull/31741.
ACKs for top commit:
ryanofsky:
Code review ACK 75d5d235a6. Looks good as a temporary fix and I think after #31741 we should be able to drop the extra argument.
Tree-SHA512: 793e095aa277ab0ea864b49542cc6fe458f9a16123e244f92a77de84fe141837dc7bcc06f543539ad23aaa207c0ea49a47f3afb4493f13fcefb3af87ad7c7c4f
This is a follow-up of #31731.
Technically this change fixes the preset configuration
execution failure as it needs multiprocess to be enabled,
so we disable it using -DWITH_MULTIPROCESS=OFF.
This code will need to be updated in PRs #31741 and #31802.
Closesbitcoin-core/gui#843.
In that issue it was brought up that users likely don't care what kind
of port forwarding is used, and the setting is opportunistic anyway, so
instead of showing an extensive warning, we can simply migrate from
UPNP to NAT-PMP+PCP. This prevents nodes dropping from the public
network.
- Change the logic for removed runtime setting `-upnp` to set `-natpmp`
instead, and only log a message.
- Also replace any lingering `upnp` in `settings.json` with `natpmp`.
Due to Base58, keys with whitespace at the beginning or
at the end are successfully parsed. This commit adds a
check into `ParsePubkeyInner` to verify whether if the
first or last char of the key is a space.
* Name the fuzz_corpora dir after its real name.
* Add missing cargo lock file.
* Use git instead of diff command to increase compatibility
* Use --help instead of --version to increase compatibility
* Use assert consistently for unexpected errors.
* Remove redundant Stdio::from.
* Fix typos.
The assumeutxo data for the fuzz target could change and invalidate the hash silently, preventing
the fuzz target from reaching some code paths.
Fix this by sanity checking the snapshot values during initialization.
e747ed989e contrib: fix read metadata related comment (rkrux)
d3095ac35a contrib: update `dumptxoutset` command in utxo_to_sqlite doc (rkrux)
Pull request description:
I noticed couple discrepancies in the documentation and comments of `utxo_to_sqlite` tool while using it, this PR fixes them. More details in the commit messages.
ACKs for top commit:
theStack:
lgtm ACK e747ed989e
Tree-SHA512: c5dab25d97c50a203ba008acfc5c93baf20cf24261a5a5127bdacc41eeeeae1c9d7b129710c2997e9fd1536ec000186e7f6395efc0188e1111b47868b7d9745a
d82dc10415 ci: Fix filtering out Qt generated files from `compile_commands.json` (Hennadii Stepanov)
Pull request description:
This PR:
1. Adjusts the regex for Qt-generated files to match the CMake build directory structure.
2. Moves the filtering command to run before `clang-tidy`, ensuring that Qt-generated files are not needlessly processed.
Fixes https://github.com/bitcoin/bitcoin/issues/31801.
ACKs for top commit:
maflcko:
ACK d82dc10415🚂
Tree-SHA512: 12ff40ffb134c56d848976e73aef7a8f895b856992b37b09ce27a1dcf6adef2cf148ba7e7d32b52ce1866c9bbeb165f9b6e6106c6c23b774d00d61bf379e13c9
c73b59d47f fuzz: implement targets for PCP and NAT-PMP port mapping requests (Antoine Poinsot)
1695c8ab5b fuzz: in FuzzedSock::GetSockName(), return a random-length name (Antoine Poinsot)
0d472c1953 fuzz: never return an uninitialized sockaddr in FuzzedSock::GetSockName (Antoine Poinsot)
39b7e2b590 fuzz: add steady clock mocking to FuzzedSock (Antoine Poinsot)
6fe1c35c05 pcp: make NAT-PMP error codes uint16_t (Antoine Poinsot)
01906ce912 pcp: make the ToString method const (Antoine Poinsot)
Pull request description:
Based on https://github.com/bitcoin/bitcoin/pull/31022, this introduces a fuzz target for `PCPRequestPortMap` and `NATPMPRequestPortMap`.
Like in #31022 we set `CreateSock` to return a `Sock` which mocks the responses from the server and uses a mocked steady clock for the `Wait`s. Except here we simply respond with fuzzer-provided data until the client stop sending requests. We also sometimes inject errors and connection failures based on fuzzer-provided data.
We reuse the existing `FuzzedSock`, so a preparatory commit is included that adds steady clock mocking to it. This may be useful for other harnesses as well.
ACKs for top commit:
laanwj:
re-ACK c73b59d47f
marcofleon:
ACK c73b59d47f
dergoegge:
utACK c73b59d47f
Tree-SHA512: 24cd4d958a0999946a0c3d164a242fc3f0a0b66770630252b881423ad0065d29fdaab765014d193b705d3eff397f201d51a88a3ca80c63fd3867745e6f21bb2b
The utxo snapshot metadata doesn't seem to contain any block height as per the
CPP code and no such value is read few lines down by the tool code as well.
Related CPP code: bitcoin/bitcoin/blob/28.x/src/node/utxo_snapshot.h#L60-L66
Running the `dumptxoutset` command without a `type` parameter leads
to the following error. Update the tool documentation to make it
easier to follow.
`Invalid snapshot type "" specified. Please specify "rollback" or "latest"`
The use of e.g. `std::underlying_type_t<T>` replaces the older `typename std::underlying_type<T>::type`.
The `_t` helper alias template (such as `std::underlying_type_t<T>`) introduced in C++14 offers a cleaner and more concise way to extract the type directly.
See https://en.cppreference.com/w/cpp/types/underlying_type for details.
-BEGIN VERIFY SCRIPT-
sed -i -E 's/(typename )?(std::[a-z_]+)(<[^<>]+>)::type\b/\2_t\3/g' $(git grep -l '::type' ./src ':(exclude)src/bench/nanobench.h' ':(exclude)src/leveldb' ':(exclude)src/minisketch' ':(exclude)src/span.h' ':(exclude)src/sync.h')
-END VERIFY SCRIPT-
7267ed0518 qt: Update `src/qt/locale/bitcoin_en.xlf` after string freeze (Hennadii Stepanov)
Pull request description:
This PR follows our [Release Process](864386a744/doc/release-process.md) and implements the ["Translation string freeze"](https://github.com/bitcoin/bitcoin/issues/31029) step.
Steps to reproduce the diff on Ubuntu:
```
$ cmake --preset dev-mode -DWITH_USDT=OFF -DWITH_MULTIPROCESS=OFF
$ cmake --build build_dev_mode --target translate
```
At the moment, the multiprocess-specific code does not introduce any new translatable strings. Therefore, there is no need to build depends with `MULTIPROCESS=1` to review this PR.
ACKs for top commit:
stickies-v:
ACK 7267ed0518 - I get the same results when building the `translate` target.
pablomartin4btc:
tACK 7267ed0518
Tree-SHA512: dc3641d3288c00cb7802714680508de517e56c615716e52181555634ad489fbed676229063995170edb8efeaa4e900ef2d3d5a0f1b8ce7cec143ef364c96e1c0
This change:
1. Collects build artifacts in dedicated locations.
2. Allows running bitcoin-chainstate.exe with bitcoinkernel.dll directly
from the build tree on Windows.
d871d77825 test: Remove non-portable IPv6 test (Hennadii Stepanov)
Pull request description:
On Illumos-based systems, such as OpenIndiana and OmniOS, the assumption that "the default zone ID of 0 can be omitted for the default scope" is incorrect. As a result, `getaddrinfo("fe80::1%0", ...)` returns the `EAI_NONAME` error instead of resolving to "fe80::1".
See: https://www.illumos.org/man/3SOCKET/getaddrinfo.
This PR removes the problematic code introduced in https://github.com/bitcoin/bitcoin/pull/19951.
ACKs for top commit:
fanquake:
ACK d871d77825
Tree-SHA512: 2ef5c22f826d16661deb1d6d005cbda64179e8b83be43d3d6ac51caff02187cf224355c9da144ff110a6ae2cb68f0338ea9b62af8e0f9f1014a518cf9dad6ab5
2c4b229c90 cmake: Introduce `FUZZ_LIBS` (Hennadii Stepanov)
ea929c0848 scripted-diff: Rename CMake helper module (Hennadii Stepanov)
8d238c1dfd cmake: Delete `check_cxx_source_links*` macros (Hennadii Stepanov)
71bf8294a9 cmake: Convert `check_cxx_source_compiles_with_flags` to a function (Hennadii Stepanov)
88ee6800c9 cmake: Delete `check_cxx_source_links_with_flags` macro (Hennadii Stepanov)
09e8fd25b1 build: Don't override CMake's default try_compile target (Hennadii Stepanov)
Pull request description:
This was requested in https://github.com/bitcoin/bitcoin/pull/31359#issuecomment-2515287092.
From https://github.com/bitcoin/bitcoin/pull/31359#issuecomment-2511246212:
> (Almost?) every CMake check internally uses the [`try_compile()`](https://cmake.org/cmake/help/latest/command/try_compile.html) command, whose behaviour, in turn, depends on the [`CMAKE_TRY_COMPILE_TARGET_TYPE`](https://cmake.org/cmake/help/latest/variable/CMAKE_TRY_COMPILE_TARGET_TYPE.html) variable:
>
> 1. The default value, `EXECUTABLE`, enables both compiler and linker checks.
>
> 2. The `STATIC_LIBRARY` value enables only compiler checks.
>
>
> To mimic Autotools' behaviour, we [disabled](d3f42fa08f/cmake/module/CheckSourceCompilesAndLinks.cmake (L9-L10)) linker checks by setting `CMAKE_TRY_COMPILE_TARGET_TYPE` to `STATIC_LIBRARY` globally (perhaps not the best design). This effectively separates the entire CMake script into regions where `CMAKE_TRY_COMPILE_TARGET_TYPE` is:
>
> * unset
>
> * set to `STATIC_LIBRARY`
>
> * set to `EXECUTABLE`
From https://github.com/bitcoin/bitcoin/pull/31359#issuecomment-2515287092:
> > This seems very fragile and unintuitive, and the fact that this could silently break at any point is not documented in any way. I don't think other bad design decisions should lead to us having to write even more boilerplate code to fix things that should "just work" (minus the upstream bugs).
>
> Agreed. I forgot that we set `CMAKE_TRY_COMPILE_TARGET_TYPE` globally. And even worse, it's buried in a module. If that upsets CMake internal tests, I think we should undo that.
This PR ensures that `CMAKE_TRY_COMPILE_TARGET_TYPE` is modified only within local scopes.
Additionally, the `FUZZ_LIBS` variable has been introduced to handle additional libraries required for linking, rather than link options, in certain build environment, such as OSS-Fuzz.
ACKs for top commit:
TheCharlatan:
Re-ACK 2c4b229c90
theuni:
utACK 2c4b229c90
Tree-SHA512: f72ffa8f50f216fc1a2f8027ba8ddfd4acd42b94ff6c1cb2138f2da51eb8f945660e97d3c247d7f3f7ec8dfebbccec3ab84347d6ae2e3f8a40f3d7aa8b14cde9
63a8791e15 contrib: fix BUILDDIR in gen-bitcoin-conf script and gen-manpages.py (jurraca)
Pull request description:
The `gen-bitcoin-conf.sh` and `gen-manpages.py` scripts assume a top level `src/` build dir, but in-tree builds are no longer allowed, nor recommended in the build steps. If a user builds `bitcoind` as recommended, these scripts fail. To fix it, we update the `BUILDDIR` env var and update the README accordingly.
Follows up on initial work and discussion in #31332 .
ACKs for top commit:
fjahr:
Code review ACK 63a8791e15
achow101:
ACK 63a8791e15
Tree-SHA512: cf4d5b0d2e8b1f5db759bec01e131d8a0c511a2fd183389d2a0488d5fe4a906db2579d944f408b5c966f619edc6b2534023c3521f1fa5f8edd0216d29f3e48db
ca6aa0b9be doc: loadwallet loads from relative walletdir (am-sq)
Pull request description:
## Why this change?
https://github.com/bitcoin/bitcoin/issues/30269 describes a need for documentation improvement with the `loadwallet` RPC. Namely, [some users have found](https://bitcoin.stackexchange.com/questions/123331/how-do-you-load-a-regtest-wallet) the usage description confusing when it comes to loading wallets that are not in the normal case of being in the default wallet directory.
The default wallet directory, depending on the machine OS, has the base directory defined here: 9c5cdf07f3/src/common/args.cpp (L699) which is then appended with `/wallets`. So for example, for MacOS, it would be `~/Library/Application Support/Bitcoin/wallets`.
## The changes implemented
1. Change the help text to indicate that the filename (or directory) passed in to `loadwallet` is relative to the base wallet directory
2. Adds additional examples to the help page showing how to fetch a wallet within a subdirectory of the base data directory for wallets, or from an absolute path
ACKs for top commit:
achow101:
ACK ca6aa0b9be
maflcko:
lgtm ACK ca6aa0b9be
rkrux:
ACK ca6aa0b9be
jonatack:
ACK ca6aa0b9be
Tree-SHA512: 123ae118c79ee1843ed65861e7a008658a53e47d4d14f2b7612561bba1b1dbdb6744f9aaac1587aac231c62d0c1804de848a6d732f1382788b437d9fe6474012
9e4a4b4832 cmake: Check `-Wno-*` compiler options for `leveldb` target (Hennadii Stepanov)
Pull request description:
Otherwise, https://cirrus-ci.com/task/4830737755537408:
```
At global scope:
cc1plus: note: unrecognized command-line option ‘-Wno-conditional-uninitialized’ may have been intended to silence earlier diagnostics
```
ACKs for top commit:
TheCharlatan:
ACK 9e4a4b4832
Tree-SHA512: 05553c80399180e01d45c3f02074ca0ce620011b29b03bef5433b87c9d88fd281fb6bf0203fc6fff590f3780c182a3fab8307002536b6350e03748420c346602
c4c5cf1748 cmake: Fix passing `APPEND_*FLAGS` to `secp256k1` subtree (Hennadii Stepanov)
eb540a2629 cmake: Remove `core_sanitizer_{cxx,linker}_flags` helper variables (Hennadii Stepanov)
Pull request description:
On the master branch @ 70e20ea024, the `APPEND_CPPFLAGS`, `APPEND_CFLAGS` and `APPEND_LDFLAGS` are not correctly applied when building C code in the `secp256k1` subtree, as intended.
This behaviour occurs due to two issues:
1. The command here: 70e20ea024/src/CMakeLists.txt (L77)
does not affect the code in `add_subdirectory(secp256k1)` above it.
2. `APPEND_LDFLAGS` is not passed to the subtree's build system at all.
This PR fixes both issues.
Additionally, the helper variables `core_sanitizer_cxx_flags` and `core_sanitizer_linker_flags` have been removed.
ACKs for top commit:
theuni:
utACK c4c5cf1748.
TheCharlatan:
ACK c4c5cf1748
Tree-SHA512: 707acfa623f0436e34e9e6ba8ce2979e0fde5e196e2242fd1cde4c50f433938549781193d8a06419a0866bbe6d69d76f8383d973afbd87d944407963b318c5c9
CMake distinguishes recommended methods for handling (1) linker options
and (2) libraries used during linking. Therefore, it is both reasonable
and consistent to introduce a dedicated variable for the latter,
particularly when a build environment, such as OSS-Fuzz, requires
linking against additional libraries.
CMake assumes the default value internally, so overriding this causes
problems. The minimal speedup of skipping the linker isn't worth the
complexity of setting it to static.
ff4ddd3d2e Revert "cmake: Ensure generated sources are up to date for `translate` target" (Hennadii Stepanov)
03b3166aac cmake: Exclude generated sources from translation (Hennadii Stepanov)
Pull request description:
This PR fixes an error encountered when building the `translate` target:
```
$ gmake -j $(nproc) -C depends MULTIPROCESS=1
$ cmake -G "Unix Makefiles" --preset dev-mode --toolchain depends/x86_64-pc-linux-gnu/toolchain.cmake -DWITH_USDT=OFF
$ cmake --build build_dev_mode -t translate
gmake[3]: *** No rule to make target 'src/test/ipc_test.capnp.c++', needed by 'src/qt/CMakeFiles/translate'. Stop.
gmake[2]: *** [CMakeFiles/Makefile2:1646: src/qt/CMakeFiles/translate.dir/all] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:1653: src/qt/CMakeFiles/translate.dir/rule] Error 2
gmake: *** [Makefile:699: translate] Error 2
```
The previous [attempt](864386a744) to address this issue worked only with Ninja generators and has been reverted.
Essentially, this PR modifies the `translate` target so that it ignores generated sources rather than attempting to update them.
At present, multiprocess-specific sources do not contain any translatable strings. Nonetheless, it is prudent to maintain a general approach.
ACKs for top commit:
TheCharlatan:
ACK ff4ddd3d2e
pablomartin4btc:
tACK ff4ddd3d2e
Tree-SHA512: 6471498a33b145e073f76bd007591b0449e5d520f141c3e3afeca02a09c160fd0f572ec7172dd84703cdc2a1175ad8f3c91e8b0bf705d671338d760786765f56
3e9b12b3e0 Revert "Merge bitcoin/bitcoin#31826: random: Check `GetRNDRRS` is supported in `InitHardwareRand` to avoid infinite loop" (Antoine Poinsot)
Pull request description:
PR #31826 was merged [despite the code not compiling](https://github.com/bitcoin/bitcoin/pull/31826#discussion_r1961315638).
#31902 was opened to fix the code but since this code is only targeting a not officially supported platform, we don't have a CI in place to compile and run tests on this platform, neither apparently reviewers do (nor does the author?), don't take more risk right before 29 and revert the original broken PR.
ACKs for top commit:
sipa:
ACK 3e9b12b3e0
achow101:
ACK 3e9b12b3e0
TheCharlatan:
ACK 3e9b12b3e0
eval-exec:
ACK 3e9b12b3e0
laanwj:
ACK 3e9b12b3e0
Tree-SHA512: e90f8ffb2eebe77e5b6f1c273fbeb29dd5bd6a76698d9a6048c33f3349033c56ea984dd9b64704698da01ecad4c47f98acac1a30312bf2499dbdd1931596953f
Signapple has been updated to sign individual binaries, and notarize app
bundles and binaries. When codesigning, all individual binaries will be
codesigned, and both the app bundle and individual binaries will be
notarized.
The wallet crashes if it processes the same block disconnection event twice in a row due
to an incompatible coinbase transaction state.
This happens because 'disconnectBlock' provides 'TxStateInactive' without the "abandoned"
flag for coinbase transactions to 'SyncTransaction', while 'AddToWallet()' internally
modifies it to retain the abandoned state.
The flow is as follows:
1) On the first disconnection, the transaction state transitions from "confirmed" to
"inactive," bypassing the state equality check since the provided state differs. Then,
'AddToWallet' internally updates the state to "inactive + abandoned"
2) On the second disconnection, as we provide only the "inactive" state
to 'SyncTransaction()', the state equality assertion fails and crashes the wallet.
09b150bb8a In `InitHardwareRand`, do trail test for `RNDRRS` by `VerifyRNDRRS` (Eval EXEC)
Pull request description:
This PR want to fix#31817 by added a maximum retry limit (`max_retries`) to the `GetRNDRRS` function to prevent it from entering an infinite loop if the hardware random number generator fails to return a valid random number. This change improves stability and ensures that the function terminates after a predefined number of retries.
ACKs for top commit:
achow101:
ACK 09b150bb8a
sipa:
utACK 09b150bb8a
Tree-SHA512: 5626b6b182a55d344a3aba11b782259ecc6bbec513771d50077874c5f70934750e68add8f63aa0bf69c6b7b313112940a85508af5447622c703cc5e92439ab4a
e4dd5a351b test: wallet, abandon coinbase txs and their descendants during startup (furszy)
474139aa9b wallet: abandon inactive coinbase tx and their descendants during startup (furszy)
Pull request description:
Since #26499, we mark coinbase transactions and their descendants as abandoned when a reorg arises through the "block disconnection" signal handler. However, this does not cover all scenarios; external wallets could contain coinbase transactions from blocks the node has not seen yet, or the user could have replaced the chain with an earlier or different version (one without the coinbase chain).
This affects balance calculation as well as mempool rebroadcast (descendants shouldn't be relayed).
Fix this by marking orphaned coinbase transactions and their descendants as abandoned during wallet startup.
ACKs for top commit:
achow101:
ACK e4dd5a351b
rkrux:
tACK e4dd5a351b
mzumsande:
Code Review ACK e4dd5a351b
Tree-SHA512: 461a43de7a6f5a580f2e6e3b56ec9bc92239cd45e850a2ff594ab5488dcd4a507f68fbbf550a33d7173b2add0de80de1e1b3841e1dfab0c95b284212d8ced08a
bb633c9407 tests: add functional test for miniscript decaying multisig (Michael Dietz)
Pull request description:
This is very closely based on [test/functional/wallet_multisig_descriptor_psbt.py](https://github.com/bitcoin/bitcoin/blob/master/test/functional/wallet_multisig_descriptor_psbt.py) both in code and concept. It should serve as some integration testing for Miniscript descriptors, and also documents a simple multisig that starts as 4-of-4 and decays to 3-of-4, 2-of-4, and finally 1-of-4 at block heights (I think in the real world aligning this to halvenings would be nice).
ACKs for top commit:
achow101:
ACK bb633c9407
rkrux:
reACK bb633c9407
hodlinator:
ACK bb633c9407
Tree-SHA512: 1f8e8e50258d45d8f2b882b5f86dcd390d86c543ff4801f397733017102e0854ac387960b6e296bb164603545615d224a4b400247cbbc07bf21b2f4b718ab2ff
Improves the documentation of help output for loadwallet
to clarify that filename is relative to the default
wallet directory. Adds examples that get a wallet from
sub-directories.
113a7a363f build: remove ENABLE_HARDENING cond from check-security (fanquake)
Pull request description:
This check is only used in release builds, where hardening should always be enabled. I can't think of a reason we'd want to silently skip these checks if hardening was inadvertently disabled.
ACKs for top commit:
maflcko:
lgtm ACK 113a7a363f
TheCharlatan:
ACK 113a7a363f
hebasto:
ACK 113a7a363f.
Tree-SHA512: 46217e8ab238e23374d758b12e5b6bdc22353d8bf272aa0d2260cdea023b5b80aba972dccaa0a4fb8da21c8c665991848f7fd79966d20ac2489d499c68d95639
`OBJECT` libraries have historically exhibited poor support for various
features, both in the past and now. For example, see one of the latest
issues:
- https://gitlab.kitware.com/cmake/cmake/-/issues/24058
Furthermore, CMake maintainers have acknowledged:
> In general, however, where there is a choice, static libraries will
> typically be the more convenient choice in CMake projects.
This change:
1. Converts the `bitcoin_clientversion` library from an `OBJECT` library
to a `STATIC` library.
2. Removes an obsolete workaround.
9cf746d663 cmake: add optional source files to crc32c directly (Daniel Pfeifer)
9c7823c5b5 cmake: add optional source files to bitcoin_crypto directly (Daniel Pfeifer)
Pull request description:
Avoid having many static libraries by adding the optional sources to the target `bitcoin_crypto` directly.
Set the necessary compile options at the source file level, rather than the target level.
fixes: #31697
ACKs for top commit:
s373nZ:
ACK 9cf746d663
hebasto:
re-ACK 9cf746d663.
TheCharlatan:
ACK 9cf746d663
Tree-SHA512: 04b468ccbd284d63fc83b382177bb8183b325369835c3b92e555e159955c73d71712a63a2e556f8da68a1232ac07d3845e11f1057c50666843db91db98fca979
fa3e409c9a contrib: Add deterministic-fuzz-coverage (MarcoFalke)
Pull request description:
The goal of this script is to detect and debug the remaining fuzz determinism and stability issues (https://github.com/bitcoin/bitcoin/issues/29018).
ACKs for top commit:
marcofleon:
Tested ACK fa3e409c9a
brunoerg:
tested ACK fa3e409c9a
Tree-SHA512: f336537d64188d6bc3c53880f4552a09cc498841c539cb7b4f14e622c9542531b970c1a6910981f7506e7bf659d2ce83471d58f5f51b0a411868f4c11eaf6b2a
This check is only used in release builds, where hardening should always
be enabled. I can't think of a reason we'd want to silently skip these
checks if hardening was inadvertently disabled.
4080b66cbe test: add test for utxo-to-sqlite conversion script (Sebastian Falbesoner)
ec99ed7380 contrib: add tool to convert compact-serialized UTXO set to SQLite database (Sebastian Falbesoner)
Pull request description:
## Problem description
There is demand from users to get the UTXO set in form of a SQLite database (#24628). Bitcoin Core currently only supports dumping the UTXO set in a binary _compact-serialized_ format, which was crafted specifically for AssumeUTXO snapshots (see PR #16899), with the primary goal of being as compact as possible. Previous PRs tried to extend the `dumptxoutset` RPC with new formats, either in human-readable form (e.g. #18689, #24202), or most recently, directly as SQLite database (#24952). Both are not optimal: due to the huge size of the ever-growing UTXO set with already more than 80 million entries on mainnet, human-readable formats are practically useless, and very likely one of the first steps would be to put them in some form of database anyway. Directly adding SQLite3 dumping support on the other hand introduces an additional dependency to the non-wallet part of bitcoind and the risk of increased maintenance burden (see e.g. https://github.com/bitcoin/bitcoin/pull/24952#issuecomment-1163551060, https://github.com/bitcoin/bitcoin/issues/24628#issuecomment-1108469715).
## Proposed solution
This PR follows the "external tooling" route by adding a simple Python script for achieving the same goal in a two-step process (first create compact-serialized UTXO set via `dumptxoutset`, then convert it to SQLite via the new script). Executive summary:
- single file, no extra dependencies (sqlite3 is included in Python's standard library [1])
- ~150 LOC, mostly deserialization/decompression routines ported from the Core codebase and (probably the most difficult part) a little elliptic curve / finite field math to decompress pubkeys (essentialy solving the secp256k1 curve equation y^2 = x^3 + 7 for y given x, respecting the proper polarity as indicated by the compression tag)
- creates a database with only one table `utxos` with the following schema:
```(txid TEXT, vout INT, value INT, coinbase INT, height INT, scriptpubkey TEXT)```
- the resulting file has roughly 2x the size of the compact-serialized UTXO set (this is mostly due to encoding txids and scriptpubkeys as hex-strings rather than bytes)
[1] note that there are some rare cases of operating systems like FreeBSD though, where the sqlite3 module has to installed explicitly (see #26819)
A functional test is also added that creates UTXO set entries with various output script types (standard and also non-standard, for e.g. large scripts) and verifies that the UTXO sets of both formats match by comparing corresponding MuHashes. One MuHash is supplied by the bitcoind instance via `gettxoutsetinfo muhash`, the other is calculated in the test by reading back the created SQLite database entries and hashing them with the test framework's `MuHash3072` module.
## Manual test instructions
I'd suggest to do manual tests also by comparing MuHashes. For that, I've written a go tool some time ago which would calculate the MuHash of a sqlite database in the created format (I've tried to do a similar tool in Python, but it's painfully slow).
```
$ [run bitcoind instance with -coinstatsindex]
$ ./src/bitcoin-cli dumptxoutset ~/utxos.dat
$ ./src/bitcoin-cli gettxoutsetinfo muhash <block height returned in previous call>
(outputs MuHash calculated from node)
$ ./contrib/utxo-tools/utxo_to_sqlite.py ~/utxos.dat ~/utxos.sqlite
$ git clone https://github.com/theStack/utxo_dump_tools
$ cd utxo_dump_tools/calc_utxo_hash
$ go run calc_utxo_hash.go ~/utxos.sqlite
(outputs MuHash calculated from the SQLite UTXO set)
=> verify that both MuHashes are equal
```
For a demonstration what can be done with the resulting database, see https://github.com/bitcoin/bitcoin/pull/24952#pullrequestreview-956290477 for some example queries. Thanks go to LarryRuane who gave me to the idea of rewriting this script in Python and adding it to `contrib`.
ACKs for top commit:
ajtowns:
ACK 4080b66cbe - light review
achow101:
ACK 4080b66cbe
romanz:
tACK 4080b66cbe on signet (using [calc_utxo_hash](8981aa3e85/calc_utxo_hash/calc_utxo_hash.go)):
tdb3:
ACK 4080b66cbe
Tree-SHA512: be8aa0369a28c8421a3ccdf1402e106563dd07c082269707311ca584d1c4c8c7b97d48c4fcd344696a36e7ab8cdb64a1d0ef9a192a15cff6d470baf21e46ee7b
a85e8c0e61 doc: Add some general documentation about negated options (Ryan Ofsky)
490c8fa178 doc: Add release notes summarizing negated option behavior changes. (Ryan Ofsky)
458ef0a11b refactor: Avoid using IsArgSet() on -connect list option (Ryan Ofsky)
752ab9c3c6 test: Add test to make sure -noconnect disables -dnsseed and -listen by default (Ryan Ofsky)
3c2920ec98 refactor: Avoid using IsArgSet() on -signetseednode and -signetchallenge list options (Ryan Ofsky)
d05668922a refactor: Avoid using IsArgSet() on -debug, -loglevel, and -vbparams list options (Ryan Ofsky)
3d1e8ca53a Normalize inconsistent -noexternalip behavior (Ryan Ofsky)
ecd590d4c1 Normalize inconsistent -noonlynet behavior (Ryan Ofsky)
5544a19f86 Fix nonsensical bitcoin-cli -norpcwallet behavior (Ryan Ofsky)
6e8e7f433f Fix nonsensical -noasmap behavior (Ryan Ofsky)
b6ab350806 Fix nonsensical -notest behavior (Ryan Ofsky)
6768389917 Fix nonsensical -norpcwhitelist behavior (Ryan Ofsky)
e03409c70f Fix nonsensical -norpcbind and -norpcallowip behavior (Ryan Ofsky)
40c4899bc2 Fix nonsensical -nobind and -nowhitebind behavior (Ryan Ofsky)
5453e66fd9 Fix nonsensical -noseednode behavior (Ryan Ofsky)
Pull request description:
The PR changes behavior of negated `-noseednode`, `-nobind`, `-nowhitebind`, `-norpcbind`, `-norpcallowip`, `-norpcwhitelist`, `-notest`, `-noasmap`, `-norpcwallet`, `-noonlynet`, and `-noexternalip` options, so negating these options just clears previously specified values doesn't have other side effects.
Negating options on the command line can be a useful way of resetting options that may have been set earlier in the command line or config file. But before this change, negating these options wouldn't fully reset them, and would have confusing and undocumented side effects (see commit descriptions for details). Now, negating these options just resets them and behaves the same as not specifying them.
Motivation for this PR is to fix confusing behaviors and also to remove incorrect usages of the `IsArgSet()` function. Using `IsArgSet()` tends to lead to negated option bugs in general, but it especially causes bugs when used with list settings returned by `GetArgs()`, because when these settings are negated, `IsArgSet()` will return true but `GetArgs()` will return an empty list. This PR eliminates all uses of `IsArgSet()` and `GetArgs()` together, and followup PR #17783 makes it an error to use `IsArgSet()` on list settings, since calling `IsArgSet()` is never actually necessary. Most of the changes here were originally made in #17783 and then moved here to be easier to review and avoid a dependency on #16545.
ACKs for top commit:
achow101:
ACK a85e8c0e61
danielabrozzoni:
re-ACK a85e8c0e61
hodlinator:
re-ACK a85e8c0e61
Tree-SHA512: dd4b19faac923aeaa647b1c241d929609ce8242b43e3b7bc32523cc48ec92a83ac0dc5aee79f1eba8794535e0314b96cb151fd04ac973671a1ebb9b52dd16697
f919d919eb fuzz: Add fuzzing for max_ret_len in DecodeBase58/DecodeBase58Check (Lőrinc)
635bc58f46 test: Fuzz Base32/Base58/Base64 roundtrip conversions (Lőrinc)
5dd3a0d8a8 test: Extend base58_encode_decode.json with edge cases (Lőrinc)
ae40cf1a8e test: Add padding tests for Base32/Base64 (Lőrinc)
Pull request description:
Added fuzzed roundtrips for `base[32|58|64]` encoding to make sure encoding/decoding are symmetric.
Note that if we omit the padding in `EncodeBase32` we won't be able to decode it with `DecodeBase32`.
Added dedicated padding tests to cover failure behavior
Also moved over the Base58 json test edge cases from https://github.com/bitcoin/bitcoin/pull/30035
ACKs for top commit:
hodlinator:
re-ACK f919d919eb
achow101:
ACK f919d919eb
Tree-SHA512: 6a6c63d0a659b70d42aad7a8f37ce6e372756e2c88c84e7be5c1ff1f2a7c58860ed7113acbe1a9658a7d19deb91f0abe2ec527ed660335845cd1e0a9380b4295
4818da809f wallet: fix rescanning inconsistency (Martin Zumsande)
Pull request description:
If the chain advances during a rescan, ScanForWalletTransactions would previously process the new blocks without adjusting `m_last_processed_block`, which would leave the wallet in an inconsistent state temporarily, and could lead to crashes in the GUI reported in #31474.
Fix this by not rescanning blocks beyond `m_last_processed_block` - for all blocks beyond that height, there will be pending BlockConnected notifications that will process them after the rescan is finished.
This means that if rescanning was triggered with `cs_wallet` permanently held (`AttachChain`), additional blocks that were connected during the rescan will only be processed with the pending `blockConnected` notifications after the lock is released.
If rescanning without a permanent `cs_wallet` lock (`RescanFromTime`), additional blocks that were connected during the rescan can be re-processed here because `m_last_processed_block` was already updated by `blockConnected`.
Fixes#31474
ACKs for top commit:
psgreco:
Not that it matters much, but UTACK 4818da809f
achow101:
ACK 4818da809f
furszy:
utACK 4818da809f
Tree-SHA512: 8e7dbc9e00019aef4f80a11776f3089cd671e0eadd3c548cc6267b5c722433f80339a9b2b338ff9b611863de75ed0a817a845e1668e729b71af70c9038b075af
c3fa043ae5 doc: build: Fix instructions for msvc gui builds (David Gumberg)
Pull request description:
If the instructions in `doc/build-windows-msvc.md` are followed as-is, and "Developer (PowerShell|Command Prompt) for VS 2022" is used to execute the suggested build commands, the root directory of vcpkg (e.g. in VS 2022 Community edition: `C:\Program Files\Microsoft Visual Studio\2022\Community\VC\vcpkg`), is too long, and when vcpkg attempts to build any of the QT packages, it will fail because of build steps that require path lengths greater than Windows' `MAX_PATH` 260 character limit. This can be avoided without needing to move the vcpkg root dir by setting [`--x-buildtrees-root`](https://learn.microsoft.com/en-us/vcpkg/commands/common-options#buildtrees-root) to a short path, like `C:\vcpkg`.
See e.g. https://github.com/microsoft/vcpkg/issues/28451, https://github.com/microsoft/vcpkg/issues/28083, https://github.com/microsoft/vcpkg/issues/24751.
ACKs for top commit:
achow101:
ACK c3fa043ae5
hebasto:
ACK c3fa043ae5.
TheCharlatan:
ACK c3fa043ae5
Tree-SHA512: 7de11d38b9125de63e72415f79d82f18818123a1b37f679f2229c4c82f5628dd7d1039dbc5dcdf1bc1c8c382e3e29de74a31d256e73872cbf1fa2687c52185ca
c9136ca906 validation: fix issue with an interrupted -reindex (Martin Zumsande)
a2675897e2 validation: Don't loop over all chainstates in LoadExternalBlock (Martin Zumsande)
Pull request description:
If a user interrupts a reindex while it is iterating over the block files, it will continue to reindex with the next node start (if the `-reindex` arg is dropped, otherwise it will start reindexing from scratch).
However, due to an early call to `ActivateBestChainState()` that only exists to connect the genesis block during
the original `-reindex`, it wil start connecting blocks immediately before having iterated through all block files.
Because later headers above the minchainwork threshold won't be loaded in this case, `-assumevalid` will not
be applied and the process is much slower due to script validation being done.
Fix this by only calling `ActivateBestChainState()` here if Genesis is not connected yet (equivalent to `ActiveHeight() == -1`).
Also simplify this spot by only doing this for the active chainstate instead of looping over all chainstates (first commit).
This issue was discussed in the thread below https://github.com/bitcoin/bitcoin/pull/31346#discussion_r1856824817, the impact on assumevalid was found by l0rinc.
The fix can be tested by manually aborting a `-reindex` e.g. on signet and observing in the debug log the order in which blockfiles are indexed / blocks are connected with this branch vs master.
ACKs for top commit:
achow101:
ACK c9136ca906
ryanofsky:
Code review ACK c9136ca906. Only comments changed since last review. Appreciate the new comments, I think they make a little clearer what things code is trying to do and what things are just side-effects.
TheCharlatan:
Re-ACK c9136ca906
Tree-SHA512: 6f34abc317ad7e605ccc0c2f4615e4ea6978223d207f80f768f39cc135a9ac0adf31681fadfa2aed45324a5d27a4f68c5e118ee7eec18ca5c40ef177caa9cc47
fa8e0956c2 rpc: Remove deprecated dummy alias for listtransactions::label (MarcoFalke)
Pull request description:
The RPC arg is not a dummy, but a label, so offering an undocumented alias is inconsistent with all other label interfaces and confusing at best, if not entirely unused.
Fix it by removing the deprecated alias.
This pull is a breaking change, but it should be limited, because it only affects someone using the deprecated named arg on this RPC. I can't imagine anyone doing this, because in all other places where label args are accepted, they are called `label`. If someone really didn't use `label` here as named arg, it would be trivial and less confusing for them to fix it up.
ACKs for top commit:
achow101:
ACK fa8e0956c2
rkrux:
tACK fa8e0956c2
ryanofsky:
Code review ACK fa8e0956c2
Tree-SHA512: 0d0f3f53237ff9fac8c065b7d0a4245f5ff86efa427dbeeca711765494b7315a9d72b44751d346c76422847daf3d7ff90dbccb5ba200b089fb96128bd95da9f0
f0e5e4cdbe test: Add test for rpcwhitelistdefault (naiyoma)
Pull request description:
This PR adds tests for `rpcwhitelistdefault.` The implementation is a continuation of this [PR](https://github.com/bitcoin/bitcoin/pull/17805).
Applied suggestions to include the tests in` rpc_whitelist.py` and to use a single node.
PR covers three test cases:
- rpcwhitelistdefault = 0, no permissions
- rpcwhitelistdefault = 1, no permissions
- rpcwhitelistdefault = 1, with user permissions
I didn't add tests for rpcwhitelistdefault = 0 with user permissions since that is already tested here: [rpc_whitelist.py#L77](https://github.com/bitcoin/bitcoin/blob/master/test/functional/rpc_whitelist.py#L77).
ACKs for top commit:
achow101:
ACK f0e5e4cdbe
ryanofsky:
Code review ACK f0e5e4cdbe. PR seems very clear and simple, moving 1 test and adding 3 new tests.
ismaelsadeeq:
Tested and Code review ACK f0e5e4cdbe
Tree-SHA512: c3652940d2f23746e769ebe834e43dee47b7af8f258cbb133e38663aa8a05a1a8d0194d3008c3a10b0c54d11b5b95420c9cad0aa761c0fc1b9559277443b0696
a759ea3e92 doc: Improve dependencies documentation (Nicola Leonardo Susca)
Pull request description:
Initially there was a distinction between the compiler dependencies and
other required dependencies (refs https://github.com/bitcoin/bitcoin/pull/23565) but the distinction was
removed (refs https://github.com/bitcoin/bitcoin/pull/24585) which is why having two distinct tables could lead
to confusion now.
ACKs for top commit:
achow101:
ACK a759ea3e92
hodlinator:
re-ACK a759ea3e92
rkrux:
ACK a759ea3e92
Tree-SHA512: 14aaf9356d65bd150c9993dcbc51b1b98c835a760b95e6d91e69460c97c18f1dd10eb52b9f1d70129e6aa5e361af3a55619fd35787ed4e1ec48909568adbb604
cd4bfaee10 net: reduce CAddress usage to CService or CNetAddr (Vasil Dimov)
Pull request description:
Using `CAddress` when only `CService` or `CNetAddr` is needed is excessive and confusing. Fix those occurrences to use the class they need:
* `CConnman::CalculateKeyedNetGroup()` needs `CNetAddr`, not `CAddress`, thus change its argument.
* Both callers of `CConnman::CreateNodeFromAcceptedSocket()` create a dummy `CAddress` from `CService`, so use `CService` instead.
* `GetBindAddress()` only needs to return `CService`.
* `CNode::addrBind` only needs to be `CService`.
ACKs for top commit:
Sjors:
ACK cd4bfaee10
achow101:
ACK cd4bfaee10
hodlinator:
ACK cd4bfaee10
laanwj:
Code review ACK cd4bfaee10
Tree-SHA512: 0b41c1519784eeeaf9926c6a4d24f583b90c3376741f37a3199a3808b0dd6d143d3f929bd7c06f87b031f4fc1c2bd7a6dfc7d715ec1f79bf36b862c00fd67085
81c174e318 cmake: Refer to the configure log instead of printing PIE test error (Hennadii Stepanov)
65a0920ca6 cmake: Add `CheckLinkerSupportsPIE` module (Hennadii Stepanov)
Pull request description:
This new module is a wrapper around CMake's `CheckPIESupported` module that fixes an upstream bug.
See: https://gitlab.kitware.com/cmake/cmake/-/issues/26463.
Fixes https://github.com/bitcoin/bitcoin/issues/30771.
ACKs for top commit:
theuni:
utACK 81c174e318.
vasild:
ACK 81c174e318
Tree-SHA512: 77d7022238551a4e69c59d1fe6b78975bb552cbbed5339459853d7ebf0086813036081f464fed230be330b3bd7d6cf8590b536b064028d2f786d6ae40f342f95
9b033bebb1 cmake: rename Kernel component to bitcoinkernel for consistency (Cory Fields)
2e0c92558e cmake: add and use install_binary_component (Cory Fields)
0264c5d86c cmake: use per-target components for bitcoin-qt and bitcoin-gui (Cory Fields)
fb0546b1c5 ci: don't try to install for a fuzz build (Cory Fields)
Pull request description:
This makes it possible to build/install only the desired binaries regardless of the configuration.
For consistency, the component names match the binary names. `Kernel` and `GUI` have been renamed.
Additionally it fixes#31762 by installing only the manpages for the configured targets (and includes them in the component installs for each).
Also fixes#31745.
Alternative to #31765 which is (imo) more correct/thorough.
Can be tested using (for ex):
```bash
$ cmake -B build
$ cmake --build build -t bitcoind -t bitcoin-cli
$ cmake --install build --component bitcoind
$ cmake --install build --component bitcoin-cli
```
ACKs for top commit:
hebasto:
ACK 9b033bebb1.
TheCharlatan:
Re-ACK 9b033bebb1
stickies-v:
re-ACK 9b033bebb1
Tree-SHA512: fd4818e76f190dbeafbf0c246b466f829771902c9d6d7111ed917093b811c8a5536a4a45e20708f73e7f581d6cb77c8e61cfa69e065788dcf0886792f553a355
dead908654 cmake: Improve compatibility with Python version managers (Hennadii Stepanov)
Pull request description:
This PR resolves the issue [highlighted](https://github.com/bitcoin/bitcoin/pull/31411#issuecomment-2516745547) in https://github.com/bitcoin/bitcoin/pull/31411:
> Here's another case where CMake just picks some other Python...
The fix leverages two [hints](https://cmake.org/cmake/help/latest/module/FindPython3.html#hints) for the CMake `FindPython3` module:
1. `Python3_FIND_FRAMEWORK` is set to `LAST`. This ensures that Unix-style package components are preferred over frameworks on macOS. As a side effect, the `FindPython3` module reports a shim or symlink (e.g., from `pyenv`) rather than the underlying framework's binary. The module's output aligns with the result of the `which` command.
2. `Python3_FIND_UNVERSIONED_NAMES` is set to `FIRST`. This supports scenarios where tools like `pyenv`—which use shims—have multiple Python versions installed.
Here are examples of output on my macOS 15.1.1 (Intel) with installed Homebrew's [Python 3.13.0](https://formulae.brew.sh/formula/python@3.13):
- without any Python version manager:
```
% which -a python3
/usr/local/bin/python3
/usr/bin/python3
% cmake -B build
<snip>
-- Found Python3: /usr/local/bin/python3 (found suitable version "3.13.0", minimum required is "3.10") found components: Interpreter
<snip>
```
- using `pyenv`:
```
% pyenv versions
system
* 3.10.14 (set by /Users/hebasto/dev/bitcoin/.python-version)
3.12.8
3.13.1
% which -a python3
/Users/hebasto/.pyenv/shims/python3
/usr/local/bin/python3
/usr/bin/python3
% cmake -B build
<snip>
-- Found Python3: /Users/hebasto/.pyenv/shims/python3 (found suitable version "3.10.14", minimum required is "3.10") found components: Interpreter
<snip>
```
Both variables, `Python3_FIND_FRAMEWORK` and `Python3_FIND_UNVERSIONED_NAMES`, can still be overridden by the user via the command line if needed.
ACKs for top commit:
theuni:
No opinion on the python selection changes themselves, but code-review ACK dead908654
willcl-ark:
ACK dead908654
Tree-SHA512: 69f8541223e5b6c35c892b4ba2a2dcfc24b41a10cf20accc75d3008b16434db8a9240c99c886c3a4566ba24269c5b0e0d856357891811f0a77b39f4afbee3634
70398ae05b mapport: make ProcessPCP void (Antoine Poinsot)
9e6cba2988 mapport: remove unnecessary 'g_mapport_enabled' (Antoine Poinsot)
8fb45fcda0 mapport: remove unnecessary 'g_mapport_current' variable (Antoine Poinsot)
1b223cb19b mapport: merge DispatchMapPort into StartMapPort (Antoine Poinsot)
9bd936fa34 mapport: drop unnecessary function (Antoine Poinsot)
2a6536ceda mapport: rename 'use_pcp' to 'enable' (Antoine Poinsot)
c4e82b854c mapport: make 'enabled' and 'current' bool (Antoine Poinsot)
Pull request description:
Followup to #31130, this does a couple cleanups to `src/mapport.*` to clarify the logic now that there is a single protocol option for port mapping.
ACKs for top commit:
laanwj:
Code review ACK 70398ae05b
TheCharlatan:
ACK 70398ae05b
Tree-SHA512: d9a3ab4fcd59a7cf4872415c40cc7ac3a98dfc5aa25e195d4df880bb588bac286c30c3471e9d9499de379a75f45dcd0a82019eba3cb9f342004ae1482d0ba075
2434aeab62 depends: avoid an unset CMAKE_OBJDUMP (fanquake)
Pull request description:
Similar to #31840, currently our Linux toolchain file contains:
```bash
set(CMAKE_AR "aarch64-linux-gnu-ar")
set(CMAKE_RANLIB "aarch64-linux-gnu-ranlib")
set(CMAKE_STRIP "aarch64-linux-gnu-strip")
set(CMAKE_OBJCOPY "aarch64-linux-gnu-objcopy")
set(CMAKE_OBJDUMP "")
```
`objdump` is currently only used for the macOS cross build, where it's `llvm-objdump`, but we should be consistent in producing a toolchain file that points to actual tools, rather than leaving variables unset.
ACKs for top commit:
hebasto:
ACK 2434aeab62.
theuni:
utACK 2434aeab62
Tree-SHA512: 65f6b7b9cae79e9c0784c108709139125e52d8f2818afbea5f719bc1b6dc338b870abbdfcb174ae541c0027a7ac07cb56012735b7a37b58b9a6e55a48c0257cf
0a02e7fdea test: deduplicates p2p_tx_download constants (Sergi Delgado Segura)
Pull request description:
Some of the networking constants defined in p2p_tx_download.py are more generally defined in p2p.py
ACKs for top commit:
i-am-yuvi:
re-ACK 0a02e7fdea
maflcko:
review ACK 0a02e7fdea🔖
danielabrozzoni:
re-ACK 0a02e7fdea
tdb3:
re ACK 0a02e7fdea
Tree-SHA512: 05fc114a32b6b42a7c57563a38f1a8921e0bb224c4b124ae9d395c3a1105ae6e9cdfc62f603f4f2dee55cef5f6a6ed400d328740ad84fbd3093c5e0f3fb2982a
bb0879ddab test: check `scanning` field from `getwalletinfo` (brunoerg)
Pull request description:
During a rescan, check that `getwalletinfo` returns properly information (the scanning field) about it.
ACKs for top commit:
maflcko:
lgtm ACK bb0879ddab
arejula27:
ACK [`bb0879d`](bb0879ddab)
achow101:
ACK bb0879ddab
BrandonOdiwuor:
Code Review ACK bb0879ddab
Prabhat1308:
re-ACK [`bb0879d`](bb0879ddab)
Tree-SHA512: 9bca1c1e813bf4f61a5621bdc0a5f5c2bcfb388ffe9dfacb821bf6954f6e0880140d72258dc93ab6b84efb54f55c682a17aebd42f6559d6cfac9998e6bc4e5b9
af76664b12 test: Test migration of a solvable script with no privkeys (Ava Chow)
17f01b0795 test: Test migration of taproot output scripts (Ava Chow)
1eb9a2a39f test: Test migration of miniscript in legacy wallets (Ava Chow)
e8c3efc7d8 wallet migration: Determine Solvables with CanProvide (Ava Chow)
fa1b7cd6e2 migration: Skip descriptors which do not parse (Ava Chow)
440ea1ab63 legacy spkm: use IsMine() to extract watched output scripts (Ava Chow)
b777e84cd7 legacy spkm: Move CanProvide to LegacyDataSPKM (Ava Chow)
b1ab927bbf tests: Test migration of additional P2WSH scripts (Ava Chow)
c39b3cfcd1 test: Extra verification that migratewallet migrates (Ava Chow)
Pull request description:
The legacy wallet `IsMine()` is essentially a black box that would tell us whether the wallet is watching an output script. In order to migrate legacy wallets to descriptor wallets, we need to be able to compute all of the output scripts that a legacy wallet would watch. The original approach for this was to understand `IsMine()` and write a function which would be its inverse. This was partially done in the original migration code, and attempted to be completed in #30328. However, further analysis of `IsMine()` has continued to reveal additional edge cases which make writing an inverse function increasingly difficult to verify correctness.
This PR instead changes migration to utilize `IsMine()` to produce the output scripts by first computing a superset of all of the output scripts that `IsMine()` would watch and testing each script against `IsMine()` to filter for the ones that actually are watched. The superset is constructed by computing all possible output scripts for the keys and scripts in the wallet - for keys, every key could be a P2PK, P2PKH, P2WPKH, and P2SH-P2WPKH; for scripts, every script could be an output script, the redeemScript of a P2SH, the witnessScript of a P2WSH, and the witnessScript of a P2SH-P2WSH.
Additionally, the legacy wallet can contain scripts that are redeemScripts and witnessScripts, while not watching for any output script utilizing that script. These are known as solvable scripts and are migrated to a separate "solvables" wallet. The previous approach to identifying these solvables was similar to identifying output scripts - finding known solvable conditions and computing the scripts. However, this also can miss scripts, so the solvables are now identified in a manner similar to the output scripts but using the function `CanProvide()`. Using the same superset as before, all output scripts which are `ISMINE_NO` are put through `CanProvide()` which will perform a dummy signing and then a key lookup to determine whether the legacy wallet could provide any solving data for the output script. The scripts that pass will have their descriptors inferred and the script included in the solvables wallet.
The main downside of this approach is that `IsMine()` and `CanProvide()` can no longer be deleted. They will need to be refactored to be migration only code instead in #28710.
Lastly, I've added 2 test cases for the edge cases that prompted this change of approach. In particular, miniscript witnessScripts and `rawtr()` output scripts are solvable and signable in a legacy wallet, although never `ISMINE_SPENDABLE`.
ACKs for top commit:
sipa:
Code review ACK af76664b12d8611b606a7e755a103a20542ee539; I did not review the tests in detail.
brunoerg:
code review ACK af76664b12
rkrux:
ACK af76664b12
Tree-SHA512: 7f58a90de6f38fe9801fb6c2a520627072c8d66358652ad0872ff59deb678a82664b99babcfd874288bebcb1487d099a77821f03ae063c2b4cbf2d316e77d141
This commit does not change behavior, it just changes code to handle -noconnect
values explicitly with IsArgNegated() instead of implicitly with IsArgSet(),
and adds comments to make it clear what behavior is intended when -noconnect is
specified.
Make sure -noconnect has same effect as -connect for disabling DNS seeding and
listening by default, and warning about -dnsseed being ignored with the -proxy
setting.
Initial implementation of https://github.com/bitcoin/bitcoin/pull/30529
accidentally broke this behavior, so having coverage may be useful to make sure
it does not break again.
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
This commit does not change behavior because negation of -signetseednode and
-signetchallenge parameters has been disallowed since these were introduced in
#18267, so calling IsArgSet() is equivalent to checking if GetArgs() returns a
non-empty list.
This commit does not change behavior, it just drops unnecessary IsArgSet()
calls for -debug, -loglevel, and -vbparams options. The calls are unnecessary
because GetArgs() already returns empty arrays if these arguments are not
specified.
Treat specifying -noexternalip the same as not specifying -externalip, instead
of causing it to soft-set the -discover default to false.
Before this change, was -noexternalip basically an undocumented synonym for
-nodiscover.
After this change, specifying -noexternalip just clears previously specifed
-externalip options, restoring default behavior as if they were not were
specified.
The previous -noexternalip behavior wasn't neccessarily bad, but it was
undocumented, redundant with the -nodiscover option, and inconsistent with
behavior of other list options.
Treat specifying -noonlynet the same as not specifying -onlynet, instead of
marking all networks unreachable.
Before this change, specifying -noonlynet cleared list of reachable networks
and did not allow connecting to any network. It was basically an undocumented
synonym for -noconnect.
After this change, specifying -nononlynet just clears previously specifed
-onlynet options and allows connecting to all networks, restoring default
behavior as if no -onlynet options were specified.
Before this change, there was no way to restore default behavior once an
-onlynet option was specified. So for example, if a config file specifed
onlynet settings, they couldn't be reset on the command line without disabling
the entire config file.
The previous -noonlynet behavior wasn't neccessarily bad, but it was
undocumented, redundant with the -noconnect option, inconsistent with behavior
of other list options, and inconsistent with being able to use the command line
to selectively override config options. It was also probably unintended,
arising from use of the IsArgSet() method and its interaction with negated
options.
Treat specifying -norpcwallet the same as not specifying any -rpcwallet option,
instead of treating it like -rpcwallet=0 with 0 as the wallet name.
This restores previous behavior before 743077544b
from https://github.com/bitcoin/bitcoin/pull/18594, which inadvertently changed
it.
Treat specifying -norpcwhitelist the same as not specifying -rpcwhitelist,
instead of behaving almost the same but flipping the default
-rpcwhitelistdefault value.
This is confusing because before this change if -norpcwhitelist was specified
it would block users from calling any RPC methods.
Treat specifying -norpcbind and -norpcallowip the same as not specifying
-rpcbind or -rpcallowip, instead of failing to bind to localhost and failing to
show warnings.
Also add code comment to clarify what intent of existing code is.
Treat specifying -noseednode the same as not specifying any -seednode value,
instead of enabling the seed node timeout and log messages, and waiting longer
to add other seeds.
Instead of a `gethostname` hack, use the official way of calling
`GetAdaptersAddresses` to get local network addresses on Windows.
As additional cleanup, move out `FromSockAddr` from MacOS and use it
everywhere appropriate.
Suggested by Ava Chow.
7afeaa2469 test: `-debug=0` and `-debug=none` behave similarly to `-nodebug` (Daniela Brozzoni)
a8fedb36a7 logging: Ensure -debug=0/none behaves consistently with -nodebug (Daniela Brozzoni)
d39d521d86 test: `-nodebug` clears previously set debug options (Daniela Brozzoni)
Pull request description:
Previously, -nodebug cleared all prior -debug configurations in the command line while allowing subsequent debug options to be applied.
However, -debug=0 and -debug=none completely disabled debugging, even for categories specified afterward.
This commit ensures consistency by making -debug=0 and -debug=none behave like -nodebug: they now clear previously set debug configurations but do not disable debugging for categories specified later.
See https://github.com/bitcoin/bitcoin/pull/30529#discussion_r1930956563
ACKs for top commit:
hodlinator:
re-ACK 7afeaa2469
ryanofsky:
Code review ACK 7afeaa2469. Nicely implemented change with test and release notes, and I like how the test is implemented as the first commit.
maflcko:
review ACK 7afeaa2469👡
Tree-SHA512: c69b17ff10da6c88636bd01918366dd408832e70f2d0a7b951e9619089e89c39282db70398ba2542d3aa69a2fe6b6a0a01638b3225aff79d234d84d3067f2caa
Measure ConnectBlock performance for
- blocks containing only schnorr sigs
- blocks containing both schnorr and ecdsa sigs
- blocks containing only ecdsa sigs
This will allow testing and measurement of performance improvement for features like
batch verification of schnorr signatures
Similar to #31840, currently our Linux toolchain file contains:
```bash
set(CMAKE_AR "aarch64-linux-gnu-ar")
set(CMAKE_RANLIB "aarch64-linux-gnu-ranlib")
set(CMAKE_STRIP "aarch64-linux-gnu-strip")
set(CMAKE_OBJCOPY "aarch64-linux-gnu-objcopy")
set(CMAKE_OBJDUMP "")
```
`objdump` is currently only used for the macOS cross build, where it's
`llvm-objdump`, but we should be consistent in producing a toolchain
file that points to actual tools, rather than leaving variables unset.
* `CConnman::CalculateKeyedNetGroup()` needs `CNetAddr`, not `CAddress`,
thus change its argument.
* Both callers of `CConnman::CreateNodeFromAcceptedSocket()` create a
dummy `CAddress` from `CService`, so use `CService` instead.
* `GetBindAddress()` only needs to return `CService`.
* `CNode::addrBind` only needs to be `CService`.
e3c0152769 cmake: Copy `cov_tool_wrapper.sh.in` to the build tree (Hennadii Stepanov)
Pull request description:
This PR ensures that `cov_tool_wrapper.sh.in` is available when invoking the `Coverage.cmake` script from any directory.
Here is an example of usage on Ubuntu 24.10 with the default GCC 14.2.0:
```
$ cmake -B build -DCMAKE_BUILD_TYPE=Coverage -DCMAKE_CXX_FLAGS="-fprofile-update=atomic" -DCMAKE_C_FLAGS="-fprofile-update=atomic"
$ cmake --build build -j $(nproc)
$ cd ..
$ cmake -DJOBS=$(nproc) -DLCOV_OPTS="--ignore-errors inconsistent,inconsistent" -P bitcoin/build/Coverage.cmake
```
Fixes https://github.com/bitcoin/bitcoin/issues/31638.
ACKs for top commit:
theuni:
utACK e3c0152769
Tree-SHA512: ccfc8e893567f199d9b05ea3916cac06fca89c5892cc7492d5251c331c35408222fd918ed08017515e2dfca10970ae3f633b3917bfb7037db539559e71d7f711
If the instructions are followed as-is, and "Developer
(PowerShell|Command Prompt) for VS 2022" is used to execute the
suggested build commands, the root directory of vcpkg (e.g. in VS 2022
Community edition: `C:\Program Files\Microsoft Visual
Studio\2022\Community\VC\vcpkg`), is too long, and when vcpkg attempts
to build any of the QT packages, it will fail because of build steps
that require path lengths greater than Windows' `MAX_PATH` 260 character
limit. This can be avoided without needing to move the vcpkg root dir by
setting `--x-buildtrees-root` to a short path, like `C:\vcpkg`.
3edaf0b428 depends: add missing Darwin objcopy (fanquake)
Pull request description:
Our CMake toolchain for a Darwin cross build currently contains:
```bash
set(CMAKE_AR "/usr/bin/llvm-ar")
set(CMAKE_RANLIB "/usr/bin/llvm-ranlib")
set(CMAKE_STRIP "/usr/bin/llvm-strip")
set(CMAKE_OBJCOPY "arm64-apple-darwin-objcopy")
set(CMAKE_OBJDUMP "/usr/bin/llvm-objdump")
```
`objcopy` isn't currently used for the Darwin build (only for Linux and splitting the debug symbols), but we shouldn't be producing a toolchain file that refers to nonexistent tools.
ACKs for top commit:
laanwj:
Code review ACK 3edaf0b428
theuni:
utACK 3edaf0b428
Tree-SHA512: b74deb9f3f053c37d03505e698419d4a14131131f12a042dab698a81f2ad76b71fd55c1d1afd5f5822cc50fdaad5acdab15a8b20626c56f705179add1165449f
ConsumeData() will always try to return a name as long as the requested size. It is more useful, and
closer to how `getsockname` would actually behave in reality, to return a random length name
instead.
This was hindering coverage in the PCP fuzz target as the addr len was set to the size of the
sockaddr_in struct and would exhaust all the provided data from the fuzzer.
Thanks to Marco Fleon for suggesting this.
Co-Authored-by: marcofleon <marleo23@proton.me>
f89f16846e depends: Fix compiling `libevent` package on NetBSD (Hennadii Stepanov)
Pull request description:
Libevent [introduced](https://github.com/libevent/libevent/pull/909) the [`typeof`](https://gcc.gnu.org/onlinedocs/gcc/Typeof.html) C language extension in the NetBSD-specific code, which was pulled into our depends in https://github.com/bitcoin/bitcoin/pull/21991.
However, GCC [states](https://gcc.gnu.org/onlinedocs/gcc/Alternate-Keywords.html):
> the various `-std` options disable certain keywords.
Due to our use of b042c4f053/depends/hosts/netbsd.mk (L1)
the `typeof` keyword is disabled, resulting in a compilation error:
```
$ gmake -C depends libevent CC=/usr/pkg/gcc14/bin/gcc CXX=/usr/pkg/gcc14/bin/g++
<snip>
[ 37%] Building C object CMakeFiles/event_core_static.dir/kqueue.c.o
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c: In function 'kq_setup_kevent':
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:56:27: error: implicit declaration of function 'typeof' [-Wimplicit-function-declaration]
56 | #define INT_TO_UDATA(x) ((typeof(((struct kevent *)0)->udata))(intptr_t)(x))
| ^~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:190:30: note: in expansion of macro 'INT_TO_UDATA'
190 | out->udata = INT_TO_UDATA(ADD_UDATA);
| ^~~~~~~~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:56:64: error: expected expression before 'intptr_t'
56 | #define INT_TO_UDATA(x) ((typeof(((struct kevent *)0)->udata))(intptr_t)(x))
| ^~~~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:190:30: note: in expansion of macro 'INT_TO_UDATA'
190 | out->udata = INT_TO_UDATA(ADD_UDATA);
| ^~~~~~~~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:56:27: error: called object is not a function or function pointer
56 | #define INT_TO_UDATA(x) ((typeof(((struct kevent *)0)->udata))(intptr_t)(x))
| ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/hebasto/dev/bitcoin/depends/work/build/x86_64-unknown-netbsd10.0/libevent/2.1.12-stable-ca6b96ec97c/kqueue.c:190:30: note: in expansion of macro 'INT_TO_UDATA'
190 | out->udata = INT_TO_UDATA(ADD_UDATA);
| ^~~~~~~~~~~~
gmake[3]: *** [CMakeFiles/event_core_static.dir/build.make:328: CMakeFiles/event_core_static.dir/kqueue.c.o] Error 1
<snip>
```
This PR resolves this issue by following GCC's [recommendation](https://gcc.gnu.org/onlinedocs/gcc/Typeof.html):
> write `__typeof__` instead of `typeof`.
ACKs for top commit:
fanquake:
ACK f89f16846e
Tree-SHA512: c0d2e535408db120535781f8518c616b0f5a39b1c6babb2a74e8e0565348aaf00b0f5a93cac0af7cf6d6bf028d5d58763fe71b3969ed9c7059fa7c3dca9d084c
The fuzz provider's `ConsumeData` may return less data than necessary
to fill the sockaddr struct and still return success. Fix this to avoid
the caller using uninitialized memory.
12fa9511b5 build: simplify dependency graph (Cory Fields)
c4e498300c build: avoid unnecessary dependencies on generated headers (Cory Fields)
Pull request description:
These changes speed up my build (default config/options/targets) by roughly 10%. I suspect the difference may be more significant in other build configs.
Before:
> $ time cmake --build build -j24
> real3m26.932s
After:
> $ time cmake --build build -j24
> real3m7.556s
Generally they allow for jobservers (either `make -jX` or `ninja`) to be better utilized. This can be verified using `top` while building and looking at the number of compiles running at any given time before/after these changes. Before, it's easy to observe periods of stalling when only one or two compiles are happening. After these changes, the compiler process count should mostly match the number of jobs given (`-jX`) until it falls off at the end.
---
The first commit sets [DEPENDS_EXPLICIT_ONLY](https://cmake.org/cmake/help/latest/command/add_custom_command.html#command:add_custom_command) for commands which generate our test header files. Without this option, `test_bitcoin`'s generated headers won't be built until all of its other dependencies have been built. This introduces a significant stall in the build, though currently only Ninja benefits from this being set, and only CMake >= 3.27 understands it.
Example from a generated `build.ninja`:
Before:
> \# Custom command for src/test/data/base58_encode_decode.json.h
>
> build src/test/data/base58_encode_decode.json.h | ${cmake_ninja_workdir}src/test/data/base58_encode_decode.json.h: CUSTOM_COMMAND /home/cory/dev/bitcoin/src/test/data/base58_encode_decode.json /home/cory/dev/bitcoin/cmake/script/GenerateHeaderFromJson.cmake || libcrc32c.a libcrc32c_sse42.a libleveldb.a libminisketch.a minisketch_clmul src/bitcoin_clientversion src/crypto/libbitcoin_crypto.a src/crypto/libbitcoin_crypto_avx2.a src/crypto/libbitcoin_crypto_sse41.a src/crypto/libbitcoin_crypto_x86_shani.a src/generate_build_info src/libbitcoin_cli.a src/libbitcoin_common.a src/libbitcoin_consensus.a src/libbitcoin_node.a src/secp256k1/src/libsecp256k1.a src/secp256k1/src/secp256k1_precomputed src/test/util/libtest_util.a src/univalue/libunivalue.a src/util/libbitcoin_util.a src/wallet/libbitcoin_wallet.a src/zmq/libbitcoin_zmq.a
After:
> \# Custom command for src/test/data/base58_encode_decode.json.h
>
> build src/test/data/base58_encode_decode.json.h | ${cmake_ninja_workdir}src/test/data/base58_encode_decode.json.h: CUSTOM_COMMAND /home/cory/dev/bitcoin/src/test/data/base58_encode_decode.json /home/cory/dev/bitcoin/cmake/script/GenerateHeaderFromJson.cmake
---
The second commit is more significant. It sets [CMAKE_OPTIMIZE_DEPENDENCIES](https://cmake.org/cmake/help/latest/prop_tgt/OPTIMIZE_DEPENDENCIES.html) globally, which allows the objects of static libs to be built in parallel when one lib depends on the other. This can be set as a per-lib property, ~but I don't see any need for that as we don't currently have any edge-cases where this wouldn't be ok. If those should arise, we could always disable on a per-lib basis~.
Edit: turns out this triggers an [upstream bug](https://gitlab.kitware.com/cmake/cmake/-/issues/24058), which I guess can be considered an edge-case until fixed in CMake. I've added 2 per-lib opt-outs as a result.
Example:
Before:
> \# Link the static library src/libbitcoin_cli.a
>
> build src/libbitcoin_cli.a: CXX_STATIC_LIBRARY_LINKER__bitcoin_cli_RelWithDebInfo src/CMakeFiles/bitcoin_cli.dir/compat/stdin.cpp.o src/CMakeFiles/bitcoin_cli.dir/rpc/client.cpp.o || src/univalue/libunivalue.a
After:
> \# Link the static library src/libbitcoin_cli.a
>
> build src/libbitcoin_cli.a: CXX_STATIC_LIBRARY_LINKER__bitcoin_cli_RelWithDebInfo src/CMakeFiles/bitcoin_cli.dir/compat/stdin.cpp.o src/CMakeFiles/bitcoin_cli.dir/rpc/client.cpp.o
>
ACKs for top commit:
l0rinc:
utACK 12fa9511b5
hebasto:
ACK 12fa9511b5.
Tree-SHA512: f85f507e70cdc06acd07542161d9f9b8edf9ba866f08c8ef17aaaed770fa11530a27521c4413456d863463a6e77d4d6983fa623a64e17bbd602c2bc70aacc112
fa952acdb6 ci: Skip read-write of default env vars (MarcoFalke)
Pull request description:
If they remain unset, they use the default anyway. Except for `USER`, but this seems unused anyway.
Can be checked via:
```
sh-5.2# touch /tmp/empty_env
sh-5.2# podman run --rm --env-file /tmp/empty_env 'ubuntu:24.04' env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
container=podman
HOME=/root
HOSTNAME=19ece5c9e052
ACKs for top commit:
0xB10C:
ACK fa952acdb6
Prabhat1308:
utACK [fa952ac](fa952acdb6)
Tree-SHA512: fe0c173b23cfda3025306303a44ffe32ecc57c2e0e1a2376594696f9887ed22f5105da84e898e790041bf15a4aa42a365fba016710ad269d439dda691977be90
fa3a4eafa1 test: Remove stale gettime test (MarcoFalke)
Pull request description:
The `gettime` test is stale:
* It was added to sanity check the `time` implementation in the mingw toolchain to catch a 32-bit vs 64-bit mismatch in commit eaafa23cbd. However, since commit 0000a63689, `std::chrono::system_clock` is used.
* Even though `system_clock` may also return incorrect values, such an error should affect *all* `GetTime<>` calls (not only the second-precision ones). (I expect such an error to lead to a signed integer overflow in the normal nanosecond precision, so it should be caught by ubsan or by the `assert(ret > 0s)`. If not, the error should be apparent on startup in the debug log.)
So remove it for now. An alternative would be to extend the test to cover `time` again, and adjust the comment to say that the test should be fixed along with the block header timestamp. Since that timestamp can't grow beyond 2106 anyway, see the `_test_y2106` functional test.
ACKs for top commit:
l0rinc:
ACK fa3a4eafa1
laanwj:
ACK fa3a4eafa1
Tree-SHA512: fd485e74962b659ee23ba2952d284fa9d6cfb9d9844a5e70013c8ead495ed77f5b784d5ca3ba0b30c492a5d27b2e81f9e1e0dbc530af7da1494789ac5e055b99
b28917be36 depends: Make default `host` and `build` comparable (Hennadii Stepanov)
Pull request description:
To detect cross-compiling, the host and build platforms are compared. The `build` variable is always an output of `config.sub`, but the `host` is not. This can lead to false results. For example, on OpenBSD:
- host=amd64-unknown-openbsd7.5
- build=x86_64-unknown-openbsd7.5
This PR sets the default value of the `host` variable to the value of `build`, ensuring cross-compiling won't be triggered when the `HOST` variable is not set.
This PR fixes needless triggering of cross-compiling for CMake-built packages in depends on OpenBSD due to this code:eb85cacd29/depends/funcs.mk (L193-L197)
No changes in Guix build.
ACKs for top commit:
laanwj:
Concept and code review ACK b28917be36
theuni:
utACK b28917be36.
Tree-SHA512: 8c5835cb8b739355b71f7cb161b350ad8b038a00e6b1def36354ba228cea3dcb9883df3c9a8e79d7d0143241f6f054129fe90772b1b2579702db51237f9d66ff
56a9b847bb build: set build type and per-build-type flags as early as possible (Cory Fields)
f605f7a9c2 build: refactor: set debug definitions in main CMakeLists (Cory Fields)
Pull request description:
This ensures that most compiler tests are not run with the wrong build type's flags. The initial c++ checks are an exception to that because many internal CMake variables are unset until a language is selected, so it's problematic to change our build type before that.
The difference can be seen in `build/CMakeFiles/CMakeConfigureLog.yaml`. Before, `Debug` was used for many of the earlly checks. After this PR, it's only the first 2 checks.
ACKs for top commit:
hebasto:
ACK 56a9b847bb.
Tree-SHA512: 87947352d6d4fd08554515822cb13634ed3be33fcda2af817c22ef943b1d0856ceb39311ddc01b8221397528fdc09f630dc7e74fc92f5a4a073f09c4ae493596
76c090145e guix: remove test-security/symbol-check scripts (fanquake)
Pull request description:
These scripts are becoming more of nuisance, than a value-add; particularly since we've been building releases using Guix. Adding new (release bin) tests can be harder, because it requires constructing a failing test, which is becoming less easy, e.g trying to disable a feature or protection that has been built into the compiler/toolchain by default.
In the pre-Guix days, these were valuable to sanity-check the environment, because we were pulling that pre-built from Ubuntu, with little control. At this point, it's less clear what these scripts are (sanity) checking.
Note that these also weren't completely ported to CMake (#31698), see also #31715 which contains other fixes that would be needed for these test-tests, to accomodate future changes.
ACKs for top commit:
hebasto:
ACK 76c090145e.
theuni:
utACK 76c090145e
Tree-SHA512: 99b5e7c0645c6966a45b17f411b5bee61df23c64d8258cce0ad9cdea4c3af4d4db32ca5fd80d0df2a3a30ba873eb772cc0d5901c345ff7f0eea13fcb971443b4
The tarballs used for codesigning are more than merely unsigned, they
also contain scripts and other data for codesigning. Rename them to
codesigning.tar.gz to distinguish from tarballs containing actually just
the unsigned binaries.
Currently the manpages are installed, but that is a bug. An upcoming commit
will avoid installing manpages for targets that aren't configured, which
removes the "install" target for fuzz builds.
0f716f2889 qa: cover PROTOCOL_ERROR variant in PCP unit tests (Antoine Poinsot)
fc700bb47f test: Add tests for PCP and NATPMP implementations (laanwj)
caf9521033 net: Use mockable steady clock in PCP implementation (laanwj)
03648321ec util: Add mockable steady_clock (laanwj)
ab1d3ece02 net: Add optional length checking to CService::SetSockAddr (laanwj)
Pull request description:
Add a NodeSteadyClock, a steady_clock that can be mocked with millisecond precision. Use this in the PCP implementation.
Then add a mock for a simple scriptable UDP server,, which is used to test various code paths (including successful mappings, timeouts and errors) in the PCP and NATPMP implementations.
Includes "net: Add optional length checking to CService::SetSockAddr" from #31014 as a prerequisite.
ACKs for top commit:
darosior:
re-ACK 0f716f2889
i-am-yuvi:
Concept ACK 0f716f2889
achow101:
ACK 0f716f2889
Tree-SHA512: 6f91b24e6fe46a3fded7a13972efd77c98e6ef235f8898e4ae44068c5df32d1cdabb22cb66c351b338dc98cb2073b624e43607a28107f4999302bfbe7a138229
2ffea09820 build: disable bitcoin-node if daemon is not built (Sjors Provoost)
Pull request description:
When building for fuzzing with multiprocess enabled, we were still trying to build `bitcoin-node`. This PR fixes that, by applying a similar check as for `bitcoin-gui`.
Before:
```
cmake -B build -DBUILD_FOR_FUZZING=ON -DWITH_MULTIPROCESS=ON
...
Configure summary
=================
Executables:
bitcoind ............................ OFF
bitcoin-node (multiprocess) ......... ON
bitcoin-qt (GUI) .................... OFF
bitcoin-gui (GUI, multiprocess) ..... OFF
...
cmake --build build
...
[ 84%] Built target bitcoin-node
```
After:
```
bitcoin-node (multiprocess) ......... OFF
```
And no `bitcoin-node` target gets built (not to be confused with `bitcoin_node`).
ACKs for top commit:
hebasto:
ACK 2ffea09820.
ryanofsky:
Code review ACK 2ffea09820
laanwj:
Code review ACK 2ffea09820
Tree-SHA512: bdb0b62049f77929d5c084bf98a076e9933de91eb30853ed89edd23cc81b3d4aec4cd57c9a9e21cf1d6930885f8c408dda830db6884b4e326c7fb348f1fbab4c
Previously, -nodebug cleared all prior -debug configurations in the
command line while allowing subsequent debug options to be applied.
However, -debug=0 and -debug=none completely disabled debugging,
even for categories specified afterward.
This commit ensures consistency by making -debug=0 and -debug=none
behave like -nodebug: they now clear previously set debug configurations
but do not disable debugging for categories specified later.
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
Our CMake toolchain for a Darwin cross build currently contains:
```bash
set(CMAKE_AR "/usr/bin/llvm-ar")
set(CMAKE_RANLIB "/usr/bin/llvm-ranlib")
set(CMAKE_STRIP "/usr/bin/llvm-strip")
set(CMAKE_OBJCOPY "arm64-apple-darwin-objcopy")
set(CMAKE_OBJDUMP "/usr/bin/llvm-objdump")
```
`objcopy` isn't currently used for the Darwin build (only for Linux and
splitting the debug symbols), but we shouldn't be producing a toolchain
file that refers to nonexistent tools.
8fe552fe6e test: add missing sync to p2p_tx_download.py (Martin Zumsande)
Pull request description:
If the node hasn't processed the inv from the outbound peer before the mocktime bump, the peer won't be preferred after the other inv timeouts, failing the test . Therefore, add a sync, just like there is one after the `send_message` calls in the previous lines.
Fixes#31833
ACKs for top commit:
maflcko:
lgtm ACK 8fe552fe6e
instagibbs:
ACK 8fe552fe6e
Tree-SHA512: fda935d8a4081b5ecae96f5a73c04f4bb91feaeb09b5c159ffd45cf16668c4345ff268c57f383ba7c7ff544ee07b21f97aa28f257ade809c18b9310837795e7a
9b7023d31a Fuzz HRP of bech32 as well (Lőrinc)
c1a5d5c100 Split out bech32 separator char to header (Lőrinc)
Pull request description:
Instead of the static "bc" human-readable part, it's now randomly generated based on https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki and the extra restrictions in the code:
> The human-readable part, which is intended to convey the type of data, or anything else that is relevant to the reader. This part MUST contain 1 to 83 US-ASCII characters, with each character having a value in the range [33-126]. HRP validity may be further restricted by specific applications.
Since `bech32::Encode` rejects uppercase letters, we're actually generating values in the `[33-126] - ['A'-'Z']` range.
Split out of https://github.com/bitcoin/bitcoin/pull/30596/files#r1706957219
ACKs for top commit:
sipa:
ACK 9b7023d31a
achow101:
ACK 9b7023d31a
marcofleon:
Code review ACK 9b7023d31a. The separation into two targets and the new `GenerateRandomHRP` seem fine to me.
brunoerg:
code review ACK 9b7023d31a
Tree-SHA512: 22a261b8e7b5516e98f4e7990811954454595438a49a10191ed4ca42b5c71c5054fcc73f2d94e23b498ea833c7f1d5adb225f537ef1a24d15b428259450cdf98
b2e9fdc00f test: expect that files may disappear from /proc/PID/fd/ (Vasil Dimov)
Pull request description:
`get_socket_inodes()` calls `os.listdir()` and then iterates on the results using `os.readlink()`. However a file may disappear from the directory after `os.listdir()` and before `os.readlink()` resulting in a `FileNotFoundError` exception.
It is expected that this may happen for `bitcoind` which is running and could open or close files or sockets at any time. Thus ignore the `FileNotFoundError` exception.
ACKs for top commit:
arejula27:
ACK [`b2e9fdc`](b2e9fdc00f)
sipa:
utACK b2e9fdc00f
achow101:
ACK b2e9fdc00f
theuni:
utACK b2e9fdc00f
hodlinator:
ACK b2e9fdc00f
Tree-SHA512: 8eb05393e4de4307a70af446c3fc7e8f7dc3f08bf9d68d74d02b0e4e900cfd4865249f297be31f1fd7b05ffea45eb855c5cfcd75704167950c1deb4f17109f33
With the exception of the first c++ checks, this ensures that compiler tests
are never run with the wrong build type's flags.
Co-Authored-By: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
This prevents intermittent failures - if the node hasn't processed
the inv from the outbound peer before the mocktime bump, the peer
won't be preferred after the other inv timeouts, failing the test .
Therefore, add a sync, just like there is one after the send_message
calls in the previous lines.
The legacy wallet will be able to solve output scripts where the
redeemScript or witnessScript is known, but does not know any of the
private keys involved in that script. These should be migrated to the
solvables wallet.
LegacySPKM would determine whether it could provide any script data to a
transaction through the use of the CanProvide function. Instead of
partially reversing signing logic to figure out the output scripts of
solvable things, we use the same candidate set approach in
GetScriptPubKeys() and instead filter the candidate set first for
things that are ISMINE_NO, and second with CanProvide(). This should
give a more accurate solvables wallet.
InferDescriptors can sometimes make descriptors which are actually
invalid and cannot be parsed. Detect and skip such descriptors by doing
a Parse() check before adding the descriptor to the wallet.
Instead of (partially) trying to reverse IsMine() to get the output
scripts that a LegacySPKM would track, we can preserve it in migration
only code and utilize it to get an accurate set of output scripts.
This is accomplished by computing a set of output script candidates from
map(Crypted)Keys, mapScripts, and setWatchOnly. This candidate set is an
upper bound on the scripts tracked by the wallet. Then IsMine() is used
to filter to the exact output scripts that LegacySPKM would track.
By changing GetScriptPubKeys() this way, we can avoid complexities in
reversing IsMine() and get a more complete set of output scripts.
ea687d2029 doc: swap CPPFLAGS for APPEND_CPPFLAGS (fanquake)
Pull request description:
`APPEND_CPPFLAGS` will be understood by our CMake, whereas `CPPFLAGS` will not. Attempting what is currently documented will just give:
```bash
CMake Warning:
Ignoring extra path from command line:
"CPPFLAGS=-DDEBUG_LOCKCONTENTION"
```
ACKs for top commit:
fjahr:
ACK ea687d2029
hebasto:
ACK ea687d2029.
Tree-SHA512: b8d3359b77a813535a4fa715619b815cd88e5440950f7d4cd045514e6b45d3f1c7f62061315c8581d0a99c0aec38340d34008be05657d198d868b241d19b7828
b448b01494 test: add a mocked Sock that allows inspecting what has been Send() to it (Vasil Dimov)
f1864148c4 test: put the generic parts from StaticContentsSock into a separate class (Vasil Dimov)
4b58d55878 test: move the implementation of StaticContentsSock to .cpp (Vasil Dimov)
Pull request description:
Put the generic parts from `StaticContentsSock` into a separate class `ZeroSock` so that they can be reused in other mocked `Sock` implementations.
Add a new `DynSock` whose `Recv()` and `Send()` methods can be controlled after the object is created. To achieve that, the caller/creator of `DynSock` provides to its constructor two pipes (FIFOs) - recv-pipe and send-pipe. Whatever data is written to recv-pipe is later received by `DynSock::Recv()` method and whatever data is written to the socket using `DynSock::Send()` can later be found in the send-pipe. For convenience there are also two methods to send and receive `CNetMessage`s.
---
This is used in https://github.com/bitcoin/bitcoin/pull/26812 (first two commits from that PR).
Extracting as a separate PR suggested here: https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1619152037.
ACKs for top commit:
Sjors:
re-ACK b448b01494
jonatack:
re-ACK b448b01494
pinheadmz:
ACK b448b01494
Tree-SHA512: 4a36f038192ec4ef63366cbe1a38ae70e7e015630c9f7c44926b756b20ab8c08138acae41801f23b30f6629c7059c1f81e001806e86584ff1bf1fa5b44d9caec
386eecff5f doc: add release notes (ismaelsadeeq)
3eaa0a3b66 miner: init: add `-blockreservedweight` startup option (ismaelsadeeq)
777434a2cd doc: rpc: improve `getmininginfo` help text (ismaelsadeeq)
c8acd4032d init: fail to start when `-blockmaxweight` exceeds `MAX_BLOCK_WEIGHT` (ismaelsadeeq)
5bb31633cc test: add `-blockmaxweight` startup option functional test (ismaelsadeeq)
2c7d90a6d6 miner: bugfix: fix duplicate weight reservation in block assembler (ismaelsadeeq)
Pull request description:
* This PR attempts to fix the duplicate coinbase weight reservation issue we currently have.
* Fixes#21950
We reserve 4000 weight units for coinbase transaction in `DEFAULT_BLOCK_MAX_WEIGHT`
7590e93bc7/src/policy/policy.h (L23)
And also reserve additional `4000` weight units in the default `BlockCreationOptions` struct.
7590e93bc7/src/node/types.h (L36-L40)
**Motivation**
- This issue was first noticed during a review here https://github.com/bitcoin/bitcoin/pull/11100#discussion_r136157411)
- It was later reported in issue #21950.
- I also came across the bug while writing a test for building the block template. I could not create a block template above `3,992,000` in the block assembler, and this was not documented anywhere. It took me a while to realize that we were reserving space for the coinbase transaction weight twice.
---
This PR fixes this by consolidating the reservation to be in a single location in the codebase.
This PR then adds a new startup option `-blockreservedweight` whose default is `8000` that can be used to lower or increase the block reserved weight for block header, txs count, coinbase tx.
ACKs for top commit:
Sjors:
ACK 386eecff5f
fjahr:
Code review ACK 386eecff5f
glozow:
utACK 386eecff5f, nonblocking nits. I do think the release notes should be clarified more
pinheadmz:
ACK 386eecff5f
Tree-SHA512: f27efa1da57947b7f4d42b9322b83d13afe73dd749dd9cac49360002824dd41c99a876a610554ac2d67bad7485020b9dcc423a8e6748fc79d6a10de6d4357d4c
fa5a02bcfa ci: Use clang-20 for sanitizer tasks (MarcoFalke)
Pull request description:
A new clang version generally comes with bugfixes, new (sanitizer) features, and deprecations.
Upgrade the sanitizer tasks to use the new version.
This was also suggested in https://github.com/bitcoin/bitcoin/pull/31691#issuecomment-2602517116
ACKs for top commit:
fanquake:
ACK fa5a02bcfa - tested 20 in some other infra, we just needed to fix the same deprecation warnings we'd seen, in cryptofuzz: 09ca550c3e.
Tree-SHA512: 6114d790b5d7145eb5f019e02da6c2c833342707ad67fb9f9c09506001afbef0c9b9beee7e51321f17f12ea692509d6428e6072ad105dba51e4d54cd057621cd
These scripts are becoming more of nuisance, than a value-add;
particularly since we've been building releases using Guix. Adding new
(release bin) tests can be harder, because it requires constructing a
failing test, which is becoming less easy e.g trying to disable a
feature or protection that has been built into the compiler/toolchain by
default.
In the pre-Guix days, these were valuable to sanity-check the environment,
because we were pulling that pre-built from Ubuntu, with little control.
At this point, it's less clear what these scripts are (sanity) checking.
Note that these also weren't completely ported to CMake (#31698), see
also #31715 which contains other fixes that would be needed for these
test-tests, to accomodate future changes.
e107bf78f9 [fuzz] TxOrphanage::SanityCheck accounting (glozow)
22dccea553 [fuzz] txorphan byte accounting (glozow)
982ce10178 add orphanage byte accounting to TxDownloadManagerImpl::CheckIsEmpty() (glozow)
c289217c01 [txorphanage] track the total number of announcements (glozow)
e5ea7daee0 [txorphanage] add per-peer weight accounting (glozow)
672c69c688 [refactor] change per-peer workset to info map within orphanage (glozow)
59cd0f0e09 [txorphanage] account for weight of orphans (glozow)
Pull request description:
Part of orphan resolution project, see #27463.
Definitions:
- **Announcement** is a unique pair (wtxid, nodeid). We can have multiple announcers for the same orphan since #31397.
- **Size** is the weight of an orphan. I'm calling it "size" and "bytes" because I think we can refine it in the future to be memusage or be otherwise more representative of the orphan's actual cost on our memory. However, I am open to naming changes.
This is part 1/2 of a project to also add limits on orphan size and count. However, this PR **does not change behavior**, just adds internal counters/tracking and a fuzzer. I will also open a second PR that adds behavior changes, which requires updating a lot of our tests and careful thinking about DoS.
ACKs for top commit:
instagibbs:
reACK e107bf78f9
marcofleon:
reACK e107bf78f9
sipa:
utACK e107bf78f9
Tree-SHA512: 855d725d5eb521d131e36dacc51990725e3ca7881beb13364d5ba72ab2202bbfd14ab83864b13b1b945a4ec5e17890458d0112270b891a41b1e27324a8545d72
f5b9a2f68c build: use CLIENT_NAME in libbitcoinkernel.pc.in (fanquake)
Pull request description:
Follows up from when the `pc.in` was added.
ACKs for top commit:
davidgumberg:
utACK f5b9a2f68c
stickies-v:
ACK f5b9a2f68c
theuni:
utACK f5b9a2f68c
hebasto:
ACK f5b9a2f68c.
Tree-SHA512: 7c32db919aa226f9894ed21baa3f794d1181d43d36ae56ba2d187e1a9bafd89feadc6209ab5b5a1b90d8a3de54fb910736397b1061ef48b232b59792ee250d55
2706c5b7c8 test: test_inv_block, use mocktime instead of waiting (Greg Sanders)
Pull request description:
Performance issue reported in https://github.com/bitcoin/bitcoin/pull/31437#issuecomment-2640221382
It seems that code as-is waits for wall-clock time to pass to synchronize mempools. Locally, sometimes the subtest takes a couple seconds, sometimes it takes an additional minute.
Just use mocktime?
ACKs for top commit:
sr-gi:
tACK [2706c5b](2706c5b7c8)
rishkwal:
tACK 2706c5b
Prabhat1308:
tACK [2706c5b](2706c5b7c8)
Tree-SHA512: 561fe3d67282c67e1ed7dd5eeb137964c083d498534ea5f749f3d782e73a3f47d23faee6cca39866eaba770fda7b7d60a9f740f16bdb451d6a5e9105417cb158
407062f2ac depends: Avoid using the `-ffile-prefix-map` compiler option (Hennadii Stepanov)
Pull request description:
This PR is similar to https://github.com/bitcoin/bitcoin/pull/31337 and applies analogous changes to all dependency packages.
The issue was [recently noticed](https://github.com/bitcoin/bitcoin/pull/31661#discussion_r1923896475) when `-ffile-prefix-map` was added to the `libevent` package, which is built in OSS-Fuzz.
This PR replaces `-ffile-prefix-map` in all packages for consistency.
Fixes https://github.com/bitcoin/bitcoin/issues/31770.
ACKs for top commit:
davidgumberg:
Tested ACK 407062f2ac.
theuni:
utACK 407062f2ac
Tree-SHA512: c501519c2397b7f11cdab13c0cd4b98a73b305817dba6ff61efc1c80c3cb44134bbd7f55eaecc1dab97f817ce44b28b6c81ccef74ea2d62c93ac43130be4efaf
faca7ac132 ci: Bump fuzz task timeout (MarcoFalke)
Pull request description:
The fuzz task seems to be the most CPU intense task (going through GB of data through all fuzz inputs for all fuzz targets).
Normally, the task takes 44 minutes (example https://cirrus-ci.com/task/5077976091459584), but under higher load, it may take longer (https://cirrus-ci.com/task/5966231095738368).
I tried to move it to GHA to see how it compares, but it will be even slower there: https://github.com/maflcko/bitcoin-core-with-ci/actions/runs/13182526514/job/36796629409.
The CI machines were recently updated to increase the CI performance, so in theory they could be updated again, but this can take some time and seems like the wrong fix anyway, because it will just hide the problem:
Ideally fuzzing is fast and when evaluating a fuzz input takes more than 10 seconds, it feels more like a slow unit test loop. So ideally fuzz timeouts should be fixed (https://github.com/bitcoin/bitcoin/issues/31066, https://github.com/bitcoin/bitcoin/issues/30498, ...). However, this can also take time.
So temporarily bump the fuzz timeout for now.
ACKs for top commit:
dergoegge:
ACK faca7ac132
Tree-SHA512: cfb06d14712d94be6b28a17eee821dcc762453e8efbd9376200f8a0e784a55c2140e45ac48bee9b71ef6e85ae7345155dddc1239cbf0cd4bc02583848fe46308
No change for now, moving from map of NodeId->workset to
NodeId->PeerOrphanInfo struct that holds the workset.
In future commits, we will start tracking more things per-peer in the
orphanage.
2f27c91086 qt: Update the `src/qt/locale/bitcoin_en.xlf` translation source file (Hennadii Stepanov)
864386a744 cmake: Ensure generated sources are up to date for `translate` target (Hennadii Stepanov)
2b51dd384b Update Transifex slug for 29.x (Hennadii Stepanov)
Pull request description:
This PR follows our [Release Process](864386a744/doc/release-process.md).
It is required to open Transifex translations for v29.0, as scheduled in https://github.com/bitcoin/bitcoin/issues/31029.
The previous similar PR: https://github.com/bitcoin/bitcoin/pull/30548.
**Notes for reviewers:**
1. This is the first release process conducted after migrating the build system to CMake. This revealed a bug, which is fixed in the second commit
2. To reproduce the diff in the third commit, follow these steps:
```
gmake -C depends -j $(nproc) MULTIPROCESS=1
cmake --preset dev-mode --toolchain depends/$(./depends/config.guess)/toolchain.cmake
cmake --build build_dev_mode --target translate
```
ACKs for top commit:
stickies-v:
ACK 2f27c91086
Tree-SHA512: 325ce2418f218b82cc3b0a6c727473963455680cdf6383a85768613ed9e485974b2e52bd5b2e7a7472ad8ebe40bccb2884764d7f9e83dc10a587cd7892e0028b
Some of the networking constants defined in p2p_tx_download.py are more generally
defined in p2p.py
Also, rename the remaining ones to match ones defined in txdownloadman
Some sources might be generated, and while they likely do not contain
any translatable strings, this change generalizes the approach to
include generated sources in the translation process as well.
1973a9e4f1 test: fixes p2p_ibd_txrelay wait time (Sergi Delgado Segura)
Pull request description:
`p2p_ibd_txrelay` expects no GETDATA to have been received by a peer after announcing a transaction. The reason is that the node is doing IBD, so transaction requests are not replied to. However, the way this is checked is wrong, and the check will pass even if the node **was not** in IBD.
This is due to the mocktime not being properly initialized, so the check is always performed earlier than it should, making it impossible for the request to be there.
This can be checked by modifying the test so the peer **is not doing IBD**, and checking how the test succeeds on that assert (even though it fails later on, given the nature of the test):
```diff
index 882f5b5c13..3a69ae5860 100755
--- a/test/functional/p2p_ibd_txrelay.py
+++ b/test/functional/p2p_ibd_txrelay.py
@@ -34,7 +34,7 @@ NORMAL_FEE_FILTER = Decimal(100) / COIN
class P2PIBDTxRelayTest(BitcoinTestFramework):
def set_test_params(self):
- self.setup_clean_chain = True
+ # self.setup_clean_chain = True
self.num_nodes = 2
self.extra_args = [
["-minrelaytxfee={}".format(NORMAL_FEE_FILTER)],
@@ -43,9 +43,11 @@ class P2PIBDTxRelayTest(BitcoinTestFramework):
def run_test(self):
self.log.info("Check that nodes set minfilter to MAX_MONEY while still in IBD")
- for node in self.nodes:
- assert node.getblockchaininfo()['initialblockdownload']
- self.wait_until(lambda: all(peer['minfeefilter'] == MAX_FEE_FILTER for peer in node.getpeerinfo()))
+ # for node in self.nodes:
+ # assert node.getblockchaininfo()['initialblockdownload']
+ # self.wait_until(lambda: all(peer['minfeefilter'] == MAX_FEE_FILTER for peer in node.getpeerinfo()))
```
ACKs for top commit:
i-am-yuvi:
ACK 1973a9e4f1
glozow:
ACK 1973a9e4f1
Tree-SHA512: c4b3afe9927c5480671ebf5c1f6ee5fc7e3aeefeb13c210fa83587a6c126e1a8e40ad8e46587537d0f4bf06a36bbf2310ca065d685d4d9286e5a446b8d5b2235
3f4b104b1b test: make sure we are on sync with a peer before checking if they have sent a message (Sergi Delgado Segura)
Pull request description:
p2p_orphan_handling checks whether a message has not been requested slightly too soon, making the check always succeed. This passes unnoticed since the expected result is for the message to not have been received, but it will make the test not catch a relevant change that should make it fail.
An easy way to check this is the case is to modify one of the test cases to force a request within the expected time, and check how the request is not seen. After the change, the test would crash as expected:
```diff
index 963d92485c..30ab5f2035 100755
--- a/test/functional/p2p_orphan_handling.py
+++ b/test/functional/p2p_orphan_handling.py
@@ -186,9 +185,12 @@ class OrphanHandlingTest(BitcoinTestFramework):
parent_inv = CInv(t=MSG_WTX, h=int(tx_parent_arrives["tx"].getwtxid(), 16))
assert_equal(len(peer_spy.get_invs()), 0)
peer_spy.assert_no_immediate_response(msg_getdata([parent_inv]))
+ txid = 0xdeadbeef
+ peer_spy.send_and_ping(msg_inv([CInv(t=MSG_WTX, h=txid)]))
# Request would be scheduled with this delay because it is not a preferred relay peer.
self.nodes[0].bumpmocktime(NONPREF_PEER_TX_DELAY)
+ peer_spy.assert_never_requested(int(txid))
peer_spy.assert_never_requested(int(tx_parent_arrives["txid"], 16))
peer_spy.assert_never_requested(int(tx_parent_doesnt_arrive["txid"], 16))
# Request would be scheduled with this delay because it is by txid.
```
It is worth noting that this is not seen in the cases where the message is expected to be received, because in such cases `assert_never_requested` is always after a `wait_....` method, which is already waiting for the node to sync on their end.
ACKs for top commit:
i-am-yuvi:
ACK 3f4b104b1b
instagibbs:
ACK 3f4b104b1b
glozow:
ACK 3f4b104b1b
Tree-SHA512: 321a6605d630bed2217b6374e999dbb84da14138263dd8adf65fe3a6cd7981a50c873beced9cf05cb6d747a912e91017c58e7d4323d25449c87d83095ff4cba9
e3622a9692 tracing: document that peer addrs can be >68 chars (0xb10c)
b19b526758 tracing: log_p2p_connections.bt example (0xb10c)
caa5486574 tracing: connection closed tracepoint (0xb10c)
b2ad6ede95 tracing: add misbehaving conn tracepoint (0xb10c)
68c1ef4f19 tracing: add inbound connection eviction tracepoint (0xb10c)
4d61d52f43 tracing: add outbound connection tracepoint (0xb10c)
85b2603eec tracing: add inbound connection tracepoint (0xb10c)
Pull request description:
This adds five new tracepoints with documentation and tests for network connections:
- established connections with `net:inbound_connection` and `net:outbound_connection`
- closed connections (both closed by us or by the peer) with `net:closed_connnection`
- inbound connections that we choose to evict with `net:evicted_inbound_connection`
- connections that are misbehaving and punished with `net:misbehaving_connection`
I've been using these tracepoints for a few months now to monitor connection lifetimes, re-connection frequency by IP and netgroup, misbehavior, peer discouragement, and eviction and more. Together with the two existing P2P message tracepoints they allow for a good overview of local P2P network activity. Also sort-of addresses https://github.com/bitcoin/bitcoin/pull/22006#discussion_r636775863.
I've been back and forth on which arguments to include. For example, `net:evicted_connection` could also include some of the eviction metrics like e.g. `last_block_time`, `min_ping_time`, ... but I've left them out for now. If wanted, this can be added here or in a follow-up. I've tried to minimize a potential performance impact by measuring executed instructions with `gdb` where possible (method described [here](https://github.com/bitcoin/bitcoin/pull/23724#issuecomment-996919963)). I don't think a few hundred extra instructions are too crucial, as connection opens/closes aren't too frequent (compared to e.g. P2P messages). Note: e.g. `CreateNodeFromAcceptedSocket()` usually executes between 80k and 90k instructions for each new inbound connection.
| tracepoint | instructions |
|----------------------------|--------------------------------------------------------|
| net:inbound_connection | 390 ins |
| net:outbound_connection | between 700 and 1000 ins |
| net:closed_connnection | 473 ins |
| net:evicted_inbound_connection | not measured; likely similar to net:closed_connnection |
| net:misbehaving_connection | not measured |
Also added a bpftrace (tested with v0.14.1) `log_p2p_connections.bt` example script that produces output similar to:
```
Attaching 6 probes...
Logging opened, closed, misbehaving, and evicted P2P connections
OUTBOUND conn to 127.0.0.1:15287: id=0, type=block-relay-only, network=0, total_out=1
INBOUND conn from 127.0.0.1:45324: id=1, type=inbound, network=0, total_in=1
MISBEHAVING conn id=1, message='getdata message size = 50001'
CLOSED conn to 127.0.0.1:15287: id=0, type=block-relay-only, network=0, established=1231006505
EVICTED conn to 127.0.0.1:45324: id=1, type=inbound, network=0, established=1612312312
```
ACKs for top commit:
laanwj:
re-ACK e3622a9692
vasild:
ACK e3622a9692
sipa:
utACK e3622a9692
Tree-SHA512: 1032dcac6fe0ced981715606f82c2db47016407d3accb8f216c978f010da9bc20453e24a167dcc95287f4783b48562ffb90f645bf230990e3df1b9b9a6d4e5d0
The `-ffile-prefix-map` compiler option implicitly enables
`-fprofile-prefix-map` in GCC or `-fcoverage-prefix-map` in Clang, which
can cause issues with coverage builds.
This change ensures that only the options necessary for build
reproducibility are applied.
723440c5b8 test framework, wallet: rename get_scriptPubKey method to get_output_script (Alfonso Roman Zubeldia)
fa0232a3e0 test: add validation for gettxout RPC response (Alfonso Roman Zubeldia)
Pull request description:
Added a new test in `test/functional/rpc_blockchain.py` to validate the gettxout RPC response. This new test ensures all response elements are verified, including `bestblock`, `confirmations`, `value`, `coinbase`, and `scriptPubKey` details.
Also renamed the method `get_scriptPubKey` from `test/functional/test_framework/wallet.py` to the modern name `get_output_script` as suggested by maflcko (https://github.com/bitcoin/bitcoin/pull/30226#discussion_r1925491846)
ACKs for top commit:
fjahr:
reACK 723440c5b8
maflcko:
lgtm ACK 723440c5b8
brunoerg:
code review ACK 723440c5b8
Tree-SHA512: 3384578909d2e7548cef302c5b8a9fed5b82dfc942892503ad4a05e73f5cceafad1eab3af9a27e54aef3db7631f8935298d6b882c70d2f02a9a75b8e3c209b6c
846a138728 func test: Expand tx download preference tests (Greg Sanders)
Pull request description:
1. Check that outbound nodes are treated the same as whitelisted connections for
the purposes of `getdata` delays
2. Add test case that demonstrates download retries are preferentially
given to outbound (preferred) connections
even when multiple announcements are
considered ready.
`NUM_INBOUND` is a magic number large enough that it should fail over 90% of the time
if the underlying outbound->preferred->PriorityComputer logic was broken. Bumping this
to 100 peers cost another 14 seconds locally for the sub-test, so I made it pretty small.
ACKs for top commit:
i-am-yuvi:
tACK 846a138728 good catch
maflcko:
ACK 846a138728🍕
marcofleon:
lgtm ACK 846a138728
Tree-SHA512: 337aa4dc33b5c0abeb4fe7e4cd5e389f7f53ae25dd991ba26615c16999872542391993020122fd255af4c7163f76c1d1feb2f2d6114f12a364c0360d4d52b8c3
7e0db87d4f test: added additional coverage to waitforblock and waitforblockheight rpc's (kevkevinpal)
Pull request description:
Similar to https://github.com/bitcoin/bitcoin/pull/31746
This adds test coverage to the `waitforblock` and `waitforblockheight` rpc's by adding a test to assert we get an rpc error if we include a negative timeout
ACKs for top commit:
Sjors:
utACK 7e0db87d4f
Prabhat1308:
ACK [7e0db87](7e0db87d4f)
brunoerg:
code review ACK 7e0db87d4f
BrandonOdiwuor:
Code Review ACK 7e0db87d4f
Tree-SHA512: c3b1b3a525e91e0092b783d73d2401126e3b8792a394be00374258f20cf3d619498e6625d3aad5e5ced29509c5eac828ee03c4ee66ba405b91e89be7e8b07311
d9c8aacce3 depends, refactor: Avoid hardcoding `host_prefix` in toolchain file (Hennadii Stepanov)
Pull request description:
This PR allows the entire `depends/<host_prefix>` directory to be relocatable.
Only `libevent` package configuration files are non-relocatable for the version `2.1.12-stable` we use now. However, this issue has been fixed upstream in 1f1593ff27 and friends.
Fixes https://github.com/bitcoin/bitcoin/issues/31050.
ACKs for top commit:
theuni:
Neat. utACK d9c8aacce3
ryanofsky:
Code review ACK d9c8aacce3
Tree-SHA512: c4c340722e63fc1da36fba2b15f025a6e1d477da1991194d678f546f2f3ea9454e2f0ff054aae6ae6c332a0781a597c3ce63b9018b46b8c258033f0d19efbef3
- The reserved weight of the coinbase transaction is an estimate and
may not reflect the exact value; it can be lower.
- It should be clear that `currentblockweight` includes the reserved coinbase transaction weight.
whereas `currentblocktx` does not account for the coinbase transaction count.
- Also clarify `m_last_block_num_txs` and `m_last_block_weight`
- This commit renamed coinbase_max_additional_weight to block_reserved_weight.
- Also clarify that the reservation is for block header, transaction count
and coinbase transaction.
faf8fc5487 lint: Call lint_commit_msg from test_runner (MarcoFalke)
fa99728b0c lint: Move commit range printing to test_runner (MarcoFalke)
fa673cf344 lint: Call lint_scripted_diff from test_runner (MarcoFalke)
Pull request description:
The lint `commit-script-check.sh` can not be called from the test_runner at all and must be called manually. Also, some checks require `COMMIT_RANGE` to be set.
Fix all issues by moving two lint checks into the test_runner. Also, the proper commit range is passed to the checks by the test_runner, so that the user no longer has to do it.
ACKs for top commit:
kevkevinpal:
reACK [faf8fc5](faf8fc5487)
willcl-ark:
tACK faf8fc5487
Tree-SHA512: 78018adc618d997508c226c9eee0a4fada3899cdfd91587132ab1c0389aea69127bafc3a900e90e30aca2c6bae9dcd6e6188ef287e91413bc63ee66fb078b1af
3e97ff9c5e gui, psbt: Use SIGHASH_DEFAULT when signing PSBTs (Ava Chow)
Pull request description:
SIGHASH_DEFAULT should be used to indicate SIGHASH_DEFAULT for taproot inputs, and SIGHASH_ALL for all other input types. This avoids adding an unnecessary byte to the end of all Taproot signatures added to PSBTs signed in the GUI.
See also bitcoin/bitcoin#22514
ACKs for top commit:
Sjors:
utACK 3e97ff9c5e
pablomartin4btc:
utACK 3e97ff9c5e
hebasto:
ACK 3e97ff9c5e, I have reviewed the code and it looks OK.
Tree-SHA512: f96f26b3a6959865cf23039afb5ffb7e454fb52ee39c510583851caf00a8a383cde69bc7e90db536addbdd498a02f4b001cbaf509d6d53c5f8601b3933786f6c
9d2d9f7ce2 rpc: Include assumeutxo as a failure reason of rescanblockchain (Fabian Jahr)
595edee169 test, assumeutxo: import descriptors during background sync (Alfonso Roman Zubeldia)
d73ae603d4 rpc: Improve importdescriptor RPC error messages (Fabian Jahr)
27f99b6d63 validation: Don't assume m_chain_tx_count in GuessVerificationProgress (Fabian Jahr)
42d5d53363 interfaces: Add helper function for wallet on pruning (Fabian Jahr)
Pull request description:
A test that is added as part of #30455 uncovered this issue: The `GuessVerificationProgress` function is used during during descriptor import and relies on `m_chain_tx_count`. In #29370 an [`Assume` was added](0fd915ee6b) expecting the `m_chaint_tx_count` to be set. However, as the test uncovered, `GuessVerificationProgress` is called with background sync blocks that have `m_chaint_tx_count = 0` when they have not been downloaded and processed yet.
The simple fix is to remove the `Assume`. Users should not be thrown off by the `Internal bug detected` error. The behavior of `importdescriptor` is kept consistent with the behavior for blocks missing due to pruning.
The test by alfonsoromanz is cherry-picked here to show that the [CI errors](https://cirrus-ci.com/task/5110045812195328?logs=ci#L2535) should be fixed by this change.
This PR also improves error messages returned by the `importdescriptors` and `rescanblockchain` RPCs. The error message now changes depending on the situation of the node, i.e. if pruning is happening or an assumutxo backgroundsync is active.
ACKs for top commit:
achow101:
ACK 9d2d9f7ce2
mzumsande:
Code Review ACK 9d2d9f7ce2
furszy:
Code review ACK 9d2d9f7ce2
Tree-SHA512: b841a9b371e5eb8eb3bfebca35645ff2fdded7a3e5e06308d46a33a51ca42cc4c258028c9958fbbb6cda9bb990e07ab8d8504dd9ec6705ef78afe0435912b365
0cdddeb224 kernel: Move block tree db open to BlockManager constructor (TheCharlatan)
7fbb1bc44b kernel: Move block tree db open to block manager (TheCharlatan)
57ba59c0cd refactor: Remove redundant reindex check (TheCharlatan)
Pull request description:
Before this change the block tree db was needlessly re-opened during startup when loading a completed snapshot. Improve this by letting the block manager open it on construction. This also simplifies the test code a bit.
The change was initially motivated to make it easier for users of the kernel library to instantiate a BlockManager that may be used to read data from disk without loading the block index into a cache.
ACKs for top commit:
maflcko:
re-ACK 0cdddeb224🏪
achow101:
ACK 0cdddeb224
mzumsande:
re-ACK 0cdddeb224
Tree-SHA512: fe3d557a725367e549e6a0659f64259cfef6aaa565ec867d9a177be0143ff18a2c4a20dd57e35e15f97cf870df476d88c05b03b6a7d9e8d51c568d9eda8947ef
93747d934b test: Added coverage to the waitfornewblock rpc (kevkevinpal)
Pull request description:
Added a test for the Negative timeout error if the rpc is given a negative value for its timeout arg
This adds coverage to the `waitfornewblock` rpc
you can check to see there is no coverage for this error by doing
`grep -nri "Negative timeout" ./test/`
and nothing shows up, you can also see by manually checking where we call `waitfornewblock` in the functional tests
ACKs for top commit:
Sjors:
tACK 93747d934b
achow101:
ACK 93747d934b
brunoerg:
code review ACK 93747d934b
tdb3:
ACK 93747d934b
Tree-SHA512: 45cf34312412d3691a39f003bcd54791ea16542aa3f5a2674d7499c9cc4039550b2cbd32cc3d4c5fe100d65cb05690594b10a0c42dfab63bcca3dac121bb195b
e1676b08f7 doc: release notes (Sjors Provoost)
0082f6acc1 rpc: have mintime account for timewarp rule (Sjors Provoost)
79d45b10f1 rpc: clarify BIP94 behavior for curtime (Sjors Provoost)
0713548137 refactor: add GetMinimumTime() helper (Sjors Provoost)
Pull request description:
#30681 fixed the `curtime` field of `getblocktemplate` to take the timewarp rule into account. However I forgot to do the same for the `mintime` field, which was hardcoded to use `pindexPrev->GetMedianTimePast()+1`.
This PR adds a helper `GetMinimumTime()` and uses it for the `mintime` field.
#31376 changed the `curtime` field to always account for the timewarp rule. This PR maintains that behavior.
Note that `mintime` now always applies BIP94, including on mainnet. This makes future softfork activation safer.
It could be backported to v28.
ACKs for top commit:
fjahr:
tACK e1676b08f7
achow101:
ACK e1676b08f7
darosior:
utACK e1676b08f7 on the code changes
tdb3:
brief code review re ACK e1676b08f7
TheCharlatan:
ACK e1676b08f7
Tree-SHA512: 0e322d8cc3b8ff770849bce211edcb5b6f55d04e5e0dee0657805049663d758f27423b047ee6363bd8f6c6fead13f974760f48b3321ea86f514f446e1b23231c
This prevents the generation of these headers from also depending on the
dependencies of the libs/binaries which consume them.
Specifically, this prevents generated test headers (such as
test/data/base58_encode_decode.json.h) from depending on the
dependencies of test_bitcoin (libcrc32c.a libcrc32c_sse42.a libleveldb.a)
Note that this is currently only relevant for Ninja.
For more detail, see:
https://cmake.org/cmake/help/latest/command/add_custom_command.html
8888ee4403 ci: Allow build dir on CI host (MarcoFalke)
Pull request description:
This is required to pass cross builds on to a different machine after the build.
See for example https://github.com/bitcoin/bitcoin/pull/31176, but this pull will also allow someone to implement it outside this repo.
ACKs for top commit:
davidgumberg:
lgtm ACK 8888ee4403
hebasto:
re-ACK 8888ee4403.
Tree-SHA512: a1e2c32bc1b95efbd0b48287ac5b49e0e1bacbf5a5800845be5352bbdd3e17fa478e90348b2e94e95cf3ae863cdf75ab444089376588f6f8eec438f73a4b5b97
152a2dcdef test: fix intermittent timeout in p2p_1p1c_network.py (Martin Zumsande)
Pull request description:
The timeout is due to outstanding txrequests with python peers, which have the same timeout (`60s`) as the mempool sync timeout.
I explained this in more detail in https://github.com/bitcoin/bitcoin/issues/31721#issuecomment-2620169640 and also mentioned there how to reproduce it.
Fix this by disconnecting the python peers after they send their txns, they aren't needed after this point anyway because the main goal of the test is the sync between the 4 full nodes.
Fixes#31721
ACKs for top commit:
achow101:
ACK 152a2dcdef
instagibbs:
reACK 152a2dcdef
marcofleon:
ACK 152a2dcdef
glozow:
reACK 152a2dcdef
Tree-SHA512: 908c58933d8e9fcca91425fce1b7c9c7cb7121a6d26840630e03a442356ad2a327d1e087df72a19caa97024ea827593e10f2ff93838f88939458e73df9857df0
Deduplicate the logic of adding the parents as announcements to
txrequest. The function can return a bool (indicating whether we're
attempting orphan resolution) instead of the delay.
e87429a2d0 ci: optionally use local docker build cache (0xb10c)
Pull request description:
By setting `DANGER_DOCKER_BUILD_CACHE_HOST_DIR`, the task-specific docker images built during the CI run can be cached. This allows, for example, ephemeral CI runners to reuse the docker images (or layers of it) from earlier runs, by persisting the image cache before the ephemeral CI runner is shut down. The cache keyed by `CONTAINER_NAME`.
As `--cache-to` doesn't remove old cache files, the existing cache is removed after a successful `docker build` and the newly cached image is moved to it's location to avoid the cache from growing indefinitely with old, unused layers.
When `--cache-from` doesn't find the directory, the cached version is a cache-miss, or the cache can't be imported for whatever other reason, it warns and `docker build` continues by building the docker image.
This feature is opt-in. The documentation for the docker build cache of `type=local` can be found on https://docs.docker.com/build/cache/backends/local/
This replaces https://github.com/bitcoin/bitcoin/pull/31377 - some of the discussion there might provide more context.
ACKs for top commit:
maflcko:
I haven't tested this, and it looks harmless and is easy to revert, if needed. So lgtm ACK e87429a2d0
achow101:
ACK e87429a2d0
TheCharlatan:
tACK e87429a2d0
willcl-ark:
ACK e87429a2d0
Tree-SHA512: 0887c395dee2e2020394933246d4c1bfb6dde7165219cbe93eccfe01379e05c75dce8920b6edd7df07364c703fcee7be4fba8fa45fd0e0e89da9e24759f67a71
cddcbaf81e RPC: improve SFFO arg parsing, error catching and coverage (furszy)
4f4cd35319 rpc: decouple sendtoaddress 'subtractfeefromamount' boolean parsing (furszy)
Pull request description:
Following changes were made:
1) Catch and signal error for duplicate string destinations.
2) Catch and signal error for invalid value type.
3) Catch and signal error for string destination not found in tx outputs.
4) Improved `InterpretSubtractFeeFromOutputInstructions()` code organization.
5) Added test coverage for all possible error failures.
Also, fixed two PEP 8 warnings at the 'wallet_sendmany.py' file:
- PEP 8: E302 expected 2 blank lines, found 1 at the SendmanyTest class declaration.
- PEP 8: E303 too many blank lines (2) at skip_test_if_missing_module() and set_test_params()
ACKs for top commit:
achow101:
ACK cddcbaf81e
murchandamus:
crACK cddcbaf81e
naiyoma:
TACK [cddcbaf81e)
ismaelsadeeq:
Code review and Tested ACK cddcbaf81e
Tree-SHA512: c9c15582b81101a93987458d155394ff2c9ca42864624c034ee808a31c3a7d7f55105dea98e86fce17d3c7b2c1a6b5b77942da66b287f8b8881a60cde78c1a3c
d45eb3964f test: compare BDB dumps of test framework parser and wallet tool (Sebastian Falbesoner)
01ddd9f646 test: complete BDB parser (handle internal/overflow pages, support all page sizes) (Sebastian Falbesoner)
Pull request description:
This PR adds missing features to our test framework's BDB parser with the goal of hopefully being able to read all legacy wallets that are created with current and past versions of Bitcoin Core. This could be useful both for making review of https://github.com/bitcoin/bitcoin/pull/26606 easier and to also possibly improve our functional tests for the wallet BDB-ro parser by additionally validating it with an alternative implementation. The second commits introduces a test that create a legacy wallet with huge label strings (in order to create overflow pages, i.e. pages needed for key/value data than is larger than the page size) and compares the dump outputs of wallet tool and the extended test framework BDB parser.
It can be exercised via `$ ./test/functional/tool_wallet.py --legacy`. BDB support has to be compiled in (obviously).
For some manual tests regarding different page sizes, the following patch can be used:
```diff
diff --git a/src/wallet/bdb.cpp b/src/wallet/bdb.cpp
index 38cca32f80..1bf39323d3 100644
--- a/src/wallet/bdb.cpp
+++ b/src/wallet/bdb.cpp
@@ -395,6 +395,7 @@ void BerkeleyDatabase::Open()
DB_BTREE, // Database type
nFlags, // Flags
0);
+ pdb_temp->set_pagesize(1<<9); /* valid BDB pagesizes are from 1<<9 (=512) to <<16 (=65536) */
if (ret != 0) {
throw std::runtime_error(strprintf("BerkeleyDatabase: Error %d, can't open database %s", ret, strFile));
```
I verified that the newly introduced test passes with all valid page sizes between 512 and 65536.
ACKs for top commit:
achow101:
ACK d45eb3964f
furszy:
utACK d45eb3964f
brunoerg:
code review ACK d45eb3964f
Tree-SHA512: 9f8ac80452545f4fcd24a17ea6f9cf91b487cfb1fcb99a0ba9153fa4e3b239daa126454e26109fdcb72eb1c76a4ee3b46fd6af21dc318ab67bd12b3ebd26cfdd
p2p_orphan_handling checks whether a message has not been requested slightly
too soon, making the check always succeed. This passes unnoticed since the
expected result is for the message to not have been received, but it will make
the test not catch a relevant change that should make it fail
p2p_ibd_txrelay expects no GETDATA to have been received by a peer after
announcing a transaction. The reason is that the node is doing IBD, so
transaction requests are not replied to. However, the way this is checked
is wrong, and the check will pass even if the node **was not** in IBD.
This is due to the mocktime not being properly initialized, so the check
is always performed earlier than it should, making it impossible for the
request to be there
The timeout is due to outstanding txrequests with
python peers. Fix this by disconnecting these peers
after they send their txns, they aren't needed after
this point anyway.
Previously in getblocktemplate only curtime took the timewarp rule into account.
Mining pool software could use either, though in general it should use curtime.
Before bip94 there was an assumption that the minimum permitted
timestamp is GetMedianTimePast() + 1.
This commit splits a helper function out of UpdateTime() to
obtain the minimum time in a way that takes the
timewarp attack rule into account.
fa8ade300f refactor: Avoid GCC false positive error (MarcoFalke)
fa40807fa8 ci: Enable DEBUG=1 for one GCC-12+ build to catch 117966 regressions (MarcoFalke)
Pull request description:
It is possible that someone accidentally removes the workaround in fa9e0489f5, or more likely that someone accidentally adds new code without the workaround.
Avoid this by adding a temporary CI check.
This can be tested by reverting the workaround and observing a failure.
ACKs for top commit:
hebasto:
ACK fa8ade300f, I've tested locally on Ubuntu 24.04.
Tree-SHA512: 7ee1538fd5304a5ab91ac8c7619a573548d7e0345592a1e9d38b3b73729e09e7c77a9ee703d64cf02a8218de3148376d7836e294abb939aa7533034ba36dfb6c
faf2f2c654 test: Avoid redundant stop and error spam on shutdown (MarcoFalke)
fae3bf6b87 test: Avoid redundant stop and error spam on startup failure (MarcoFalke)
fa0dc09b90 test: Remove --noshutdown flag (MarcoFalke)
fad441fba0 test: Treat leftover process as error (MarcoFalke)
Pull request description:
The `--noshutdown` flag is brittle, confusing, and redundant:
* Someone wanting to inspect the state after a test failure will likely also want to debug the state on the python side, so the option is redundant with `--pdbonfailure`. If there was a use case to replicate `--pdbonfailure` without starting pdb, a dedicated flag could be added for that use case.
* It is brittle to use the flag for a passing test, because it will disable checks in the test. For example, on shutdown LSan will perform a leak check, and the test framework will check that the node did not crash, and it will check that the node did not print errors to stderr.
Fix all issues by removing it.
Also, tidy up startup error messages to be less confusing as a result.
ACKs for top commit:
hodlinator:
re-ACK faf2f2c654
pablomartin4btc:
re tACK faf2f2c654
Tree-SHA512: 46d7ae59c7be88b93f1f9e0b6be21af0fc101e646512e2c5e725682cb18bfec8aa010e0ebe89ce9ffe239e5caac0da5f81cc97b79e738d26ca5fa31930e8e4e3
f5883286e3 Add a fuzz test for Num3072 multiplication and inversion (Pieter Wuille)
a26ce62894 Safegcd based modular inverse for Num3072 (Pieter Wuille)
91ce8cef2d Add benchmark for MuHash finalization (Pieter Wuille)
Pull request description:
This implements a safegcd-based modular inverse for MuHash3072. It is a fairly straightforward translation of [the libsecp256k1 implementation](https://github.com/bitcoin-core/secp256k1/pull/831), with the following changes:
* Generic for 32-bit and 64-bit
* Specialized for the specific MuHash3072 modulus (2^3072 - 1103717).
* A bit more C++ish
* Far fewer sanity checks
A benchmark is also included for MuHash3072::Finalize. The new implementation is around 100x faster on x86_64 for me (from 5.8 ms to 57 μs); for 32-bit code the factor is likely even larger.
For more information:
* [Original paper](https://gcd.cr.yp.to/papers.html) by Daniel J. Bernstein and Bo-Yin Yang
* [Implementation](https://github.com/bitcoin-core/secp256k1/pull/767) for libsecp256k1 by Peter Dettman; and the [final](https://github.com/bitcoin-core/secp256k1/pull/831) version
* [Explanation](https://github.com/bitcoin-core/secp256k1/blob/master/doc/safegcd_implementation.md) of the algorithm using Python snippets
* [Analysis](https://github.com/sipa/safegcd-bounds) of the maximum number of iterations the algorithm needs
* [Formal proof in Coq](https://medium.com/blockstream/a-formal-proof-of-safegcd-bounds-695e1735a348) by Russell O'Connor (for the 256-bit version of the algorithm; here we use a 3072-bit one).
ACKs for top commit:
achow101:
ACK f5883286e3
TheCharlatan:
Re-ACK f5883286e3
dergoegge:
tACK f5883286e3
Tree-SHA512: 275872c61d30817a82901dee93fc7153afca55c32b72a95b8768f3fd464da1b09b36f952f30e70225e766b580751cfb9b874b2feaeb73ffaa6943c8062aee19a
the cmake build steps suggest a build/ directory, which breaks these
scripts. Additionally, in-tree builds are no longer allowed, so it makes
sense to update the code and the README accordingly.
These comments are there since wallet encryption was first introduced
(see commit 4e87d341f7, PR #352), but scrypt
was actually never implemented as a derivation method.
1b51616f2e test: improve rogue calls in mining functions (i-am-yuvi)
Pull request description:
#31403 follow-up, see [comment](https://github.com/bitcoin/bitcoin/pull/31403#pullrequestreview-2498806354)
- Rename `invalid_call` parameter to `called_by_framework` in `generateblock`, `generatetoaddress` and `generatetodescriptor` mining methods to better express its intended usage.
- Add explicit assertion message clarifying that these functions should only be called by TestFramework itself to maintain proper node synchronization.
ACKs for top commit:
maflcko:
lgtm ACK 1b51616f2e
achow101:
ACK 1b51616f2e
hodlinator:
re-ACK 1b51616f2e
Prabhat1308:
ACK [1b51616](1b51616f2e)
Tree-SHA512: 56832626fe54dcaa07dacb4f9c960c0a83fad3fb12272155114ac697856c59b7f44805e1152eddeec7a5e8f7daf487382dc01b5b9ae2e74b62b2df6bd1f81f77
92787dd52c test: raise an error when target_vsize is below tx virtual size (ismaelsadeeq)
a8780c937f test: raise an error if output value is <= 0 in `create_self_transfer` (ismaelsadeeq)
f6e88931f0 test: test that `create_self_transfer_multi` respects `target_vsize` (ismaelsadeeq)
Pull request description:
This is a simple test PR that does two things:
1. Raise an exception in `_bulk_tx_` when `target_vsize` is too low, i.e., below the tx vsize.
2. Addresses some review comments from https://github.com/bitcoin/bitcoin/pull/30162, which are:
- Raise an error if the output value is less than or equal to zero in `create_self_transfer`.
This prevents creating transactions with a value of 0 or less.
- Add a test to verify that `create_self_transfer_multi` also respects the passed `target_vsize`.
ACKs for top commit:
achow101:
ACK 92787dd52c
theStack:
ACK 92787dd52c
rkrux:
reACK 92787dd52c
glozow:
ACK 92787dd52c
Tree-SHA512: 1f2767f2cf715ed65074c5fff347eec160b142685777d833d5e872cfef364d3dc1916b52ee442e99c7b9a8d514ff62bc67a9899d8854f65a4b93ac3ae300d18e
18619b4732 wallet: remove BDB dependency from wallet migration benchmark (furszy)
Pull request description:
Part of the legacy wallet removal working path #20160.
Stops creating a bdb database in the wallet migration benchmark.
Instead, the benchmark now creates the db in memory and re-uses it for the migration process.
ACKs for top commit:
achow101:
ACK 18619b4732
brunoerg:
code review ACK 18619b4732
theStack:
Code-review ACK 18619b4732
Tree-SHA512: a107deee3d2c00b980e3606be07d038ca524b98251442956d702a7996e2ac5e2901f656482018cacbac8ef6a628ac1fb03f677d1658aeaded4036d834a95d7e0
2656a5658c tests: add a test for the new blocksdir lock (Cory Fields)
bdc0a68e67 init: lock blocksdir in addition to datadir (Cory Fields)
cabb2e5c24 refactor: introduce a more general LockDirectories for init (Cory Fields)
1db331ba76 init: allow a new xor key to be written if the blocksdir is newly created (Cory Fields)
Pull request description:
This probably should've been included in #12653 when `-blocksdir` was introduced. Credit TheCharlatan for noticing that it's missing.
This guards against 2 processes running with separate datadirs but the same blocksdir. I didn't add `walletdir` as I assume sqlite has us covered there.
It's not likely to happen currently, but may be more relevant in the future with applications using the kernel. Note that the kernel does not currently do any dir locking, but it should.
ACKs for top commit:
maflcko:
review ACK 2656a5658c 🏼
kevkevinpal:
ACK [2656a56](2656a5658c)
achow101:
ACK 2656a5658c
tdb3:
Code review and light test ACK 2656a5658c
Tree-SHA512: 3ba17dc670126adda104148e14d1322ea4f67d671c84aaa9c08c760ef778ca1936832c0dc843cd6367e09939f64c6f0a682b0fa23a5967e821b899dff1fff961
8996fef8ae test: p2p: check that INV messages not matching wtxidrelay are ignored (Sebastian Falbesoner)
e0b3336822 test: p2p: fix sending of manual INVs in tx download test (Sebastian Falbesoner)
Pull request description:
The `test_inv_block` sub-test in p2p_tx_download.py has a subtle bug: the manual msg_inv announcements from peers currently have no effect, since they don't match the wtxidrelay setting (=true by default for `P2PInterface` instances) and are hence ignored by the nodes (since 2d282e0c / PR #18044):
e7c4794955/src/net_processing.cpp (L3904-L3911)
Though the test still passes on master, it does so without the intended scenario of asking an additional peer (triggering the GETDATA_TX_INTERVAL delay). Fix this by sending the INV message with MSG_WTX instead of MSG_TX. This increases the test run time by about one minute intentionally.
It might be good to avoid issues like this in the future, happy to add test framework improvements if someone has a concrete idea.
(Got into the topic of tx/wtx announcements via the discussion https://github.com/bitcoin/bitcoin/pull/31397#discussion_r1904121487)
ACKs for top commit:
maflcko:
ACK 8996fef8ae😸
danielabrozzoni:
ACK 8996fef8ae
mzumsande:
Code Review ACK 8996fef8ae
Tree-SHA512: 3da26f9539c89d64c3b0d0579d9af2a6a4577615eed192506e1fb4318421b235f99a6672a497dea3050fba85dad32678f37fd2cda9ecb70cbf52982db37982e8
fad83e759a doc: Fix incorrect send RPC docs (MarcoFalke)
Pull request description:
It would be good to have accurate RPC docs, so that humans and machines can read them and rely on them.
This fixes one issue.
ACKs for top commit:
fjahr:
utACK fad83e759a
rkrux:
tACK fad83e759a
luke-jr:
tACK fad83e759a
Tree-SHA512: 65d0cc18a62ef44833621464d74b743d24ffe2b853596dce2c4f423df0495142d50387c02ba1b54f5ca77d4ddb083d55116a8ac92698aa6558762d841664911e
The Miniscript code contains assertions to prevent ending up in an insane state or prevent UB, but
also to enforce logical invariants. For the latter it is not necessary to crash the program if they
are broken. Raising an exception suffices, especially as this code is often called through the RPC
interface which can in turn handle the exception and the user can report it to developers.
This is based on previous work from Pieter Wuille.
5c3e4d8b29 doc: add a section about using MSan (Antoine Poinsot)
Pull request description:
Just a couple lines in a subsection of the sanitizers section mentioning that using the memory sanitizer is a bit more involve than other sanitizers, describing the steps and pointing to an example.
ACKs for top commit:
fanquake:
ACK 5c3e4d8b29
dergoegge:
ACK 5c3e4d8b29
Tree-SHA512: 4ff73c2dd0f25cb96148e54bd867b8d340bd0fbc9b9a736a705125039352eb1d40bd724f9f262a44d3dbd1bea8f03166cf30e571d882fec02ceb1dd399ef7422
Trying to shut down a node after a test failure may fail and lead to an
RPC error.
Also, it is confusing to sidestep the existing fallback to kill any
leftover nodes on a test failure.
So just rely on the fallback.
Idea by Hodlinator.
Co-Authored-By: Hodlinator <172445034+hodlinator@users.noreply.github.com>
4da7bfdcc9 test: add coverage for unknown address type for `createwalletdescriptor` (brunoerg)
Pull request description:
Calling `createwalletdescriptor` RPC with an unknown address type throws an error. This PR adds test coverage for it as done for other RPCs (`getnewaddress `, `getrawchangeaddress`, etc).
ACKs for top commit:
maflcko:
lgtm ACK 4da7bfdcc9
rkrux:
tACK 4da7bfdcc9
Tree-SHA512: 490bc3ffeb70b0f26db0a44d3950d7410fef35d4056487f2e82c081fb14ca277a18943c487235e0163a29f90fc741a262c29835beb9f41936affa4e73ddad25f
fa9aced800 test: Check that reindex with prune wipes blk files (MarcoFalke)
fa9593efc2 test: Use high-level python types (MarcoFalke)
Pull request description:
This adds missing test coverage for `CleanupBlockRevFiles`.
ACKs for top commit:
TheCharlatan:
Re-ACK fa9aced800
l0rinc:
ACK fa9aced800
tdb3:
re ACK fa9aced800
Tree-SHA512: b31ff8a896ce344437715e7fb7efdb8cd7e11470e8465d8972fddfdb58ffd78257786c4060e8596cc53b6278f8ac6a9b6eb05a06e9df58b8b240bdaa719a8e5b
d38ade7bc4 qa: Use `sys.executable` when invoking other Python scripts (Hennadii Stepanov)
Pull request description:
This PR fixes the `rpc_signer.py` and `wallet_signer.py` functional tests on systems where `python3` is not available in the `PATH`, causing the shebang `#!/usr/bin/env python3` to fail.
Here are logs on NetBSD 10.0:
- without this PR:
```
$ python3.12 ./build/test/functional/test_runner.py rpc_signer.py wallet_signer.py
Temporary test directory at /tmp/test_runner_₿_🏃_20241219_160538
Remaining jobs: [rpc_signer.py, wallet_signer.py --descriptors]
1/2 - rpc_signer.py failed, Duration: 1 s
stdout:
2024-12-19T16:05:40.012000Z TestFramework (INFO): PRNG seed is: 1833166631173850775
2024-12-19T16:05:40.012000Z TestFramework (INFO): Initializing test directory /tmp/test_runner_₿_🏃_20241219_160538/rpc_signer_1
2024-12-19T16:05:40.754000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/home/hebasto/dev/bitcoin/test/functional/test_framework/util.py", line 160, in try_rpc
fun(*args, **kwds)
File "/home/hebasto/dev/bitcoin/test/functional/test_framework/coverage.py", line 50, in __call__
return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hebasto/dev/bitcoin/test/functional/test_framework/authproxy.py", line 146, in __call__
raise JSONRPCException(response['error'], status)
test_framework.authproxy.JSONRPCException: RunCommandParseJSON error: process(/home/hebasto/dev/bitcoin/test/functional/mocks/signer.py enumerate) returned 127: env: python3: No such file or directory
(-1)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/hebasto/dev/bitcoin/test/functional/test_framework/test_framework.py", line 135, in main
self.run_test()
File "/home/hebasto/dev/bitcoin/build/test/functional/rpc_signer.py", line 72, in run_test
assert_raises_rpc_error(-1, 'fingerprint not found',
File "/home/hebasto/dev/bitcoin/test/functional/test_framework/util.py", line 151, in assert_raises_rpc_error
assert try_rpc(code, message, fun, *args, **kwds), "No exception raised"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hebasto/dev/bitcoin/test/functional/test_framework/util.py", line 166, in try_rpc
raise AssertionError(
AssertionError: Expected substring not found in error message:
substring: 'fingerprint not found'
error message: 'RunCommandParseJSON error: process(/home/hebasto/dev/bitcoin/test/functional/mocks/signer.py enumerate) returned 127: env: python3: No such file or directory
'.
2024-12-19T16:05:40.756000Z TestFramework (INFO): Stopping nodes
2024-12-19T16:05:40.873000Z TestFramework (WARNING): Not cleaning up dir /tmp/test_runner_₿_🏃_20241219_160538/rpc_signer_1
2024-12-19T16:05:40.873000Z TestFramework (ERROR): Test failed. Test logging available at /tmp/test_runner_₿_🏃_20241219_160538/rpc_signer_1/test_framework.log
2024-12-19T16:05:40.873000Z TestFramework (ERROR):
2024-12-19T16:05:40.873000Z TestFramework (ERROR): Hint: Call /home/hebasto/dev/bitcoin/test/functional/combine_logs.py '/tmp/test_runner_₿_🏃_20241219_160538/rpc_signer_1' to consolidate all logs
2024-12-19T16:05:40.873000Z TestFramework (ERROR):
2024-12-19T16:05:40.873000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024-12-19T16:05:40.873000Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
2024-12-19T16:05:40.873000Z TestFramework (ERROR):
stderr:
Remaining jobs: [wallet_signer.py --descriptors]
2/2 - wallet_signer.py --descriptors failed, Duration: 1 s
stdout:
2024-12-19T16:05:40.014000Z TestFramework (INFO): PRNG seed is: 7530764367977090686
2024-12-19T16:05:40.014000Z TestFramework (INFO): Initializing test directory /tmp/test_runner_₿_🏃_20241219_160538/wallet_signer_0
2024-12-19T16:05:40.526000Z TestFramework (ERROR): JSONRPC error
Traceback (most recent call last):
File "/home/hebasto/dev/bitcoin/test/functional/test_framework/test_framework.py", line 135, in main
self.run_test()
File "/home/hebasto/dev/bitcoin/build/test/functional/wallet_signer.py", line 66, in run_test
self.test_valid_signer()
File "/home/hebasto/dev/bitcoin/build/test/functional/wallet_signer.py", line 83, in test_valid_signer
self.nodes[1].createwallet(wallet_name='hww', disable_private_keys=True, descriptors=True, external_signer=True)
File "/home/hebasto/dev/bitcoin/test/functional/test_framework/test_node.py", line 935, in createwallet
return self.__getattr__('createwallet')(wallet_name, disable_private_keys, blank, passphrase, avoid_reuse, descriptors, load_on_startup, external_signer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hebasto/dev/bitcoin/test/functional/test_framework/coverage.py", line 50, in __call__
return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hebasto/dev/bitcoin/test/functional/test_framework/authproxy.py", line 146, in __call__
raise JSONRPCException(response['error'], status)
test_framework.authproxy.JSONRPCException: RunCommandParseJSON error: process(/home/hebasto/dev/bitcoin/test/functional/mocks/signer.py enumerate) returned 127: env: python3: No such file or directory
(-1)
2024-12-19T16:05:40.528000Z TestFramework (INFO): Stopping nodes
2024-12-19T16:05:40.645000Z TestFramework (WARNING): Not cleaning up dir /tmp/test_runner_₿_🏃_20241219_160538/wallet_signer_0
2024-12-19T16:05:40.646000Z TestFramework (ERROR): Test failed. Test logging available at /tmp/test_runner_₿_🏃_20241219_160538/wallet_signer_0/test_framework.log
2024-12-19T16:05:40.646000Z TestFramework (ERROR):
2024-12-19T16:05:40.646000Z TestFramework (ERROR): Hint: Call /home/hebasto/dev/bitcoin/test/functional/combine_logs.py '/tmp/test_runner_₿_🏃_20241219_160538/wallet_signer_0' to consolidate all logs
2024-12-19T16:05:40.646000Z TestFramework (ERROR):
2024-12-19T16:05:40.646000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024-12-19T16:05:40.646000Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
2024-12-19T16:05:40.646000Z TestFramework (ERROR):
stderr:
TEST | STATUS | DURATION
rpc_signer.py | ✖ Failed | 1 s
wallet_signer.py --descriptors | ✖ Failed | 1 s
ALL | ✖ Failed | 2 s (accumulated)
Runtime: 1 s
```
- with this PR:
```
$ python3.12 ./build/test/functional/test_runner.py rpc_signer.py wallet_signer.py
Temporary test directory at /tmp/test_runner_₿_🏃_20241219_160011
Remaining jobs: [rpc_signer.py, wallet_signer.py --descriptors]
1/2 - rpc_signer.py passed, Duration: 2 s
Remaining jobs: [wallet_signer.py --descriptors]
2/2 - wallet_signer.py --descriptors passed, Duration: 3 s
TEST | STATUS | DURATION
rpc_signer.py | ✓ Passed | 2 s
wallet_signer.py --descriptors | ✓ Passed | 3 s
ALL | ✓ Passed | 5 s (accumulated)
Runtime: 3 s
```
ACKs for top commit:
maflcko:
lgtm ACK d38ade7bc4
stickies-v:
ACK d38ade7bc4 . I have a minor concern about `sys.executable` not being guaranteed to return a valid Python path, but this patch seems good enough as is so no blocker.
Tree-SHA512: 91fe0abc0b7e2b599c5562f8b225ba60f94c5bd6baa77d8df532155ef4d3ef6c6a862cee7f4a7f565ed4bb3251adcda813b4a4f79be1aa6a4ffdfda8b4e53415
733fa0b0a1 miner: never create a template which exploits the timewarp bug (Antoine Poinsot)
Pull request description:
This check was introduced in #30681 but only enabled for testnet4. To avoid potentially creating an invalid block template if a soft fork to fix the timewarp attack were to activate in the future, we should have this check on all networks. It also seems wise for our miner to not support it whether or not a soft fork activates to fix it at the consensus level.
ACKs for top commit:
Sjors:
ACK 733fa0b0a1
fjahr:
utACK 733fa0b0a1
TheCharlatan:
ACK 733fa0b0a1
Tree-SHA512: 9b3bc8b26a57f93425b17dda80bcfac4ecb750a3d26bc3eb8df619135634e369ac15982fac0c9770b1df207bd2e418ffe02a98f37968f024e55262d97715a4f5
Allowing to call the check from the test_runner allows for consistent
error messages and better UX by having a single test_runner for all
checks.
This requires the env var to be set for now. The next commit makes the
commit range optional.
c31166ac77 cmake: Fail if `Libmultiprocess` is missing when `WITH_MULTIPROCESS=ON` (Hennadii Stepanov)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/31708:
```
$ cmake -B build -DWITH_MULTIPROCESS=ON
-- The CXX compiler identification is GNU 13.3.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found SQLite3: /usr/include (found suitable version "3.45.1", minimum required is "3.7.17")
CMake Error at CMakeLists.txt:146 (find_package):
By not providing "FindLibmultiprocess.cmake" in CMAKE_MODULE_PATH this
project has asked CMake to find a package configuration file provided by
"Libmultiprocess", but CMake did not find one.
Could not find a package configuration file provided by "Libmultiprocess"
with any of the following names:
LibmultiprocessConfig.cmake
libmultiprocess-config.cmake
Add the installation prefix of "Libmultiprocess" to CMAKE_PREFIX_PATH or
set "Libmultiprocess_DIR" to a directory containing one of the above files.
If "Libmultiprocess" provides a separate development package or SDK, be
sure it has been installed.
-- Configuring incomplete, errors occurred!
```
ACKs for top commit:
vasild:
ACK c31166ac77
ryanofsky:
Code review ACK c31166ac77
TheCharlatan:
ACK c31166ac77
Tree-SHA512: 503e6d7ff253c9ae95b13ff0649af7db97c74a97c04ca6fe88130defae251b94bfe9f4466300d3fab16397c7c8346b392a80a7b80a2d6517464a4eabe3aa40db
a4df12323c doc: add release notes (Sjors Provoost)
c75872ffdd test: use DIFF_1_N_BITS in tool_signet_miner (tdb3)
4131f322ac test: check difficulty adjustment using alternate mainnet (Sjors Provoost)
c4f68c12e2 Use OP_0 for BIP34 padding in signet and tests (Sjors Provoost)
cf0a62878b rpc: add next to getmininginfo (Sjors Provoost)
2d18a078a2 rpc: add target and bits to getchainstates (Sjors Provoost)
f153f57acc rpc: add target and bits to getblockchaininfo (Sjors Provoost)
baa504fdfa rpc: add target to getmininginfo result (Sjors Provoost)
2a7bfebd5e Add target to getblock(header) in RPC and REST (Sjors Provoost)
341f932516 rpc: add GetTarget helper (Sjors Provoost)
d20d96fa41 test: use REGTEST_N_BITS in feature_block (tdb3)
7ddbed4f9f rpc: add nBits to getmininginfo (Sjors Provoost)
ba7b9f3d7b build: move pow and chain to bitcoin_common (Sjors Provoost)
c4cc9e3e9d consensus: add DeriveTarget() to pow.h (Sjors Provoost)
Pull request description:
**tl&dr for consensus-code only reviewers**: the first commit splits `CheckProofOfWorkImpl()` in order to create a `DeriveTarget()` helper. The rest of this PR does not touch consensus code.
There are three ways to represent the proof-of-work in a block:
1. nBits
2. Difficulty
3. Target
The latter notation is useful when you want to compare share work against either the pool target (to get paid) or network difficulty (found an actual block). E.g. for difficulty 1 which corresponds to an nBits value of `0x00ffff`:
```
share hash: f6b973257df982284715b0c7a20640dad709d22b0b1a58f2f88d35886ea5ac45
target: 7fffff0000000000000000000000000000000000000000000000000000000000
```
It's immediately clear that the share is invalid because the hash is above the target.
This type of logging is mostly done by the pool software. It's a nice extra convenience, but not very important. It impacts the following RPC calls:
1. `getmininginfo` displays the `target` for the tip block
2. `getblock` and `getblockheader` display the `target` for a specific block (ditto for their REST equivalents)
The `getdifficulty` method is a bit useless in its current state, because what miners really want to know if the difficulty for the _next_ block. So I added a boolean argument `next` to `getdifficulty`. (These values are typically the same, except for the first block in a retarget period. On testnet3 / testnet4 they change when no block is found after 20 minutes).
Similarly I added a `next` object to `getmininginfo` which shows `bit`, `difficulty` and `target` for the next block.
In order to test the difficulty transition, an alternate mainnet chain with 2016 blocks was generated and used in `mining_mainnet.py`. The chain is deterministic except for its timestamp and nonce values, which are stored in `mainnet_alt.json`.
As described at the top, this PR introduces a helper method `DeriveTarget()` which is split out from `CheckProofOfWorkImpl`. The proposed `checkblock` RPC in #31564 needs this helper method internally to figure out the consensus target.
Finally, this PR moves `pow.cpp` and `chain.cpp` from `bitcoin_node` to `bitcoin_common`, in order to give `rpc/util.cpp` (which lives in `bitcoin_common`) access to `pow.h`.
ACKs for top commit:
ismaelsadeeq:
re-ACK a4df12323c
tdb3:
code review re ACK a4df12323c
ryanofsky:
Code review ACK a4df12323c. Only overall changes since last review were dropping new `gettarget` method and dropping changes to `getdifficulty`, but there were also various internal changes splitting and rearranging commits.
Tree-SHA512: edef5633590379c4be007ac96fd1deda8a5b9562ca6ff19fe377cb552b5166f3890d158554c249ab8345977a06da5df07866c9f42ac43ee83dfe3830c61cd169
fa3c787b62 fuzz: Abort when global PRNG is used before SeedRand::ZEROS (MarcoFalke)
Pull request description:
This adds one more check to abort when global PRNG is used before SeedRand::ZEROS in fuzz tests. This is achieved by carving out the two remaining uses. First, `g_rng_temp_path_init`, and second the random fallback for `RANDOM_CTX_SEED`, which isn't used in fuzz tests anyway.
Requested in https://github.com/bitcoin/bitcoin/pull/31521#issuecomment-2554669015
Can be tested by reverting fadd568931 and observing an abort when running the `utxo_total_supply` fuzz target.
ACKs for top commit:
marcofleon:
ACK fa3c787b62
hodlinator:
re-ACK fa3c787b62
ryanofsky:
Code review ACK fa3c787b62. This adds a new check to make that sure that RNG is never seeded during fuzzing after the RNG has been used. Together with existing checks which ensure RNG can only be seeded with zeroes during fuzzing, and that RNG must was seeded at some point if used after fuzzing, this implies it must have been seeded by zeros before being used.
Tree-SHA512: 2614928d31c310309bd9021b3e5637b35f64196020fbf9409e978628799691d0efd3f4cf606be9a2db0ef60b010f890c2e70c910eaa2934a7fbf64cd1598fe22
223081ece6 scripted-diff: rename block and undo functions for consistency (Lőrinc)
baaa3b2846 refactor,blocks: remove costly asserts and modernize affected logs (Lőrinc)
fa39f27a0f refactor,blocks: deduplicate block's serialized size calculations (Lőrinc)
dfb2f9d004 refactor,blocks: inline `WriteBlockToDisk` (Lőrinc)
42bc491465 refactor,blocks: inline `UndoWriteToDisk` (Lőrinc)
86b85bb11f bench: add SaveBlockBench (Lőrinc)
34f9a0157a refactor,bench: rename bench/readblock.cpp to bench/readwriteblock.cpp (Lőrinc)
Pull request description:
`UndoWriteToDisk` and `WriteBlockToDisk` were delegating a subset of their functionality to single-use methods that didn't optimally capture a meaningful chunk of the algorithm, resulting in calculating things twice (serialized size, header size).
This change inlines the awkward methods (asserting that all previous behavior was retained), and in separate commits makes the usages less confusing.
Besides making the methods slightly more intuitive, the refactorings reduce duplicate calculations as well.
The speed difference is insignificant for now (~0.5% for the new `SaveBlockToDiskBench`), but are a cleanup for follow-ups such as https://github.com/bitcoin/bitcoin/pull/31539
ACKs for top commit:
ryanofsky:
Code review ACK 223081ece6. Since last review, "Save" was renamed to "Write", uint32_t references were dropped, some log statements and comments were improved as suggested, and a lot of tweaks made to commits and commit messages which should make this easier to review.
hodlinator:
ACK 223081ece6
TheCharlatan:
ACK 223081ece6
andrewtoth:
ACK 223081ece6
Tree-SHA512: 951bc8ad3504c510988afd95c561e3e259c6212bd14f6536fe56e8eb5bf5c35c32a368bbdb1d5aea1acc473d7e5bd9cdcde02008a148b05af1f955e413062d5c
e94c9d1712 [doc] Amend notes on benchmarking (dergoegge)
Pull request description:
This gives some more context on the motivation and larger picture of benchmarks.
ACKs for top commit:
l0rinc:
ACK e94c9d1712
instagibbs:
reACK e94c9d1712
darosior:
reACK e94c9d1712
brunoerg:
reACK e94c9d1712
Tree-SHA512: 2cbf51f283f2efc0938e7021ae48db51fe89caf9ef9780821e99fa745dff839e2d202ca956ce6cc48b8319db304069728e77883feefe486264eb1783a0610c93
This is similar in structure to test/functional/wallet_multisig_descriptor_psbt.py
both in code and concept. It should serve as some integration testing for
Miniscript descriptors, and also documents a simple multisig that starts as 4-of-4
and decays to 3-of-4, 2-of-4, and finally 1-of-4 at block heights (I think in the
real world aligning this to halvenings would be nice).
For blocks 1 through 15 the script_BIP34_coinbase_height appends OP_1
to comply with BIP34 and avoid bad-cb-length.
This is inconsistent with BlockAssembler::CreateNewBlock() which adds
OP_0 instead.
The utxo_total_supply fuzzer and MinerTestingSetup::Block also use OP_0.
Changing it is required to import the test vectors in the next commit.
It also ensures the test vectors can be regenerated using the CPU miner
at https://github.com/pooler/cpuminer without patches (it uses OP_0).
The same helper is used by the signet miner, so this will impact newly
bootstrapped signets.
66d21d0eb6 qa: check parsed multipath descriptors dont share references (Antoine Poinsot)
09a1875ad8 miniscript: Make NodeRef a unique_ptr (Ava Chow)
9ccb46f91a miniscript: Ensure there is no NodeRef copy constructor or assignment operator (Ava Chow)
6d11c9c60b descriptor: Add proper Clone function to miniscript::Node (Ava Chow)
Pull request description:
Multipath descriptors requires performing a deep copy, so a Clone function that does that is added to miniscript::Node instead of the current shallow copy.
Fixes#30864
ACKs for top commit:
darosior:
re-ACK 66d21d0eb6
hodlinator:
re-ACK 66d21d0eb6🚀
brunoerg:
reACK 66d21d0eb6
Tree-SHA512: bea017497ed3cc0b2da2df7e3ccae1fa4a324769b7da1065963da131235bd8bfdcdfe337a3fabbb3ab4d3822611211fca6a9772e18e2ee1cb3d853e831ff6f88
Split CheckProofOfWorkImpl() to introduce a helper function
DeriveTarget() which converts the nBits value to the target.
The function takes pow_limit as an argument so later commits can
avoid having to pass ChainstateManager through the call stack.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Multipath descriptors requires performing a deep copy, so a Clone
function that does that is added to miniscript::Node instead of the
current shallow copy.
Co-Authored-By: Antoine Poinsot <darosior@protonmail.com>
d44626a9c2 depends: Override default build type for `libevent` (Hennadii Stepanov)
Pull request description:
This PR fixes a regression for the `libevent` package introduced in https://github.com/bitcoin/bitcoin/pull/29835.
The `libevent` package defaults to the "Release" build type, which overrides our per-build-type optimization flags with `-O3`.
To prevent this behavior, set `CMAKE_BUILD_TYPE` to "None", consistent with how other packages are handled.
ACKs for top commit:
fanquake:
ACK d44626a9c2
Tree-SHA512: 77abd2e28ad8dda86eb0548d8e49ecf23bac08a2e07dc35c71db62539aa659d471c863d361534c3cf693f9945c1b4f12de7e04eef05d11f8cc5e86d6eff5242d
fa80a7dac4 test: Bump sync_mempools timeout in p2p_1p1c_network.py (MarcoFalke)
1111b0ac19 ci: Add missing --combinedlogslen to test-each-commit task (MarcoFalke)
Pull request description:
This should address the two issues that happened in https://github.com/bitcoin/bitcoin/actions/runs/12885576442/job/35924329657?pr=25832#step:6:7601:
* The combined log isn't printed on a test failure.
* The timeout is too strict for the GHA virtual machines.
For reference, the output was:
```
...
149/315 - rpc_blockchain.py --v2transport passed, Duration: 10 s
150/315 - p2p_addrfetch.py passed, Duration: 1 s
151/315 - p2p_1p1c_network.py failed, Duration: 31 s
stdout:
2025-01-21T12:05:49.465000Z TestFramework (INFO): PRNG seed is: 6581340712385622842
2025-01-21T12:05:49.466000Z TestFramework (INFO): Initializing test directory /tmp/test_runner_₿_🏃_20250121_120233/p2p_1p1c_network_207
2025-01-21T12:05:52.408000Z TestFramework (INFO): Fill mempools with large transactions to raise mempool minimum feerates
2025-01-21T12:05:52.408000Z TestFramework (INFO): Fill the mempool until eviction is triggered and the mempoolminfee rises
2025-01-21T12:05:59.692000Z TestFramework (INFO): Pre-send some transactions to nodes
2025-01-21T12:06:00.203000Z TestFramework (INFO): Submit full packages to node0
2025-01-21T12:06:00.220000Z TestFramework (INFO): Wait for mempools to sync
2025-01-21T12:06:20.384000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/home/runner/work/bitcoin/bitcoin/test/functional/test_framework/test_framework.py", line 135, in main
self.run_test()
File "/home/runner/work/bitcoin/bitcoin/build/test/functional/p2p_1p1c_network.py", line 153, in run_test
self.sync_mempools(timeout=20)
File "/home/runner/work/bitcoin/bitcoin/test/functional/test_framework/test_framework.py", line 803, in sync_mempools
raise AssertionError("Mempool sync timed out after {}s:{}".format(
AssertionError: Mempool sync timed out after 20s:
...
ACKs for top commit:
l0rinc:
utACK fa80a7dac4
glozow:
ACK fa80a7dac4
Tree-SHA512: b326b7906b184fb47abc50d0d7ec91a6c90d324997f2abc40f156f588090e8d89bd8486bb8950cac604e77b1b336142a47b53ad463b2670d81222814eeb313d4
The `libevent` package defaults to the "Release" build type, which
overrides our per-build-type optimization flags with `-O3`.
To prevent this behavior, set `CMAKE_BUILD_TYPE` to "None", consistent
with how other packages are handled.
c0045e6cee Add test for multipath miniscript expression (David Gumberg)
b4ac48090f descriptor: Use InferXOnlyPubkey for miniscript XOnly pubkey from script (Ava Chow)
4c50c21f6b tests: Check ExpandPrivate matches for both parsed descriptors (Ava Chow)
092569e858 descriptor: Try the other parity in ConstPubkeyProvider::GetPrivKey() (Ava Chow)
Pull request description:
When a `ConstPubkeyProvider` is xonly, the stored pubkey does not necessarily have the correct parity bit. `ToPrivateString()` is correctly handling this by looking up the keys for both parity bits, but `GetPrivKey` does not. This results in not finding the private key when it is actually available if its pubkey has the other parity bit value.
To fix this, this key finding is refactored into `GetPrivKey()` so that its behavior is corrected, and `ToPrivateString()` is changed to use `GetPrivKey()` as well.
Additionally, the descriptor test checks are updated to include a check for `ExpandPrivate()` to verify that both the parsed public and private descriptors produce `SigningProvider`s with the same contents.
Fixes#31589
ACKs for top commit:
Pttn:
ACK c0045e6cee
davidgumberg:
utACK c0045e6cee
kevkevinpal:
Concept and Code review ACK [c0045e6](c0045e6cee)
furszy:
ACK c0045e6cee
theStack:
re-ACK c0045e6cee
rkrux:
Concept ACK c0045e6cee
Tree-SHA512: 3dcf2a802b996e0680a3f819075e5a689eb22e484c81ea79b40ec04197ee4ba3f6b9c87c45dfe8a847c9b805b2fd0fad77ffb92a93e65dc3aad74d69d9e3d97f
Make the block db open RAII style by calling it in the BlockManager
constructor.
Before this change the block tree db was needlessly re-opened during
startup when loading a completed snapshot. Improve this by letting the
block manager open it on construction. This also simplifies the test
code a bit.
The change was initially motivated to make it easier for users of the
kernel library to instantiate a BlockManager that may be used to read
data from disk without loading the block index into a cache.
This commit is done in preparation for the next commit. Here, the block
tree options are moved to the blockmanager options and the block tree is
instantiated through a helper method of the BlockManager, which is
removed again in the next commit.
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
01df180bfb depends: add mold & ld.lld to gen_id (fanquake)
d032ac8063 depends: add *FLAGS to gen_id (fanquake)
Pull request description:
The depends cache should be busted when flags change, the same as any other tooling change. I'd also like to start passing `*FLAGS` into depends inside the Guix env, which, without this change, doesn't bust the cache.
ACKs for top commit:
hebasto:
ACK 01df180bfb.
Tree-SHA512: 3809359fe763af9dde484e0c6bd3e262c4c09fcbe2f96ccf64194f5f9f840f5476b9c9929cf7bda7b8c14efeffd369cdb8c233625b79a944e1380df20698246f
faaabfaea7 ci: Bump centos stream 10 (MarcoFalke)
Pull request description:
This is a follow-up to fa47baa03b, which bumped the gcc version to avoid a warning bloat in the CI log. However, it is also required to bump python3, see https://github.com/bitcoin/bitcoin/issues/31476#issue-2735206340
> This will uncover an issue in the centos task that the correct python version is missing. I guess this should be fixed by installing and activating an acceptable python version.
Instead of bumping the packages individually in centos stream 9, just bump to stream 10.
ACKs for top commit:
fanquake:
ACK faaabfaea7
Tree-SHA512: a564ff3a2a0dc4d39874e87540e67072f293bbed82c8eca22266fcadc16c5571e0e41d38576a63e466b64d13f7e3acbd95be10cf2420de33127aa420eca3b928
6e29de2101 ci: Supply `platform` argument to docker commands. (David Gumberg)
Pull request description:
I ran into this issue when following the instructions in `ci/README.md` for running CI locally.
Newer versions of docker require a `--platform` argument when building from a platform-specific image that differs from the host platform, I'm not sure when this change took place, but trying to build any of the cross-platform CI images on Docker 27.5.0 fails in the following manner:
```console
$ # From ci/README.md
$ env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64/v4) and no specific platform was requested
Creating docker.io/arm64v8/debian:bookworm container to run in
+ docker build --file $BITCOIN_SRC/ci/test_imagefile --build-arg CI_IMAGE_NAME_TAG=docker.io/arm64v8/debian:bookworm --build-arg FILE_ENV=./ci/test/00_setup_env_arm.sh --label=bitcoin-ci-test --tag=ci_arm_linux $BITCOIN_SRC
[+] Building 0.6s (2/2) FINISHED docker:default
=> [internal] load build definition from test_imagefile 0.0s
=> => transferring dockerfile: 600B 0.0s
=> WARN: InvalidDefaultArgInFrom: Default value for ARG ${CI_IMAGE_NAME_TAG} results in empty or invalid base image name (line 8) 0.0s
=> ERROR [internal] load metadata for docker.io/arm64v8/debian:bookworm 0.5s
------
> [internal] load metadata for docker.io/arm64v8/debian:bookworm:
------
1 warning found (use docker --debug to expand):
- InvalidDefaultArgInFrom: Default value for ARG ${CI_IMAGE_NAME_TAG} results in empty or invalid base image name (line 8)
test_imagefile:8
--------------------
6 |
7 | ARG CI_IMAGE_NAME_TAG
8 | >>> FROM ${CI_IMAGE_NAME_TAG}
9 |
10 | ARG FILE_ENV
--------------------
ERROR: failed to solve: docker.io/arm64v8/debian:bookworm: failed to resolve source metadata for docker.io/arm64v8/debian:bookworm: no match for platform in manifest: not found
```
This branch fixes this by setting the `--platform` argument of `docker build` and `docker run` with an environment variable `CI_IMAGE_PLATFORM` for each platform specific job, and `linux/{$cpuarch}` for any native jobs.
Thi
## Steps to reproduce
1. Install relevant dependencies, on Ubuntu:
```bash
sudo apt install bash docker.io python3 qemu-user-static
```
2. Run one of the platform-specific CI images, e.g.:
```bash
env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
```
ACKs for top commit:
maflcko:
lgtm ACK 6e29de2101
hebasto:
ACK 6e29de2101
Tree-SHA512: 81b9fa8ec1f3d21619d37d864047c8d7917ef2c8536851f80facf7f1973dfe14628d7755f12d2a9c6edebb6cb16877c582d4d41cdab52b73b23c44f08c6e6b30
- Move "Clang" and "GCC" from the table to a new "Compiler" heading,
indicating either is required.
- Move "CMake" into the required table.
- Move "Python" into the optional table.
- Merge the optional dependencies into one table. Removed sub-headers
are put into parentheses behind the dependency name in the first
column.
- Replace the whitespace in the "Minimum required" column of "qrencode"
with "N/A" for consistency.
- Add missing info for the "systemtap" dependency.
- Add context for "Linux Kernel" dependency in parentheses behind the
dependency name.
Base the unit test directly on `VersionBitsConditionChecker`, slightly
improving coverage, in particular adding coverage for the the logic
regarding setting the TOP_BITS.
Replaces State() (which returned ACTIVE/STARTED/etc) with IsActiveAfter()
which just returns a bool, as this was all State was actually used
for. Drops Mask(), which was only used in tests and can be replaced with
`1<<bit`, and also drops StateSinceHeight() and Statistics(), which are
now only used internally for Info().
Rather than having the RPC code have knowledge about how BIP9 is
implemented, create a reporting function in the versionbits code, and
limit the RPC code to coverting the result of that into the appropriate
output for getblocktemplate.
Rather than having the RPC code have knowledge about how BIP9 is
implemented, create a reporting function in the versionbits code, and
limit the RPC code to coverting the result of that into Univalue/JSON.
Rather than having the rule change period/threshold be constant for all
potential deployments on a chain, have it be specific to the deployment
itself. This both matches history (BIP 9 specified a 2016 block period
and 1916 block threshold; BIP 91 specified a 336 block period and 269
block threshold; and BIP 341 specified a 2016 block period and 1815
block threshold), and allows the code to be simplified, as only the
BIP9Deployment structure is needed, not the full Consensus::Params
structure.
31a0e5f090 depends: Qt 5.15.16 (fanquake)
Pull request description:
Contains a handful of miscellaneous bug fixes.
We can drop a few of our patches.
See https://github.com/qt/qtbase/compare/v5.15.14-lts-lgpl...v5.15.16-lts-lgpl.
ACKs for top commit:
hebasto:
ACK 31a0e5f090.
TheCharlatan:
ACK 31a0e5f090
Tree-SHA512: dd7b3332dd6ecb95189bc72364883425fb8869e03850791d2ee92555a37046c7abaaee16575a0396f1ce9674856b894563dbd36868c2cf46f9fee48028fd967b
fabefd9915 ci: Turn CentOS task into native one (MarcoFalke)
Pull request description:
Cross-compiling to `i686-pc-linux-gnu` on CentOS in the CI is mostly redundant with the `ci/test/00_setup_env_i686_multiprocess.sh` task (albeit it using clang):
35bf426e02/ci/test/00_setup_env_i686_multiprocess.sh (L9-L12)
One task seems sufficient as a sanity check, given that there seems to be no real demand for this architecture anyway.
Turning the task into a native one makes it possible to run the task natively on aarch64 or any other supported architecture.
Also, remove the install of the `lbzip2` package, which is unused since commit a46065e36c
Also, remove the `CONFIG_SHELL` env var, which is unused since the cmake migration. (`CONFIG_SHELL` in depends is still kept).
ACKs for top commit:
davidgumberg:
ACK fabefd9915
hebasto:
ACK fabefd9915, tested locally on Ubuntu 24.10.
Tree-SHA512: 5a7b3131b379d11ef602e5821165861e9bdf61d605014bf8fcb33b8e12d8823450798af2d3289b96f7559dfa47b839bf939ddc0b3725efecfeac7ae570a981e7
160c27ec07 doc: Update dependency installation for Debian/Ubuntu and CI (Adlai Chandrasekhar)
Pull request description:
This is similar to the recently-pushed 8d20348 and results in slightly cleaner systems for future Debian/Ubuntu builds.
According to the description for pkg-config, "pkgconf is a replacement for pkg-config, providing additional functionality while also maintaining compatibility. This package only provides a dependency link to the pkgconf package to help with package upgrades. It can be safely removed."
Thus the relevant sections of `doc/build-unix.md` and `depends/README.md` are updated.
ACKs for top commit:
maflcko:
weak ACK 160c27ec07
fanquake:
ACK 160c27ec07 - seems correct for modern distro versions, and using pkgconf on older ones also seems to work fine.
Tree-SHA512: fadeffe464073df91b706e30f560bfe332ce676521cc5d2044d3bf499f08d986ccaab0a10dd1178f626a90bbac3a4f8c445fe4f8e3a63960721664a247b758f7
This guards against 2 processes running with separate datadirs but the same
blocksdir.
It's not likely to happen currently, but may be more relevant in the future
with applications using the kernel.
Note that the kernel does not currently do any dir locking, but it should.
A subsequent commit will add a .lock file to this dir at startup, meaning that
the blocksdir is never empty by the time the xor key is being read/written.
Ignore all hidden files when determining if this is the first run.
f6a6d91205 test: add check for getting SigningProvider for a CPubKey (Sebastian Falbesoner)
62a95f5af9 test: refactor: move `CreateDescriptor` helper to wallet test util module (Sebastian Falbesoner)
493656763f desc spkm: Return SigningProvider only if we have the privkey (Ava Chow)
Pull request description:
If we know about a pubkey that's in our descriptor, but we don't have the private key, don't return a SigningProvider for that pubkey.
This is specifically an issue for Taproot outputs that use the H point as the resulting PSBTs may end up containing irrelevant information because the H point was detected as a pubkey each unrelated descriptor knew about.
Split from #29675
ACKs for top commit:
fjahr:
ACK f6a6d91205
theStack:
re-ACK f6a6d91205
furszy:
utACK f6a6d91205. Only reviewed the actual change in detail, not the test commit.
Tree-SHA512: 30a196e611a0c5d9ebe5baf6d896caaa6af66f1615463dbb0c31e52604d53cf342922bb9967b3c697b47083d76b0485c77a5f545bd6381247c8bc44321c70f97
b30cc71e85 doc: fix typos (Adlai Chandrasekhar)
Pull request description:
In the unrelated PR #31621 the linter reported a few typos, that are fixed in this commit. I used the "doc" prefix as it only modifies comments, so none of the more significant prefixes seem appropriate.
ACKs for top commit:
maflcko:
lgtm ACK b30cc71e85
Tree-SHA512: 7bba2d928fc0b98f62f96d9abf6dba98f699b386b75730271fa3e7b57a8a220df2265b699007f066e585e1db2ee3cbe5a272b74a8c153f6f8814c01e6de7a3ee
The check for whether the block tree db has been wiped before calling
NeedsRedownload() is confusing. The boolean is set in case of a reindex.
It was originally introduced to guard NeedsRedownload in case of a
reindex in #21009. However NeedsRedownload already returns early if the
chain's tip is not loaded. Since that is the case during a reindex, the
pre-check is redundant.
According to the description for pkg-config, "pkgconf is a
replacement for pkg-config, providing additional functionality
while also maintaining compatibility. This package only provides
a dependency link to the pkgconf package to help with package
upgrades. It can be safely removed."
Thus several scripts and markdown files are updated.
2a92702baf init: Use size_t consistently for cache sizes (TheCharlatan)
65cde3621d kernel: Move default cache constants to caches (TheCharlatan)
8826cae285 kernel: Move non-kernel db cache size constants (TheCharlatan)
e758b26b85 kernel: Move kernel-specific cache size options to kernel (TheCharlatan)
d5e2c4a409 fuzz: Add fuzz test for checked and saturating add and left shift (TheCharlatan)
c03a2795a8 util: Add integer left shift helpers (TheCharlatan)
8bd5f8a38c [refactor] init: Simplify coinsdb cache calculation (TheCharlatan)
5db7d4d3d2 doc: Correct docstring describing max block tree db cache (TheCharlatan)
Pull request description:
Carrying non-kernel related fields in the cache sizes for the indexes is confusing for kernel library users. The cache sizes are set currently with magic numbers in bitcoin-chainstate. The comments for the cache size calculations are not completely clear. The constants for the cache sizes are also currently in `txdb.h`, which is not an ideal place for holding all cache size related constants.
Solve these things by moving the kernel-specific cache size fields to their own struct and moving the constants to either the node or the kernel cache sizes.
This slightly changes the way the cache is allocated if (and only if) the txindex and/or blockfilterindex is used. Since they are now given precedence over the block tree db cache, this results in a bit less cache being allocated to the block tree db, coinsdb and coins caches. The effect is negligible though, i.e. cache sizes with default dbcache reported through the logs are:
master:
```
Cache configuration:
* Using 2.0 MiB for block index database
* Using 56.0 MiB for transaction index database
* Using 49.0 MiB for basic block filter index database
* Using 8.0 MiB for chain state database
* Using 335.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
```
this PR:
```
Cache configuration:
* Using 2.0 MiB for block index database
* Using 56.2 MiB for transaction index database
* Using 49.2 MiB for basic block filter index database
* Using 8.0 MiB for chain state database
* Using 334.5 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
```
---
This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).
ACKs for top commit:
stickies-v:
re-ACK 2a92702baf
ryanofsky:
Code review ACK 2a92702baf. Changes since last review are fixing size options to use size_t instead of int64_t again, simplifying CheckedLeftShift more, and making other minor suggested cleanups
hodlinator:
re-ACK 2a92702baf
Tree-SHA512: 98376eaa0660b1b8c096a5ce1f3e7c8c30e7cd6644de36856c2d3e573108cfc9473c93ebb3952b7881047b5ae6c85c5b096e6726f30f35be58b98eca07c8c785
Add a mock for a simple scriptable UDP server, and use this to test
various code paths (including successful mappings, timeouts and errors)
in the PCP and NATPMP implementations.
86d7135e36 [p2p] only attempt 1p1c when both txns provided by the same peer (glozow)
f7658d9b14 [cleanup] remove p2p_inv from AddTxAnnouncement (glozow)
063c1324c1 [functional test] getorphantxs reflects multiple announcers (glozow)
0da693f7e1 [functional test] orphan handling with multiple announcers (glozow)
b6ea4a9afe [p2p] try multiple peers for orphan resolution (glozow)
1d2e1d709c [refactor] move creation of unique_parents to helper function (glozow)
c6893b0f0b [txdownload] remove unique_parents that we already have (glozow)
163aaf285a [fuzz] orphanage multiple announcer functions (glozow)
22b023b09d [unit test] multiple orphan announcers (glozow)
96c1a822a2 [unit test] TxOrphanage EraseForBlock (glozow)
04448ce32a [txorphanage] add GetTx so that orphan vin can be read (glozow)
e810842acd [txorphanage] support multiple announcers (glozow)
62a9ff1870 [refactor] change type of unique_parents to Txid (glozow)
6951ddcefd [txrequest] GetCandidatePeers (glozow)
Pull request description:
Part of #27463.
(Transaction) **orphan resolution** is a process that kicks off when we are missing UTXOs to validate an unconfirmed transaction. We currently request missing parents by txid; BIP 331 also defines a way to [explicitly request ancestors](https://github.com/bitcoin/bips/blob/master/bip-0331.mediawiki#handle-orphans-better).
Currently, when we find that a transaction is an orphan, we only try to resolve it with the peer who provided the `tx`. If this doesn't work out (e.g. they send a `notfound` or don't respond), we do not try again. We actually can't, because we've already forgotten who else could resolve this orphan (i.e. all the other peers who announced the transaction).
What is wrong with this? It makes transaction download less reliable, particularly for 1p1c packages which must go through orphan resolution in order to be downloaded.
Can we fix this with BIP 331 / is this "duct tape" before the real solution?
BIP 331 (receiver-initiated ancestor package relay) is also based on the idea that there is an orphan that needs resolution, but it's just a new way of communicating information. It's not inherently more honest; you can request ancestor package information and get a `notfound`. So ancestor package relay still requires some kind of procedure for retrying when an orphan resolution attempt fails. See the #27742 implementation which builds on this orphan resolution tracker to keep track of what packages to download (it just isn't rebased on this exact branch). The difference when using BIP 331 is that we request `ancpkginfo` and then `pkgtxns` instead of the parent txids.
Zooming out, we'd like orphan handling to be:
- Bandwidth-efficient: don't have too many requests out at once. As already implemented today, transaction requests for orphan parents and regular download both go through the `TxRequestTracker` so that we don't have duplicate requests out.
- Not vulnerable to censorship: don't give up too easily, use all candidate peers. See e.g. https://bitcoincore.org/en/2024/07/03/disclose_already_asked_for/
- Load-balance between peers: don't overload peers; use all peers available. This is also useful for when we introduce per-peer orphan protection, since each peer will have limited slots.
The approach taken in this PR is to think of each peer who announces an orphan as a potential "orphan resolution candidate." These candidates include:
- the peer who sent us the orphan tx
- any peers who announced the orphan prior to us downloading it
- any peers who subsequently announce the orphan after we have started trying to resolve it
For each orphan resolution candidate, we treat them as having "announced" all of the missing parents to us at the time of receipt of this orphan transaction (or at the time they announced the tx if they do so after we've already started tracking it as an orphan). We add the missing parents as entries to `m_txrequest`, incorporating the logic of typical txrequest processing, which means we prefer outbounds, try not to have duplicate requests in flight, don't overload peers, etc.
ACKs for top commit:
marcofleon:
Code review ACK 86d7135e36
instagibbs:
reACK 86d7135e36
dergoegge:
Code review ACK 86d7135e36
mzumsande:
ACK 86d7135e36
Tree-SHA512: 618d523b86e60c3ea039e88326d50db4e55e8e18309c6a20e8f2b10ed9e076f1de0315c335fd3b8abdabcc8b53cbceb66fb59147d05470ea25b83a2b4bd9c877
fabeca3458 refactor: Avoid UB in SHA3_256::Write (MarcoFalke)
fad4032b21 refactor: Drop unused UCharCast (MarcoFalke)
Pull request description:
It is UB to apply a distance to a pointer or iterator further than the
end itself, even if the distance is (partially) revoked later on.
Fix the issue by advancing the data pointer at most to the end.
This fix is required to adopt C++ safe buffers https://github.com/bitcoin/bitcoin/issues/31272.
Also included is a somewhat unrelated commit.
ACKs for top commit:
sipa:
utACK fabeca3458
theuni:
utACK fabeca3458
hebasto:
ACK fabeca3458.
Tree-SHA512: 78c53691322b72c3ba9c25ec94eec275dbbbc3049b0ad45e7d9fb2df0afbbaa905b0d8fa7106a3582f937bb1dc15a7592c4ad2d80fe4cff1062a3acfd3638f08
fa3efb5729 refactor: Introduce struct to hold a runtime format string (MarcoFalke)
fa6adb0134 lint: Remove unused and broken format string linter (MarcoFalke)
fadc6b9bac refactor: Check translatable format strings at compile-time (MarcoFalke)
fa1d5acb8d refactor: Use TranslateFn type consistently (MarcoFalke)
eeee6cf2ff refactor: Delay translation of _() literals (MarcoFalke)
Pull request description:
All translatable format strings are fixed. This change surfaces errors in them at compile-time.
The implementation achieves this by allowing to delay the translation (or `std::string` construction) that previously happened in `_()` by returning a new type from this function. The new type can be converted to `bilingual_str` where needed.
This can be tested by adding a format string error in an original string literal and observing a new compile-time failure.
Fixes https://github.com/bitcoin/bitcoin/issues/30530
ACKs for top commit:
stickies-v:
re-ACK fa3efb5729
ryanofsky:
Code review ACK fa3efb5729. Since last review added TranslateFn commit, clarified FormatStringCheck documentation, dropped redundant `inline` keyword
Tree-SHA512: 28fa1db11e85935d998031347bd519675d75c171c8323b0ed6cdd0b628c95250bb86b30876946cc48840ded541e95b8a152696f9f2b13a5f28f5673228ee0509
This avoids having to rely on implicit casts when passing them to the
various functions allocating the caches.
This also ensures that if the requested amount of db_cache does not fit
in a size_t, it is clamped to the maximum value of a size_t.
Also take this opportunity to make the total amounts of cache in the
chainstate manager a size_t too.
They are not related to the txdb, so a better place for them is the
new kernel and node cache file. Re-use the default amount of kernel
cache for the default node cache.
Carrying non-kernel related fields in the cache sizes for the indexes is
confusing for kernel library users. The cache sizes also are set
currently with magic numbers in bitcoin-chainstate. The comments for the
cache size calculations are also not completely clear.
Solve these things by moving the kernel-specific cache size fields to
their own struct.
This slightly changes the way the cache is allocated if the txindex
and/or blockfilterindex is used. Since they are now given precedence
over the block tree db cache, this results in a bit less cache being
allocated to the block tree db, coinsdb and coins caches. The effect is
negligible though, i.e. cache sizes with default dbcache reported
through the logs are:
master:
Cache configuration:
* Using 2.0 MiB for block index database
* Using 56.0 MiB for transaction index database
* Using 49.0 MiB for basic block filter index database
* Using 8.0 MiB for chain state database
* Using 335.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
this branch:
Cache configuration:
* Using 2.0 MiB for block index database
* Using 56.2 MiB for transaction index database
* Using 49.2 MiB for basic block filter index database
* Using 8.0 MiB for chain state database
* Using 334.5 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
The helpers are used in the following commits to increase the safety of
conversions during cache size calculations.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: stickies-v <stickies-v@protonmail.com>
This brings the format types closer to the standard library types:
* FormatStringCheck corresponds to std::basic_format_string, with
compile-time checks done via ConstevalFormatString
* RuntimeFormat corresponds to std::runtime_format, with no compile-time
checks done.
Also, it documents where no compile-time checks are done.
The linter has many implementation bugs and missing features.
Also, it is completely redundant with FormatStringCheck, which
constructs from ConstevalFormatString or a runtime format string.
The `test_inv_block` sub-test in p2p_tx_download.py has a subtle bug:
the manual msg_inv announcements from peers currently have no effect,
since they don't match the wtxidrelay setting (=true by default for
`P2PInterface` instances) and are hence ignored by the nodes (since
2d282e0c / PR #18044). Though the test still passes, it does so without
the intended scenario of asking an additional peer (triggering the
GETDATA_TX_INTERVAL delay). Fix this by sending the INV message with
MSG_WTX instead of MSG_TX. This increases the test run time by about one
minute.
2ed161c5ce test: avoid generating non-loopback traffic from p2p_dns_seeds.py (Vasil Dimov)
a5746dc559 test: avoid generating non-loopback traffic from feature_config_args.py (Vasil Dimov)
6b3f6eae70 test: avoid generating non-loopback traffic from p2p_seednode.py (Vasil Dimov)
Pull request description:
Avoid generating outbound traffic on a non-loopback interface during tests. Fix all tests, including ones that generate DNS traffic.
---
This is a subset of https://github.com/bitcoin/bitcoin/pull/31349 containing only the changes to the tests, without the CI changes to detect future regressions.
ACKs for top commit:
kevkevinpal:
light code review ACK [2ed161c](2ed161c5ce)
brunoerg:
code review ACK 2ed161c5ce
BrandonOdiwuor:
Code Review ACK 2ed161c5ce
jonatack:
ACK 2ed161c5ce
Tree-SHA512: 34dcd4a4d0c4edaa68cc7263540af01afd6ef6e90fd6a43dcd1e989dbd7d32cb2c24ad9e68fde75866a935e6dbfe10d45c10f0bc674f40f9ac72ef964e5a380a
It is UB to apply a distance to a pointer or iterator further than the
end itself, even if the distance is (partially) revoked later on.
Fix the issue by advancing the data pointer at most to the end.
`p2p_dns_seeds.py` would try to connect to the DNS server configured on
the machine and resolve `dummySeed.invalid`.
To block that configure an unavailable proxy which will be used also to
connect to the name server. The test needs 2 successful connections to
other peers (two Python `P2PInterface`s) and they work in spite of the
unavailable proxy because they are on `127.0.0.1` (`NET_UNROUTABLE`) and
the proxy is not used for that.
`feature_config_args.py` uses a proxy address of `1.2.3.4`. This results
in actually trying to open TCP connections over the internet to
`1.2.3.4:9050`.
The test does not need those to succeed so use `127.0.0.1:1` instead.
Also avoid `-noconnect=0` because that is interpreted as `-connect=1`
which is interpreted as `-connect=0.0.0.1` and a connection to
`0.0.0.1:18444` is attempted.
`p2p_seednode.py` would try to connect to `0.0.0.1` and `0.0.0.2` as
seed nodes. This sends outbound TCP packets on a non-loopback interface
to the default router.
Configure an unavailable proxy for all executions of `bitcoind` during
this test. Also change `0.0.0.1` and `0.0.0.2` because connecting to
them would skip the `-proxy=` setting because for such an address:
* `CNetAddr::IsLocal()` is true, thus
* `CNetAddr::IsRoutable()` is false, thus
* `CNetAddr::GetNetwork()` is `NET_UNROUTABLE`, even though
`CNetAddr::m_net` is `NET_IPV4`.
This speeds up the execution time of `p2p_seednode.py`
from 12.5s to 2.5s.
In almost all cases (the only exception is `getifaddrs`), we know the
size of the data passed into SetSockAddr, so we can check this to be
what is expected.
69e95c2b4f tests: Test cleanup of mkeys from wallets without privkeys (Andrew Chow)
2b9279b50a wallet: Remove unused encryption keys from watchonly wallets (Andrew Chow)
813a16a463 wallet: Add HasCryptedKeys (Andrew Chow)
Pull request description:
An earlier version allowed users to create watchonly wallets (wallets without private keys) that were "encrypted". Such wallets would have a stored encryption keys, but nothing would actually be encrypted with them. This can cause unexpected behavior such as https://github.com/bitcoin-core/gui/issues/772.
We can detect such wallets as they will have the disable private keys flag set, no encrypted keys, and encryption keys. For such wallets, we can remove those encryption keys thereby avoiding any issues that may result from this unexpected situation.
ACKs for top commit:
sipa:
utACK 69e95c2b4f.
laanwj:
Code review re-ACK 69e95c2b4f
furszy:
Code review ACK 69e95c2b4f
Tree-SHA512: 901932cd709c57e66c598f011f0105a243b5a8b539db2ef3fcf370dca4cf35ae09bc1110e8fca8353be470f159468855a4dd96b99bc9c1112adc86ccc50e1b9d
fa029a7878 doc: Clarify min macOS and Xcode version (MarcoFalke)
Pull request description:
Two minor doc fixups:
* Clarify that `macOS 13.0+` means `macOS 13+`, indicating that on any major version, only the latest security release is supported.
* Clarify that the Xcode version was selected based on the minimum required macOS version and the minimum required clang version.
ACKs for top commit:
jarolrod:
ACK fa029a7878
hebasto:
re-ACK fa029a7878.
theuni:
ACK fa029a7878
Tree-SHA512: d34910fcc22e57021d7642938e5886419d2b711e1062cbc4fc3da48baf07377231f9d7b394e22ccb17e830d058c8c797dbd1bbffcc7c8828601bb500e1154a9e
fb37acd932 ci: build msan's libc++ with _LIBCPP_ABI_BOUNDED_* (Vasil Dimov)
Pull request description:
For the task `MSan, depends (Cirrus CI)` we build a custom libc++ for which we already use `-DLIBCXX_HARDENING_MODE=debug`. Compile it also with `_LIBCPP_ABI_BOUNDED_*` to enable further checks.
Docs at: https://libcxx.llvm.org/Hardening.html#abi-options
ACKs for top commit:
maflcko:
review ACK fb37acd932
Tree-SHA512: 7687b47e86c524c947dd4311289cdd9bc3dd25e31e844375781a37c110f8ab65bdfcc485f17fd3b20f070cc93187f0ba2ad45089451220f31309c143bb21cc3f
We use `lld` when cross-compiling for macOS, and it's version should
be tied to LLVM. However someone compiling with GCC and `-fuse-ld=lld`
would not see a cache bust if the LLVM toolchain was updated.
We don't use `mold` directly, but I'm aware of it's usage in
infrastructure, along with depends, used to test the project.
The depends cache should be busted when flags change, the same as any
other tooling change. Id also like to start passing *FLAGS into depends
inside the Guix env, which, without this change, doesn't bust the cache.
e04be3731f init,log: Unify block index and chainstate loading log line (Lőrinc)
Pull request description:
The line has been present since the beginning.
Removed redundant duration as well since it can be recovered from the timestamps.
Example logs before the change:
```
2025-01-07T11:58:33Z Verification progress: 99%
2025-01-07T11:58:33Z Verification: No coin database inconsistencies in last 6 blocks (18905 transactions)
2025-01-07T11:58:33Z block index 31892ms
2025-01-07T11:58:33Z Setting NODE_NETWORK on non-prune mode
```
ACKs for top commit:
maflcko:
lgtm ACK e04be3731f
TheCharlatan:
ACK e04be3731f
danielabrozzoni:
tACK e04be3731f
BrandonOdiwuor:
Code Review ACK e04be3731f
Tree-SHA512: cbe4569a17f56ff23e829b837a083c2f730cc490b47bee3bac12126e2257e0ba9ebe9b4384deb03203a0a60aac3b8d283c5d31a6d0481635ba011ac6e2c61ad1
f93f0c9396 tracing: Rename the `MIN` macro to `_TRACEPOINT_TEST_MIN` in log_raw_p2p_msgs (0xb10c)
Pull request description:
Inspired by: 00c1dbd26d (#31419)
Unless there's a reason we *don't* want the same change here...?
ACKs for top commit:
maflcko:
review ACK f93f0c9396🔶
0xB10C:
tested ACK f93f0c9396
Tree-SHA512: 2af2c21e575f496b966928bcffeb92847d1acab8d5e7442d0e08e27358228df326783eb576f0364001b666e956fd8efde1c50dab67d7750a0a6b65b7acec12ae
8a46286da6 depends: Fix spacing issue (Hennadii Stepanov)
Pull request description:
This PR resolves an issue where a missing space caused the value of the `build_AR` variable to be concatenated with the "NM=" string. This resulted in subsequent calls to `${AR}` and `${NM}` failing.
Here is a diff for the `make -C depends print-build_id DEBUG=1` output:
```diff
@@ -110,50 +110,18 @@
CXX_STANDARD=c++20
END CXX
BEGIN AR
-ar: invalid option -- '='
-Usage: ar [emulation options] [-]{dmpqrstx}[abcDfilMNoOPsSTuvV] [--plugin <name>] [member-name] [count] archive-file file...
- ar -M [<mri-script]
- commands:
- d - delete file(s) from the archive
- m[ab] - move file(s) in the archive
- p - print file(s) found in the archive
- q[f] - quick append file(s) to the archive
- r[ab][f][u] - replace existing or insert new file(s) into the archive
- s - act as ranlib
- t[O][v] - display contents of the archive
- x[o] - extract file(s) from the archive
- command specific modifiers:
- [a] - put file(s) after [member-name]
- [b] - put file(s) before [member-name] (same as [i])
- [D] - use zero for timestamps and uids/gids (default)
- [U] - use actual timestamps and uids/gids
- [N] - use instance [count] of name
- [f] - truncate inserted file names
- [P] - use full path names when matching
- [o] - preserve original dates
- [O] - display offsets of files in the archive
- [u] - only replace files that are newer than current archive contents
- generic modifiers:
- [c] - do not warn if the library had to be created
- [s] - create an archive index (cf. ranlib)
- [l <text> ] - specify the dependencies of this library
- [S] - do not build a symbol table
- [T] - deprecated, use --thin instead
- [v] - be verbose
- [V] - display the version number
- @<file> - read options from <file>
- --target=BFDNAME - specify the target object format as BFDNAME
- --output=DIRNAME - specify the output directory for extraction operations
- --record-libdeps=<text> - specify the dependencies of this library
- --thin - make a thin archive
- optional:
- --plugin <p> - load the specified plugin
- emulation options:
- No emulation specific options
-ar: supported targets: elf64-x86-64 elf32-i386 elf32-iamcu elf32-x86-64 pei-i386 pe-x86-64 pei-x86-64 elf64-little elf64-big elf32-little elf32-big elf64-littleaarch64 elf64-bigaarch64 elf32-littleaarch64 elf32-bigaarch64 elf32-littlearm elf32-bigarm pei-aarch64-little pe-aarch64-little elf64-alpha ecoff-littlealpha elf32-littlearm-fdpic elf32-bigarm-fdpic elf32-hppa-linux elf32-hppa elf64-ia64-little elf64-ia64-big pei-ia64 elf64-loongarch elf32-loongarch pei-loongarch64 elf32-m32r-linux elf32-m32rle-linux elf32-m68k elf32-tradbigmips elf32-tradlittlemips ecoff-bigmips ecoff-littlemips elf32-ntradbigmips elf64-tradbigmips elf32-ntradlittlemips elf64-tradlittlemips elf32-powerpc aixcoff-rs6000 elf32-powerpcle ppcboot elf64-powerpc elf64-powerpcle aixcoff64-rs6000 aix5coff64-rs6000 elf64-littleriscv elf32-littleriscv elf32-bigriscv elf64-bigriscv pei-riscv64-little elf32-s390 elf64-s390 elf32-sh-linux elf32-shbig-linux elf32-sh-fdpic elf32-shbig-fdpic elf32-sparc elf64-sparc pe-bigobj-x86-64 pe-i386 pdb srec symbolsrec verilog tekhex binary ihex plugin
+GNU ar (GNU Binutils for Ubuntu) 2.42
+Copyright (C) 2024 Free Software Foundation, Inc.
+This program is free software; you may redistribute it under the terms of
+the GNU General Public License version 3 or (at your option) any later version.
+This program has absolutely no warranty.
END AR
BEGIN NM
-bash: line 1: --version: command not found
+GNU nm (GNU Binutils for Ubuntu) 2.42
+Copyright (C) 2024 Free Software Foundation, Inc.
+This program is free software; you may redistribute it under the terms of
+the GNU General Public License version 3 or (at your option) any later version.
+This program has absolutely no warranty.
END NM
BEGIN RANLIB
GNU ranlib (GNU Binutils for Ubuntu) 2.42
@@ -321,5 +289,5 @@
NO_HARDEN=
END NO_HARDEN
END ALL
-build_id=b7effe2aa166e73f6d2587fb4805ea1cca4d3f1e5c3aae2cfd59c592816b05e3
+build_id=4173a5f75182c792550652e621f6b4a68cc27c8909385580d4efc7bc7a769f51
make: Leaving directory '/home/hebasto/git/bitcoin/depends'
```
It was accidentally introduced in https://github.com/bitcoin/bitcoin/pull/29249.
ACKs for top commit:
theuni:
Nice catch. utACK 8a46286da6
TheCharlatan:
ACK 8a46286da6
Tree-SHA512: f50f3dea1f5fa545316743e61f69ad1a3b7de674604a560fd2a8d7095788cddfae4f88bee19eb2eed2e27800f94ec12bd8ee7e17d65f2a6839530d3646e5440d
a96b84cb1b fuzz: Abort when calling system time without setting mock time (marcofleon)
ff21870e20 fuzz: Add SetMockTime() to necessary targets (marcofleon)
Pull request description:
This PR expands the `CheckGlobals` utility that was introduced in https://github.com/bitcoin/bitcoin/pull/31486 and should help with fuzz stability (https://github.com/bitcoin/bitcoin/issues/29018).
System time shouldn't be used when running a fuzz test, as it is likely to introduce instability (non-determinism). This PR identifies and fixes the targets that were calling system time without setting mock time at the start of an iteration.
Removing`SetMockTime()` from any one of these targets should result in a crash and a message describing the issue.
ACKs for top commit:
achow101:
ACK a96b84cb1b
dergoegge:
Code review ACK a96b84cb1b
brunoerg:
crACK a96b84cb1b
Tree-SHA512: e093a9feb8a397954f7b1416dfa8790b2733f09d5ac51fda5a9d225a55ebd8f99135aa52bdf5ab531653ad1a3739c4ca2b5349c1d989bb4b009ec8eaad684f7d
fd2d96d908 build, test: Build `db_tests.cpp` regardless of `USE_BDB` (Hennadii Stepanov)
Pull request description:
When the building of `db_tests.cpp` was made conditional on `USE_BDB` in commit a58b719cf7, all `db_tests` were indeed specific to BDB wallets.
However, the tests have since been [extended](ba616b932c) to include SQLite wallets as well.
On the master branch @ 433412fd84, tests specific to SQLite wallets are not built and run if configured with `WITH_BDB=OFF` (the default option).
This PR resolves this issue by guarding BDB-specific code in `db_tests.cpp` and ensuring this source file is compiled regardless of the `WITH_BDB` option.
ACKs for top commit:
achow101:
ACK fd2d96d908
maflcko:
review ACK fd2d96d908🔺
theuni:
utACK fd2d96d908
Tree-SHA512: bd9eddf16af60c568e931467d39e9e23a268e82e367ab630c23ac3cfd37e6007c6d78579b69ccbeebc1911c749cdbe75794fd56d7fbdb30c6fea6d2ab11017a3
589ed1a8ea wallet: migration, avoid loading wallet after failure when it wasn't loaded before (furszy)
Pull request description:
Fixes#31447.
During migration failure, only load wallet back into memory when the wallet was
loaded prior to migration. This fixes the case where BDB is not supported, which
implies that no legacy wallet can be loaded into memory due to the lack of db
writing functionality.
Link to error description https://github.com/bitcoin/bitcoin/issues/31447#issuecomment-2528757140.
This PR also improves migration backup related comments to better document the
current workflow.
ACKs for top commit:
achow101:
ACK 589ed1a8ea
rkrux:
ACK 589ed1a8ea
pablomartin4btc:
tACK 589ed1a8ea
Tree-SHA512: c7a489d2b253c574ee0287b691ebe29fe8d026f659f68a3f6108eca8b4e1e420c67ca7803c6bd70c1e1440791833fabca3afbcf8fe8524c6c9fc08de95b618d0
1ea7e45a1f test: raise explicit error if any of the needed release binaries is missing (Sebastian Falbesoner)
Pull request description:
If the `releases` directory exists, but still only a subset of the necessary previous release binaries are available, the test fails by throwing an exception (sometimes leading to follow-up exceptions like `AssertionError: [node 0] Error: no RPC connection`) and printing out a stack trace, which can be confusing and at a first glance suggests that the node crashed or some alike.
Improve this by checking and printing out *all* of the missing release binaries and failing with an explicit error in this case. Also add an info on how to download previous releases binaries. Noticed while testing #30328.
Can be tested by e.g.
```
$ rm -rf ./releases
$ ./test/get_previous_releases.py -b
$ rm -rf ./releases/v28.0/
$ ./build/test/functional/wallet_migration.py
```
master:
<details>
<summary>Long test fail output</summary>
```
...
2024-12-10T18:48:54.067000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/home/thestack/bitcoin/test/functional/test_framework/test_framework.py", line 590, in start_nodes
node.start(extra_args[i], *args, **kwargs)
File "/home/thestack/bitcoin/test/functional/test_framework/test_node.py", line 257, in start
self.process = subprocess.Popen(self.args + extra_args, env=subp_env, stdout=stdout, stderr=stderr, cwd=cwd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 971, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/lib/python3.10/subprocess.py", line 1863, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/thestack/bitcoin/releases/v28.0/bin/bitcoind'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/thestack/bitcoin/test/functional/test_framework/test_framework.py", line 131, in main
self.setup()
File "/home/thestack/bitcoin/test/functional/test_framework/test_framework.py", line 315, in setup
self.setup_network()
File "/home/thestack/bitcoin/test/functional/test_framework/test_framework.py", line 409, in setup_network
self.setup_nodes()
File "/home/thestack/bitcoin/./build/test/functional/wallet_migration.py", line 54, in setup_nodes
self.start_nodes()
File "/home/thestack/bitcoin/test/functional/test_framework/test_framework.py", line 595, in start_nodes
self.stop_nodes()
File "/home/thestack/bitcoin/test/functional/test_framework/test_framework.py", line 610, in stop_nodes
node.stop_node(wait=wait, wait_until_stopped=False)
File "/home/thestack/bitcoin/test/functional/test_framework/test_node.py", line 396, in stop_node
self.stop(wait=wait)
File "/home/thestack/bitcoin/test/functional/test_framework/test_node.py", line 215, in __getattr__
assert self.rpc_connected and self.rpc is not None, self._node_msg("Error: no RPC connection")
AssertionError: [node 0] Error: no RPC connection
2024-12-10T18:48:54.118000Z TestFramework (INFO): Stopping nodes
Traceback (most recent call last):
File "/home/thestack/bitcoin/./build/test/functional/wallet_migration.py", line 1097, in <module>
WalletMigrationTest(__file__).main()
File "/home/thestack/bitcoin/test/functional/test_framework/test_framework.py", line 159, in main
exit_code = self.shutdown()
File "/home/thestack/bitcoin/test/functional/test_framework/test_framework.py", line 331, in shutdown
self.stop_nodes()
File "/home/thestack/bitcoin/test/functional/test_framework/test_framework.py", line 610, in stop_nodes
node.stop_node(wait=wait, wait_until_stopped=False)
File "/home/thestack/bitcoin/test/functional/test_framework/test_node.py", line 396, in stop_node
self.stop(wait=wait)
File "/home/thestack/bitcoin/test/functional/test_framework/test_node.py", line 215, in __getattr__
assert self.rpc_connected and self.rpc is not None, self._node_msg("Error: no RPC connection")
AssertionError: [node 0] Error: no RPC connection
[node 0] Cleaning up leftover process
...
```
</details>
PR:
```
...
2025-01-01T20:26:27.999000Z TestFramework (INFO): PRNG seed is: 4570383538468804512
2025-01-01T20:26:28.000000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_lz66_zcu
2025-01-01T20:26:28.003000Z TestFramework (ERROR): Binary not found: /home/thestack/bitcoin/releases/v28.0/bin/bitcoind
2025-01-01T20:26:28.003000Z TestFramework (ERROR): Binary not found: /home/thestack/bitcoin/releases/v28.0/bin/bitcoin-cli
2025-01-01T20:26:28.003000Z TestFramework (INFO): Previous releases binaries can be downloaded via `test/get_previous_releases.py -b`.
2025-01-01T20:26:28.003000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/home/thestack/bitcoin/test/functional/test_framework/test_framework.py", line 131, in main
self.setup()
File "/home/thestack/bitcoin/test/functional/test_framework/test_framework.py", line 315, in setup
self.setup_network()
File "/home/thestack/bitcoin/test/functional/test_framework/test_framework.py", line 409, in setup_network
self.setup_nodes()
File "/home/thestack/bitcoin/./build/test/functional/wallet_migration.py", line 50, in setup_nodes
self.add_nodes(self.num_nodes, versions=[
File "/home/thestack/bitcoin/test/functional/test_framework/test_framework.py", line 537, in add_nodes
raise AssertionError("At least one release binary is missing")
AssertionError: At least one release binary is missing
2025-01-01T20:26:28.061000Z TestFramework (INFO): Stopping nodes
...
```
ACKs for top commit:
fjahr:
re-ACK 1ea7e45a1f
kevkevinpal:
ACK [1ea7e45](1ea7e45a1f)
maflcko:
lgtm ACK 1ea7e45a1f
achow101:
ACK 1ea7e45a1f
pablomartin4btc:
tACK 1ea7e45a1f
Tree-SHA512: b621c3ce044ca8fc8715a4f4b1f96a8592a470c319a64444cced9ba692d315cfd4885a066679bf377b19136fa3530d9cff6f18894a45aa9c716d39b12719baa0
On FreeBSD, the `shasum` utility is provided by the `perl5` port, which
is not part of the base system and must be installed separately.
Note that this requirement is currently not documented in
`depends/README.md`.
This change switches to using the `sha256sum` utility, which is included
in the base system.
If the chain advances during a rescan, ScanForWalletTransactions
would previously process the new blocks without adjusting m_last_processed_block,
which would leave the wallet in an inconsistent state temporarily, and could lead
to crashes in the GUI.
Fix this by not rescanning blocks beyond the last_processed_block -
for all blocks beyond that height, there will be pending BlockConnected
notifications that will process them after the rescan is finished.
Co-authored-by: Pablo Greco <psgreco@gmail.com>
When the behavior was changes in a previous commit (caching `GetSerializeSize` and avoiding `AutoFile.tell`), (static)asserts were added to make sure the behavior was kept - to make sure reviewers and CI validates it.
We can safely remove them now.
Logs were also slightly modernized since they were trivial to do.
Co-authored-by: Anthony Towns <aj@erisian.com.au>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
For consistency `UNDO_DATA_DISK_OVERHEAD` was also extracted to avoid the constant's ambiguity.
Asserts were added to help with the review - they are removed in the next commit.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
This change resolves an issue where a missing space caused the value of
the `build_AR` variable to be concatenated with the "NM=" string. This
resulted in subsequent calls to `${AR}` and `${NM}` failing.
Example logs before the change:
```
2025-01-07T11:58:33Z Verification progress: 99%
2025-01-07T11:58:33Z Verification: No coin database inconsistencies in last 6 blocks (18905 transactions)
2025-01-07T11:58:33Z block index 31892ms
2025-01-07T11:58:33Z Setting NODE_NETWORK on non-prune mode
2025-01-07T11:58:33Z block tree size = 878086
2025-01-07T11:58:33Z nBestHeight = 878085
```
Removed redundant duration as well since it can be recovered from the timestamps.
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Similarly, `WriteBlockToDisk` wasn't really extracting a meaningful subset of the `SaveBlockToDisk` functionality, it's tied closely to the only caller (needs the header size twice, recalculated block serializes size, returns multiple branches, mutates parameter).
The inlined code should only differ in these parts (modernization will be done in other commits):
* renamed `blockPos` to `pos` in `SaveBlockToDisk` to match the parameter name;
* changed `return false` to `return FlatFilePos()`.
Also removed remaining references to `SaveBlockToDisk`.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
`UndoWriteToDisk` wasn't really extracting a meaningful subset of the `WriteUndoDataForBlock` functionality, it's tied closely to the only caller (needs the header size twice, recalculated undo serializes size, returns multiple branches, modifies parameter, needs documentation).
The inlined code should only differ in these parts (modernization will be done in other commits):
* renamed `_pos` to `pos` in `WriteUndoDataForBlock` to match the parameter name;
* inlined `hashBlock` parameter usage into `hasher << block.pprev->GetBlockHash()`;
* changed `return false` to `return FatalError`;
* capitalize comment.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
(total_cache / 4) + (1 << 23) is at least 8 MiB and nMaxCoinsDBCache is
also 8 MiB, so the minimum between the two will always be
nMaxCoinsDBCache. This is just a simplification and not changing the
result of the calculation.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
7c123c08dd miner: add package feerate vector to CBlockTemplate (ismaelsadeeq)
Pull request description:
This PR enables `BlockAssembler` to add all selected packages' fee and virtual size to a vector, and then return the vector as a member of `CBlockTemplate` struct.
This PR is the first step in the https://github.com/bitcoin/bitcoin/issues/30392 project.
The packages' vsize and fee are used in #30157 to select a percentile fee rate of the top block in the mempool.
ACKs for top commit:
rkrux:
tACK 7c123c08dd
ryanofsky:
Code review ACK 7c123c08dd. Changes since last review are rebasing due to a test conflict, giving the new field a better name and description, resolving the test conflict, and renaming a lot of test variables. The actual code change is still one-line change.
glozow:
reACK 7c123c08dd
Tree-SHA512: 767b0b3d4273cf1589fd2068d729a66c7414c0f9574b15989fbe293f8c85cd6c641dd783cde55bfabab32cd047d7d8a071d6897b06ed4295c0d071e588de0861
Trying to immediately shut down a node after a startup failure without
waiting for the RPC to be fully up will in most cases just fail and lead
to an RPC error.
Also, it is confusing to sidestep the existing fallback to kill any
leftover nodes on a test failure.
So just rely on the fallback.
Printing to stderr instead of stdout makes the test_runner.py fail on
leftover processes. This is desired and fine, because a leftover process
should only happen on a test failure anyway.
- The package feerates are ordered by the sequence in which
packages are selected for inclusion in the block template.
- The commit also tests this new behaviour.
Co-authored-by: willcl-ark <will@256k1.dev>
For the task `MSan, depends (Cirrus CI)` we build a custom libc++ for
which we already use `-DLIBCXX_HARDENING_MODE=debug`. Compile it also
with `_LIBCPP_ABI_BOUNDED_*` to enable further checks.
Docs at: https://libcxx.llvm.org/Hardening.html#abi-options
`get_socket_inodes()` calls `os.listdir()` and then iterates on the
results using `os.readlink()`. However a file may disappear from the
directory after `os.listdir()` and before `os.readlink()` resulting in a
`FileNotFoundError` exception.
It is expected that this may happen for `bitcoind` which is running and
could open or close files or sockets at any time. Thus ignore the
`FileNotFoundError` exception.
If the `releases` directory exists, but still only a subset of the
necessary previous release binaries are available, the test fails by
throwing an exception (sometimes leading to follow-up exceptions like
"AssertionError: [node 0] Error: no RPC connection") and printing out
a stack trace, which can be confusing and at a first glance suggests
that the node crashed or some alike.
Improve this by checking and printing out *all* of the missing release
binaries and failing with an explicit error in this case. Also add an
info on how to download previous releases binaries.
Noticed while testing #30328.
Can be tested by e.g.
$ ./test/get_previous_releases.py -b
$ rm -rf ./releases/v28.0/
$ ./build/test/functional/wallet_migration.py
3e0a992a3f doc: Clarify comments about endianness after #30526 (Ryan Ofsky)
Pull request description:
This is a documentation-only change following up on suggestions made in the #30526 review.
Motivation for this change is that I was recently reviewing #31583, which reminded me how confusing the arithmetic blob code was and made me want to write better comments.
ACKs for top commit:
achow101:
ACK 3e0a992a3f
TheCharlatan:
ACK 3e0a992a3f
Sjors:
ACK 3e0a992a3f
BrandonOdiwuor:
LGTM ACK 3e0a992a3f
Tree-SHA512: 90d5582a25a51fc406d83ca6b8c4e5e4d3aee828a0497f4b226b2024ff89e29b9b50d0ae8ddeac6abf2757fe78548d58cf3dd54df4b6d623f634a2106048091d
04249682e3 test: use Mining interface in miner_tests (Sjors Provoost)
Pull request description:
Needed for both #31283 and #31564.
By using the Mining interface in `miner_tests.cpp` we increase its coverage in unit tests.
ACKs for top commit:
achow101:
ACK 04249682e3
ryanofsky:
Code review ACK 04249682e3, just minor suggested changes (renames, comments, BOOST_REQUIREs) since last review and some more extra clarifications and checks added to the CreateNewBlock_validity test. The CreateNewBlock_validity changes seem clear and easy to understand now.
vasild:
ACK 04249682e3
tdb3:
ACK 04249682e3
Tree-SHA512: 2761cb7555d759670e40d8f37b96a079f8e12a588ac43313b9e63c69afd478321515873a8896ea56784f0100dac4476b0c0e0ef8b5418f8aea24d9965cace4d4
SIGHASH_DEFAULT should be used to indicate SIGHASH_DEFAULT for taproot
inputs, and SIGHASH_ALL for all other input types. This avoids adding an
unnecessary byte to the end of all Taproot signatures added to PSBTs
signed in the GUI.
GetPrivKey() needs the same handling of all keyids for xonly keys that
ToPrivateString() does. Refactor that into GetPrivKey() and reuse it in
ToPrivateString() to resolve this.
fa397177ac util: Add missing types in make_secure_unique (MarcoFalke)
Pull request description:
The return type of `std::forward` depends on the template type, and can not be recovered from the args. Attempting to do so will result in a compile failure. For example, `make_secure_unique<std::string>(std::string{});` does not compile on current master, but does with this pull.
Another example would be `make_secure_unique<std::pair<std::string, std::unique_ptr<int>>>(std::string{}, std::make_unique<int>(21));`
ACKs for top commit:
hodlinator:
ACK fa397177ac
hebasto:
ACK fa397177ac.
TheCharlatan:
ACK fa397177ac
Tree-SHA512: cc902c1111c929a79a6f806b5097136a465e8c727474176bad30a5777ebbb30bedb0bd35273b43bf839d2c00492500ddec724bd17349250451f6b329cb71e6f2
Now that we track all announcers of an orphan, it's not helpful to
consider an orphan provided by a peer that didn't send us this parent.
It can only hurt our chances of finding the right orphan when there are
multiple candidates.
Adapt the 2 tests in p2p_opportunistic_1p1c.py that looked at 1p1c
packages from different peers. Instead of checking that the right peer
is punished, we now check that the package is not submitted. We can't
use the functional test to see that the package was not considered
because the behavior is indistinguishable (except for the logs).
This means we no longer return parents we already have in the
m_unique_parents result from MempoolRejectedTx.
We need to separate the loop that checks AlreadyHave parents from the
loop that adds parents as announcements, because we may do the latter
loop multiple times for different peers.
Add ability to add and track multiple announcers per orphan transaction,
erasing announcers but not the entire orphan.
The tx creation code in orphanage_tests needs to be updated so that each
tx is unique, because the CountOrphans() check assumes that calling
EraseForPeer necessarily means its orphans are deleted.
Unused for now.
Needed for a later commit adding logic to ask the TxRequestTracker for a
list of announcers. These announcers should know the parents of the
transaction they announced.
e8f0e6efaf lint: output-only - Avoid repeated arrows, trim (Hodlinator)
fa9aacf614 lint: Move assertion linter into lint runner (MarcoFalke)
Pull request description:
On failure, this makes the output more consistent with the other linters. Each failure will be marked with an '⚠️ ' emoji and explanation, making it easier to spot.
Also, add --line-number to the filesystem linter.
Also, add newlines after each failing check, to visually separate different failures from each other.
Can be reviewed with:
`--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space`
ACKs for top commit:
davidgumberg:
crACK e8f0e6efaf
hodlinator:
re-ACK e8f0e6efaf
TheCharlatan:
ACK e8f0e6efaf
Tree-SHA512: 9896ff882af9d673ec3e6d2718f877b2fdc8514faba50942fcebacb9de95b1f5b4a5db595e1338fa7f505d06df2df304897350cc55c558c7a85232800e5fd804
0a76c292ac doc: Install `net/py-pyzmq` port on FreeBSD for `interface_zmq.py` (Hennadii Stepanov)
Pull request description:
On FreeBSD, Python's `zmq` module is provided as a separate port.
This PR updates the FreeBSD Build Guide to include this port, enabling the `interface_zmq.py` functional test.
ACKs for top commit:
maflcko:
lgtm ACK 0a76c292ac
vasild:
ACK 0a76c292ac
Tree-SHA512: c13eada3e870149f47348145d6a29f41125ac75efd88eabe6dd2d0429e0377ed280e76a764cfaf627498c1d07b9135a995cc644146fa666bc3bfa0eb2c86e88b
fa0411ee30 ci: Run functional tests in msan task (MarcoFalke)
Pull request description:
Now that the CI machines have a bit more CPU, it seems good to run the functional tests as well under msan. (Also, bump the llvm minor version)
ACKs for top commit:
TheCharlatan:
ACK fa0411ee30
Tree-SHA512: 0dbb2b934485ed54b8caafb5bcd96ddef87088b148dab72a584f721c398bb7fda4095fb720b9ad602dc71f8f40a1e0f29e1b08b2879b78b90b29d46604df36c3
29bca9713d test: fix typo in mempool_ephemeral_dust (epysqyli)
Pull request description:
The `test_node_restart` test in `test/functional/mempool_ephemeral_dust.py` has a repetition in the comment.
ACKs for top commit:
maflcko:
lgtm ACK 29bca9713d
Tree-SHA512: 9828d23ca27e24d64031cd103ce9f9bd9e997ef9b63e6122ad6573073fb3c956964a72cd23dfa5773e52e195eee668762ab470bb540e686a4abd3d7561b40c59
Different values are used for max_ret_len throughout the codebase (e.g., 21, 34, 78).
Theoretically, negative and zero values are also permitted. Let's stress-test those as well.
Co-authored-by: brunoerg <brunoely.gc@gmail.com>
This commit introduces symmetric encode-decode roundtrips for all bases.
Minor refactors were also included:
• Split each base into a separate fuzz target.
• Added symmetric encode-decode roundtrip tests for all bases.
• Removed trim testing for encoded_string, as Base58 does not use whitespace padding.
• Made comparisons stricter by removing unnecessary lowercase conversions for bases that have mixed-case alphabets.
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
faf7eac364 test: clang-format -i src/univalue/test/unitester.cpp (MarcoFalke)
fafa9cc7a5 test: Embed univalue json tests in binary (MarcoFalke)
fa044857ca test: Re-enable univalue test fail18.json (MarcoFalke)
63b6b638aa build: Use character literals for generated headers to avoid narrowing (Lőrinc)
Pull request description:
All other benchmarks and tests have their data embedded, except for the univalue json tests.
This is not only confusing, but also problematic, when the test binary is moved to a different system for testing, because one has to put the test files in the source dir that was used at compile-time.
Fix all issues by embedding them. Also, re-enable a disabled test. Also, fix an issue in the GenerateHeaderFromJson.cmake.
Requested in https://github.com/bitcoin/bitcoin/pull/31434/files#r1876000910
ACKs for top commit:
l0rinc:
ACK faf7eac364
fjahr:
tACK faf7eac364
achow101:
ACK faf7eac364
TheCharlatan:
Re-ACK faf7eac364
hebasto:
Re-ACK faf7eac364. The commit, which modifies CMake scripts, has been replaced with the one from https://github.com/bitcoin/bitcoin/pull/31547, and a formatting commit has been added since my recent [review](https://github.com/bitcoin/bitcoin/pull/31542#pullrequestreview-2517189261).
Tree-SHA512: 72ad202125746f32ccf07411ad3efd2771f27a40525c204cba3c9c83b3ca46d05dd18f6fa5985720c6684bdcbb4c4853fc609ced095ddd1a124832318dd8a55d
Verify that the DescriptorSPKM method `GetSigningProvider` should
only return a signing provider for the passed public key if its
corresponding private key of the passed public key is available.
It is always applied in the same way, no matter how the txindex is
setup. This was no longer accurate after 8181db8, where their
initialization was made independent.
fa83bec78e refactor: Allow std::byte in Read(LE/BE) (MarcoFalke)
Pull request description:
Starting with C++17, `std::byte` is often (not always) a better choice over `uint8_t` for new code.
However, the existing codebase discourages the use of `std::byte`, when helpers such as `ReadLE32` are used. This is because calling code will be cluttered with byte-casts.
Fix it by allowing `std::byte` pointers in `ReadLE32` (and friends).
ACKs for top commit:
sipa:
utACK fa83bec78e
fjahr:
Code review ACK fa83bec78e
theuni:
utACK fa83bec78e
l0rinc:
ACK fa83bec78e
Tree-SHA512: 83604dc9df9ad447ad1b6f81f1e1844554c2c5331fcb78bdba1300e050d9dcbe9cf7a1b2dd250772bb23a8bf02a4ec26441012fe2f4bcc670ef31c15151adb15
This is a documentation-only change following up on suggestions made in the
#30526 review.
Motivation for this change is that I was recently reviewing #31583, which
reminded me how confusing the arithmetic blob code was and made me want to
write better comments.
b6f0593f43 doc: add release note about testmempoolaccept debug-message (Matthew Zipkin)
f9cac63523 test: cover testmempoolaccept debug-message in RBF test (Matthew Zipkin)
f9650e18ea rbf: remove unecessary newline at end of error string (Matthew Zipkin)
221c789e91 rpc: include verbose reject-details field in testmempoolaccept response (Matthew Zipkin)
Pull request description:
Adds a new field `reject-details` in `testmempoolaccept` responses to include `m_debug_message` from `ValidationState`. This string is the complete error message thrown by the mempool in response to `sendrawtransaction`.
The extra verbosity is helpful to consumers of `testmempoolaccept`, which is sort of a debug tool anyway.
example:
>
> {
> "txid": "07d7a59a7bdad4c3a5070659ea04147c9b755ad9e173c52b6a38e017abf0f5b8",
> "wtxid": "5dc243b1b92ee2f5a43134eb3e23449be03d1abb3d7f3c03c836ed0f13c50185",
> "allowed": false,
> "reject-reason": "insufficient fee",
> "reject-details": "insufficient fee, rejecting replacement 07d7a59a7bdad4c3a5070659ea04147c9b755ad9e173c52b6a38e017abf0f5b8; new feerate 0.00300000 BTC/kvB <= old feerate 0.00300000 BTC/kvB"
> }
ACKs for top commit:
rkrux:
re-ACK b6f0593f43
glozow:
ACK b6f0593f43
Tree-SHA512: 340b8023d59cefa84598879c4efdb7c399a3f62da126e87c595523f302e53d33098fc69da9c5f8c92b7580dc75466c66cea372051f935b197265648fe15c43a3
1. Update the documented NetBSD version.
2. Add the optional ZeroMQ package to align the guide with other *BSD
systems.
3. Update the Python version to meet the minimum requirement specified
in https://github.com/bitcoin/bitcoin/pull/30527.
4. Install `net/py-zmq` package to enable the `interface_zmq.py`
functional test.
5. Fix a formatting issue.
366ae00b77 descriptor: Assume `ParseScript` is not being called with a P2WPKH context (brunoerg)
e366408590 descriptor: remove unreachable verification for `pkh` (brunoerg)
Pull request description:
This PR removes an unreachable verification in the `ParseScript` function. It returns an error if `pkh` is not being used at top level, sh, wsh or tr. However, any usage of `pkh` without these contexts will not reach this verification but other ones like "invalid keys" (e.g. `wpkh(pkh(L4gM1FBdyHNpkzsFh9ipnofLhpZRp2mwobpeULy1a6dBTvw8Ywtd))`).
ACKs for top commit:
davidgumberg:
crACK 366ae00b77
achow101:
ACK 366ae00b77
tdb3:
cr ACK 366ae00b77
sipa:
crACK 366ae00b77
Tree-SHA512: b954221a77eed623aeed5eb54f14e82c49540a151d3388831924caa7a784e48a2a975e418af1e13d491e4f8cded3b1797aa39e0e4e39e302a991105df09cdec0
b29d68f942 test: descriptor: fix test for `MaxSatisfactionWeight` (brunoerg)
Pull request description:
To get the maximum size of a satisfaction for a descriptor with no max sig, the parameter `use_max_sig` should be false.
ACKs for top commit:
fjahr:
utACK b29d68f942
achow101:
ACK b29d68f942
tdb3:
re ACK b29d68f942
furszy:
utACK b29d68f942
Tree-SHA512: 8559718d126e60ce21a34183f74d227546108b43e3897e49622d6677ed9e7707caa962fd811d8787bd4dafc48a0e779ef11050d5990293faa2f91ded4aaa4f4b
fa63b8232f test: generateblocks called by multiple threads (MarcoFalke)
fa62c8b1f0 rpc: Extend scope of validation mutex in generateblock (MarcoFalke)
Pull request description:
The mutex (required by TestBlockValidity) must be held after creating the block, until TestBlockValidity is called. Otherwise, it is possible that the chain advances in the meantime and leads to a crash in TestBlockValidity: `Assertion failed: pindexPrev && pindexPrev == chainstate.m_chain.Tip() (validation.cpp: TestBlockValidity: 4338)`
Fixes#31562
ACKs for top commit:
davidgumberg:
reACK fa63b8232f
achow101:
ACK fa63b8232f
ismaelsadeeq:
re-ACK fa63b8232f
mzumsande:
utACK fa63b8232f
Tree-SHA512: 3dfda1192af52546ab11fbffe44af8713073763863f4a63fbcdbdf95b1c6cbeb003dc4b8b29e7ec67362238ad15e07d8f6855832a0c68dc5370254f8cbf9445c
bc43ecaf6d test: add functional test for balance after snapshot completion (Martin Zumsande)
226d03dd61 validation: Send correct notification during snapshot completion (Martin Zumsande)
Pull request description:
After AssumeUtxo background sync is completed in a `ActivateBestChain()` call, the `GetRole()` function called with `BlockConnected()` returns `ChainstateRole::NORMAL` instead of `ChainstateRole::BACKGROUND` for this chainstate.
This would make the wallet (which ignores `BlockConnected` notifications for the background chainstate) process it, change `m_last_block_processed_height` to the (ancient) snapshot height, and display an incorrect balance.
Fix this by caching the chainstate role before calling `ActivateBestChainStep()`.
Also contains a test for this situation that fails on master.
Fixes#31546
ACKs for top commit:
fjahr:
re-ACK bc43ecaf6d
achow101:
ACK bc43ecaf6d
furszy:
Code review ACK bc43ecaf6d
TheCharlatan:
lgtm ACK bc43ecaf6d
Tree-SHA512: c5db677cf3fbab3a33ec127ec6c27c8812299e8368fd3c986bc34d0e515c4eb256f6104479f27829eefc098197de3af75d64ddca636b6b612900a0e21243e4f2
fa0998f0a0 test: Avoid intermittent error in assert_equal(pruneheight_new, 248) (MarcoFalke)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/31446
The test uses the P2P network to sync blocks, which has no inherent guarantee that the blocks are sent and received in the right order, assuming the headers are received first.
This can mean that the first block file is flushed with block at height 249 and block at height 248 is added to the second file. In the log it looks like: `Leaving block file 0: CBlockFileInfo(blocks=249, size=65319, heights=0...249, time=2011-02-02...2024-12-03) (onto 1) (height 248)`. The test assumes that the height of the last pruned block in the first file is 248, expecting it to look like: `Leaving block file 0: CBlockFileInfo(blocks=249, size=65319, heights=0...248, time=2011-02-02...2024-12-09) (onto 1) (height 249) `.
Fix the issue by using a linear dumb sync.
ACKs for top commit:
achow101:
ACK fa0998f0a0
mzumsande:
Code Review ACK fa0998f0a0
i-am-yuvi:
Code Review ACK fa0998f0a0
fjahr:
Code review ACK fa0998f0a0
Tree-SHA512: 59cb4317be6cf9012c9bf7a3e9f5ba96b8b114b30bd2ac42af4fe742cd26a634d685b075f04a84bd782b2a43a342d75bb20a042bd82ad2831dbf844d39517ca2
fa6e599cf9 test: Call generate through test framework only (MarcoFalke)
Pull request description:
The generate RPCs are special in that they should only be called by the test framework itself. This way, they will call the sync function on the nodes, which can avoid intermittent test issues. Also, when the sync is disabled, it will happen explicitly by setting the `sync_fun`.
Apply this rule here, so that all generate calls are written consistently.
ACKs for top commit:
achow101:
ACK fa6e599cf9
rkrux:
tACK fa6e599cf9
hodlinator:
ACK fa6e599cf9
i-am-yuvi:
Tested ACK fa6e599cf9
Tree-SHA512: 31079997f1e17031ecd577904457e0560388aa53cadb1bbda281865271e8e4cf244bc6bf315838a717bf9d6620c201093e30039aa0007bec3629f7ca56abfba3
a2c45ae548 test: report failure during utf8 response decoding (furszy)
Pull request description:
Useful for debugging issues such https://github.com/bitcoin/bitcoin/pull/31241#issuecomment-2462816933.
Prints the entire response content instead of printing only the position of the byte it can't be decoded.
The diff between the error messages can be seen by running the `wallet_migration.py` functional test with the following patch applied:
```
diff --git a/src/wallet/rpc/wallet.cpp b/src/wallet/rpc/wallet.cpp
--- a/src/wallet/rpc/wallet.cpp(revision d65918c5da52c7d5035b4151dee9ffb2e94d4761)
+++ b/src/wallet/rpc/wallet.cpp(date 1731005254673)
@@ -801,7 +801,7 @@
}
UniValue r{UniValue::VOBJ};
- r.pushKV("wallet_name", res->wallet_name);
+ r.pushKV("wallet_name", "\xc3\x28");
if (res->watchonly_wallet) {
r.pushKV("watchonly_name", res->watchonly_wallet->GetName());
}
```
ACKs for top commit:
achow101:
ACK a2c45ae548
theStack:
re-ACK a2c45ae548
rkrux:
tACK a2c45ae548
ismaelsadeeq:
utACK a2c45ae548
Tree-SHA512: 6abb524b5a215c51ec881eea91ebe8174140a88ff3874c8c88676157edae7818801356586a904dbb21b45053183315a6d71dbf917d753611d8e413776b57c484
fa494a1d53 refactor: Specify const in std::span constructor, where needed (MarcoFalke)
faaf4800aa Allow std::span in stream serialization (MarcoFalke)
faa5391f77 refactor: test: Return std::span from StringBytes (MarcoFalke)
fa86223475 refactor: Avoid passing span iterators when data pointers are expected (MarcoFalke)
faae6fa5f6 refactor: Simplify SpanPopBack (MarcoFalke)
facc4f120b refactor: Replace fwd-decl with proper include (MarcoFalke)
fac3a782ea refactor: Avoid needless, unsafe c-style cast (MarcoFalke)
Pull request description:
The `std::span` type is already used in some parts of the codebase, and in most contexts can implicitly convert to and from `Span`. However, the two types are not identical in behavior and trying to use one over the other can result in compile failures in some contexts.
Fix all those issues by allowing either `Span` or `std::span` in any part of the codebase.
All of the changes are also required for the scripted-diff to replace `Span` with `std::span` in https://github.com/bitcoin/bitcoin/pull/31519
ACKs for top commit:
sipa:
utACK fa494a1d53
fjahr:
Code review ACK fa494a1d53
achow101:
ACK fa494a1d53
theuni:
utACK fa494a1d53.
adamandrews1:
utACK fa494a1d53
Tree-SHA512: 9440941823e884ff5d7ac161f58b9a0704d8e803b4c91c400bdb5f58f898e4637d63ae627cfc7330e98a721fc38285a04641175aa18d991bd35f8b69ed1d74c4
e56fc7ce6a rpc: increase the defaults for -rpcthreads and -rpcworkqueue (Vasil Dimov)
Pull request description:
`rpcthreads` was introduced with a default of 4 in 2013 in 21eb5adadb
`rpcworkqueue` was introduced with a default of 16 in 2015 in 40b556d374
Resolves: https://github.com/bitcoin/bitcoin/issues/29386
---
Just bump the ancient default values. There is no perfect default that would fit everybody. This could lead to https://bikeshed.com/
ACKs for top commit:
achow101:
ACK e56fc7ce6a
andrewtoth:
ACK e56fc7ce6a
storopoli:
ACK e56fc7ce6a
tdb3:
ACK e56fc7ce6a
Tree-SHA512: ba3ea7392fda57950daa6b4c4d38ecdef9eebe5e786824d25f8b5cea03e760ffff7f77f3acd8eb6c6178b1e92b282e02cabb940ed7222eec7f73efdb819eef06
ecaa786cc1 rpc: add signet_challenge field to getblockchaininfo and getmininginfo (Ash Manning)
Pull request description:
Signet challenges are currently only available via `getblocktemplate` RPC.
`getblockchaininfo` and `getmininginfo` both provide inadequate information to distinguish signets. Since these are the RPCs used to determine the current network, they should also provide the signet challenge for signets.
Test coverage is included in `test/functional/feature_signet.py`.
ACKs for top commit:
sipa:
utACK ecaa786cc1
achow101:
ACK ecaa786cc1
i-am-yuvi:
Concept ACK ecaa786cc1
Sjors:
ACK ecaa786cc1
zaidmstrr:
Tested ACK [ecaa786](ecaa786cc1)
Tree-SHA512: 9ccf4ae634ee74353a2a895efb881fdc62ae703a134ccd219da2cd6080c7d38319e689054584722457a7cc79004bd6022292a3b0b90eaab9f7003564665e1ea4
b9766c9977 Remove unused variable assignment (yancy)
Pull request description:
The variable is conditionally assigned toward the end of the loop and not used after. It's then set back to its default value at the beginning of the loop.
ACKs for top commit:
theuni:
utACK b9766c9977
achow101:
ACK b9766c9977
hodlinator:
crACK b9766c9977
danielabrozzoni:
code review ACK b9766c9977
murchandamus:
ACK b9766c9977
Tree-SHA512: 45e62b0dd561a473f5ae21bfa91db494940b752886669c85b63a83b68d2a157a301e9450082635e921f3dc812e6307f4ad1674806b74b3e7e0f9f4db543ad93d
On Illumos-based systems, such as OpenIndiana and SmartOS, the
assumption that "the default zone ID of 0 can be omitted for the default
scope" is incorrect. As a result, `getaddrinfo("fe80::1%0", ...)`
returns the `EAI_NONAME` error.
See: https://www.illumos.org/man/3SOCKET/getaddrinfo.
5709718b83 coins: warn on shutdown for big UTXO set flushes (Lőrinc)
Pull request description:
Split out of https://github.com/bitcoin/bitcoin/pull/30611#issuecomment-2549027130
Setting a large `-dbcache` size postpones the index writes until the coins cache size exceeds the specified limit. This causes the final flush after manual termination to seemingly hang forever (e.g. tens of minutes for 20 GiB); Now that the `dbcache` upper cap has been lifted, this will become even more apparent, so a warning will be shown when large UTXO sets are flushed (currently >1 GiB), such as:
> 2024-12-18T18:25:03Z Flushed fee estimates to fee_estimates.dat.
> 2024-12-18T18:25:03Z [warning] Flushing large (1 GiB) UTXO set to disk, it may take several minutes
> 2024-12-18T18:25:09Z Shutdown: done
---
You can reproduce it by starting `bitcoind` with a large `-dbcache`:
> mkdir demo && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j$(nproc) && build/src/bitcoind -datadir=demo **-dbcache=10000**
Waiting until the used memory is over 1 GiB
> 2024-12-18T18:25:02Z UpdateTip: [...] progress=0.069009 cache=**1181.1MiB**(8827981txo)
And cancelling the process from the terminal:
> ^C2024-12-18T18:25:03Z tor: Thread interrupt
> [...]
> 2024-12-18T18:25:03Z **[warning] Flushing large (1 GiB) UTXO set to disk, it may take several minutes*
ACKs for top commit:
sipa:
utACK 5709718b83
tdb3:
re ACK 5709718b83
1440000bytes:
ACK 5709718b83
danielabrozzoni:
tACK 5709718b83
Tree-SHA512: 608cf797de788501ccb2986508c155f5660c5f6f7a414524bfcc2820cfa9ebe3da558d13f2317d1f121a82d49ffe1e711a1152c743c22dab9f9807363f4ed8d5
06443b8f28 net: clarify if we ever sent or received from peer (Sjors Provoost)
1d01ad4d73 net: add LogIP() helper, use in net_processing (Sjors Provoost)
937ef9eb40 net_processing: use CNode::DisconnectMsg helper (Sjors Provoost)
ad224429f8 net: additional disconnection logging (Sjors Provoost)
Pull request description:
While debugging unexpected disconnections, possibly related to #28331, I found some additional [net] logging to be useful.
All cases where we disconnect now come with a log message that has the word `disconnecting`:
* all calls to `CloseSocketDisconnect()` log `disconnecting peer=…`
* wherever we set `pnode->fDisconnect = true;`
* for all `InactivityCheck` cases (which in turn sets `fDisconnect`)
* replaces "dropping" with "disconnecting" in `Network not active, dropping peer=…`
A few exceptions are listed here: https://github.com/bitcoin/bitcoin/pull/28521#discussion_r1890824361
I changed `CloseSocketDisconnect()` to no longer log `disconnecting`, and instead have all the call sites do so.
This PR introduces two helper functions on `CNode`: `DisconnectMsg` and `LogIP`. The second and third commit use these helpers in `net_processing.cpp` so these disconnect messages are more consistent now (e.g. some didn't log the IP). No new messages are added there though.
The `LogIP()` helper is rarely used outside of a disconnect event, but it's available for future use.
Any `LogPrint` this PR touches is replaced with `LogDebug` (superseded by #30750), and every `LogPrintf ` with `LogInfo`.
ACKs for top commit:
davidgumberg:
reACK 06443b8f28
vasild:
ACK 06443b8f28
danielabrozzoni:
ACK 06443b8f28
hodlinator:
ACK 06443b8f28
Tree-SHA512: 525f4c11568616e1d48455a3fcab9e923da7432377fe9230468c15403d2e9b7ce712112df8fbd547cfec01dce0d1f26107cfc1b90f78cfc1fe13e08d57b08464
If AssumeUtxo background sync is completed in this
ActivateBestChain() call, the GetRole() function
returns "normal" instead of "background" for this chainstate.
This would make the wallet (which ignores BlockConnected
notifcation for the background chainstate) process it, change
m_last_block_processed_height, and display an incorrect
balance.
The mutex (required by TestBlockValidity) must be held after creating
the block, until TestBlockValidity is called. Otherwise, it is possible
that the chain advances in the meantime and leads to a crash in
TestBlockValidity:
Assertion failed: pindexPrev && pindexPrev == chainstate.m_chain.Tip() (validation.cpp: TestBlockValidity: 4338)
The diff can be reviewed with the git options
--ignore-all-space --function-context
Setting a large `-dbcache` size postpones the index writes until the coins cache size exceeds the specified limit.
This causes the final flush after manual termination to seemingly hang forever (e.g. tens of minutes for 20 GiB);
Now that the `dbcache` upper cap has been lifted, this will become even more apparent, so a warning will be shown when large UTXO sets are flushed (currently >1 GiB), such as:
> 2024-12-18T18:25:03Z Flushed fee estimates to fee_estimates.dat.
> 2024-12-18T18:25:03Z [warning] Flushing large (1 GiB) UTXO set to disk, it may take several minutes
> 2024-12-18T18:25:09Z Shutdown: done
Note that the related BCLog::BENCH units were also converted to `KiB` from `kB` to unify the bases.
Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
By setting DANGER_DOCKER_BUILD_CACHE_HOST_DIR, the task-specific
docker images built during the CI run can be cached. This allows,
for example, ephemeral CI runners to reuse the docker images (or
layers of it) from earlier runs, by persisting the image cache
before the ephemeral CI runner is shut down. The cache keyed by
`CONTAINER_NAME`.
As --cache-to doesn't remove old cache files, the existing cache
is removed after a successful `docker build` and the newly cached
image is moved to it's location to avoid the cache from growing
indefinitly with old, unused layers.
When --cache-from doesn't find the directory, the cached version is
a cache-miss, or the cache can't be imported for whatever other reason,
it warns and `docker build` continues by building the docker image.
This feature is opt-in. The documentation for the cache type=local
can be found https://docs.docker.com/build/cache/backends/local/
This replaces https://github.com/bitcoin/bitcoin/pull/31377
b8710201fb guix: disable timezone tools & profiling in glibc (fanquake)
23b8a424fb guix: bump glibc 2.31 to 7b27c450c34563a28e634cccb399cd415e71ebfe (fanquake)
Pull request description:
An additional commit has been backported to the 2.31 branch:
https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/release/2.31/master.
Pass `--disable-timezone-tools`: removes `var/profiles/x86_64-linux-gnu/sbin/zdump`.
Pass `--disable-profile`: profiling is disabled by default, but make that explicit.
ACKs for top commit:
theuni:
utACK b8710201fb
hebasto:
ACK b8710201fb.
Tree-SHA512: 0d9a0e7451cc42384bbdd0b46c740c7aa964dc12e3f0376de586bf90e57799ebb04675892861cb38a53b5ca0e265061fa7111596cf1c94171303d0d048785ab4
be1a2e5dfb doc: Install `py3-zmq` port on OpenBSD for `interface_zmq.py` (Hennadii Stepanov)
Pull request description:
On OpenBSD, Python's `zmq` module is provided as a separate [port](https://www.ports.to/path/net/py-zmq,python3.html).
This PR updates the OpenBSD Build Guide to include this port, enabling the `interface_zmq.py` functional test.
Also updates the documented OpenBSD version.
ACKs for top commit:
theStack:
Tested ACK be1a2e5dfb
Tree-SHA512: 4d560385b94e8c7491aa19d2157d8a799617e08136601dc565a909d4c74e12582a1d273bc97ad7c2d0e57c5cf7377560ba02ef58c12f8991652322553740d2ba
e196190a28 cmake: Remove unused `BUILD_TESTING` variable from "dev-mode" preset (Hennadii Stepanov)
Pull request description:
On the master branch @ bb57017b29:
```
$ cmake -B build --preset dev-mode -DWITH_MULTIPROCESS=OFF
<snip>
-- Configuring done (12.0s)
-- Generating done (0.1s)
CMake Warning:
Manually-specified variables were not used by the project:
BUILD_TESTING
-- Build files have been written to: /home/hebasto/git/bitcoin/build
```
This PR resolves the issue.
The removed `BUILD_TESTING` variable is a part of the [`CTest`](https://cmake.org/cmake/help/latest/module/CTest.html) module, which we do not include in the project.
ACKs for top commit:
TheCharlatan:
ACK e196190a28
Tree-SHA512: 8110a0f5bdcdd0844ce7dd75160a61d8b3aff95e12da1ec4d55c56c82da41145736da0fad072adeb97551c99e46683a3493435c3bac7d8e4e62ea6086f60fb7a
Also, extend the pass2.json test to the maximum depth possible. The two
tests are now similar to fail45.json and pass4.json, except for the
string element in the inner-most array.
Also, sort.
Use character literals instead of integer hex values (i.e. `'\x5b','\x0a', ...` instead of `0x5b, 0x0a, ...`) for generated headers.
This avoids C++11 narrowing warnings in a more concise way than using explicit char casts.
Extra whitespace is also removed between elements for brevity.
fadd568931 fuzz: Fix misplaced SeedRand::ZEROS (MarcoFalke)
Pull request description:
After commit fae63bf130 this must be placed even before test_setup. This is nice, because it makes the usage consistently appear in the first line.
The change is moving a `SeedRandomForTest(SeedRand::ZEROS)` to happen earlier. This is fine, because it will either have no effect, or make the code more deterministic, because after commit fae63bf, no other re-seeding other than `ZEROS` can happen in fuzz tests.
ACKs for top commit:
marcofleon:
Re ACK fadd568931
brunoerg:
code review ACK fadd568931
hodlinator:
ACK fadd568931
Tree-SHA512: 54eadf19a1e850157a280fb252ece8797f37a9a50d3b0a01aa2c267bacbe8ef4ddea6cf3faadcbaa4ab9f53148edf08e3cee5dfb3eae928db582adf8373a5206
81cea5d4ee Ensure m_tip_block is never ZERO (Sjors Provoost)
e058544d0e Make m_tip_block an std::optional (Sjors Provoost)
Pull request description:
Suggested in https://github.com/bitcoin/bitcoin/pull/31297#discussion_r1844244309
ACKs for top commit:
fjahr:
re-ACK 81cea5d4ee
tdb3:
code review re ACK 81cea5d4ee
l0rinc:
ACK 81cea5d4ee
Tree-SHA512: 31a75ba29e3d567bab32e4e7925a419d9d7a4d2d85ed1c1012116d8d22adc14d31d5b4ce5f6c499c994188dcd26a01cced05be74f94c892fc90ae17a6783a472
The std::span constructor requires std::ranges::borrowed_range, which
tries to protect against dangling references.
One way to disable the check is to specify the std::span's element type
as const in the constructor call.
Otherwise, a compile error will look like:
include/c++/span: note: candidate constructor not viable: no known conversion from 'std::vector<unsigned char>' to 'const span<unsigned char>' for 1st argument
| span(const span&) noexcept = default;
| ^ ~~~~~~~~~~~
...
include/c++/span: note: candidate template ignored: constraints not satisfied [with _Range = std::vector<unsigned char>]
| span(_Range&& __range)
| ^
include/c++/span: note: because 'std::vector<unsigned char>' does not satisfy 'borrowed_range'
| && (ranges::borrowed_range<_Range> || is_const_v<element_type>)
| ^
include/c++/bits/ranges_base.h: note: because 'std::vector<unsigned char>' does not satisfy '__maybe_borrowed_range'
| = range<_Tp> && __detail::__maybe_borrowed_range<_Tp>;
| ^
include/c++/bits/ranges_base.h: note: because 'is_lvalue_reference_v<std::vector<unsigned char> >' evaluated to false
| = is_lvalue_reference_v<_Tp>
| ^
include/c++/bits/ranges_base.h: note: and 'enable_borrowed_range<remove_cvref_t<vector<unsigned char, allocator<unsigned char> > > >' evaluated to false
| || enable_borrowed_range<remove_cvref_t<_Tp>>;
| ^
include/c++/span: note: and 'is_const_v<element_type>' evaluated to false
| && (ranges::borrowed_range<_Range> || is_const_v<element_type>)
| ^
This is possible and safe, because std::span can implicitly convert into
Span, if needed.
Changing this function is required, because std::span requires the
extent template parameter to be specified as well.
Instead of explicilty specifying them, just let the compiler derive the
template parameters correctly.
Otherwise, there would be a compile error later on:
src/wallet/test/db_tests.cpp:39:37: error: no matching function for call to ‘as_bytes<const char>(<brace-enclosed initializer list>)’
...
/usr/include/c++/11/span:420:5: note: candidate: ...
| as_bytes(span<_Type, _Extent> __sp) noexcept
| ^~~~~~~~
/usr/include/c++/11/span:420:5: note: template argument deduction/substitution failed:
src/wallet/test/db_tests.cpp:39:37: note: couldn’t deduce template parameter ‘_Extent’
| return std::as_bytes<const char>({str.data(), str.size()});
| ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~
For Span, iterators are just raw data pointers. However, for std::span
they are not.
This change makes it explicit where data pointers are expected.
Otherwise, there could be a compile error later on:
No known conversion from 'iterator' (aka '__normal_iterator<const std::byte *, std::span<const std::byte, 18446744073709551615>>') to 'std::byte *'.
For an abstract class, specifying parameters in detail serves no point;
and for the concrete implementation, changing the consensus parameters
between invocations doesn't make sense. So simplify the class by removing
the consensus params from the method arguments, and just make it a member
variable in the concrete object where needed. This also allows dropping
dummy parameters from the unit/fuzz tests.
c991cea1a0 Remove processNewBlock() from mining interface (Sjors Provoost)
9a47852d88 Remove getTransactionsUpdated() from mining interface (Sjors Provoost)
bfc4e029d4 Remove testBlockValidity() from mining interface (Sjors Provoost)
Pull request description:
There are three methods in the mining interface that can be dropped. The Template Provider doesn't need them and other application should probably not use them either.
1. `processNewBlock()` was added in 7b4d3249ce, but became unnecessary with the introduction of interfaces::BlockTemplate::submitSolution in 7b4d3249ce.
Dropping it was suggested in https://github.com/bitcoin/bitcoin/pull/30200#issuecomment-2404460342
2. `getTransactionsUpdated()`: this is used in the implementation of #31003 `waitFeesChanged`. It's not very useful generically because the mempool updates very frequently.
3. `testBlockValidity()`: it might be useful for mining application to have a way to check the validity of a block template they modified, but the Stratum v2 Template Provider doesn't do that, and this method is a bit brittle (e.g. the block needs to build on the tip).
ACKs for top commit:
TheCharlatan:
Re-ACK c991cea1a0
ryanofsky:
Code review ACK c991cea1a0. Since last review, just rebased to avoid conflicts in surrounding code, and edited a commit message
tdb3:
code review ACK c991cea1a0
Tree-SHA512: 2138e54f920b26e01c068b24498c6a210c5c4358138dce0702ab58185d9ae148a18f04c97ac9f043646d40f8031618d80a718a176b1ce4779c237de6fb9c4a67
facb4d010c refactor: Move GuessVerificationProgress into ChainstateManager (MarcoFalke)
Pull request description:
Currently the function is standalone, which means any passed-in data like `TxData` or the block pointer needs to be taken from the `ChainstateManager` and passed in. This is currently verbose and may become even more verbose if the function is reworked in the future. As the function can not be called without a `ChainstateManager` in production code anyway, make it a member function on the class.
ACKs for top commit:
ryanofsky:
Code review ACK facb4d010c. Nice cleanup, that should make this code less awkward to work with
TheCharlatan:
ACK facb4d010c
danielabrozzoni:
reACK facb4d010c
Tree-SHA512: b17977e12cd7c6e308c47e6a1aa920acecd4442696e46d1f30bd7c201e9898ca2d581ff0bf2cc9f7334e146c1b0c50925adb849c8c17f65dcdf6877be1c5f776
processNewBlock was added in 7b4d3249ce, but became unnecessary with the introduction of interfaces::BlockTemplate::submitSolution in 7b4d3249ce.
getTransactionsUpdated() is only needed by the implementation of waitFeesChanged() (not yet part of the interface).
fa9e0489f5 refactor: Use immediate lambda to work around GCC bug 117966 (MarcoFalke)
Pull request description:
Currently the libstdc++ debug mode can only be used with version 11, or 15 (and later), due to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=117966
This seems restrictive.
Add a temporary workaround for now, which makes the global (temporary) `std::span` local to a lambda.
ACKs for top commit:
theuni:
utACK fa9e0489f5
hebasto:
ACK fa9e0489f5, tested on Ubuntu 24.10.
vasild:
ACK fa9e0489f5
Tree-SHA512: 0cc54f089f329592f7a92a6f938b7de46c92d5362615310748225a42789e858e871432721e3101271b00871d523af5fbaadba2f52433fe79e928b1d1253931f6
fae63bf130 fuzz: Clarify that only SeedRandomStateForTest(SeedRand::ZEROS) is allowed (MarcoFalke)
fa18acb457 fuzz: Abort when using global PRNG without re-seed (MarcoFalke)
fa7809aeab fuzz: Add missing SeedRandomStateForTest(SeedRand::ZEROS) (MarcoFalke)
Pull request description:
This is the first step toward improving fuzz stability and determinism (https://github.com/bitcoin/bitcoin/issues/29018).
A fuzz target using the global test-only PRNG will now abort if the seed is re-used across fuzz inputs.
Also, temporarily add `SeedRandomStateForTest(SeedRand::ZEROS)` to all affected fuzz targets. This may slow down the libfuzzer leak detector, but it will disable itself after some time, or it can be disabled explicitly with `-detect_leaks=0`.
In a follow-up, each affected fuzz target can be stripped of the global random use and a local `RandomMixin` (or similar) can be added instead.
(Can be tested by removing any one of the re-seed calls and observing a fuzz abort)
ACKs for top commit:
hodlinator:
ACK fae63bf130
dergoegge:
utACK fae63bf130
marcofleon:
Tested ACK fae63bf130
Tree-SHA512: 4a0db69af7f715408edf4f8b08b44f34ce12ee2c79d33b336ad19a6e6bd079c4ff7c971af0a3efa428213407c1171f4e2837ec6a2577086c2f94cd15618a0892
f86678156a Check leaves size maximum in MerkleComputation (Sjors Provoost)
4d57288246 refactor: use CTransactionRef in submitSolution (Sjors Provoost)
2e81791d90 Drop TransactionMerklePath default position arg (Sjors Provoost)
39d3b538e6 Rename merkle branch to path (Sjors Provoost)
Pull request description:
This PR implements the refactors suggested in https://github.com/bitcoin/bitcoin/pull/30955#pullrequestreview-2354931253.
ACKs for top commit:
tdb3:
code review re-ACK f86678156a
itornaza:
re ACK f86678156a
ryanofsky:
Code review ACK f86678156a only changes since last review are a whitespace change and adding an Assume statement to check for size_t -> uint32_t overflows
Tree-SHA512: 661b5d5d0e24b2269bf33ab1484e37c36e67b32a7796d77ca3b1856d3043378b081ad43c32a8638b46fa8c0de51c823fd9747dd9fc81f958f20d327bf330a47c
52fd1511a7 test: drop scriptPubKeyIn arg from CreateNewBlock (Sjors Provoost)
ff41b9e296 Drop script_pub_key arg from createNewBlock (Sjors Provoost)
7ab733ede4 rpc: rename coinbase_script to coinbase_output_script (Sjors Provoost)
Pull request description:
Providing a script for the coinbase transaction is only done in test code and for (unoptimized) CPU solo mining.
Production miners use the `getblocktemplate` RPC which omits the coinbase transaction entirely from its block template, leaving it to external (pool) software to construct it.
This commit removes the `script_pub_key argument` from `createNewBlock()` in the Mining interface.
A coinbase script can still be passed via `BlockCreateOptions` instead. Tests are modified to do so.
ACKs for top commit:
ryanofsky:
Code review ACK 52fd1511a7. No change since last review other than rebase
TheCharlatan:
Re-ACK 52fd1511a7
vasild:
ACK 52fd1511a7
Tree-SHA512: c4b3a53774d9a5dc90950e77f47a64dbb68f971baffbb9a0d8f59332ef8e52d0c039130c925bde73135b3d0e79e65d91d1df30dc4cff13f32d8a72e5c56669d8
If a reindex was interrupted while it was iterating
through the block files, genesis will already be connected
when the reindex resumes at the next startup.
In this case, a call to ActivateBestChainState() is not only unnecessary,
but it would connect multiple blocks without applying
-assumevalid, which is much slower.
This is because assumevalid requires us to have a header above
the minimum chainwork, but that header is unknown to us if it's in
a later blockfile not indexed yet.
This simplifies the code. The only reason to call ActivateBestChain()
here is to allow the main init thread to finish startup in a case of
-reindex. In this situation no second chainstate can exist anyway
because -reindex would have deleted any snapshot chainstate earlier.
This could change behavior slightly if -loadblocks was used when there is a
snapshot chainstate. In this case, there is no reason to call
ActivateBestChain() for that chainstate here - it will be called in
ImportBlocks() after all blocks have been indexed.
fa0e30b93a fuzz: Fix test_runner error reporting (MarcoFalke)
Pull request description:
The error reporting is confusing, because right now it prints:
https://cirrus-ci.com/task/4846031060336640?logs=ci#L4931
```
...
Traceback (most recent call last):
File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 411, in <module>
main()
File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 199, in main
run_once(
File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 376, in run_once
assert len(done_stat) == 1
^^^^^^^^^^^^^^^^^^^
AssertionError
```
This is harmless, but confusing.
Fix it by collecting statistics only when the program has not aborted. (Can be reviewed with `--color-moved=dimmed-zebra`)
Also, reword the error message to align it with error messages in other test_runners in this repo.
ACKs for top commit:
dergoegge:
utACK fa0e30b93a
brunoerg:
code review ACK fa0e30b93a
marcofleon:
Tested ACK fa0e30b93a. Prints out the error for the target that crashed. Much clearer than the current error message.
Tree-SHA512: 5e8d3fc0e4837b3264ff0c3cb322fe7fe2ec7af48d35e2a14f82080d03ace793963c3314611b0a170a38e200497d7ba703d9c35c9a7ed3272d93e43f0f0e4c2b
2b9ff4a66d build: use `-mbig-obj` for mingw-w64 Debug builds (fanquake)
Pull request description:
Windows cross builds using `-O0` (`-DCMAKE_BUILD_TYPE=Debug`) currently fail to compile, as some objects have too many sections. As a convenience, add `-mbig-obj` to our compile flags when using the `Debug` build type, so that if someone tries to build this way, it will work.
This would also be needed if we switched the depends flags to -O0. (maybe in #29796).
`-mbig-obj`
> On PE/COFF target this option forces the use of big object
> file format, which allows more than 32768 sections.
Closes#28109. Seems unlikely that we are going to break up the relevant object files, and the main issue is still the inclusion of Boost.
ACKs for top commit:
theuni:
utACK 2b9ff4a66d
hebasto:
ACK 2b9ff4a66d, tested in the following scenarios:
Tree-SHA512: 9ad36de172629a8b7e5371fe3cd75ac2f3c29856040569052cc59e42825eec9121e012dd2178e00b163173c98e78f79dd16b8cee2c93daa2ee0d7e99799325cd
a10bb400e8 depends: Fix CXXFLAGS on NetBSD (Hennadii Stepanov)
Pull request description:
This PR corrects an issue where `CXXFLAGS` were mistakenly overridden by `CFLAGS`. This behaviour was introduced in 7e7b3e42fa (from https://github.com/bitcoin/bitcoin/pull/22380).
On the master branch:
```
$ gmake --no-print-directory -C depends print-x86_64_netbsd_CXXFLAGS
x86_64_netbsd_CXXFLAGS=-pipe -std=c11
```
With this PR:
```
$ gmake --no-print-directory -C depends print-x86_64_netbsd_CXXFLAGS
x86_64_netbsd_CXXFLAGS=-pipe -std=c++20
```
ACKs for top commit:
theuni:
utACK a10bb400e8
Tree-SHA512: 0c842db2965ebb0a58693394715922810235d9e5f2a7416fe258eb252dbd68ec04f90a0f7948abe938caf94a9194cca7deb53a08335c4404cce3a40c5cb44944
46e207d329 cmake: Link `bitcoin_consensus` as a library (Hennadii Stepanov)
Pull request description:
The [`TARGET_OBJECTS`](https://cmake.org/cmake/help/latest/manual/cmake-generator-expressions.7.html#genex:TARGET_OBJECTS) generator expression was introduced in the staging branch when we aimed to build the libbitcoinconsensus shared library. However, `bitcoin_consensus` is a `STATIC` library, not an `OBJECT` library.
This change updates the build system to link `bitcoin_consensus` normally to `test_bitcoin`, resolving [linking issues](https://github.com/bitcoin/bitcoin/issues/31456#issuecomment-2538798107) when building with clang-cl.
ACKs for top commit:
TheCharlatan:
ACK 46e207d329
theuni:
utACK 46e207d329
Tree-SHA512: b5400be8e8350f80c9fc8b66c4a22032a51578e409eb1817309116fbf0bddeb5fcadd5fda685c98859730ee6cc904adb29d54207387732c8b574a1feb2be906f
3353d4a5e9 depends: Ignore prefix directory on OpenBSD (Hennadii Stepanov)
Pull request description:
On OpenBSD, the prefix directory is named as follows:
```
$ gmake --no-print-directory -C depends print-x86_64_openbsd_prefix
x86_64_openbsd_prefix=/home/hebasto/dev/bitcoin/depends/amd64-unknown-openbsd7.6
```
This name does not match any pattern in `depends/.gitignore`.
This PR resolves this issue.
ACKs for top commit:
tdb3:
ACK 3353d4a5e9
theuni:
utACK 3353d4a5e9
theStack:
Tested ACK 3353d4a5e9🐟
Tree-SHA512: 82dfff1af974aa43c21e5e5a4483256d5ab4efdf1a15073fb864e635eff52eb8414346cda125f097af59e3342ac031a52683529f4e64df9fc60c8783fcd85e74
To avoid future code changes from reintroducing the ambiguity fixed
by the previous commit, mark m_tip_block private and Assume that
it's not set to uint256::ZERO.
Belt and suspenders for future code changes.
Currently this function is only called from TransactionMerklePath() which sets leaves to the block transactions, so the Assume always holds.
The TARGET_OBJECTS generator expression was introduced in the staging
branch when we aimed to build the libbitcoinconsensus shared library.
However, `bitcoin_consensus` is a STATIC library, not an OBJECT library.
This change updates the build system to link `bitcoin_consensus`
normally to `test_bitcoin`, resolving linking issues when building with
clang-cl.
The variable is conditionally assigned toward the end of the loop and
not used after. It's then set back to its default value at the beginning
of the loop.
1dd3af8fbc Add release note for #31223 (Martin Zumsande)
997757dd2b test: add functional test for -port behavior (Martin Zumsande)
0e2b12b92a net, init: derive default onion port if a user specified a -port (Martin Zumsande)
Pull request description:
This resolves#31133 (setups with multiple local nodes each using a different `-port` no longer working with v28.0, see the issue description for more details) by deriving the default onion listening port to be the value specified by `-port` incremented by 1 (idea by vasild / laanwj).
Note that with this fix, the chosen `-port` values of two local nodes cannot be adjacent, otherwise there will be port collisions again.
From the discussion in the linked issue, this was the most popular option, followed by doing nothing and telling affected users to change their setups to use `-bind` instead of `-port`. But more opinions are certainly welcome!
I think that if we decide to do something about the problem described in the issue, we should do so soon (in 28.1.), so I opened this PR.
Fixes#31133
ACKs for top commit:
achow101:
ACK 1dd3af8fbc
laanwj:
Tested ACK 1dd3af8fbc
tdb3:
Code review ACK 1dd3af8fbc
Tree-SHA512: 37fda2b23bbedcab5df3a401cf5afce66ae5318fb78f9660f83e3fd075b528e8156d7a0903f9a12ffe97ab5d83860587116b74af28670a1f4c2f0d1be4999f40
- No empty line separating errors and arrows ("^^^"). Keeping them together signals they are related.
- No empty line separating error message and linter failure line (not completely empty, it contains several spaces left over from Rust multi-line literal).
- Keep the linter description on the same line as the failure line, otherwise it looks like it's a description for the following step.
Windows cross builds using `-O0` currently fail to compile, as some
objects have too many sections. As a convenience, add `-mbig-obj` to
our compile flags when using the `Debug` build type, so that if someone
tries to build this way, it will work.
This would also be needed if we switched the depends flags to -O0.
`-mbig-obj`
> On PE/COFF target this option forces the use of big object
> file format, which allows more than 32768 sections.
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
fa47baa03b ci: Bump centos gcc (MarcoFalke)
Pull request description:
Currently the centos stream9 CI task is using gcc-11. This is fine, because this is also the minimum supported.
However:
* There is already a CI task that is checking the minimum supported version: 62bd61de11/ci/test/00_setup_env_native_previous_releases.sh (L11-L12)
* The CI log is a bit useless, because it is mostly just `#warning _FORTIFY_SOURCE > 2 is treated like 2 on this platform [-Werror=cpp]`. This makes it harder to spot real warnings, such as https://github.com/bitcoin/bitcoin/issues/31476
Fix both issues by using gcc-12.
ACKs for top commit:
hebasto:
ACK fa47baa03b.
Tree-SHA512: 573618efc949437d33365a24f77a26a9b68457f7fb9bd603ee92bc5f17fec73ccba114cafb900eddee3531af47508ce5c246def93268787cdfa2b99e6f45a13d
5cd9e95eea depends: update capnproto to 1.0.2 (fanquake)
Pull request description:
This fixes compilation on FreeBSD:
```bash
-- Build files have been written to: /tmp/cirrus-ci-build/bitcoin-core/depends/work/build/x86_64-unknown-freebsd14.0/native_capnp/1.0.1-867405dd2c4
Building native_capnp...
gmake[1]: Entering directory '/tmp/cirrus-ci-build/bitcoin-core/depends/work/build/x86_64-unknown-freebsd14.0/native_capnp/1.0.1-867405dd2c4'
[ 1%] Building CXX object src/kj/CMakeFiles/kj.dir/array.c++.o
[ 2%] Building CXX object src/kj/CMakeFiles/kj.dir/cidr.c++.o
/tmp/cirrus-ci-build/bitcoin-core/depends/work/build/x86_64-unknown-freebsd14.0/native_capnp/1.0.1-867405dd2c4/src/kj/cidr.c++:112:71: error: member access into incomplete type 'const struct sockaddr_in6'
otherBits = reinterpret_cast<const struct sockaddr_in6*>(addr)->sin6_addr.s6_addr;
^
/tmp/cirrus-ci-build/bitcoin-core/depends/work/build/x86_64-unknown-freebsd14.0/native_capnp/1.0.1-867405dd2c4/src/kj/cidr.c++:112:51: note: forward declaration of 'sockaddr_in6'
otherBits = reinterpret_cast<const struct sockaddr_in6*>(addr)->sin6_addr.s6_addr;
^
/tmp/cirrus-ci-build/bitcoin-core/depends/work/build/x86_64-unknown-freebsd14.0/native_capnp/1.0.1-867405dd2c4/src/kj/cidr.c++:123:63: error: member access into incomplete type 'const struct sockaddr_in'
&reinterpret_cast<const struct sockaddr_in*>(addr)->sin_addr.s_addr);
^
/tmp/cirrus-ci-build/bitcoin-core/depends/work/build/x86_64-unknown-freebsd14.0/native_capnp/1.0.1-867405dd2c4/src/kj/cidr.c++:123:44: note: forward declaration of 'sockaddr_in'
&reinterpret_cast<const struct sockaddr_in*>(addr)->sin_addr.s_addr);
^
/tmp/cirrus-ci-build/bitcoin-core/depends/work/build/x86_64-unknown-freebsd14.0/native_capnp/1.0.1-867405dd2c4/src/kj/cidr.c++:133:69: error: member access into incomplete type 'const struct sockaddr_in6'
otherBits = reinterpret_cast<const struct sockaddr_in6*>(addr)->sin6_addr.s6_addr;
^
/tmp/cirrus-ci-build/bitcoin-core/depends/work/build/x86_64-unknown-freebsd14.0/native_capnp/1.0.1-867405dd2c4/src/kj/cidr.c++:133:49: note: forward declaration of 'sockaddr_in6'
otherBits = reinterpret_cast<const struct sockaddr_in6*>(addr)->sin6_addr.s6_addr;
^
3 errors generated.
```
See: 1c19c362b4.
ACKs for top commit:
kevkevinpal:
Concept ACK [5cd9e95](5cd9e95eea)
theuni:
utACK 5cd9e95eea
ryanofsky:
Code review ACK 5cd9e95eea. Downloaded the file and checked the hash. Also followed theuni's lead and looked at the source changes which were very minor. It did look like thousands of lines changed in the autotools build, but this should not affect us as we are using the cmake build.
Tree-SHA512: 5d78887a9e950c8532c427b17969128de0c6d466ec5ffba85241457e8e19673c22ddb3493cdfce5086f57ba760eac5e91f703992b2f70f2a7c82ba885255279c
62b2d23edb wallet: Migrate non-HD keys to combo() descriptor (Ava Chow)
Pull request description:
Non-HD keys do not have an HD seed ID associated with them, so if this value is the null value (all 0s), then we should not perform any seed ID comparison that would result in excluding the keys from combo() migration.
This changes the migration of non-HD wallets (or blank wallets with imported private keys) to make a single combo() descriptors for the non-HD/imported keys, rather than pk(), pkh(), sh(wpkh()), and wpkh() descriptors for the keys.
Implements https://github.com/bitcoin/bitcoin/pull/31374#discussion_r1876650074
ACKs for top commit:
laanwj:
Concept and code review ACK 62b2d23edb
brunoerg:
code review ACK 62b2d23edb
furszy:
Nice catch. ACK 62b2d23edb
theStack:
ACK 62b2d23edb
rkrux:
tACK 62b2d23edb
Tree-SHA512: 86a80b7dcc1598ab18068a2572ff4b4920b233178b760f7b76c5b21a9e6608005ac872f90e082a8f99b51daab0b049e73e4bee5b8e0b537d56ed0d34122a1f49
On failure, this makes the output more consistent with the other linter.
Each failure will be marked with an '⚠️ ' emoji and explanation, making
it easier to spot.
Also, add --line-number to the filesystem linter.
Also, add newlines after each failing check, to visually separate
different failures from each other.
Can be reviewed with:
"--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space"
e2d3372e55 lint: Disable signature output in git log (Hodlinator)
Pull request description:
Necessary for users that have signature output enabled by default, since the script would stumble on them and error out.
---
### Testing setup
Set local repo config to show signatures in log by default, simulating a user having that setting turned on globally.
```
₿ git config set log.showSignature true
```
### Command under test
```
₿ ( cd ./test/lint/test_runner/ && COMMIT_RANGE='HEAD^..HEAD' cargo run )
```
#### Before
```
...
fatal: invalid object name 'gpg'.
Traceback (most recent call last):
File "/home/hodlinator/bitcoin/test/lint/lint-git-commit-check.py", line 52, in <module>
main()
File "/home/hodlinator/bitcoin/test/lint/lint-git-commit-check.py", line 42, in main
commit_info = check_output(["git", "log", "--format=%B", "-n", "1", hash], text=True, encoding="utf8").splitlines()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/wfbjq35kxs6x83c3ncpfxdyl5gbhdx4h-python3-3.12.6/lib/python3.12/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nix/store/wfbjq35kxs6x83c3ncpfxdyl5gbhdx4h-python3-3.12.6/lib/python3.12/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'log', '--format=%B', '-n', '1', 'gpg: Signature made ons 11 dec 2024 10:46:34 CET']' returned non-zero exit status 128.
^---- ⚠️ Failure generated from lint-git-commit-check.py
...
```
#### After
(No failure generated by *lint-git-commit-check.py*).
ACKs for top commit:
maflcko:
lgtm ACK e2d3372e55
willcl-ark:
ACK e2d3372e55
Tree-SHA512: 584ccece1e6e0f4691683a2b1816eff33b88f48e9ead9272e2dc73ea9c637b182632108fbeddea1ffc8ed6ba5a5838d7eac7a9f33dfda5bdf325dd7a41e43365
b7ec69c25c depends: add -g to *BSD_debug flags (fanquake)
Pull request description:
To match the other HOST_debug_flags. Pulled out of #29796.
ACKs for top commit:
theuni:
utACK b7ec69c25c
Tree-SHA512: 654a6dc2c1e295021380f18565379ccde5c5bebcbb5e48ab0364aa79c6f15d301b4acf058d75629a4b217483c6788a0ecb60560e8701882e09490b92c4c346d0
During migration failure, only load wallet back into memory when the
wallet was loaded prior to migration. This fixes the case where BDB
is not supported, which implies that no legacy wallet can be loaded
into memory due to the lack of db writing functionality.
This commit also improves migration backup related comments to better
document the current workflow.
Co-authored-by: Ava Chow <github@achow101.com>
f6496a8388 guix: disable gcov in base-linux-gcc (fanquake)
Pull request description:
In a `x86_64-linux-gnu` build, this drops:
```bash
x86_64-linux-gnu/bin/x86_64-linux-gnu-gcov
x86_64-linux-gnu/bin/x86_64-linux-gnu-gcov-dump
x86_64-linux-gnu/bin/x86_64-linux-gnu-gcov-tool
x86_64-linux-gnu/lib/gcc/x86_64-linux-gnu/12.4.0: libgcov.a
```
For mingw-w64-gcc, `--disable-gcov` is currently passed for this target in Guix, due to issues with mingw-w64, see
8bed031e58/gnu/packages/gcc.scm (L99-L102). However we'll add it in any case, in case it's re-enabled in future, when the underlying issues are fixed.
ACKs for top commit:
TheCharlatan:
ACK f6496a8388
Tree-SHA512: ad6de53f63e7bb658cac05fb023fb1f8e76103073c7dffb4267412d3046148e1389df8848010128c1bd3d428f05e1587b656ef2cad8c7d9078ebec83a68bad49
c93bf0e6e2 test: Add missing %c character test (Hodlinator)
76cca4aa6f test: Document non-parity between tinyformat and ConstevalFormatstring (Hodlinator)
533013cba2 test: Prove+document ConstevalFormatString/tinyformat parity (Hodlinator)
b81a465995 refactor test: Profit from using namespace + using detail function (Hodlinator)
Pull request description:
Clarifies and puts the extent of parity under test.
Broken out from #30546 based on https://github.com/bitcoin/bitcoin/pull/30546#discussion_r1755013263 and https://github.com/bitcoin/bitcoin/pull/30546#discussion_r1756495304.
ACKs for top commit:
maflcko:
re-ACK c93bf0e6e2 🗜
l0rinc:
ACK c93bf0e6e2
ryanofsky:
Code review ACK c93bf0e6e2. Just a few cleanups tweaking function declarations and commit comments and consolidating some test cases since last review.
Tree-SHA512: 5ecc893b26cf2761c0009861be392ec4c4fceb0ef95052a2f6f9df76b2e459cfb3f9e257f61be07c3bb2ecc6e525e72c5ca853be1f63b70b52785323d3db6b42
988721d37a test: avoid internet traffic in rpc_net.py (Sebastian Falbesoner)
Pull request description:
In order to avoid connecting to the internet in the functional test `rpc_net.py`, specify a non-working proxy (parameter `-proxy=127.0.0.1:1`, same approach as in #31142) for the nodes. There is at least one known instance where this is currently happening on master where a connection attempt to a public IP is made (see also the discussion in #31339):
17834bd197/test/functional/rpc_net.py (L253)
Can be tested by running
```
$ sudo tcpdump -i eth0 host 11.22.33.44
```
both on master and the PR branch and verifying that no packets appear in the tcpdump in the latter anymore.
ACKs for top commit:
achow101:
ACK 988721d37a
tdb3:
ACK 988721d37a
vasild:
ACK 988721d37a
Tree-SHA512: 0f51fedccbfac0f80a7e6f9c5ba9193d0c20b5a788553c7cd7e583225df7b1151b86cd848d6ccf61f7b2de848f0ac98d73d7b5db100aa54fe8cbeeb4c0549106
ec777917d6 test: Fix intermittent issue in wallet_backwards_compatibility.py (Randall Naar)
Pull request description:
When creating and replacing a transaction using `bumpfee`, an async update is sent in the form of the `TransactionAddedToMempool` and `TransactionRemovedFromMempool` signals. When `wallet_backwards_compatibility.py` creates `tx3_id` this way and replaces it with `tx4_id`, the `abandontransaction` rpc is called right after. In some cases the `TransactionAddedToMempool` and `TransactionRemovedFromMempool` is handled after the transaction is abandoned in the wallet, and overwrites the transaction's `abandoned` flag. This PR forces the signals to get handled before `abandontransaction` is called by invoking `self.sync_mempools` which calls `syncwithvalidationinterfacequeue` on every node's rpc connection.
This will mitigate the immediate inconsistency observed with the abandontransaction call, but the potential race conditions between the signals and wallet operations may also be useful to note in a separate issue (if it's okay to not address it in this one).
Fixes#29806
ACKs for top commit:
achow101:
ACK ec777917d6
tdb3:
ACK ec777917d6
Tree-SHA512: e75bc2c1f7fefc4f4910bb353654848fed5661c1436416798a5f4e0c5a76bde15617a5af04c2384464005953326317b8f273039e47508d5124677908cf36d31e
bb7e686341 fuzz: add cstdlib to FuzzedDataProvider (fanquake)
Pull request description:
Same as https://github.com/llvm/llvm-project/pull/113951.
Avoids compile failures under clang-20 & `D_LIBCPP_REMOVE_TRANSITIVE_INCLUDES`:
```bash
In file included from /bitcoin/src/test/fuzz/addition_overflow.cpp:5:
/bitcoin/src/test/fuzz/FuzzedDataProvider.h:209:5: error: use of undeclared identifier 'abort'
209 | abort();
| ^
/bitcoin/src/test/fuzz/FuzzedDataProvider.h:250:5: error: use of undeclared identifier 'abort'
250 | abort();
```
ACKs for top commit:
dergoegge:
utACK bb7e686341
brunoerg:
ACK bb7e686341
Tree-SHA512: 22efd5505273ec7254e8dccbb275e648fe02107397c45eff6752e4a6ea787d9d2e45eb0f2ee309df431e9b92ffd14cbcba4b0f4b11a127664466e20be43c383e
Non-HD keys in legacy wallets without a HD seed ID were being migrated
to separate pk(), pkh(), sh(wpkh()), and wpkh() descriptors for each key.
These could be more compactly represented as combo() descriptors, so
migration should make combo() for them.
It is possible that existing non-HD wallets that were migrated, or
wallets that started blank and had private keys imported into them have
run into this issue. However, as the 4 descriptors produce the same output
scripts as the single combo(), so any previously migrated wallets are
not missing any output scripts. The only observable difference should be
performance related, and the wallet size on disk.
cdd207c0e4 test: add coverage for migrating standalone imported keys (furszy)
297a876c98 test: add coverage for migrating watch-only script (furszy)
932cd1e92b wallet: fix crash during watch-only wallet migration (furszy)
Pull request description:
The crash occurs because we assume the cached scripts structure will not be empty,
but it can be empty for watch-only wallets that start blank.
This also adds test coverage for standalone imported keys, which were also crashing
because pubkey imports are treated the same way as hex script imports through
`importaddress()`.
Testing Notes:
This can be verified by cherry-picking and running any of the test commits on master.
It will crash there but pass on this branch.
ACKs for top commit:
theStack:
re-ACK cdd207c0e4
brunoerg:
reACK cdd207c0e4
achow101:
ACK cdd207c0e4
Tree-SHA512: e05c77cf3e9f35f10f122a73680b3f131f683c56685c1e26b5ffc857f95195b64c8c9d4535960ed3d6f931935aa79b0b1242537462006126bdb68251f0452954
Same as https://github.com/llvm/llvm-project/pull/113951.
Avoids compile failures under clang-20 &
`D_LIBCPP_REMOVE_TRANSITIVE_INCLUDES`:
```bash
In file included from /bitcoin/src/test/fuzz/addition_overflow.cpp:5:
/bitcoin/src/test/fuzz/FuzzedDataProvider.h:209:5: error: use of undeclared identifier 'abort'
209 | abort();
| ^
/bitcoin/src/test/fuzz/FuzzedDataProvider.h:250:5: error: use of undeclared identifier 'abort'
250 | abort();
```
In a `x86_64-linux-gnu` build, this drops:
```bash
x86_64-linux-gnu/bin/x86_64-linux-gnu-gcov
x86_64-linux-gnu/bin/x86_64-linux-gnu-gcov-dump
x86_64-linux-gnu/bin/x86_64-linux-gnu-gcov-tool
x86_64-linux-gnu/lib/gcc/x86_64-linux-gnu/12.4.0: libgcov.a
```
For mingw-w64-gcc, `--disable-gcov` is currently passed for this
target in Guix, due to issues with mingw-w64, see
8bed031e58/gnu/packages/gcc.scm (L99-L102).
However we'll add it in any case, in case it's re-enabled in future,
when the underlying issues are fixed.
1. Check that outbound nodes are treated
the same as whitelisted connections for
the purposes of getdata delays
2. Add test case that demonstrates
download retries are preferentially
given to outbound (preferred) connections
even when multiple announcements are
considered ready.
31e59d94c6 iwyu: Drop backported mapping (Hennadii Stepanov)
fe9bc5abef ci: Update Clang in "tidy" job (Hennadii Stepanov)
Pull request description:
This PR switches to the latest [IWYU 0.23](https://github.com/include-what-you-use/include-what-you-use/releases/tag/0.23), which is compatible with Clang 19.
New "bugprone-use-after-move" and "modernize-use-starts-ends-with" warnings that emerged have been addressed.
ACKs for top commit:
maflcko:
lgtm ACK 31e59d94c6
l0rinc:
ACK 31e59d94c6
theuni:
ACK 31e59d94c6
Tree-SHA512: ae0ca150673e1bfa78664f2ef35dbc965094b32374cafeeae390c6d368c28169a7f7790debe9a6eeb5efc39c9a468f5032d92f30cc4032b09d8265f6a75de882
- For "%n", which is supposed to write to the argument for printf.
- For string/integer mismatches of width/precision specifiers.
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
Also adds BOOST_CHECK_NO_THROW() while touching that line, clarifying part of what we are checking for.
Also removed redundant inline from template functions in .cpp file.
The crash occurs because we assume the cached scripts
structure will not be empty, but it can be empty when
the legacy wallet contained only watch-only and
solvable but not spendable scripts
Stops creating a bdb database in the wallet migration benchmark.
Instead, the benchmark now creates the db in memory and re-uses
it for the migration process.
0184d33b3d scripted-diff: Replace strprintf(Untranslated) with Untranslated(strprintf) (Ryan Ofsky)
006e4d1d59 refactor: Use + instead of strformat to concatenate translated & untranslated strings (Ryan Ofsky)
831d2bfcf9 refactor: Don't embed translated string in untranslated string. (Ryan Ofsky)
058021969b refactor: Avoid concatenation of format strings (Ryan Ofsky)
Pull request description:
This PR cleans up string formatting in the codebase so other PRs adding compile time checking can be simpler and easier to review (specifically #30928, #31061, #31074, and #31149).
Currently these PRs are hard to review because in addition to changing formatting APIs, they have to update callers that are using the API's in unusual ways. Clean up these callers now so later PRs can be simpler. Specifically:
- Use string literals instead of `std::string` format strings to enable more compile-time checking.
- Avoid using untranslated bilingual strings as format strings. Use originals so they can by checked at compile time.
- Favor `Untranslated(strprintf(...))` over `strprintf(Untranslated(...), ...)` for consistency and to prevent translated and untranslated strings from being unintentionally combined.
ACKs for top commit:
maflcko:
lgtm ACK 0184d33b3d🔹
l0rinc:
ACK 0184d33b3d - no overall difference because of the rebase
Tree-SHA512: 37eb771187d30977f5f054eddb82af6dd69878ace89cede72032bb389e57200898119f9fe486ce6903ebb00cb24648be215ab3e44842b3d206b35e26038da750
7239ddb7ce test: make sure node has all transactions (brunoerg)
ee1b9bef00 test: replace `is not` to `!=` when comparing block hash (brunoerg)
Pull request description:
`test_invalid_tx_in_compactblock` tests that we don't get disconnected if we relay a compact block with valid header, but invalid transactions.
In this test, after sending the block with invalid transactions, this test checks two things: the tip in the receiver node did not advance and the sender did not get disconnected. However, even if the block contains only valid transactions, the tip would not advance because the receiver does not have all transactions to reconstruct the valid and would request them back. This PR fixes it by sending all the transactions.
Also, comparing block hash (int) using `is not` can lead to subtle bugs, this PR fixes it by replacing it to `!=`.
--------------
Can be tested by applying:
```diff
diff --git a/test/functional/p2p_compactblocks.py b/test/functional/p2p_compactblocks.py
index 274ef9532c..419153a32f 100755
--- a/test/functional/p2p_compactblocks.py
+++ b/test/functional/p2p_compactblocks.py
@@ -723,11 +723,8 @@ class CompactBlocksTest(BitcoinTestFramework):
utxo = self.utxos[0]
block = self.build_block_with_transactions(node, utxo, 5)
- del block.vtx[3]
block.hashMerkleRoot = block.calc_merkle_root()
# Drop the coinbase witness but include the witness commitment.
- add_witness_commitment(block)
- block.vtx[0].wit.vtxinwit = []
block.solve()
# Make sure node has the transactions to reconstruct the block
```
ACKs for top commit:
instagibbs:
ACK 7239ddb7ce
glozow:
ACK 7239ddb7ce
lucasbalieiro:
Tested ACK for commit [7239ddb](7239ddb7ce)
Tree-SHA512: 6d04fb7c50b5e635c83ede75c12130cbd8e1b229887a86a2e1bfe747e4208731faecc7265cae063c1ace187b20c5f37080d5116760766fa2948f38971e5f6fbf
fae76393bd test: Avoid F541 (f-string without any placeholders) (MarcoFalke)
Pull request description:
An extra `f` string-prefix is mostly harmless, but could be confusing or hint to a mistake where a format argument was forgotten.
Try to avoid the confusion and mistakes by applying the `F541` linter rule.
ACKs for top commit:
lucasbalieiro:
**Tested ACK** [fae7639](fae76393bd)
danielabrozzoni:
ACK fae76393bd
tdb3:
Code review ACK fae76393bd
Tree-SHA512: 4992a74fcf0c19b32e4d95f7333e087b4269b5c5259c556789fb86721617db81c7a4fe210ae136c92824976f07f71ad0f374655e7008b1967c02c73324862d9a
97a18c8545 cmake: Fix `IF_CHECK_PASSED` option handling (Hennadii Stepanov)
Pull request description:
`IF_CHECK_PASSED` is a multi-value keyword, resulting in a list value. Convert it to a string before applying any `string()` command.
Split from https://github.com/bitcoin/bitcoin/pull/30861.
No current CMake code is affected by this bug.
ACKs for top commit:
theuni:
utACK 97a18c8545
Tree-SHA512: d2556ca38c35a8992175e9f948c2028a789e71c2b2d5fdf365b31710c8ed3d5edf5d0363853c5d750d29abb58cfda3c78cdc2971a627e5b4c61aca4ec2a33356
edb41e4814 util: use explicit cast in MultiIntBitSet::Fill() (Vasil Dimov)
Pull request description:
The current code does not have a bug, but is implicitly casting -1 to 65535 and the sanitizer has no way to know whether we intend that or not.
```
FUZZ=bitset src/test/fuzz/fuzz /tmp/fuz
error: implicit conversion from type 'int' of value -1 (32-bit, signed)
to type 'value_type' (aka 'unsigned short') changed the value to 65535
(16-bit, unsigned)
Base64: Qv7bX/8=
```
https://api.cirrus-ci.com/v1/task/5685829642747904/logs/ci.log
ACKs for top commit:
sipa:
ACK edb41e4814
maflcko:
lgtm ACK edb41e4814
Empact:
ACK edb41e4814
tdb3:
code review ACK edb41e4814
Tree-SHA512: a53835d654d9a7246ec0dab30fa5fbc08155dadb40d9bee3297060aa90816e0ce3d3e92dbdcd7af9474446d842d03f2781b7645a68ffef7fb5fc32ee02545112
55347a5018 test: Rework migratewallet to use previous release (v28.0) (Ava Chow)
f42ec0f3bf wallet: Check specified wallet exists before migration (Ava Chow)
Pull request description:
This PR reworks wallet_migration.py to use previous releases to produce legacy wallets for testing so that the test will continue to work once legacy wallets are removed.
Split from #28710
ACKs for top commit:
maflcko:
re-ACK 55347a5018🥊
rkrux:
re-ACK 55347a5
Tree-SHA512: f90a2f475febc73d29e8ad3cb20d134c368a40a3b5934c3e4aaa77ae704af6314d4dd2e85c261142bd60a201902ac4ba00b8e2443d3cef7c8cc45d23281fa831
cccca8a77f test: Avoid logging error when logging error (MarcoFalke)
Pull request description:
Currently a logging error in the form of `--- Logging error ---` happens when an error is logged in the `_on_data` helper.
Fix it by properly logging the error.
Also, treat pylint errors as errors, to avoid this problem in the future.
Can be tested by running `p2p_addrv2_relay.py` with the following example diff:
```diff
diff --git a/test/functional/test_framework/p2p.py b/test/functional/test_framework/p2p.py
index 523e1bd068..0f1eb29d13 100755
--- a/test/functional/test_framework/p2p.py
+++ b/test/functional/test_framework/p2p.py
@@ -137,7 +137,7 @@ MESSAGEMAP = {
b"notfound": msg_notfound,
b"ping": msg_ping,
b"pong": msg_pong,
- b"sendaddrv2": msg_sendaddrv2,
+ #b"sendaddrv2": msg_sendaddrv2,
b"sendcmpct": msg_sendcmpct,
b"sendheaders": msg_sendheaders,
b"sendtxrcncl": msg_sendtxrcncl,
ACKs for top commit:
fanquake:
ACK cccca8a77f
Tree-SHA512: dd19f3feed0093246cb205903529fb9ebd5ad9a6c9330cfc5987c0154253c9dcec8d0e25ff99e4ac806a464ff58c3787a205378b8dfb7a1a521da25eac429136
f6afca46a1 lint: use clearer wording on error message (willcl-ark)
811a65d3c6 lint: bump MLC to v0.19.0 (willcl-ark)
Pull request description:
Fixes: #31044
This MLC update includes a change which will ignore files being ignored by git, and help avoid false-positives when linting in this repo.
Top commit has no ACKs.
Tree-SHA512: d3edd0125f719c7a4456f7089e298dc851352a082b8119bbd8d642de518bb193827af9994ba416dd18a6a6f1359ee96122d95a31232da1623c679db39b370370
The current code does not have a bug, but is implicitly casting -1 to
65535 and the sanitizer has no way to know whether we intend that or
not.
```
FUZZ=bitset src/test/fuzz/fuzz /tmp/fuz
error: implicit conversion from type 'int' of value -1 (32-bit, signed)
to type 'value_type' (aka 'unsigned short') changed the value to 65535
(16-bit, unsigned)
Base64: Qv7bX/8=
```
This change switches to the latest IWYU 0.23, which is compatible with
Clang 19.
Fixed new "modernize-use-starts-ends-with" warnings.
The new "bugprone-use-after-move" warning in `result_tests.cpp` is a
false positive caused by a bug in Boost.Test versions < 1.87. This has
been addressed by introducing a local variable.
See upstream references:
- Issue: https://github.com/boostorg/test/issues/343
- Fix: https://github.com/boostorg/test/pull/348
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
0f84cdd266 func: test orphan parent is re-requested from 2nd peer (Greg Sanders)
Pull request description:
Small test which I couldn't find coverage for.
ACKs for top commit:
glozow:
lgtm ACK 0f84cdd266
tdb3:
code review ACK 0f84cdd266
theStack:
ACK 0f84cdd266
marcofleon:
tACK 0f84cdd266. Removing `node.bumpmocktime(GETDATA_TX_INTERVAL)` results in failure.
Tree-SHA512: fe8cb9d56aabc8f2ef1f49b6cd4e87e28a51ada8070c698f60c5fd945a28d849f0c5793f2e3e29f013e610168b860e0bf1c0aa010eec5b339688269d2b9e69af
Fixes: #31044
This MLC update includes a change which will ignore files being ignored
by git, and help avoid false-positives when linting in this repo.
1807df3d9f test: addrman: tried 3 times and never a success so `isTerrible=true` (brunoerg)
Pull request description:
This PR adds test coverage for the following verification:
```cpp
if (TicksSinceEpoch<std::chrono::seconds>(m_last_success) == 0 && nAttempts >= ADDRMAN_RETRIES) { // tried N times and never a success
return true;
}
```
If we've tried an address for 3 or more times and were unsuccessful, this address should be pointed out as "terrible".
-------
You can test this by applying:
```diff
diff --git a/src/addrman.cpp b/src/addrman.cpp
index 054a9bee32..93a9521b59 100644
--- a/src/addrman.cpp
+++ b/src/addrman.cpp
@@ -81,7 +81,7 @@ bool AddrInfo::IsTerrible(NodeSeconds now) const
}
if (TicksSinceEpoch<std::chrono::seconds>(m_last_success) == 0 && nAttempts >= ADDRMAN_RETRIES) { // tried N times and never a success
- return true;
+ return false;
}
```
ACKs for top commit:
jonatack:
re-ACK 1807df3d9f
naumenkogs:
ACK 1807df3d9f
achow101:
ACK 1807df3d9f
Tree-SHA512: e3cc43c98bddfe90f585d5b4bd00543be443b77ecaf038615261aa8cc4d14fc2f1006d0b00c04188040eaace455c5c6dbb3bb200a2c0f29c3b4ef5128bf0973a
This makes code more consistent and makes it easier to add compile-time checking to
enforce that format strings contain the right specifiers, because it stops
using Untranslated() to create the format string, so the Untranslated()
function will not need to get involved in formatting.
-BEGIN VERIFY SCRIPT-
quote='"[^"]+"'
quotes="(?:$quote|\\s)*"
nonparens="[^()]*"
single_level_paren="\($nonparens\)"
double_level_paren="\($nonparens\($nonparens\)$nonparens\)"
exprs="(?:$double_level_paren|$single_level_paren|$nonparens)*"
git grep -l 'Untranslated' | xargs perl -0777 -i -pe "s/strprintf\((\\W*)Untranslated\(($quotes)\)($exprs)(\))/Untranslated(\1strprintf(\2\3))/gs"
-END VERIFY SCRIPT-
50cce20013 test, refactor: Compact ccoins_access and ccoins_spend (Lőrinc)
0a159f0914 test, refactor: Remove remaining unbounded flags from coins_tests (Lőrinc)
c0b4b2c1ee test: Validate error messages on fail (Lőrinc)
d5f8d607ab test: Group values and states in tests into CoinEntry wrappers (Lőrinc)
ca74aa7490 test, refactor: Migrate GetCoinsMapEntry to return MaybeCoin (Lőrinc)
15aaa81c38 coins, refactor: Remove direct GetFlags access (Lőrinc)
6b733699cf coins, refactor: Assume state after SetClean in AddFlags to prevent dangling pointers (Lőrinc)
fc8c282022 coins, refactor: Make AddFlags, SetDirty, SetFresh static (Lőrinc)
cd0498eabc coins, refactor: Split up AddFlags to remove invalid states (Lőrinc)
Pull request description:
Similarly to https://github.com/bitcoin/bitcoin/pull/30849, this cleanup is intended to de-risk https://github.com/bitcoin/bitcoin/pull/30673#discussion_r1739909068 by simplifying the coin cache public interface.
`CCoinsCacheEntry` provided general access to its internal flags state, even though, in reality, it could only be `clean`, `fresh`, `dirty`, or `fresh|dirty` (in the follow-up, we will remove `fresh` without `dirty`).
Once it was marked as `dirty`, we couldn’t set the state back to clean with `AddFlags(0)`—tests explicitly checked against that.
This PR refines the public interface to make this distinction clearer and to make invalid behavior impossible, rather than just checked by tests. We don't need extensive access to the internals of `CCoinsCacheEntry`, as many tests were simply validating invalid combinations in this way.
The last few commits contain significant test refactorings to make `coins_tests` easier to change in follow-ups.
ACKs for top commit:
andrewtoth:
Code Review ACK 50cce20013
laanwj:
Code review ACK 50cce20013
ryanofsky:
Code review ACK 50cce20013. Looks good! Thanks for the followups.
Tree-SHA512: c0d65f1c7680b4bb9cd368422b218f2473c2ec75a32c7350a6e11e8a1601c81d3c0ae651b9f1dae08400fb4e5d43431d9e4ccca305a718183f9a936fe47c1a6c
This change manually removes two strprintf(Untranslated...) calls. All
remaining calls are removed in the next scripted-diff commit.
Removing these calls makes code more consistent and makes it easier to
implement compile-time checking enforcing that format strings contain valid
specifiers, by avoiding the need for the Untranslated() function to be involved
in formatting.
Additionally, using + and += instead of strprintf here makes code a little
shorter, and more type-safe because + unlike strprintf only works on strings of
the same type, making it less likely english strings and bilingual strings will
be unintentionally combined.
This could produce an english error message containing non-english string
fragments if PopulateAndValidateSnapshot started returning any translated
strings in the future. This change is also needed to make the next
scripted-diff commit work.
Instead just concatenate already formatted strings. This allows untranslated
format strings to be checked at compile time now, and translated format strings
to be checked at compile time in #31061.
95a0104f2e test: Add tests for directories in place of config files (Hodlinator)
e85abe92c7 args: Catch directories in place of config files (Hodlinator)
e4b6b1822c test: Add tests for -noconf (Hodlinator)
483f0dacc4 args: Properly support -noconf (Hodlinator)
312ec64cc0 test refactor: feature_config_args.py - Stop nodes at the end of tests, not at the beginning (Hodlinator)
7402658bc2 test: -norpccookiefile (Hodlinator)
39cbd4f37c args: Support -norpccookiefile for bitcoind and bitcoin-cli (Hodlinator)
e82ad88452 logs: Use correct path and more appropriate macros in cookie-related code (Hodlinator)
6e28c76907 test: Harden testing of cookie file existence (Hodlinator)
75bacabb55 test: combine_logs.py - Output debug.log paths on error (Hodlinator)
bffd92f00f args: Support -nopid (Hodlinator)
12f8d848fd args: Disallow -nodatadir (Hodlinator)
6ff9662760 scripted-diff: Avoid printing version information for -noversion (Hodlinator)
e8a2054edc doc args: Document narrow scope of -color (Hodlinator)
Pull request description:
- Document `-color` as only applying to `-getinfo`, to be less confusing for bitcoin-cli users.
- No longer print version information when getting passed `-noversion`.
- Disallow `-nodatadir` as we cannot run without one. It was previously interpreted as a mix of unset and as a relative path of "0".
- Support `-norpccookiefile`
- Support `-nopid`
- Properly support `-noconf` (instead of working by accident). Also detect when directories are specified instead of files.
Prompted by investigation in https://github.com/bitcoin/bitcoin/pull/16545#pullrequestreview-2316714013.
ACKs for top commit:
l0rinc:
utACK 95a0104f2e
achow101:
ACK 95a0104f2e
ryanofsky:
Code review ACK 95a0104f2e. Looks good! Thanks for all your work on this breaking the changes down and making them simple.
Tree-SHA512: 5174251e6b9196a9c6d135eddcb94130295c551bcfccc78e633d9e118ff91523b1be0d72828fb49603ceae312e6e1f8ee2651c6a2b9e0f195603a73a9a622785
fa3e074304 refactor: Tidy fixups (MarcoFalke)
fa72646f2b move-only: Detail_CheckNumFormatSpecifiers and G_TRANSLATION_FUN (MarcoFalke)
faff8403f0 refactor: Pick translated string after format (MarcoFalke)
Pull request description:
The changes are required for https://github.com/bitcoin/bitcoin/pull/31061, however they also make sense on their own. For example, they are fixing up an `inline namespace`, which lead to compile errors otherwise (can be tested by observing the compile error after reverting the changes to `src/util/strencodings.h`). Also, a unit test comment is fixed.
ACKs for top commit:
ryanofsky:
Code review ACK fa3e074304. Nice changes! These should allow related PRs to be simpler.
l0rinc:
ACK fa3e074304
hodlinator:
cr-ACK fa3e074304
Tree-SHA512: 37371181a348610442186b5fbb7a6032d0caf70aae566002ad60be329a3131a2b89f28f6c51e10872079f987986925dc8c0611bde639057bee4f572d2b9ba92a
faf70cc994 Remove wallet::ParseISO8601DateTime, use ParseISO8601DateTime instead (MarcoFalke)
2222aecd5f util: Implement ParseISO8601DateTime based on C++20 (MarcoFalke)
Pull request description:
`boost::posix_time` in `ParseISO8601DateTime` has many issues:
* It parses random strings that are clearly invalid and returns a time value for them, see [1] below.
* None of the separators `-`, or `:`, or `T`, or `Z` are validated.
* It may crash when running under a hardened C++ library, see https://github.com/bitcoin/bitcoin/issues/28917.
* It has been unmaintained for years, so reporting or fixing any issues will most likely be useless.
* It pulls in a third-party dependency, when the functionality is already included in vanilla C++20.
Fix all issues by replacing it with a simple helper function written in C++20.
Fixes https://github.com/bitcoin/bitcoin/issues/28917.
[1] The following patch passes on current master:
```diff
diff --git a/src/wallet/test/rpc_util_tests.cpp b/src/wallet/test/rpc_util_tests.cpp
index 32f6f5ab46..c1c94c7116 100644
--- a/src/wallet/test/rpc_util_tests.cpp
+++ b/src/wallet/test/rpc_util_tests.cpp
@@ -12,6 +12,14 @@ BOOST_AUTO_TEST_SUITE(wallet_util_tests)
BOOST_AUTO_TEST_CASE(util_ParseISO8601DateTime)
{
+ BOOST_CHECK_EQUAL(ParseISO8601DateTime("964296"), 242118028800);
+ BOOST_CHECK_EQUAL(ParseISO8601DateTime("244622"), 15023836800);
+ BOOST_CHECK_EQUAL(ParseISO8601DateTime("+INfINITy"), 9223372036854);
+ BOOST_CHECK_EQUAL(ParseISO8601DateTime("7000802 01"), 158734166400);
+ BOOST_CHECK_EQUAL(ParseISO8601DateTime("7469-2 +INfINITy"), 9223372036854);
+ BOOST_CHECK_EQUAL(ParseISO8601DateTime("maXimum-datE-time"), 253402300799);
+ BOOST_CHECK_EQUAL(ParseISO8601DateTime("577737 114maXimum-datE-time"), 253402300799);
+
BOOST_CHECK_EQUAL(ParseISO8601DateTime("1970-01-01T00:00:00Z"), 0);
BOOST_CHECK_EQUAL(ParseISO8601DateTime("1960-01-01T00:00:00Z"), 0);
BOOST_CHECK_EQUAL(ParseISO8601DateTime("2000-01-01T00:00:01Z"), 946684801);
```
ACKs for top commit:
hebasto:
ACK faf70cc994, I have reviewed the code and it looks OK.
dergoegge:
utACK faf70cc994
Tree-SHA512: 9dd745a356d04acf6200e13a6af52c51a9e2a0eeccea110093ce5da147b3c669c0eda918e46db0164c081a78c8feae3fe557a4759bea18449a8ff2d090095931
Providing a script for the coinbase transaction is only done in test code
and for CPU solo mining.
Production miners use the getblocktemplate RPC which omits the coinbase
transaction entirely from its block template, leaving it to external (pool)
software to construct it.
A coinbase script can still be passed via BlockCreateOptions instead.
A temporary overload is added so that the test can be modified in the
next commit.
8f85d36d68 refactor: Clamp worker threads in ChainstateManager constructor (TheCharlatan)
Pull request description:
This ensures the options are applied consistently from contexts where they might not pass through the args manager, such as in some tests, or when used through the kernel library.
This is similar to the patch applied in 09ef322acc, used to make applying the mempool options consistent.
---
This is part of the libbitcoinkernel project https://github.com/bitcoin/bitcoin/issues/27587
ACKs for top commit:
maflcko:
ACK 8f85d36d68 🛳
achow101:
ACK 8f85d36d68
furszy:
Code ACK 8f85d36d68
stickies-v:
ACK 8f85d36d68
Tree-SHA512: 32d7cc177d6726ee9df62ac9eb43e49ba676f35bfcff47834bd97a1e33f2a9ea7be65d0a8a37be149de04e58c9c500ecef730e498f4e3909042324d3136160e9
32fc59796f rpc: Allow single transaction through submitpackage (glozow)
Pull request description:
There's no particular reason to restrict single transaction submissions with submitpackage. This change relaxes the RPC checks as enables the `AcceptPackage` flow to accept packages of a single transaction.
Resolves#31085
ACKs for top commit:
naumenkogs:
ACK 32fc59796f
achow101:
ACK 32fc59796f
glozow:
ACK 32fc59796f
Tree-SHA512: ffed353bfdca610ffcfd53b40b76da05ffc26df6bac4b0421492e067bede930380e03399d2e2d1d17f0e88fb91cd8eb376e3aabebbabcc724590bf068d09807c
73db95c65c kernel: Make bitcoin-chainstate's block validation mirror submitblock's (TheCharlatan)
bb53ce9bda tests: Add functional test for submitting a previously pruned block (Greg Sanders)
1f7fc73825 rpc: Remove submitblock duplicate pre-check (TheCharlatan)
e62a8abd7d rpc: Remove submitblock invalid-duplicate precheck (TheCharlatan)
36dbebafb9 rpc: Remove submitblock coinbase pre-check (TheCharlatan)
Pull request description:
With the introduction of a mining ipc interface and the potential future introduction of a kernel library API it becomes increasingly important to offer common behaviour between them. An example of this is ProcessNewBlock, which is used by ipc, rpc, net_processing and (potentially) the kernel library. Having divergent behaviour on suggested pre-checks and checks for these functions is confusing to both developers and users and is a maintenance burden.
The rpc interface for ProcessNewBlock (submitblock) currently pre-checks if the block has a coinbase transaction and whether it has been processed before. While the current example binary for how to use the kernel library, bitcoin-chainstate, imitates these checks, the other interfaces do not.
The coinbase check is repeated again early during ProcessNewBlock. Pre-checking it may also shadow more fundamental problems with a block. In most cases the block header is checked first, before validating the transactions. Checking the coinbase first therefore masks potential issues with the header. Fix this by removing the pre-check.
Similary the duplicate checks are repeated early in the contextual checks of ProcessNewBlock. If duplicate blocks are detected much of their validation is skipped. Depending on the constitution of the block, validating the merkle root of the block is part of the more intensive workload when validating a block. This could be an argument for moving the pre-checks into block processing. In net_processing this would have a smaller effect however, since the block mutation check, which also validates the merkle root, is done before.
Testing spamming a node with valid, but duplicate unrequested blocks seems to exhaust a CPU thread, but does not seem to significantly impact keeping up with the tip. The benefits of adding these checks to net_processing are questionable, especially since there are other ways to trigger the more CPU-intensive checks without submitting a duplicate block. Since these DOS concerns apply even less to the RPC interface, which does not have banning mechanics built in, remove them too.
Finally, also remove the pre-checks from `bitcoin-chainstate.cpp`.
---
This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).
ACKs for top commit:
Sjors:
re-utACK 73db95c65c
achow101:
ACK 73db95c65c
instagibbs:
ACK 73db95c65c
mzumsande:
ACK 73db95c65c
Tree-SHA512: 2d02e851cf402ecf6a1968c058df3576aac407e200cbf922a1a6391b7f97b4f42c6d9f6b0a78b9d1af0a6d40bdd529a7b11a1e6d88885bd7b8b090f6d1411861
492e1f0994 [validation] merge all ConnectBlock debug logging code paths (Pieter Wuille)
b49df703f0 [validation] include all logged information in BlockValidationState (Pieter Wuille)
7b267c034f [validation] Add detailed txin/txout information for script error messages (Pieter Wuille)
146a3d5426 [validation] Make script error messages uniform for parallel/single validation (Pieter Wuille)
1ac1c33f3f [checkqueue] support user-defined return type through std::optional (Pieter Wuille)
Pull request description:
~~Builds on top of #31097~~ (now merged). Fixes#30960.
So far, detailed information about script validation failures is only reported when running with `-par=1`, due to a lack of ability to transfer information from the script validation threads to the validation thread. Fix this by extending the `CCheckQueue` functionality to pass more results through than just success/failure, and use this to report the exact Script error, as well as the transaction input in which it occurred.
ACKs for top commit:
achow101:
ACK 492e1f0994
furszy:
Code review ACK 492e1f0
maflcko:
re-ACK 492e1f0994🍈
dergoegge:
ACK 492e1f0994
instagibbs:
ACK 492e1f0994
mzumsande:
Code Review ACK 492e1f0994
Tree-SHA512: 234f2e7dfd03bdcd2a56200875fe370962f211ea7ed334038a6a9279a758030bf94bb6246f60d06dd0473dac4b9dbf050d9a32ecaa4176f7727eff63572bf4fd
fa5e706459 ci: Skip broken Wine64 tests by default (MarcoFalke)
Pull request description:
I don't think the unit tests run in Wine after the Windows cross-compilation have ever shown a true positive since the MSVC task was added. However, they are a source of frequent false-positives.
Thus, disable them by default for now. Anyone can still enable them by setting `RUN_UNIT_TESTS=true`.
A follow-up could run them on real Windows, see https://github.com/bitcoin/bitcoin/pull/31176.
Conceptually there are many other nightly tasks, which rarely find issues and are not run by default, like the valgrind or s390x tasks. So putting the Wine unit tests in the same bucket should be fine.
ACKs for top commit:
hebasto:
ACK fa5e706459, to avoid false-positives.
willcl-ark:
ACK fa5e706459
Tree-SHA512: 6bd54470e4d5ce18923c5d724aba0dbf475d053d7097d3f87e822a455cc537b6ce5f0dfcc8ccd0719c12c5d0c8fc7355a6c84185a6c9b5d484d98aee763d0c49
-noconf would previously lead to an ifstream "successfully" being opened to the ".bitcoin"-directory (not a file). (Guards against the general case of directories as configs are added in grandchild commit to this one).
Other users of AbsPathForConfigVal() in combination with negated args have been updated earlier in this PR ("args: Support -nopid" and "args: Support -norpccookiefile...").
This ensures we don't needlessly start the node, and reduces implicit dependencies between test functions.
test_seed_peers() - Move assert calling RPC to verify correct chain after our own function actually started the node.
This makes the debug output mostly the same for -par=1 and parallel validation runs. Of course,
parallel validation is non-deterministic in what error it may encounter first if there are
multiple issues. Also, the way certain script-related and non-script-related checks are
performed differs between the two modes still, which may result in discrepancies.
The check type function now needs to return a std::optional<R> for some type R,
and the check queue overall will return std::nullopt if all individual checks
return that, or one of the non-nullopt values if there is at least one.
For most tests, we use R=int, but for the actual validation code, we make it return
the ScriptError.
409d0d6293 test: enable running individual independent functional test methods (ismaelsadeeq)
Pull request description:
- Some test methods in the functional test framework are independent and do not require any prior context or setup in `run_test`.
- This commit adds a new option for running these specific methods within a test file, allowing them to be executed individually without running the entire test suite.
- Using this option reduces the time you need to wait before the test you are interested in starts executing.
- The functionality added by this PR can be achieved manually by commenting out code, but having a pragmatic option to do this is more convenient.
Note: Running test methods that require arguments or context will fail.
**Example Usage**:
```zsh
build/test/functional/feature_reindex.py --test_methods continue_reindex_after_shutdown
```
```zsh
build/test/functional/feature_config_args.py --test_methods test_log_buffer test_args_log test_connect_with_seednode
```
ACKs for top commit:
maflcko:
review ACK 409d0d6293
rkrux:
reACK 409d0d6293
ryanofsky:
Code review ACK 409d0d6293. This seems like a good step towards making it easy to run independent tests quickly. I think ideally there would be some naming convention or @ annotation added to test methods that can run independently, so the test framework could provide more functionality like being able to list test methods, being able to show command lines to quickly reproduce problems when tests fails, and calling test methods automatically instead of requiring individual tests to call them. But these ideas are all compatible with the new `--test_methods` option
Tree-SHA512: b0daac7c3b322e6fd9b946962335d8279e8cb004ff76f502c8d597b9c4b0073840945be198a79d44c5aaa64bda421429829d5c84ceeb8c6139eb6ed079a35878
19f49c7489 doc: Use more precise anchor link to codesigning docs (Jeremy Rand)
Pull request description:
The "Codesigning" section is what users presumably are looking for when they follow this link.
ACKs for top commit:
fanquake:
ACK 19f49c7489
Tree-SHA512: 0e25cf0d7160db7d564d67d3e3ac614f9bd209b2399414f1278fa01cfc1ff827aa8311f7c1c2666924d5ac2dc23fe9bc258b80ed8025d5b8d5b11bcf1d12b28c
62f6d9e1a4 test: simple ordering optimization to reduce runtime (tdb3)
Pull request description:
Noticed in #31371 that the position of `mempool_ephemeral_dust` within `BASE_SCRIPTS` was lengthening total test runtime. Instead of moving only that test, looked for others to move to reduce runtime.
This is a quick optimization that was found to reduce overall functional test runtime of up to around 20% (depending on jobs and machine characteristics). Since it seems like test ordering could be done in many different ways, with many variables, and bike shedding could creep in, a relatively straightforward approach was taken for now that minimized changes to test_runner.
ACKs for top commit:
maflcko:
lgtm ACK 62f6d9e1a4
TheCharlatan:
ACK 62f6d9e1a4
Tree-SHA512: 6f93fbe4de3fce202383d9f84aa0e96961af3de3c02b8cab73589339d701f32c5e1b57a191eeebf4b06b5cd7a82617f63f24110732940be1a5a4d9237813a570
The `ccoins_add` and `ccoins_write` tests check the actual exception error messages now instead of just that they fail for the given parameters.
This enables us testing different exceptions in a more fine-grained way in later changes.
8bf1b3039c doc: Use more precise anchor links to Xcode SDK extraction (Jeremy Rand)
Pull request description:
The "SDK Extraction" section is what users presumably are looking for when they follow these links.
ACKs for top commit:
fanquake:
ACK 8bf1b3039c
Tree-SHA512: 38669a6b171aa102bb80f5b3a343bd6a067c6921c454f6d18087c5add8016eea2ba8196036f9968f0a9b7df1f642c96ff6c657338c32e775beb04038497cde1f
faa16ed4b9 test: Add missing node.setmocktime(self.mocktime) to p2p_ibd_stalling.py (MarcoFalke)
Pull request description:
This was forgotten by myself in commit fa5b58ea01.
This time, there is a diff to test, which fails on current master and passes with this pull request.
```diff
diff --git a/src/net_processing.cpp b/src/net_processing.cpp
index e503a68382..16438ebd08 100644
--- a/src/net_processing.cpp
+++ b/src/net_processing.cpp
@@ -112,9 +112,9 @@ static_assert(MAX_BLOCKTXN_DEPTH <= MIN_BLOCKS_TO_KEEP, "MAX_BLOCKTXN_DEPTH too
* want to make this a per-peer adaptive value at some point. */
static const unsigned int BLOCK_DOWNLOAD_WINDOW = 1024;
/** Block download timeout base, expressed in multiples of the block interval (i.e. 10 min) */
-static constexpr double BLOCK_DOWNLOAD_TIMEOUT_BASE = 1;
+static constexpr double BLOCK_DOWNLOAD_TIMEOUT_BASE = .05; // 30 sec
/** Additional block download timeout per parallel downloading peer (i.e. 5 min) */
-static constexpr double BLOCK_DOWNLOAD_TIMEOUT_PER_PEER = 0.5;
+static constexpr double BLOCK_DOWNLOAD_TIMEOUT_PER_PEER = 0.;
/** Maximum number of headers to announce when relaying blocks with headers message.*/
static const unsigned int MAX_BLOCKS_TO_ANNOUNCE = 8;
/** Minimum blocks required to signal NODE_NETWORK_LIMITED */
diff --git a/test/functional/p2p_ibd_stalling.py b/test/functional/p2p_ibd_stalling.py
index fa07873929..f8cdd8998c 100755
--- a/test/functional/p2p_ibd_stalling.py
+++ b/test/functional/p2p_ibd_stalling.py
@@ -82,6 +82,7 @@ class P2PIBDStallingTest(BitcoinTestFramework):
# Need to wait until 1023 blocks are received - the magic total bytes number is a workaround in lack of an rpc
# returning the number of downloaded (but not connected) blocks.
bytes_recv = 172761 if not self.options.v2transport else 169692
+ time.sleep(31);
self.wait_until(lambda: self.total_bytes_recv_for_blocks() == bytes_recv)
self.all_sync_send_with_ping(peers)
ACKs for top commit:
brunoerg:
ACK faa16ed4b9
Tree-SHA512: 5a670e2dcf828ac83b721a3e20d897744cca50080b0583a8460a0d0c7bf2c2c988cf7e35f688dde6a3349f1c21cc83a16ea5242ed06a59d59a04130416690737
We don't need so much access to the internals of CCoinsCacheEntry, since many tests are just exercising invalid combinations this way.
This implies that `AddFlags` has private access now.
CCoinsCacheEntry provided general access to its internal flags state, even though in reality it could only be clean, fresh, dirty or fresh|dirty.
After it got dirtied we couldn't set the state back to clean by AddFlags(0) - tests were explicitly checking against that.
This commit cleans up the public interface to make this distinction cleaner and invalid behavior impossible instead of just checked by tests.
This includes the removal of redundant `inline` qualifiers (we're inside a struct).
Also renamed `self` to `pair` to simplify the upcoming commits.
Also modernized `EmplaceCoinInternalDANGER` since it was already modified.
Co-authored-by: Andrew Toth <andrewstoth@gmail.com>
ab5c63edcc cmake: Build `secp256k1` only when required (Hennadii Stepanov)
76a3a540a4 cmake: Ensure script correctness when no targets are specified (Hennadii Stepanov)
Pull request description:
When no build targets are specified, it is reasonable to expect the configuration step to succeed and produce a build system that does not build any targets.
This PR updates the code to ensure this behaviour:
```
$ cmake -B build -G "Ninja" -DBUILD_DAEMON=OFF -DBUILD_CLI=OFF -DBUILD_TX=OFF -DBUILD_UTIL=OFF -DENABLE_WALLET=OFF -DBUILD_TESTS=OFF
$ cmake --build build
ninja: no work to do.
```
ACKs for top commit:
TheCharlatan:
ACK ab5c63edcc
tdb3:
light test ACK ab5c63edcc
Tree-SHA512: 1b13f406c58b02768d9ba831413aeae1d7e03659e7101de8e598f906ba220f479ac06707965c96a14468ce4ba49011a1ab9adee9cee34ab1e8622f690b94dad8
16b140f225 doc: correct libfuzzer-nosan preset flag (Niklas Gögge)
Pull request description:
`--prefix` is not the correct option for using a preset (it's not an option at all).
ACKs for top commit:
maflcko:
lgtm ACK 16b140f225
Tree-SHA512: 8c5fad4f8573bd9ef972447b2847ede61a3b6af9650a599f6ff7e90a2c009e4422715164261b424c08170c9e179cce241a3ca31ddc234f446316f24fc2c353b1
b73d331937 dbwrapper: Bump max file size to 32 MiB (Maciej S. Szmigiero)
Pull request description:
The default max file size for LevelDB is 2 MiB, which results in the LevelDB compaction code generating ~4 disk cache flushes per second when syncing with the Bitcoin network.
These disk cache flushes are triggered by `fdatasync()` syscall issued by the LevelDB compaction code when reaching the max file size.
If the database is on a HDD this flush rate brings the whole system to a crawl.
It also results in very slow throughput since 2 MiB * 4 flushes per second is about 8 MiB / second max throughput, while even an old HDD can pull 100 - 200 MiB / second streaming throughput.
Increase the max file size for LevelDB to 128 MiB instead so the flush rate drops to about 1 flush / 2 seconds and the system no longer gets so sluggish.
The max file size value chosen also matches the `MAX_BLOCKFILE_SIZE` file size setting already used by the block storage.
ACKs for top commit:
l0rinc:
ACK b73d331937
davidgumberg:
ACK b73d331937
andrewtoth:
ACK b73d331937
TheCharlatan:
ACK b73d331937
willcl-ark:
ACK b73d331937
tdb3:
ACK b73d331937
laanwj:
ACK b73d331937
Tree-SHA512: 5d8fb9ad1ea643fb3e42a9c59f6fc90cc5cc3b82c06d9b8d59de3a5a926fabaeb78efb51b608b1e7925f49d82dfcbd5b72c552993879789f33201efe57c278f3
935973b315 Remove `src/config` directory (Hennadii Stepanov)
Pull request description:
The `src/config` directory has not been used since the migration to CMake, which disables in-source builds.
ACKs for top commit:
TheCharlatan:
ACK 935973b315
BrandonOdiwuor:
ACK 935973b315
Tree-SHA512: cc5b405e39387673fa2fd1e96680295b6eb3dc49a5f9a4d288580b8ea83efba04c96132811ca2cec14bcca509dbaf20c390cd47dabeea2a6ebc973e364e7a43b
The default max file size for LevelDB is 2 MiB, which results in the
LevelDB compaction code generating ~4 disk cache flushes per second when
syncing with the Bitcoin network.
These disk cache flushes are triggered by fdatasync() syscall issued by the
LevelDB compaction code when reaching the max file size.
If the database is on a HDD this flush rate brings the whole system to a
crawl.
It also results in very slow throughput since 2 MiB * 4 flushes per second
is about 8 MiB / second max throughput, while even an old HDD can pull
100 - 200 MiB / second streaming throughput.
Increase the max file size for LevelDB to 32 MiB instead so the flush rate
drops significantly and the system no longer gets so sluggish.
The new max file size value chosen is a compromise between the one that
works best for HDD and SSD performance, as determined by benchmarks done by
various people.
160799d913 test: refactor: introduce `create_ephemeral_dust_package` helper (Sebastian Falbesoner)
61e18dec30 doc: ephemeral policy: add missing closing double quote (Sebastian Falbesoner)
Pull request description:
This small PR contains ephemeral dust follow-ups mentioned in #30329 that were not tackled in the first follow-up PR #31279:
https://github.com/bitcoin/bitcoin/pull/30239#discussion_r1828577696https://github.com/bitcoin/bitcoin/pull/30239#discussion_r1825279952
Happy to add more if I missed some or anyone has concrete commits to add.
ACKs for top commit:
rkrux:
tACK 160799d913
instagibbs:
ACK 160799d913
tdb3:
Code review ACK 160799d913
Tree-SHA512: e9a80c6733f1e7fe9e834d81b404f6e8ef7a61fe986f61b3dcdbda1a0bc547145fc279ec02f54361df56cb4e62a6fedaa0f3991c6e084c3a703ed1b1bfbdbe4e
ee6185372f gen-manpages: Prompt error if no binaries are found (Andre)
299e2220e9 gen-manpages: implement --skip-missing-binaries (Andre Alves)
Pull request description:
Instead of stopping the execution of gen-manpages.py when a binary is not found, continue generating manpages for the available binaries and skip the missing ones.
A new argument, `--skip-missing-binaries`, has been added to enable this behavior.
```sh
➜ bitcoin git:(fix-gen-manpages) ✗ ./contrib/devtools/gen-manpages.py --help
usage: gen-manpages.py [-h] [-s]
options:
-h, --help show this help message and exit
-s, --skip-missing-binaries
skip generation for binaries that are not found
```
closes#30985
This PR also includes an error prompt if no binaries are found in the build path.
ACKs for top commit:
achow101:
ACK ee6185372f
laanwj:
re-ACK ee6185372f
Tree-SHA512: af4a0a5e26e508a51ab63f8aa9f98a6d6af9d7682a16791d8a6a61d49e44cb0147453f628ad5910f65d4efa6e3c7b6605c007259c23230b54888845bfaeb050c
37a5c5d836 doc: update descriptors.md for getdescriptoractivity (James O'Beirne)
ee3ce6a4f4 test: rpc: add no address case for getdescriptoractivity (James O'Beirne)
811f76f3a5 rpc: add getdescriptoractivity (James O'Beirne)
25fe087de5 rpc: move-only: move ScriptPubKeyDoc to utils (James O'Beirne)
Pull request description:
The RPC command `scanblocks` provides a useful way to get a set of blockhashes that have activity relevant to a set of descriptors (`relevant_blocks`). However actually extracting the activity from those blocks is left as an exercise to the end user.
This process involves not only generating the (potentially ranged) set of scripts for the descriptor set on the client side (maybe via `deriveaddresses`), but then the user must retrieve each block's contents one-by-one using `getblock <hash>`, which is transmitted over a network link. And that's all before they perform the actual search over block content. There's even more work required to incorporate unconfirmed transactions.
This PR introduces an RPC `getdescriptoractivity` that [dovetails](https://bitcoin-irc.chaincode.com/bitcoin-core-dev/2024-08-16#1046393;) with `scanblocks` output, handling the process described above. Users specify the blockhashes (perhaps from `relevant_blocks`) and a set of descriptors; they are then given all spend/receive activity in that set of blocks.
This is a very useful tool when implementing lightweight wallets that want neither to require a third-party indexer like electrs, nor the overhead of creating and managing watch-only wallets in Core. This allows Core to be more easily used in a "stateless" manner by wallets, with potentially many nodes interchangeably acting as backends.
### Example usage
```
% ./src/bitcoin-cli scanblocks start \
'["addr(bc1p0cp0vyag6snlta2l7c4am3rue7eef9f72l7uhx52m4v27vfydx9s8tfs7t)"]' \
857263
{
"from_height": 857263,
"to_height": 858263,
"relevant_blocks": [
"00000000000000000002bc5cc78f5b0913a5230a8f4b0d5060bc9a60900a5a88",
"00000000000000000001c5291ed6a40c06d3db5c8fb738567654b24a14b24ecb"
],
"completed": true
}
% ./src/bitcoin-cli getdescriptoractivity \
'["00000000000000000002bc5cc78f5b0913a5230a8f4b0d5060bc9a60900a5a88", "00000000000000000001c5291ed6a40c06d3db5c8fb738567654b24a14b24ecb"]' \
'["addr(bc1p0cp0vyag6snlta2l7c4am3rue7eef9f72l7uhx52m4v27vfydx9s8tfs7t)"]'
{
"activity": [
{
"type": "receive",
"amount": 0.00002900,
"blockhash": "00000000000000000002bc5cc78f5b0913a5230a8f4b0d5060bc9a60900a5a88",
"height": 857907,
"txid": "c9d34f202c1f66d80cae76f305350f5fdde910b97cf6ae6bf79f5bcf2a337d06",
"vout": 254,
"output_spk": {
"asm": "1 7e02f613a8d427f5f55ff62bddc47ccfb394953e57fdcb9a8add58af3124698b",
"desc": "rawtr(7e02f613a8d427f5f55ff62bddc47ccfb394953e57fdcb9a8add58af3124698b)#yewcd80j",
"hex": "51207e02f613a8d427f5f55ff62bddc47ccfb394953e57fdcb9a8add58af3124698b",
"address": "bc1p0cp0vyag6snlta2l7c4am3rue7eef9f72l7uhx52m4v27vfydx9s8tfs7t",
"type": "witness_v1_taproot"
}
},
{
"type": "spend",
"amount": 0.00002900,
"blockhash": "00000000000000000001c5291ed6a40c06d3db5c8fb738567654b24a14b24ecb",
"height": 858260,
"spend_txid": "7f61d1b248d4ee46376f9c6df272f63fbb0c17039381fb23ca5d90473b823c36",
"spend_vin": 0,
"prevout_txid": "c9d34f202c1f66d80cae76f305350f5fdde910b97cf6ae6bf79f5bcf2a337d06",
"prevout_vout": 254,
"prevout_spk": {
"asm": "1 7e02f613a8d427f5f55ff62bddc47ccfb394953e57fdcb9a8add58af3124698b",
"desc": "rawtr(7e02f613a8d427f5f55ff62bddc47ccfb394953e57fdcb9a8add58af3124698b)#yewcd80j",
"hex": "51207e02f613a8d427f5f55ff62bddc47ccfb394953e57fdcb9a8add58af3124698b",
"address": "bc1p0cp0vyag6snlta2l7c4am3rue7eef9f72l7uhx52m4v27vfydx9s8tfs7t",
"type": "witness_v1_taproot"
}
}
]
}
```
ACKs for top commit:
instagibbs:
reACK 37a5c5d836
achow101:
ACK 37a5c5d836
tdb3:
Code review and light retest ACK 37a5c5d836
rkrux:
re-ACK 37a5c5d836
Tree-SHA512: 04aa51e329c6c2ed72464b9886281d5ebd7511a8a8e184ea81249033a4dad535a12829b1010afc2da79b344ea8b5ab8ed47e426d0bf2eb78ab395d20b1da8dbb
e8f50c5deb guix: swap moreutils for just sponge (fanquake)
Pull request description:
Switch to building the only `moreutils` utility we actually need (`sponge`). This results in having less unused stuff in the Guix environment (i.e all the other `moreutils` utilities), and, the dependency graph is simplified. i.e we no-longer have a dependency on `perl`, `docbook` etc, for this package.
Current `moreutils` dependency graph:

In the Guix env, `chronic`, `combine`, `errno`, `ifdata`, `ifne`, `isutf8`, `lckdo`, `mispipe`, `parallel`, `pee`, `ts`, `vidir`, `vipe` & `zrun` (plus their `*.real` variants) are removed.
ACKs for top commit:
hebasto:
ACK e8f50c5deb.
TheCharlatan:
Re-ACK e8f50c5deb
Tree-SHA512: 3687ec4a821ff79c26ee839d2af879166edb7e179287a9574eca8fbf34bed1fea8fcdad822a2140d0a0089e1820f3fef29a6100e0e8da788896e1f7bac5ec3e6
ee1128ead8 doc: update stack-clash-protection comment re mingw-w64 (fanquake)
bf47448f15 test: drop check for Windows < 10 (fanquake)
35b898c47f release: target Windows 10 or later (fanquake)
398754e70b depends: target Windows 10 when building for mingw-w64 (fanquake)
Pull request description:
Follows up to https://github.com/bitcoin/bitcoin/pull/31048#discussion_r1803165670.
We definitely cannot claim that Bitcoin Core is "supported and extensively tested on" on Windows 7.
Note that #30997 is also increasing the minimum required Windows version (for the GUI) to 10.
ACKs for top commit:
hodlinator:
cr-ACK ee1128ead8
davidgumberg:
ACK ee1128ead8
achow101:
ACK ee1128ead8
hebasto:
re-ACK ee1128ead8, only rebased, a commit message and a comment have been amended since my recent [review](https://github.com/bitcoin/bitcoin/pull/31172#pullrequestreview-2415452160).
TheCharlatan:
ACK ee1128ead8
Tree-SHA512: 245e0bac3d63414d919a1948661fef4ff79359faaacaf19d64abd91cc62e822797fb1cf3379e340bfdf9a85c0b88fd99a90eda450dd4218b6213ab78aefb1374
11f3bc229c refactor: Reserve vectors in fuzz tests (Lőrinc)
152fefe7a2 refactor: Preallocate PrevectorFillVector(In)Direct without vector resize (Lőrinc)
a774c7a339 refactor: Fix remaining clang-tidy performance-inefficient-vector errors (Lőrinc)
Pull request description:
PR inspired by https://github.com/bitcoin/bitcoin/pull/29608#issuecomment-2437847307 (and https://github.com/bitcoin/bitcoin/pull/29458, https://github.com/bitcoin/bitcoin/pull/29606, https://github.com/bitcoin/bitcoin/pull/29607, https://github.com/bitcoin/bitcoin/pull/30093).
The `clang-tidy` check can be run via:
```bash
cmake -B build -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DBUILD_BENCH=ON -DBUILD_FUZZ_BINARY=ON -DBUILD_FOR_FUZZING=ON && cmake --build build -j$(nproc)
run-clang-tidy -quiet -p build -j $(nproc) -checks='-*,performance-inefficient-vector-operation' | grep -v 'clang-tidy'
```
which revealed 3 tests and 1 prod warning (+ fuzz and benching, found by hebasto).
Even though the tests aren't performance critical, getting rid of these warnings (for which the checks were already enabled via https://github.com/bitcoin/bitcoin/blob/master/src/.clang-tidy#L18, see below), the fix was quite simple.
<details>
<summary>clang-tidy -list-checks</summary>
```bash
cd src && clang-tidy -list-checks | grep 'vector'
performance-inefficient-vector-operation
```
</details>
<details>
<summary>Output before the change</summary>
```
src/test/rpc_tests.cpp:434:9: error: 'emplace_back' is called inside a loop; consider pre-allocating the container capacity before the loop [performance-inefficient-vector-operation,-warnings-as-errors]
433 | for (int64_t i = 0; i < 100; i++) {
434 | feerates.emplace_back(1 ,1);
| ^
src/test/checkqueue_tests.cpp:366:13: error: 'emplace_back' is called inside a loop; consider pre-allocating the container capacity before the loop [performance-inefficient-vector-operation,-warnings-as-errors]
365 | for (size_t i = 0; i < 3; ++i) {
366 | tg.emplace_back(
| ^
src/test/cuckoocache_tests.cpp:231:9: error: 'emplace_back' is called inside a loop; consider pre-allocating the container capacity before the loop [performance-inefficient-vector-operation,-warnings-as-errors]
228 | for (uint32_t x = 0; x < 3; ++x)
229 | /** Each thread is emplaced with x copy-by-value
230 | */
231 | threads.emplace_back([&, x] {
| ^
src/rpc/output_script.cpp:127:17: error: 'push_back' is called inside a loop; consider pre-allocating the container capacity before the loop [performance-inefficient-vector-operation,-warnings-as-errors]
126 | for (unsigned int i = 0; i < keys.size(); ++i) {
127 | pubkeys.push_back(HexToPubKey(keys[i].get_str()));
| ^
```
And the fuzz and benchmarks, noticed by hebasto: https://github.com/bitcoin/bitcoin/pull/31305#issuecomment-2483124499
</details>
ACKs for top commit:
maflcko:
review ACK 11f3bc229c🎦
achow101:
ACK 11f3bc229c
theuni:
ACK 11f3bc229c
hebasto:
ACK 11f3bc229c, tested with clang 19.1.5 + clang-tidy.
Tree-SHA512: 41691c19f35c63b922a95407617a54f9bff1af3f95f99d15642064f321df038aeb1ae5f061f854ed913f69036807cc28fa6222b2ff4c24ef43b909027fa0f9b3
c288c790cd interpreter: Use the same type for SignatureHash in the definition (TheCharlatan)
Pull request description:
This was missed during the original PR switching the nHashType argument to int32_t in SignatureHash in bc52cda1f3.
The problem was discovered after running into a linker error when attempting to link this code as a static library using the header as a declaration with a riscv32 bare metal toolchain. The compiler would error with:
```
/opt/riscv-ilp32/lib/gcc/riscv32-unknown-elf/13.2.0/../../../../riscv32-unknown-elf/bin/ld: build_kernel_riscv/src/libbitcoin_consensus.a(interpreter.cpp.o): in function `GenericTransactionSignatureChecker<CTransaction>::CheckECDSASignature(std::vector<unsigned char, std::allocator<unsigned char> > const&, std::vector<unsigned char, std::allocator<unsigned char> > const&, CScript const&, SigVersion) const':
/home/user/bitcoin/build_kernel_riscv/./script/interpreter.cpp:2043:(.text._ZNK34GenericTransactionSignatureCheckerI12CTransactionE19CheckECDSASignatureERKSt6vectorIhSaIhEES6_RK7CScript10SigVersion[_ZNK34GenericTransactionSignatureCheckerI12CTransactionE19CheckECDSASignatureERKSt6vectorIhSaIhEES6_RK7CScript10SigVersion]+0xee): undefined reference to `uint256 SignatureHash<CTransaction>(CScript const&, CTransaction const&, unsigned int, int, long long const&, SigVersion, PrecomputedTransactionData const*)'
```
With this patch it is possible to link against the static consensus library and produce a fully static executable.
ACKs for top commit:
l0rinc:
ACK c288c790cd
maflcko:
review ACK c288c790cd🐺
achow101:
ACK c288c790cd
theuni:
Obvious fix ACK c288c790cd.
BrandonOdiwuor:
Code Review ACK c288c790cd
Tree-SHA512: 74f283637f0a9cd0cab65d3502f2f8fc4fb983c7672f24e7a76ba2eb6e53b4a81cca0aacb610ef39ac0a454305be594ab440a697ae3718987bf5dbcbc7146a31
b031b7910d [ci] Split out native fuzz jobs for macOS and windows (dergoegge)
Pull request description:
Split out two new CI jobs (for native macOS and windows) that run the fuzz tests on the qa-assets input corpora.
In both jobs the fuzz binary is built with `-DBUILD_FOR_FUZZING` to enable `Assume` assertions as well as `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION`.
ACKs for top commit:
maflcko:
re-lgtm ACK b031b7910d
achow101:
ACK b031b7910d
hebasto:
ACK b031b7910d.
Tree-SHA512: 7d0dc5a9cb299f6f4596dd9a5526b6aaf82efc6eba308bdc9d8b0a45f79dea87204fb6cd4b2ea2a1bd952466b2e958d64021999296d110d7a83c1667f4de51fe
This is not a pure refactor:
1. It slightly changes the log messages, as reflected in the test changes
2. It adds the IP address to all disconnect logging (when fLogIPs is set)
And under the hood suppoert single transactions
in AcceptPackage. This simplifies user experience
and paves the way for reducing number of codepaths
for transaction acceptance in the future.
Co-Authored-By: instagibbs <gsanders87@gmail.com>
* Since the main LIMITED_WHILE stated `outpoints.size() < 200'000`, I've presized outpoints accordingly.
* `tx_mut.vin` and `tx_mut.vout` weren't caught by the clang-tidy, but addressed them anyway.
This was missed during the original PR switching the nHashType argument
to int32_t in SignatureHash in bc52cda1f3.
The problem was discovered after running into a linker error when
attempting to link this code as a static library using the header as a
declaration with a riscv32 bare metal toolchain. The compiler would
error with:
/opt/riscv-ilp32/lib/gcc/riscv32-unknown-elf/13.2.0/../../../../riscv32-unknown-elf/bin/ld: build_kernel_riscv/src/libbitcoin_consensus.a(interpreter.cpp.o): in function `GenericTransactionSignatureChecker<CTransaction>::CheckECDSASignature(std::vector<unsigned char, std::allocator<unsigned char> > const&, std::vector<unsigned char, std::allocator<unsigned char> > const&, CScript const&, SigVersion) const':
/home/user/bitcoin/build_kernel_riscv/./script/interpreter.cpp:2043:(.text._ZNK34GenericTransactionSignatureCheckerI12CTransactionE19CheckECDSASignatureERKSt6vectorIhSaIhEES6_RK7CScript10SigVersion[_ZNK34GenericTransactionSignatureCheckerI12CTransactionE19CheckECDSASignatureERKSt6vectorIhSaIhEES6_RK7CScript10SigVersion]+0xee): undefined reference to `uint256 SignatureHash<CTransaction>(CScript const&, CTransaction const&, unsigned int, int, long long const&, SigVersion, PrecomputedTransactionData const*)'
Can be tested by running
```
$ sudo tcpdump -i eth0 host 11.22.33.44
```
and verifying that no packets appear in the tcpdump output.
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
We build the only moreutils utility we actually need (sponge), have less
unused stuff in the Guix environment, and, the dependency graph is
simplified. i.e we no-longer have a dependency on perl, docbook etc, for
this package.
The `-ffile-prefix-map` compiler option implies `-fprofile-prefix-map`
on GCC or `-fcoverage-prefix-map` on Clang, which can lead to issues
with coverage builds.
This change applies only the options necessary for build reproducibility
and accurate source location messages.
This tests the new submitblock behaviour that is introduced in the
previous commit: Submitting a previously pruned block should persist the
block's data again.
The duplicate checks are repeated early in the contextual checks of
ProcessNewBlock. If duplicate blocks are detected much of their
validation is skipped. Depending on the constitution of the block,
validating the merkle root of the block is part of the more intensive
workload when validating a block. This could be an argument for moving
the pre-checks into block processing. In net_processing this would have
a smaller effect however, since the block mutation check, which also
validates the merkle root, is done before.
A side effect of this change is that a duplicate block is persisted
again on disk even when pruning is activated. This is similar to the
behaviour with getblockfrompeer. Add a release note for this change in
behaviour.
Testing spamming a node with valid, but duplicate unrequested blocks
seems to exhaust a CPU thread, but does not seem to significantly impact
keeping up with the tip. The benefits of adding these checks to
net_processing are questionable, especially since there are other ways
to trigger the more CPU-intensive checks without submitting a duplicate
block. Since these DOS concerns apply even less to the RPC interface,
which does not have banning mechanics built in, remove them too.
---
With the introduction of a mining ipc interface and the potential future
introduction of a kernel library API it becomes increasingly important
to offer common behaviour between them. An example of this is
ProcessNewBlock, which is used by ipc, rpc, net_processing and
(potentially) the kernel library. Having divergent behaviour on
suggested pre-checks and checks for these functions is confusing to both
developers and users and is a maintenance burden.
The rpc interface for ProcessNewBlock (submitblock) currently pre-checks
if the block has a coinbase transaction and whether it has been
processed before. While the current example binary for how to use the
kernel library, bitcoin-chainstate, imitates these checks, the other
interfaces do not.
ProcessNewBlock fails if an invalid duplicate block is passed in through
its call to AcceptBlock and AcceptBlockHeader. The failure in
AcceptBlockHeader makes AcceptBlock return early. This makes the
pre-check in submitblock redundant.
---
With the introduction of a mining ipc interface and the potential future
introduction of a kernel library API it becomes increasingly important
to offer common behaviour between them. An example of this is
ProcessNewBlock, which is used by ipc, rpc, net_processing and
(potentially) the kernel library. Having divergent behaviour on
suggested pre-checks and checks for these functions is confusing to both
developers and users and is a maintenance burden.
The rpc interface for ProcessNewBlock (submitblock) currently pre-checks
if the block has a coinbase transaction and whether it has been
processed before. While the current example binary for how to use the
kernel library, bitcoin-chainstate, imitates these checks, the other
interfaces do not.
The coinbase check is repeated again early during ProcessNewBlock.
Pre-checking it may also shadow more fundamental problems with a block.
In most cases the block header is checked first, before validating the
transactions. Checking the coinbase first therefore masks potential
issues with the header. Fix this by removing the pre-check.
The pre-check was likely introduced on top of
ada0caa165 to fix UB in
GetWitnessCommitmentIndex in case a block's transactions are empty. This
code path could only be reached because of the call to
UpdateUncommittedBlockStructures in submitblock, but cannot be reached
through net_processing.
Add some functional test cases to cover the previous conditions that
lead to a "Block does not start with a coinbase" json rpc error being
returned.
---
With the introduction of a mining ipc interface and the potential future
introduction of a kernel library API it becomes increasingly important
to offer common behaviour between them. An example of this is
ProcessNewBlock, which is used by ipc, rpc, net_processing and
(potentially) the kernel library. Having divergent behaviour on
suggested pre-checks and checks for these functions is confusing to both
developers and users and is a maintenance burden.
The rpc interface for ProcessNewBlock (submitblock) currently pre-checks
if the block has a coinbase transaction and whether it has been
processed before. While the current example binary for how to use the
kernel library, bitcoin-chainstate, imitates these checks, the other
interfaces do not.
92d3d691f0 fuzz: Implement G_TEST_GET_FULL_NAME (Hodlinator)
Pull request description:
Catching up to bench & unit tests. Makes for more orderly paths for fuzz tests using `BasicTestingSetup`.
### Before
```
/tmp/test_common bitcoin/0748ae43ef8fa80703bc/regtest/blocks/xor.dat
```
### After
```
/tmp/test_common bitcoin/tx_pool_standard/f18b3744625e0600eb0c/regtest/blocks/xor.dat
```
ACKs for top commit:
kevkevinpal:
ACK [92d3d69](92d3d691f0)
furszy:
utACK 92d3d691f0
tdb3:
ACK 92d3d691f0
dergoegge:
utACK 92d3d691f0
brunoerg:
code review ACK 92d3d691f0
Tree-SHA512: 5e83970b111232adece10f79e3a43d0c3c49ab635763e2a4b420f1336cbb8fee94aab751264ddec01ac8363166636e3b29cfe3b2969fc28c8dd6b31bda351950
fe3457ccff ci: note that we should install pkgconf in future (fanquake)
8d203480b3 doc: migrate from pkg-config to pkgconf in macOS build docs (fanquake)
Pull request description:
Migrate the macOS build docs and CI from `pkg-config` to `pkgconf`. As the former now just redirects to the later.
Upstream is currently mass-migrating its formula. i.e https://github.com/Homebrew/homebrew-core/pull/198317.
Fixes#31334.
ACKs for top commit:
maflcko:
ACK fe3457ccff🍭
hebasto:
re-ACK fe3457ccff.
Tree-SHA512: 6e337acb6767d163491149b6ae7181d7d7042bc11cdc745eb6f52d4df6d7a19c4f6daa000b314acd9178f97e670aba145f829e48b1b3033117d7e39cdd3af177
Recently added mempool_util implementation probably evolved in parallel with the package RBF one before being submitted as part of ephemeral dust in e2e30e89ba.
Brew has migrated to using the later:
```bash
brew info pkg-config
==> pkgconf: stable 2.3.0 (bottled), HEAD
Package compiler and linker metadata toolkit
https://github.com/pkgconf/pkgconf
```
9aa50152c1 Add destroy to BlockTemplate schema (Sjors Provoost)
Pull request description:
This ensures that if a client no longer needs a block template, the node can clear its memory as soon as possible.
A block template may hold on to transactions that are no longer in the mempool, so this can be significant.
This has a trivial silent merge conflict with #31283 because it also used the number `@9` (gaps are not allowed). I'll rebase whichever is merged last.
ACKs for top commit:
TheCharlatan:
Re-ACK 9aa50152c1
ryanofsky:
Code review ACK 9aa50152c1
Tree-SHA512: 393571b4455969cba71c7572feaeff4503738205331ab198b9181c156c75eb65933a8e5ceff66fc06d1efb8f2528bdb254e5eee7df75735b912526de1e7b166d
When BasicTestingSetup is used in fuzz-tests it will now create test directories containing the fuzz target names. Example:
/tmp/test_common bitcoin/tx_package_eval/153d7906294f7d0606a7/
This is already implemented for bench and unit tests.
5736d1ddac tracing: pass if replaced by tx/pkg to tracepoint (0xb10c)
a4ec07f194 doc: add comments for CTxMemPool::ChangeSet (Suhas Daftuar)
83f814b1d1 Remove m_all_conflicts from SubPackageState (Suhas Daftuar)
d3c8e7dfb6 Ensure that we don't add duplicate transactions in rbf fuzz tests (Suhas Daftuar)
d7dc9fd2f7 Move CalculateChunksForRBF() to the mempool changeset (Suhas Daftuar)
284a1d33f1 Move prioritisation into changeset (Suhas Daftuar)
446b08b599 Don't distinguish between direct conflicts and all conflicts when doing cluster-size-2-rbf checks (Suhas Daftuar)
b53041021a Duplicate transactions are not permitted within a changeset (Suhas Daftuar)
b447416fdd Public mempool removal methods Assume() no changeset is outstanding (Suhas Daftuar)
2b30f4d36c Make RemoveStaged() private (Suhas Daftuar)
18829194ca Enforce that there is only one changeset at a time (Suhas Daftuar)
7fb62f7db6 Apply mempool changeset transactions directly into the mempool (Suhas Daftuar)
34b6c5833d Clean up FinalizeSubpackage to avoid workspace-specific information (Suhas Daftuar)
57983b8add Move LimitMempoolSize to take place outside FinalizeSubpackage (Suhas Daftuar)
01e145b975 Move changeset from workspace to subpackage (Suhas Daftuar)
802214c083 Introduce mempool changesets (Suhas Daftuar)
87d92fa340 test: Add unit test coverage of package rbf + prioritisetransaction (Suhas Daftuar)
15d982f91e Add package hash to package-rbf log message (Suhas Daftuar)
Pull request description:
part of cluster mempool: #30289
It became clear while working on cluster mempool that it would be helpful for transaction validation if we could consider a full set of proposed changes to the mempool -- consisting of a set of transactions to add, and a set of transactions (ie conflicts) to simultaneously remove -- and perform calculations on what the mempool would look like if the proposed changes were to be applied. Two specific examples of where we'd like to do this:
- Determining if ancestor/descendant/TRUC limits would be violated (in the future, cluster limits) if either a single transaction or a package of transactions were to be accepted
- Determining if an RBF would make the mempool "better", however that idea is defined, both in the single transaction and package of transaction cases
In preparation for cluster mempool, I have pulled this reworking of the mempool interface out of #28676 so it can be reviewed on its own. I have not re-implemented ancestor/descendant limits to be run through the changeset, since with cluster mempool those limits will be going away, so this seems like wasted effort. However, I have rebased #28676 on top of this branch so reviewers can see what the new mempool interface could look like in the cluster mempool setting.
There are some minor behavior changes here, which I believe are inconsequential:
- In the package validation setting, transactions would be added to the mempool before the `ConsensusScriptChecks()` are run. In theory, `ConsensusScriptChecks()` should always pass if the `PolicyScriptChecks()` have passed and it's just a belt-and-suspenders for us, but if somehow they were to diverge then there could be some small behavior change from adding transactions and then removing them, versus never adding them at all.
- The error reporting on `CheckConflictTopology()` has slightly changed due to no longer distinguishing between direct conflicts and indirect conflicts. I believe this should be entirely inconsequential because there shouldn't be a logical difference between those two ideas from the perspective of this function, but I did have to update some error strings in some tests.
- Because, in a package setting, RBFs now happen as part of the entire package being accepted, the logging has changed slightly because we do not know which transaction specifically evicted a given removed transaction.
- Specifically, the "package hash" is now used to reference the set of transactions that are being accepted, rather than any single txid. The log message relating to package RBF that happen in the `TXPACKAGES` category has been updated as well to include the package hash, so that it's possible to see which specific set of transactions are being referenced by that package hash.
- Relatedly, the tracepoint logging in the package rbf case has been updated as well to reference the package hash, rather than a transaction hash.
ACKs for top commit:
naumenkogs:
ACK 5736d1ddac
instagibbs:
ACK 5736d1ddac
ismaelsadeeq:
reACK 5736d1ddac
glozow:
ACK 5736d1ddac
Tree-SHA512: 21810872e082920d337c89ac406085aa71c5f8e5151ab07aedf41e6601f60a909b22fbf462ef3b735d5d5881e9b76142c53957158e674dd5dfe6f6aabbdf630b
a6ca8f3243 fuzz: Fix difficulty target generation in p2p_headers_presync (marcofleon)
fa327c77e3 util: Add ConsumeArithUInt256InRange fuzzing helper (marcofleon)
Pull request description:
In the `p2p_headers_presync` fuzz target, this assertion failed:
```
assert(total_work < chainman.MinimumChainWork());
```
Input that triggered the failure: [p2ppresync_crash.txt](https://github.com/user-attachments/files/17620203/p2ppresync_crash.txt)
The test previously used `ConsumeIntegralInRange` to generate header difficulty targets within a hardcoded range. The fuzzer found specific values in that range that correspond to very low thresholds due to how [`SetCompact`][setcompact-link] works. The total work of a long enough test chain ended up exceeding `MinimumChainWork`.
Fix this by adding a new `ConsumeArithUInt256InRange` helper function and use it in the fuzz test to generate target values within the originally intended range. The target is then converted to an `nBits` value using `GetCompact()`.
For some more context, see https://github.com/bitcoin/bitcoin/pull/30918.
[setcompact-link]: 6463117a29/src/arith_uint256.h (L251-L271)
ACKs for top commit:
instagibbs:
ACK a6ca8f3243
dergoegge:
Code review ACK a6ca8f3243
brunoerg:
code review ACK a6ca8f3243
Tree-SHA512: 92013d9d37bd3f11992ee678ba9745196efbdc4d773fd14994116629260bea46ffc9fa3923d443af7b623d39c6211900ce98a349c62ad1976e12312c37ef9df0
637f437a16 doc: remove PR Review Club frequency (Gabriele Bocchi)
Pull request description:
The PR Review Club is mentioned as weekly in the CONTRIBUTING.md file, but it is held monthly as per the official [Bitcoin Core PR Review Club website](https://bitcoincore.reviews/). This PR updates the documentation to just remove the frequency.
ACKs for top commit:
fanquake:
ACK 637f437a16
Tree-SHA512: 27bf8a0e32edd8bedb5301ceb3c744ff4629403292a7ad00b633921f36278443ae297cd53708a533b1d6e6eab863b831e11247b95277b94cce28e3d5ddb7d9b9
fa7857ccda build: Enable -Wbidi-chars=any (MarcoFalke)
Pull request description:
I don't see a use-case for UTF-8 bidirectional control characters in this codebase. So disable them for now.
Ref: https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wbidi-chars_003d
ACKs for top commit:
fanquake:
ACK fa7857ccda
Tree-SHA512: 29cf78a2bd0fd94f919f4cd1d1038009a574b4d011146c69bf94d3c06951606200b7d1f827ac6f2fb4540e8f45118ba72b3ccf6c20ef8048e819974371d8f67a
bcd82b13f4 Remove pkgconfig from toolchain file (TheCharlatan)
319a4e8261 depends: drop sqlite pkgconfig file (fanquake)
a8fe1fd38b depends: better cleanup after fontconfig (fanquake)
17e79c9260 depends: fully remove libtool archives from Qt build (fanquake)
8ca85651c8 guix: move pkg-config to Linux builds (fanquake)
e3e648cf41 depends: drop pkg-config option from Qt build (fanquake)
0d185bd99f doc: update depends doc to prefer .cmake outputs (fanquake)
Pull request description:
After #31181, `pkg-config` is no-longer needed for macOS or Windows Guix builds. It's still needed for Linux, as it's used by a Qt subdependency (fontconfig to find freetype). However we should also no-longer need it for Qt itself, when building using depends.
ACKs for top commit:
TheCharlatan:
ACK bcd82b13f4
Tree-SHA512: 89ae68281030d43fcb6c5c96429cd038a21f13a8ca19ea828ada47e8f9f0aa7407854a67c9003652817e47ab9565573b7028342e3e11bb1cca1d823c483081cd
faaaf59f71 test: Make g_rng_temp_path rand, not dependent on SeedRandomForTest (MarcoFalke)
fa80b08fef test: Revert to random path element (MarcoFalke)
Pull request description:
The randomness in the path element is required to allow a single fuzz test to run in parallel. Previous releases used a uint256 random value, but 10 random bytes should be sufficient as well, while avoiding a `MAX_PATH` violation on Windows.
The issue was introduced by myself, by suggesting to use the current time in https://github.com/bitcoin/bitcoin/pull/31000#discussion_r1835351305.
ACKs for top commit:
kevkevinpal:
reACK faaaf59f71
hodlinator:
ACK faaaf59f71
tdb3:
re ACK faaaf59f71
dergoegge:
ACK faaaf59f71
Tree-SHA512: f12256c8b353618291030f71bf36eab97a25ffeaa28e36a5f2c6718dfc1fbbc8548c71475edec53d59026f2a779a05778db83f0530dd3e1d1faf6e4fc0ee7d70
111465d72d test: Remove unused attempts parameter from wait_until (Fabian Jahr)
5468a23eb9 test: Add check_interval parameter to wait_until (Fabian Jahr)
16c87d91fd test: Introduce ensure_for helper (Fabian Jahr)
Pull request description:
A repeating pattern in the functional tests is that the test sleeps for a while to ensure that a certain condition is still true after some amount of time has elapsed. Most recently a new case of this was added in #30807. This PR here introduces an `ensure` helper to streamline this functionality.
Some approach considerations:
- It is possible to construct this by reusing `wait_until` and wrapping it in `try` internally. However, the logger output of the failing wait would still be printed which seems irritating. So I opted for simplified but similar internals to `wait_until`.
- This implementation starts for a failure in the condition right away which has the nice side-effect that it might give feedback on a failure earlier than is currently the case. However, in some cases, it may be expected that the condition may still be false at the beginning and then turns true until time has run out, something that would work when the test sleeps without checking in a loop. I decided against this design (and even against adding it as an option) because such a test design seems like it would be racy either way.
- I have also been going back and forth on naming. To me `ensure` works well but I am also not a native speaker, happy consider a different name if others don't think it's clear enough.
ACKs for top commit:
maflcko:
re-ACK 111465d72d🍋
achow101:
ACK 111465d72d
tdb3:
code review re ACK 111465d72d
furszy:
utACK 111465d72d
Tree-SHA512: ce01a4f3531995375a6fbf01b27d51daa9d4c3d7cd10381be6e86ec5925d2965861000f7cb4796b8d40aabe3b64c4c27e2811270e4e3c9916689575b8ba4a2aa
b2d5361002 build: Temporarily disable compiling `fuzz/utxo_snapshot.cpp` with MSVC (Hennadii Stepanov)
Pull request description:
This PR suggests a temporary workaround for a compiler bug [introduced](https://github.com/bitcoin/bitcoin/issues/31303) in Visual Studio 2022 version 17.12.
This workaround is required to fix the CI until the upstream compiler bug is resolved.
ACKs for top commit:
maflcko:
lgtm ACK b2d5361002
TheCharlatan:
ACK b2d5361002
brunoerg:
ACK b2d5361002
Tree-SHA512: 8f8b772253f6f61d9e24c6ae8692511c7c1229c7d20a45fe680ad6a3909c59b58b504589f06d3135d9708a61684be4aba6a426f7f5f60020551f52b9090d3030
This ensures the options are applied consistently from contexts where
they might not pass through the args manager, such as in some tests, or
when used through the kernel library.
This is similar to the patch applied in 09ef322acc.
This ensures that if a client no longer needs a block template,
the node can clear its memory as soon as possible.
A block template may hold on to transactions that are no longer
in the mempool, so this can be significant.
Requested by clang-tidy:
src/wallet/salvage.cpp:119:18: error: use emplace_back instead of push_back [modernize-use-emplace,-warnings-as-errors]
119 | warnings.push_back(Untranslated("Salvage: Database salvage found errors, all data may not be recoverable."));
| ^~~~~~~~~~
| emplace_back(
This is required for a future commit. Can be reviewed via the git
options --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
Also move util::detail::Hex to a proper namespace instead of an inline
namespace so it doesn't conflict with the new util::detail namespace, and
won't create other problems for callers trying to use the inline namespaces.
Also fix a misleading comment in util_string_tests.cpp.
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
fa1177e3d7 refactor: Avoid std::string format strings (MarcoFalke)
Pull request description:
This changes some unchecked `std::string` format strings to use string literals, which are `consteval` checked at compile-time.
Split out, because it is used in several pull requests.
ACKs for top commit:
l0rinc:
ACK fa1177e3d7
tdb3:
code review and light test ACK fa1177e3d7
rkrux:
tACK fa1177e3d7
ryanofsky:
Code review ACK fa1177e3d7
Tree-SHA512: 7cc70a49b07dadc386336687b463043e79e94a46d18db0184c9813218536e87e954a1afeb8739d5d8706e7b2f355d3f7984048c7de2725851b463985f1c5369f
0bd53d913c test: add test for getchaintips behavior with invalid chains (Martin Zumsande)
ccd98ea4c8 test: cleanup rpc_getchaintips.py (Martin Zumsande)
f5149ddb9b validation: mark blocks building on an invalid block as BLOCK_FAILED_CHILD (Martin Zumsande)
783cb7337f validation: call RecalculateBestHeader in InvalidChainFound (Martin Zumsande)
9275e9689a rpc: call RecalculateBestHeader as part of reconsiderblock (Martin Zumsande)
a51e91783a validation: add RecalculateBestHeader() function (Martin Zumsande)
Pull request description:
`m_best_header` (the most-work header not known to be on an invalid chain) can be wrong in the context of invalidation / reconsideration of blocks. This can happen naturally (a valid header is received and stored in our block tree db; when the full block arrives, it is found to be invalid) or triggered by the user with the `invalidateblock` / `reconsiderblock` rpc.
We don't currently use `m_best_header` for any critical things (see OP of #16974 for a list that still seems up-to-date), so it being wrong affects mostly rpcs.
This PR proposes to recalculate it if necessary by looping over the block index and finding the best header. It also suggest to mark headers between an invalidatetd block and the previous `m_best_header` as invalid, so they won't be considered in the recalculation.
It adds tests to `rpc_invalidateblock.py` and `rpc_getchaintips.py` that fail on master.
One alternative to this suggested in the past would be to introduce a continuous tracking of header tips (#12138).
While this might be more performant, it is also more complicated, and situations where we need this data are only be remotely triggerable by paying the cost of creating a valid PoW header for an invalid block.
Therefore I think it isn't necessary to optimise for performance here, plus the solution in this PR doesn't perform any extra steps in the normal node operation where no invalidated blocks are encountered.
Fixes #26245
ACKs for top commit:
fjahr:
reACK 0bd53d913c
achow101:
ACK 0bd53d913c
TheCharlatan:
Re-ACK 0bd53d913c
Tree-SHA512: 23c2fc42d7c7bb4f9b4ba4949646b3d0031dd29ed15484e436afd66cd821ed48e0f16a1d02f45477b5d0d73a006f6e81a56b82d9721e0dee2e924219f528b445
8610bcef9d ci: skip Github CI on branch pushes for forks (Sjors Provoost)
Pull request description:
When a contributor maintains a fork of the repo, any pull request they make to their own fork, or to the main repository, will trigger two CI runs one for the branch push and one for the pull request.
After this PR when `SKIP_BRANCH_PUSH` is set, pushes made to git branches inside fork repositories will no longer trigger CI runs, unless the git branches are associated with PRs in the fork repository, or the main repository.
The same behaviour was added for Cirrus in e9bfbb5414.
Note to maintainers: `SKIP_BRANCH_PUSH=true` needs to be set for the GUI repo to maintain existing behaviour.
ACKs for top commit:
m3dwards:
ACK 8610bcef9d
achow101:
ACK 8610bcef9d
vasild:
ACK 8610bcef9d
Tree-SHA512: 4055153f03f0cb60a97ce26157ab9db40a4609dee9f060ed7b06aa8841df5bd8e1a42ff2ac0f20bd69e221e8e67bff062a9a361a291124070a03dd51c609e845
42066f45ff Refactor SipHash_32b benchmark to improve accuracy and avoid optimization issues (Lőrinc)
Pull request description:
This PR stems from the discussions in https://github.com/bitcoin/bitcoin/pull/30317#discussion_r1649187336
The previous benchmark for `SipHash` was slightly less accurate in representing real-world usage and allowed for potential compiler optimizations that could invalidate the benchmark.
This change aims to ensure the benchmark produces more realistic results.
By modifying the initial values and only incrementing the bytes of `val`, the benchmark should reflects a more typical usage patterns - and prevent the compiler from optimizing away the calculations.
-------
On my M1 processor the benchmark's speed changed significantly (but the CI seems to produce the same result as before):
> cmake -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_BENCH=ON && cmake --build build -j10 &&
./build/src/bench/bench_bitcoin --filter=SipHash_32b --min-time=1000
Before:
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 35.15 | 28,445,856.66 | 0.2% | 1.10 | `SipHash_32b`
After (note that only the benchmark changed):
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 22.05 | 45,350,886.64 | 0.3% | 1.10 | `SipHash_32b`
ACKs for top commit:
maflcko:
review ACK 42066f45ff
achow101:
ACK 42066f45ff
hodlinator:
ACK 42066f45ff
Tree-SHA512: 6bbe9d725d4c3396642e55ce48c31baa5339e56838d6d5fb377fb1069daa9292375e7020ceff7da0d78befffc1e984f717b5232217fe911989613480adaa937e
192dac1d33 [refactor] Cleanup BlockAssembler mempool usage (TheCharlatan)
Pull request description:
The `addPackageTxs` method of the `BlockAssembler` currently has access to two mempool variables, as an argument and as a member. Clean this up and clarify that they both are the same mempool instance by removing the argument and instead only using the member variable in the method.
This was noticed in this PR review: https://github.com/bitcoin/bitcoin/pull/25223#discussion_r898164322.
ACKs for top commit:
achow101:
ACK 192dac1d33
danielabrozzoni:
re-ACK 192dac1
stickies-v:
ACK 192dac1d33
Tree-SHA512: a5ae7d6d771fbb5b54f23624b4d3429acf07bbe38179a462a078c825d60c89a725ad4e13fe7067eebea7dfec63c56c8f39b5077b0d949d594f500affcc1272d1
After port collisions are no longer tolerated but lead to
a startup failure in v28.0, local setups of multiple nodes,
each with a different -port value would not be possible anymore
due to collision of the onion default port - even if the nodes
were using tor or not interested in receiving onion inbound connections.
Fix this by deriving the onion listening port to be -port + 1.
(idea by vasild / laanwj)
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Pass literal format strings instead of std::string so formats can be
checked at compile time.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: stickies-v <stickies-v@protonmail.com>
ac286e0d1b doc: Fix grammatical errors in multisig-tutorial.md (secp512k2)
Pull request description:
This pull request fixes grammatical errors in the `multisig-tutorial.md` document.
ACKs for top commit:
Abdulkbk:
ACK ac286e0d1b
Tree-SHA512: 684fe16e802431109957b9cde441353edeb16ffffde4282310c1a6f104adffc53347d00a2cf3a5969a78803f3177d6056ca37d3b7e8be52c2ec43ec0b9fcf4d9
fe39acf88f tinyformat: Add compile-time checking for literal format strings (Ryan Ofsky)
184f34f2d0 util: Support dynamic width & precision in ConstevalFormatString (Ryan Ofsky)
Pull request description:
Add compile-time checking for literal format strings passed to `strprintf` and `tfm::format` to make sure the right number of format arguments are passed.
There is still no compile-time checking if non-literal `std::string` or `bilingual_str` format strings are passed, but this is improved in other PRs:
- [#31061](https://github.com/bitcoin/bitcoin/pull/31061) implements compile-time checking for translated strings
- [#31072](https://github.com/bitcoin/bitcoin/pull/31072) increases compile-time checking by using literal strings as format strings, instead of `std::string` and `bilingual_str`
- [#31149](https://github.com/bitcoin/bitcoin/pull/31149) may drop the `std::string` overload for `strprintf` to [require](https://github.com/bitcoin/bitcoin/pull/31149#issuecomment-2444579999) compile-time checking
ACKs for top commit:
maflcko:
re-ACK fe39acf88f🕐
l0rinc:
ACK fe39acf88f
hodlinator:
re-ACK fe39acf88f
Tree-SHA512: f1ddef0c96b9468c5ffe31b42dc19f1922583dd14f2e180b618d992c98614c5cc7db9f9cd917ef503f833bbc7dbec78e4489d0035416dce6840837e1d66d87cb
e80e4c6ff9 validation: Remove RECENT_CONSENSUS_CHANGE validation result (TheCharlatan)
Pull request description:
The *_RECENT_CONSENSUS_CHANGE variants in the validation result enumerations were always unused. They seem to have been kept around speculatively for a soft fork after segwit, however they were never used for taproot either. This points at them not having a clear purpose. Based on the original pull requests' comments their usage was never entirely clear:
https://github.com/bitcoin/bitcoin/pull/11639#issuecomment-370234133https://github.com/bitcoin/bitcoin/pull/15141#discussion_r271039747
Since they are part of the validation interface and need to be exposed by the kernel library keeping them around may also be confusing to future users of the library.
ACKs for top commit:
sipa:
ACK e80e4c6ff9
naumenkogs:
ACK e80e4c6ff9
dergoegge:
ACK e80e4c6ff9
ajtowns:
ACK e80e4c6ff9
Tree-SHA512: 0af17c4435bb1b5a4f43600da30545cbbe95a7d642419cabdefabfb82b9335d92262c1c48be7ca2f2a024078ae9447161228b6f951d2f508a51159a31947fb54
With --skip-missing-binaries, instead of stopping the execution of
gen-manpages.py when a binary is not found, continue generating
manpages for the available binaries and skip the missing ones.
fa66e0887c bench: add support for custom data directory (furszy)
ad9c2cceda test, bench: specialize working directory name (furszy)
Pull request description:
Expands the benchmark framework with the existing `-testdatadir` arg,
enabling the ability to change the benchmark data directory.
This is useful for running benchmarks on different storage devices, and
not just under the OS `/tmp/` directory.
A good use case is #28574, where we are benchmarking the wallet
migration process on an HDD.
ACKs for top commit:
maflcko:
re-ACK fa66e0887c
achow101:
ACK fa66e0887c
tdb3:
re ACK fa66e0887c
hodlinator:
re-ACK fa66e0887c
pablomartin4btc:
re-ACK fa66e0887c
Tree-SHA512: 4e87206c07e26fe193c07074ae9eb0cc9c70a58aeea8cf27d18fb5425d77e4b00dbe0e6d6a75c17b427744e9066458b9a84e5ef7b0420f02a4fccb9c5ef4dacc
The mempool:replaced tracepoint now reports either a txid or a
package hash (previously it always was a txid). To let users know
if a txid or package hash is passed, a boolean argument is added
the the tracepoint.
In the functional test, a ctypes.Structure class for MempoolReplaced
is introduced as Python warns the following when not explcitly
casting it to a ctype:
Type: 'bool' not recognized. Please define the data with ctypes manually.
Rather than individually calling addUnchecked for each transaction added in a
changeset (after removing all the to-be-removed transactions), instead we can
take advantage of boost::multi_index's splicing features to extract and insert
entries directly from the staging multi_index into mapTx.
This has the immediate advantage of saving allocation overhead for mempool
entries which have already been allocated once. This also means that the memory
locations of mempool entries will not change when transactions go from staging
to the main mempool.
Additionally, eliminate addUnchecked and require all new transactions to enter
the mempool via a CTxMemPoolChangeSet.
9c5775c331 addrman: cap the `max_pct` to not exceed the maximum number of addresses (brunoerg)
Pull request description:
Fixes#31234
This PR fixes a bad alloc issue in `GetAddresses` by capping the value `max_pct`. In practice, values greater than 100 should be treated as 100 since it's the percentage of addresses to return. Also, it limites the value `max_pct` in connman target to exercise values between 0 and 100.
ACKs for top commit:
adamandrews1:
Code Review ACK 9c5775c331
marcofleon:
Tested ACK 9c5775c331. Reproduced the crash on master and checked that this fixed it. The checks added to `GetAddr_` look reasonable.
mzumsande:
Code Review ACK 9c5775c331
vasild:
ACK 9c5775c331
Tree-SHA512: 2957ae561ccc37df71f43c1863216d2e563522ea70b9a4baee6990e0b4a1ddadccabdcb9115c131a9a57480367b5ebdd03e0e3d4c8583792e2b7d1911a0a06d3
The previous error message for non-existent wallets of "Already a
descriptor wallet" is misleading. Return a more specific error when a
non-existent wallet is specified.
5c2e291060 bench: Add basic CheckEphemeralSpends benchmark (Greg Sanders)
3f6559fa58 Add release note for ephemeral dust (Greg Sanders)
71a6ab4b33 test: unit test for CheckEphemeralSpends (Greg Sanders)
21d28b2f36 fuzz: add ephemeral_package_eval harness (Greg Sanders)
127719f516 test: Add CheckMempoolEphemeralInvariants (Greg Sanders)
e2e30e89ba functional test: Add ephemeral dust tests (Greg Sanders)
4e68f90139 rpc: disallow in-mempool prioritisation of dusty tx (Greg Sanders)
e1d3e81ab4 policy: Allow dust in transactions, spent in-mempool (Greg Sanders)
04b2714fbb functional test: Add new -dustrelayfee=0 test case (Greg Sanders)
Pull request description:
A replacement for https://github.com/bitcoin/bitcoin/pull/29001
Now that we have 1P1C relay, TRUC transactions and sibling eviction, it makes sense to retarget this feature more narrowly by not introducing a new output type, and simple focusing on the feature of allowing temporary dust in the mempool.
Users of this can immediately use dust outputs as:
1. Single keyed anchor (can be shared by multiple parties)
2. Single unkeyed anchor, ala P2A
Which is useful when the parent transaction cannot have fees for technical or accounting reasons.
What I'm calling "keyed" anchors would be used anytime you don't want a third party to be able to run off with the utxo. As a motivating example, in Ark there is the concept of a "forfeit transaction" which spends a "connector output". The connector output would ideally be 0-value, but you would not want that utxo spend by anyone, because this would cause financial loss for the coordinator of the service: https://arkdev.info/docs/learn/concepts#forfeit-transaction
Note that this specific use-case likely doesn't work as it involves a tree of dust, but the connector idea in general demonstrates how it could be used.
Another related example is connector outputs in BitVM2: https://bitvm.org/bitvm2.html .
Note that non-TRUC usage will be impractical unless the minrelay requirement on individual transactions are dropped in general, which should happen post-cluster mempool.
Lightning Network intends to use this feature post-29.0 if available: https://github.com/lightning/bolts/issues/1171#issuecomment-2373748582
It's also useful for Ark, ln-symmetry, spacechains, Timeout Trees, and other constructs with large presigned trees or other large-N party smart contracts.
ACKs for top commit:
glozow:
reACK 5c2e291060 via range-diff. Nothing but a rebase and removing the conflict.
theStack:
re-ACK 5c2e291060
Tree-SHA512: 88e6a6b3b91dc425de47ccd68b7668c8e98c5683712e892c588f79ad639ae95c665e2d5563dd5e5797983e7542cbd1d4353bc90a7298d45a1843b05a417f09f5
The hardcoded nBits range would occasionally produce values for
the difficulty target that were too low, causing the total work
of the test chain to exceed MinimumChainWork. This fix uses
ConsumeArithUInt256InRange to properly generate targets that
will produce header chains with less work than MinimumChainWork.
Also known as Ephemeral Dust.
We try to ensure that dust is spent in blocks by requiring:
- ephemeral dust tx is 0-fee
- ephemeral dust tx only has one dust output
- If the ephemeral dust transaction has a child,
the dust is spent by by that child.
0-fee requirement means there is no incentive to mine
a transaction which doesn't have a child bringing its
own fees for the transaction package.
83fab3212c test: Add combinerawtransaction test to rpc_createmultisig (Ava Chow)
Pull request description:
The only coverage of combinerawtransaction is in a legacy wallet only test. So also use it in rpc_createmultisig so that this RPC remains tested after the legacy wallet is removed.
Split from #28710
ACKs for top commit:
maflcko:
re-ACK 83fab3212c
BrandonOdiwuor:
Re-ACK 83fab3212c
Abdulkbk:
ACK 83fab3212c
brunoerg:
code review ACK 83fab3212c
rkrux:
tACK 83fab3212c
Tree-SHA512: 383d88ff6c9b54337ed81c714026e527b0fed41d976959fd5c6863b49d0defa4ea13fdc3d984885c86a2b6380825cd66c17842cc31f20fbec4bc42d86aecbbfa
Expands the benchmark framework with the existing '-testdatadir' arg,
enabling the ability to change the benchmark data directory.
This is useful for running benchmarks on different storage devices, and
not just under the OS /tmp/ directory.
Since G_TEST_GET_FULL_NAME is not initialized in the benchmark framework,
benchmarks using the unit test setup run in the same directory without
any clear distinction between them.
This poses an extra complication for locating any specific benchmark
directory during a failure.
In master, unit tests and benchmarks run in the following path:
/<OS_tmp_dir>/test_common bitcoin/<random_uint256>/
After this commit, unit tests and benchmarks are contained within its
own directory:
/<OS_tmp_dir>/test_common bitcoin/<test_name>/<time_in_nanoseconds>/
This makes it easier to find any benchmark run when a failure occurs.
5a96767e3f depends, libevent: Do not install *.pc files and remove patches for them (Hennadii Stepanov)
ffda355b5a cmake, refactor: Move `HAVE_EVHTTP_...` to `libevent` interface (Hennadii Stepanov)
b619bdc330 cmake: Revamp `FindLibevent` module (Hennadii Stepanov)
Pull request description:
This PR generalizes the use of `find_package` / `pkg_check_modules`, prioritizing the former.
Addresses https://github.com/bitcoin/bitcoin/pull/30903#issuecomment-2444700876:
> We should also follow up with refactoring the libevent module, to more generically use CMake/pkg-config, rather than restricting the CMake usage to `vcpkg`. At that point, we'd likely be able to dump pkg-config for the depends path entirely.
Similar to https://github.com/bitcoin/bitcoin/pull/30903.
ACKs for top commit:
fanquake:
ACK 5a96767e3f
Tree-SHA512: 181020c16ccd2821e718c73f264badcdc5e62980c4a8d9691e759efe2ea00da2326e26308d1dcfdeac01e9e27930428ecace9f36941deee951b751b138d7266c
ec375de39f doc: Add missing 'blank=true' option in offline-signing-tutorial.md (secp512k2)
Pull request description:
Issue:
The text mentions that the `createwallet` command should use the options `disable_private_keys=true, blank=true`, but the provided command only includes `disable_private_keys=true`, missing the `blank=true` option.
Correction:
Added `blank=true` to the command to match the options described in the text.
Explanation:
The `blank=true` option is necessary to create a blank wallet. Including this option ensures the command matches the options specified in the text.
ACKs for top commit:
fanquake:
ACK ec375de39f
Tree-SHA512: 8c145e3ef1598c5e13f2aa89e496f76bfe2fc6f47d5e740963acad18dd1f782655a822dc234862af8e5a08060ab7fe1039a3dcfa68455a9143fe2d849975927c
- Some test methods in the functional test framework are independent
and do not require any previous context or setup defined in `run_test`.
- This commit adds a new option for running these specific methods within a test file,
allowing them to be executed individually without running the entire test suite.
- running test methods that require an argument or context will fail.
5e3b444022 doc: Fix missing comma in JSON example in REST-interface.md (secp512k2)
Pull request description:
This pull request addresses a minor issues in the REST-interface.md documentation:
Missing Comma in JSON Example: In the "Query UTXO set" section, a missing comma after the "desc" field in the JSON example has been added to ensure valid JSON syntax.
ACKs for top commit:
maflcko:
lgtm ACK 5e3b444022
Abdulkbk:
ACK 5e3b444022
Tree-SHA512: d2d479c8a991d3380d16b7b140a375a90dca0fce0a024a4b8ccf842d703398fde14ae972349f5fbd2e0ce26aa6cd6d07c0262d9c09ddc4c6c466527cfbe0e1f1
fa729ab4a2 doc: Fixup bitcoin-wallet manpage chain selection args (MarcoFalke)
Pull request description:
The sentence is missing `-testnet4` and `-chain`. Instead of duplicating the full list (and having to keep it in sync), just refer to them as `(test)chain selection arguments`.
ACKs for top commit:
willcl-ark:
utACK fa729ab4a2
tdb3:
Code Review ACK fa729ab4a2
rkrux:
crACK fa729ab4a2
Tree-SHA512: e2cb6e2dd778a34e6c7e8ccde9794ab601e68bad68fe110f41cd73ac12ac3c5d0632fb59a48355f03ef0909f77ec5afd7ea50f301a998cb3ec76e115969f3e7e
4120c7543e scripted-diff: get rid of remaining "command" terminology in protocol.{h,cpp} (Sebastian Falbesoner)
Pull request description:
The confusing "command" terminology for the 12-byte field in the (v1) p2p message header was replaced with the more proper term "message type" in other modules already years ago, see eg #18533, #18937, #24078, #24141. This PR does the same for the protocol.{h,cpp} module to complete the replacements. Note that "GetCommand" is a method name also used in the `ArgsManager` (there it makes much more sense), so the scripted-diff lists for this replacement the files explicitly, rather than using `$(git grep -l ...)`.
ACKs for top commit:
maflcko:
review ACK 4120c7543e🛒
fjahr:
Code review ACK 4120c7543e
rkrux:
tACK 4120c7543e
Tree-SHA512: 7b4dd30136392a145da95d2f3ba181c18c155ba6f3158e49e622d76811c6a45ef9b5c7539a979a04d8404faf18bb27f11457aa436d4e2998ece3deb2c9e59748
36a22e5683 ci: make ctest stop on failure (furszy)
Pull request description:
Make `ctest` stops when the first failure happens.
Wasting less resources and notifying the developer faster when a failure occurs.
ACKs for top commit:
maflcko:
lgtm ACK 36a22e5683
tdb3:
code review and test ACK 36a22e5683
Tree-SHA512: 3abdb330e76aa312f7a5432e3d447a654e6689fc56e067b8e4d07ed8d677fc92f836e603aab0b2f175a6c039a5d50e5fd1160d503164321c1af44ad902f09605
0de3e96e33 tracing: use bitcoind pid in bcc tracing examples (0xb10c)
411c6cfc6c tracing: only prepare tracepoint args if attached (0xb10c)
d524c1ec06 tracing: dedup TRACE macros & rename to TRACEPOINT (0xb10c)
Pull request description:
Currently, if the tracepoints are compiled (e.g. in depends and release builds), we always prepare the tracepoint arguments regardless of the tracepoints being used or not. We made sure that the argument preparation is as cheap as possible, but we can almost completely eliminate any overhead for users not interested in the tracepoints (the vast majority), by gating the tracepoint argument preparation with an `if(something is attached to this tracepoint)`. To achieve this, we use the optional semaphore feature provided by SystemTap.
The first commit simplifies and deduplicates our tracepoint macros from 13 TRACEx macros to a single TRACEPOINT macro. This makes them easier to use and also avoids more duplicate macro definitions in the second commit.
The Linux tracing tools I'm aware of (bcc, bpftrace, libbpf, and systemtap) all support the semaphore gating feature. Thus, all existing tracepoints and their argument preparation is gated in the second commit. For details, please refer to the commit messages and the updated documentation in `doc/tracing.md`.
Also adding unit tests that include all tracepoint macros to make sure there are no compiler problems with them (e.g. some varadiac extension not supported).
Reviewers might want to check:
- Do the tracepoints still work for you? Do the examples in `contrib/tracing/` run on your system (as bpftrace frequently breaks on every new version, please test master too if it should't work for you)? Do the CI interface tests still pass?
- Is the new documentation clear?
- The `TRACEPOINT_SEMAPHORE(event, context)` macros places global variables in our global namespace. Is this something we strictly want to avoid or maybe move to all `TRACEPOINT_SEMAPHORE`s to a separate .cpp file or even namespace? I like having the `TRACEPOINT_SEMAPHORE()` in same file as the `TRACEPOINT()`, but open for suggestion on alternative approaches.
- Are newly added tracepoints in the unit tests visible when using `readelf -n build/src/test/test_bitcoin`? You can run the new unit tests with `./build/src/test/test_bitcoin --run_test=util_trace_tests* --log_level=all`.
<details><summary>Two of the added unit tests demonstrate that we are only processing the tracepoint arguments when attached by having a test-failure condition in the tracepoint argument preparation. The following bpftrace script can be used to demonstrate that the tests do indeed fail when attached to the tracepoints.</summary>
`fail_tests.bt`:
```c
#!/usr/bin/env bpftrace
usdt:./build/src/test/test_bitcoin:test:check_if_attached {
printf("the 'check_if_attached' test should have failed\n");
}
usdt:./build/src/test/test_bitcoin:test:expensive_section {
printf("the 'expensive_section' test should have failed\n");
}
```
Run the unit tests with `./build/src/test/test_bitcoin` and start `bpftrace fail_tests.bt -p $(pidof test_bitcoin)` in a separate terminal. The unit tests should fail with:
```
Running 594 test cases...
test/util_trace_tests.cpp(31): error: in "util_trace_tests/test_tracepoint_check_if_attached": check false has failed
test/util_trace_tests.cpp(51): error: in "util_trace_tests/test_tracepoint_manual_tracepoint_active_check": check false has failed
*** 2 failures are detected in the test module "Bitcoin Core Test Suite"
```
</details>
These links might provide more contextual information for reviewers:
- [How SystemTap Userspace Probes Work by eklitzke](https://eklitzke.org/how-sytemtap-userspace-probes-work) (actually an example on Bitcoin Core; mentions that with semaphores "the overhead for an untraced process is effectively zero.")
- [libbpf comment on USDT semaphore handling](1596a09b5d/src/usdt.c (L83-L92)) (can recommend the whole comment for background on how the tracepoints and tracing tools work together)
- https://sourceware.org/systemtap/wiki/UserSpaceProbeImplementation#Semaphore_Handling
ACKs for top commit:
willcl-ark:
utACK 0de3e96e33
laanwj:
re-ACK 0de3e96e33
jb55:
utACK 0de3e96e33
vasild:
ACK 0de3e96e33
Tree-SHA512: 0e5e0dc5e0353beaf5c446e4be03d447e64228b1be71ee9972fde1d6fac3fac71a9d73c6ce4fa68975f87db2b2bf6eee2009921a2a145e24d83a475d007a559b
The *_RECENT_CONSENSUS_CHANGE variants in the validation result
enumerations were always unused. They seem to have been kept around
speculatively for a soft fork after segwit, however they were never used
for taproot either. This points at them not having a clear purpose.
Based on the original pull requests' comments their usage was never
entirely clear:
https://github.com/bitcoin/bitcoin/pull/11639#issuecomment-370234133https://github.com/bitcoin/bitcoin/pull/15141#discussion_r271039747
Since they are part of the validation interface and need to exposed by
the kernel library keeping them around may also be confusing to future
users of the library.
This pull request addresses a minor issues in the REST-interface.md documentation:
Missing Comma in JSON Example: In the "Query UTXO set" section, a missing comma after the "desc" field in the JSON example has been added to ensure valid JSON syntax.
c189eec848 doc: release note for mempoolrullrbf removal (Greg Sanders)
d47297c6aa rpc: Mark fullrbf and bip125-replaceable as deprecated (Greg Sanders)
04a5dcee8a docs: remove requirement to signal bip125 (Greg Sanders)
111a23d9b3 Remove -mempoolfullrbf option (Greg Sanders)
Pull request description:
Given https://github.com/bitcoin/bitcoin/pull/30493 and the related discussion on network uptake it's probably not helpful to have an option for a feature that will not be respected by the network in any meaningful way.
Wallet changes can be done in another PR on its own cadence to account for possible fingerprinting, waiting for fullrbf logic to permeate the network, etc.
ACKs for top commit:
stickies-v:
re-ACK c189eec848
achow101:
ACK c189eec848
murchandamus:
ACK c189eec848
rkrux:
reACK c189eec848
Tree-SHA512: 9447f88f8f291c56c5bde70af0a91b0a4f5163aaaf173370fbfdaa3c3fd0b44120b14d3a1977f7ee10e27ffe9453f8a70dd38aad0ffb8c39cf145049d2550730
The only coverage of combinerawtransaction is in a legacy wallet only
test. So also use it in rpc_createmultisig so that this RPC remains
tested after the legacy wallet is removed.
While we will only outwardly support Windows 10+, due to an issue in
mingw-w64, we can't set the *-subsystem-version values higher than to
target Windows 8, so do that as a best effort.
e2ba823671 depends: Specify CMake generator explicitly (Hennadii Stepanov)
Pull request description:
Building packages in depends implies using GNU Make. However, this assumption can be wrong in environments where the [`CMAKE_GENERATOR`](https://cmake.org/cmake/help/latest/envvar/CMAKE_GENERATOR.html) variable is set.
This change explicitly makes CMake use the "Unix Makefiles" generator.
Can be tested as follows:
```
$ env CMAKE_GENERATOR=Ninja make -C depends
```
ACKs for top commit:
fanquake:
ACK e2ba823671 - Going forward I think we should look at making this work without having to hard code anything.
Tree-SHA512: e14ed1cec192434fe089d36a83e1e150727a3b299fada80a61fa5b44b0c50e014a774ef1e6cd6df189e25f7a13042a20d4f9605f6ccd32e7782f10adaf5e788f
If we know about a pubkey that's in our descriptor, but we don't have
the private key, don't return a SigningProvider for that pubkey.
This is specifically an issue for Taproot outputs that use the H point
as the resulting PSBTs may end up containing irrelevant information
because the H point was detected as a pubkey each unrelated descriptor
knew about.
fa461d7a43 fuzz: Limit wallet_notifications iterations (MarcoFalke)
Pull request description:
I don't think the fuzz target has ever found a real issue. The closest being https://github.com/bitcoin/bitcoin/pull/25869
It is also, by far, the slowest fuzz target. For example, looking at https://cirrus-ci.com/task/5533338067271680?logs=ci#L3974, it takes more than one hour:
```
Run wallet_notifications with args ['/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/fuzz', '-runs=1', PosixPath('/ci_container_base/ci/scratch/qa-assets/fuzz_corpora/wallet_notifications')]INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 1096115652
INFO: Loaded 1 modules (625824 inline 8-bit counters): 625824 [0x5628396d9138, 0x562839771dd8),
INFO: Loaded 1 PC tables (625824 PCs): 625824 [0x562839771dd8,0x56283a0fe7d8),
INFO: 1287 files found in /ci_container_base/ci/scratch/qa-assets/fuzz_corpora/wallet_notifications
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1047827 bytes
INFO: seed corpus: files: 1287 min: 1b max: 1047827b total: 11616898b rss: 172Mb
#16pulse cov: 14328 ft: 25341 corp: 14/239b exec/s: 5 rss: 204Mb
#64pulse cov: 19179 ft: 58412 corp: 61/3587b exec/s: 5 rss: 320Mb
#128pulse cov: 19692 ft: 85738 corp: 125/16Kb exec/s: 3 rss: 544Mb
#256pulse cov: 19923 ft: 107490 corp: 253/72Kb exec/s: 2 rss: 556Mb
#512pulse cov: 20107 ft: 124704 corp: 509/330Kb exec/s: 2 rss: 590Mb
Slowest unit: 10 s:
artifact_prefix='./'; Test unit written to ./slow-unit-9fa5f7d7e4afa1626622ef1b3c70a7563eecf11d
#1024pulse cov: 20360 ft: 136324 corp: 1009/2488Kb exec/s: 0 rss: 726Mb
Slowest unit: 23 s:
artifact_prefix='./'; Test unit written to ./slow-unit-5d99a20de2c2b6bedb0cbaf0ba3743ae3ba13c7c
Slowest unit: 26 s:
artifact_prefix='./'; Test unit written to ./slow-unit-8889ecb61bdc0650355e0d0d27c012f3239d07a4
Slowest unit: 42 s:
artifact_prefix='./'; Test unit written to ./slow-unit-d16c084282ac1a85fcdc43c48e49836b08446686
#1289INITED cov: 20409 ft: 138281 corp: 1245/10323Kb exec/s: 0 rss: 880Mb
#1289DONE cov: 20409 ft: 138281 corp: 1245/10323Kb lim: 1047827 exec/s: 0 rss: 880Mb
Done 1289 runs in 3813 second(s)
```
Looking at the flame graphs, it looks like the slow runs spend most of their time in the Knapsack solver. This seems reasonable, because it may run 1000 inner Knapsack iterations 200 times. So reduce the fuzz iterations from 200 to 20 to avoid fuzz timeouts and wasted resources.
ACKs for top commit:
brunoerg:
code review ACK fa461d7a43
dergoegge:
lgtm ACK fa461d7a43
Tree-SHA512: bee707a3398ab0c729f335f00d8cad63135939831454dd863830fc957b4b51b27064224be0ed15eb76cfcc39de972e4e79b0802940934fbac516840ddc475ab9
d22a234ed2 net: Use actual memory size in receive buffer accounting (laanwj)
047b5e2af1 streams: add DataStream::GetMemoryUsage (laanwj)
c3a6722f34 net: Use DynamicUsage(m_type) in CSerializedNetMsg::GetMemoryUsage (laanwj)
c6594c0b14 memusage: Add DynamicUsage for std::string (laanwj)
7596282a55 memusage: Allow counting usage of vectors with different allocators (laanwj)
Pull request description:
Add a method `CNetMessage::GetMemoryUsage` and use this for accounting of the size of the process receive queue instead of the raw message size (like we already do for the send buffer and `CSerializedNetMsg`).
This ensures that allocation and deserialization overhead is better taken into account.
On average, this counts about ~100 extra bytes per packet on x86_64:
```
2024-10-27T09:50:12Z [net] 24 bytes -> 112 bytes
2024-10-27T10:36:37Z [net] 61 bytes -> 176 bytes
2024-10-27T10:36:38Z [net] 1285 bytes -> 1392 bytes
2024-10-27T09:50:21Z [net] 43057 bytes -> 43168 bytes
```
ACKs for top commit:
l0rinc:
ACK d22a234ed2
achow101:
ACK d22a234ed2
i-am-yuvi:
ACK d22a234ed2
danielabrozzoni:
Light ACK d22a234ed2 - code looks good to me, but I'm not very familiar with C++ memory management specifics
Tree-SHA512: ef09707e77b67bdbc48e9464133e4fccfa5c05051c1022e81ad84f20ed41db83ac5a9b109ebdb8d38f70785c03c5d6bfe51d32dc133d49e52d1e6225f6f8e292
Issue:
The text mentions that the `createwallet` command should use the options `disable_private_keys=true, blank=true`, but the provided command only includes `disable_private_keys=true`, missing the `blank=true` option.
Correction:
Added `blank=true` to the command to match the options described in the text.
Explanation:
The `blank=true` option is necessary to create a blank wallet. Including this option ensures the command matches the options specified in the text.
5161c2618c ci: add second_deadlock_stack=1 to TSAN options (fanquake)
Pull request description:
This is mentioned in the developer notes, but isn't present in `TSAN_OPTIONS`, resulting in:
```bash
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=60508)
Cycle in lock order graph: M0 (0xffff98e02208) => M1 (0xffff98e0cbe8) => M2 (0xffff98e0cd98) => M0
<snip>
Hint: use TSAN_OPTIONS=second_deadlock_stack=1 to get more informative warning message
```
Add it, for (potentially) more informative output, when failures occur. Checked that adding does output more information.
ACKs for top commit:
maflcko:
lgtm ACK 5161c2618c
hebasto:
ACK 5161c2618c.
Tree-SHA512: 57bfa24d248ed76ba0db537edff425453a0765c4768fc1b6f59a87876d4acf63ed38bb3c20f369a008ae256472d9d24e58d76729d423f662dfdb2952afc46cb0
Previously this assertion checked MAX_PEER_TX_REQUEST_IN_FLIGHT was not
exceeded. However, this property is not actually enforced; it is just
used to determine when a peer is overloaded.
9e5089dbb0 build, msvc: Enable `libqrencode` vcpkg package (Hennadii Stepanov)
30089b0cb6 cmake: Add `FindQRencode` module (Hennadii Stepanov)
Pull request description:
This PR introduces the `FindQRencode` CMake module, following the official CMake [guidelines](https://cmake.org/cmake/help/latest/manual/cmake-developer.7.html#find-modules) for managing [upstream libraries](https://github.com/fukuchi/libqrencode) that lack a config file package. This module enhances flexibility in locating the `libqrencode` library by making the use of `pkg-config` optional.
With this update, `libqrencode` can be detected on systems where either `pkg-config` or the `libqrencode.pc` file is unavailable, such as Windows environments using the vcpkg package manager. However, if `libqrencode.pc` is available, it remains beneficial as the only direct source of the library's version information.
Additionally, the `libqrencode` vcpkg package is enabled for MSVC builds.
Here is a diff for configuration output on Ubuntu 24.10:
```diff
-- Detecting CXX compile features - done
-- Found SQLite3: /usr/include (found suitable version "3.46.1", minimum required is "3.7.17")
-- Found PkgConfig: /usr/bin/pkg-config (found version "1.8.1")
--- Checking for module 'libqrencode'
--- Found libqrencode, version 4.1.1
+-- Found QRencode: /usr/lib/x86_64-linux-gnu/libqrencode.so (found version "4.1.1")
-- Found Qt: /usr/lib/x86_64-linux-gnu/cmake/Qt5 (found suitable version "5.15.15", minimum required is "5.11.3")
-- Performing Test CXX_SUPPORTS__WERROR
-- Performing Test CXX_SUPPORTS__WERROR - Success
```
ACKs for top commit:
fanquake:
ACK 9e5089dbb0
Tree-SHA512: bb9baca64386772f2f4752b1cbff1230792562ca6b2e37c56ad28580b55b1ae6ff65c2cf0d8ab026111d7b5a056d7ac672496a3cfd1a81e4fdd2b84c8cf75fff
97235c446e build: Disable secp256k1 musig module (Ava Chow)
2d46a89386 Squashed 'src/secp256k1/' changes from 2f2ccc46954..0cdc758a563 (Ava Chow)
Pull request description:
v0.6.0 was just released, main change is that it has the musig module which #29675 needs.
ACKs for top commit:
hebasto:
ACK 97235c446e, verified by updating the secp256k1 subtree locally.
laanwj:
ACK 97235c446e
Tree-SHA512: af92da26fc9afb55399b73d80198c0d2aa1adfae7b91f0ad20ffeb519135baf7e78243049b9bd45a2027943931b2d657c944f93151e5200d95a6f3c90b831f31
This is mentioned in the developer notes, but isn't present in
`TSAN_OPTIONS`, resulting in:
```bash
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=60508)
Cycle in lock order graph: M0 (0xffff98e02208) => M1 (0xffff98e0cbe8) => M2 (0xffff98e0cd98) => M0
<snip>
Hint: use TSAN_OPTIONS=second_deadlock_stack=1 to get more informative warning message
```
Add it, for (potentially) more informative output, when failures
occur. Checked that adding does output more information.
fabe90c824 ci: Use clang-19 from apt.llvm.org (MarcoFalke)
Pull request description:
A new clang version generally comes with bugfixes, new sanitizer features, deprecations, as well as new features.
Upgrade the sanitizer tasks to use the new version.
ACKs for top commit:
TheCharlatan:
ACK fabe90c824
hebasto:
ACK fabe90c824, I have reviewed the code and it looks OK.
Tree-SHA512: 637452e3cbc8ad97a3650976e2dbf4bcd032f2c454e12b48514eb5d252b3e106448674fe2d3bf45d11c0782874250d7a79e34fbb0aaa990499a156fcafd50329
This pull request fixes grammatical errors in the 'multisig-tutorial.md' document.
Corrections:
1. Incorrect Phrase "As can been seen":
- Before:
There are discussions about eliminating this redundancy, as can been seen in the issue #17190 (https://github.com/bitcoin/bitcoin/issues/17190).
- After:
There are discussions about eliminating this redundancy, as can be seen in the issue #17190 (https://github.com/bitcoin/bitcoin/issues/17190).
2. Clarity Improvement in a Sentence:
- Before:
Note that at least two descriptors are usually used, one for internal derivation paths and external ones.
- After:
Note that at least two descriptors are usually used, one for internal derivation paths and one for external ones.
Explanation:
- Corrected "been" to "be" to fix the grammatical error.
- Added "one for" before "external ones" to improve clarity and parallel structure in the sentence.
These minor corrections enhance the readability and professionalism of the documentation. Thank you for considering this pull request.
f6577b7174 build, msvc: Update vcpkg manifest baseline (Hennadii Stepanov)
16e16013bf build, msvc: Document `libevent` version pinning (Hennadii Stepanov)
ec47cd2b50 build, msvc: Drop no longer needed `liblzma` version pinning (Hennadii Stepanov)
9a0734df5f build, msvc: Reorder keys in `vcpkg.json` (Hennadii Stepanov)
Pull request description:
This PR updates the vcpkg manifest baseline from the [2023.08.09 Release ](https://github.com/microsoft/vcpkg/releases/tag/2023.08.09) to the [2024.09.30 Release](https://github.com/microsoft/vcpkg/releases/tag/2024.09.30), with the following package changes:
- `boost`: 1.82.0#2 --> 1.85.0#1,2
- `qt5`: 5.15.10#5 -> 5.15.15
- `sqlite3`: 3.42.0#1 --> 3.46.1
- `zeromq`: 2023-06-20#1 --> 4.3.5#2
The previous update was made in https://github.com/bitcoin/bitcoin/pull/28938.
For additional minor improvements, please refer to the commit messages.
ACKs for top commit:
fanquake:
ACK f6577b7174
Tree-SHA512: bfd6f995d97cd3222573ac1c3626c13ee68cf3e2de344869a2d91775090d60f63ef2b17d9a59eba46620eedd51d6787aebe3aeed1189ec55379211a186c21b4e
9f71cff6ab doc: Use relative hyperlinks in release-process.md (Jeremy Rand)
Pull request description:
Improves usability with offline clones of the documentation.
Refs
https://github.com/bitcoin/bitcoin/pull/30025#issuecomment-2093081127
ACKs for top commit:
maflcko:
review ACK 9f71cff6ab
Tree-SHA512: 475603556e111ec21e656e9d105f742e6881fbfce220347951c96406ffe8a71da0b10a0631dd2da89f59e9b76d5d9980b3e5c8f97a9c2562ff58422ae41d1343
44939e5de1 doc: Fix word order in developer-notes.md (secp512k2)
Pull request description:
This pull request fixes a word order error in developer-notes.md.
ACKs for top commit:
fanquake:
ACK 44939e5de1
Tree-SHA512: f8c8f2a976940c6fb3483c13c048accd073a8486e0d614ca9da15bf166c41f1f1c1cd57678359af04a32c5ffa15afa647a3f015f89c83997c69803a62fad8de6
4747f03095 depends, doc: List packages required to build `qt` package separately (Hennadii Stepanov)
Pull request description:
Suggested in https://github.com/bitcoin/bitcoin/pull/30997#discussion_r1791789613:
> There's probably enough GUI-only stuff here, i.e `bison`, `ninja-build`, `python3`, `xz-utils`, that this could be moved to it's own `#### Gui` section.
ACKs for top commit:
fanquake:
ACK 4747f03095
Tree-SHA512: 090af77606e9c1f87b3466d6a6c97745af456943495bc7df46cdb5e955f641c39da8a6f7590fd1cc0ea816e320d7c336a860faffc2b35b0d5014dabbc490d9f9
fafbf8acf4 Make G_FUZZING constexpr, require -DBUILD_FOR_FUZZING=ON to execute a fuzz target (MarcoFalke)
fae3cf0ffa ci: Temporarily disable macOS/Windows fuzz step (MarcoFalke)
Pull request description:
`g_fuzzing` is used inside `Assume` at runtime, causing significant overhead in hot paths. See https://github.com/bitcoin/bitcoin/issues/31178
One could simply remove the `g_fuzzing` check from the `Assume`, but this would make fuzzing a bit less useful. Also, it would be unclear if `g_fuzzing` adds a runtime overhead in other code paths today or in the future.
Fix all issues by making `G_FUZZING` equal to the build option `BUILD_FOR_FUZZING`, and for consistency in fuzzing, require it to be set when executing any fuzz target.
Fixes https://github.com/bitcoin/bitcoin/issues/31178
Temporarily this drops fuzzing from two CI tasks, but they can be re-added in a follow-up with something like https://github.com/bitcoin/bitcoin/pull/31073
ACKs for top commit:
marcofleon:
Tested ACK fafbf8acf4
davidgumberg:
I still ACK fafbf8acf4 for fixing the regression measured in #31178.
ryanofsky:
Code review ACK fafbf8acf4 but approach -0, because this approach means libraries built for fuzz testing do not function correctly if used in a release, and libraries built for releases are mostly useless for fuzz testing. So I would like to at least consider other solutions to this problem even if we go with this one.
dergoegge:
utACK fafbf8acf4
Tree-SHA512: 124fc2e8b35e0c4df414436556a7a0a36cd1bec4b3000b40dcf2ab8c85f32e0610bf7f70d2fd79223d62f3c3665b6c09da21241654c7b9859461b8ca340d5421
This pull request fixes a word order error in developer-notes.md.
Before:
"In cases where do you call .c_str(), you might want to additionally check that the string does not contain embedded '\0' characters..."
After:
"In cases where you do call .c_str(), you might want to additionally check that the string does not contain embedded '\0' characters..."
Explanation:
The sentence had incorrect word order, making it grammatically incorrect. Rearranging "do you" to "you do" corrects the sentence, improving the readability and clarity of the documentation.
87532fe558 netinfo: allow setting an outbound-only peer list (Jon Atack)
681ebcceca netinfo: rename and hoist max level constant to use in top-level help (Jon Atack)
e7d307ce8c netinfo: clarify relaytxes and addr_relay_enabled help docs (Jon Atack)
eef2a9d406 netinfo: add peer services column (Jon Atack)
Pull request description:
Been using this since May 2023.
- add a peer services column (considered displaying the p2p_v2 flag as "p" or "2"; proposing "2" here for continuity with the "v" column, but "p" is fine for me as well)
- clarify in the help that "relaytxes" and "addr_relay_enabled" are from getpeerinfo
- hoist (and rename) the max level constant to use in top-level help, to avoid overlooking to update the top-level help if the value of the constant changes (as caught by Larry Ruane in review below)
- add an optional "outonly" (or "o") argument for an outbound-only peer list, as suggested by Vasil Dimov in his review below. Several people have requested this, to keep the output within screen limits when running netinfo as a live dashboard (i.e. with `watch`) on a node with many peers. While doing this, also permit passing "h" for the help in addition to "help".
ACKs for top commit:
achow101:
ACK 87532fe558
rkrux:
tACK 87532fe558
tdb3:
cr re ACK 87532fe558
brunoerg:
crACK 87532fe558
Tree-SHA512: 35b1b0de28dfecaad58bf5af194757a5e0f563553cf69ea4d76f2e1963f8d662717254df2549114c7bba4a041bf5282d5cb3fba8d436b2807f2a00560787d64c
0cdc758a563 Merge bitcoin-core/secp256k1#1631: release: prepare for 0.6.0
39d5dfd542a release: prepare for 0.6.0
df2eceb2790 build: add ellswift.md and musig.md to release tarball
a306bb7e903 tools: fix check-abi.sh after cmake out locations were changed
145868a84d2 Do not export `secp256k1_musig_nonce_gen_internal`
b161bffb8bf Merge bitcoin-core/secp256k1#1579: Clear sensitive memory without getting optimized out (revival of #636)
a38d879a1a6 Merge bitcoin-core/secp256k1#1628: Name public API structs
7d48f5ed02e Merge bitcoin-core/secp256k1#1581: test, ci: Lower default iteration count to 16
694342fdb71 Name public API structs
0f73caf7c62 test, ci: Lower default iteration count to 16
9a8db52f4e9 Merge bitcoin-core/secp256k1#1582: cmake, test: Add `secp256k1_` prefix to test names
765ef53335a Clear _gej instances after point multiplication to avoid potential leaks
349e6ab916b Introduce separate _clear functions for hash module
99cc9fd6d01 Don't rely on memset to set signed integers to 0
97c57f42ba8 Implement various _clear() functions with secp256k1_memclear()
9bb368d1466 Use secp256k1_memclear() to clear stack memory instead of memset()
e3497bbf001 Separate between clearing memory and setting to zero in tests
d79a6ccd43a Separate secp256k1_fe_set_int( . , 0 ) from secp256k1_fe_clear()
1c081262227 Add secp256k1_memclear() for clearing secret data
1464f15c812 Merge bitcoin-core/secp256k1#1625: util: Remove unused (u)int64_t formatting macros
980c08df80a util: Remove unused (u)int64_t formatting macros
9b7c59cbb90 Merge bitcoin-core/secp256k1#1624: ci: Update macOS image
096e3e23f63 ci: Update macOS image
e7d384488e8 Don't clear secrets in pippenger implementation
68b55209f1b Merge bitcoin-core/secp256k1#1619: musig: ctimetests: fix _declassify range for generated nonce points
f0868a9b3d8 Merge bitcoin-core/secp256k1#1595: build: 45839th attempt to fix symbol visibility on Windows
1fae76f50c0 Merge bitcoin-core/secp256k1#1620: Remove unused scratch space from API
8be3839fb2e Remove unused scratch space from API
57eda3ba300 musig: ctimetests: fix _declassify range for generated nonce points
87384f5c0f2 cmake, test: Add `secp256k1_` prefix to test names
e59158b6eb7 Merge bitcoin-core/secp256k1#1553: cmake: Set top-level target output locations
18f9b967c25 Merge bitcoin-core/secp256k1#1616: examples: do not retry generating seckey randomness in musig
5bab8f6d3c4 examples: make key generation doc consistent
e8908221a45 examples: do not retry generating seckey randomness in musig
70b6be1834e extrakeys: improve doc of keypair_create (don't suggest retry)
01b5893389e Merge bitcoin-core/secp256k1#1599: #1570 improve examples: remove key generation loop
cd4f84f3ba8 Improve examples/documentation: remove key generation loops
a88aa935063 Merge bitcoin-core/secp256k1#1603: f can never equal -m
3660fe5e2a9 Merge bitcoin-core/secp256k1#1479: Add module "musig" that implements MuSig2 multi-signatures (BIP 327)
168c92011f5 build: allow enabling the musig module in cmake
f411841a46b Add module "musig" that implements MuSig2 multi-signatures (BIP 327)
0be79660f38 util: add constant-time is_zero_array function
c8fbdb1b972 group: add ge_to_bytes_ext and ge_from_bytes_ext
ef7ff03407f f can never equal -m
c232486d84e Revert "cmake: Set `ENVIRONMENT` property for examples on Windows"
26e4a7c2146 cmake: Set top-level target output locations
4c57c7a5a95 Merge bitcoin-core/secp256k1#1554: cmake: Clean up testing code
447334cb06d include: Avoid visibility("default") on Windows
472faaa8ee6 Merge bitcoin-core/secp256k1#1604: doc: fix typos in `secp256k1_ecdsa_{recoverable_,}signature` API description
292310fbb24 doc: fix typos in `secp256k1_ecdsa_{recoverable_,}signature` API description
85e224dd97f group: add ge_to_bytes and ge_from_bytes
7c987ec89e6 cmake: Call `enable_testing()` unconditionally
6aa576515ef cmake: Delete `CTest` module
git-subtree-dir: src/secp256k1
git-subtree-split: 0cdc758a56360bf58a851fe91085a327ec97685a
Add a method CNetMessage::GetMemoryUsage and use this for accounting of
the size of the process receive queue instead of the raw message size.
This ensures that allocation and deserialization overhead is taken into
account.
It was only necessary for switching between mapping protocols. It's also used to return
in ThreadMapPort but we can just use the interrupt for this purpose.
- Modify `SipHash_32b` benchmark to use `FastRandomContext` for generating initial values.
- Cycle through and modify each byte of the `uint256` value to ensure no part of it can be optimized away.
The lack of "recursion" (where the method call overwrites the used inputs partially) and the systematic modification of each input byte makes the benchmark usage more reliable and thorough.
d7fd766feb test: added test to assert TX decode rpc error on submitpackage rpc (kevkevinpal)
Pull request description:
This PR adds coverage for this line https://github.com/bitcoin/bitcoin/blob/master/src/rpc/mempool.cpp#L996
If you run the following you will get no results for `submitpackage`
`grep -nri "TX decode failed" ./test/functional`
ACKs for top commit:
achow101:
ACK d7fd766feb
instagibbs:
reACK d7fd766feb
tdb3:
ACK d7fd766feb
rkrux:
reACK d7fd766feb
Tree-SHA512: e92e0e2621a4efab35625d8da3ac61ccb7fa65c378aa977112bc132fd3b42431f8c3ceb081f7c9903ed2833c229042b65bdb11444e1d6367354ae65dc7504231
a1b3ccae4b init: warn, don't error, when '-upnp' is set (Antoine Poinsot)
Pull request description:
It prevented the GUI from starting when its settings.json had the -upnp option set. This also doesn't prevent the node from running, so this error didn't need to be fatal.
Thanks to Sjors for bringing attention to what i broke and to Maflcko for suggesting a simple short term fix.
Fixes https://github.com/bitcoin-core/gui/issues/843.
ACKs for top commit:
maflcko:
lgtm ACK a1b3ccae4b
kevkevinpal:
Concept ACK [a1b3cca](a1b3ccae4b)
achow101:
ACK a1b3ccae4b
tdb3:
ACK a1b3ccae4b
Tree-SHA512: ceb1513bf532698e5143d64430a065f39626ef0d2708103ffc8ab7f81e8393f488af2350c5a299bc80f966add82a3951b4d81ae8b0e3070c0d15c94e8db4badd
54d07dd37d ci: Do not error on unused-member-function in test each commit (Sergi Delgado Segura)
Pull request description:
After https://github.com/bitcoin/bitcoin/pull/31045, an unused method in a commit will trigger a compilation error, even if that method is later used in a following commit within the same PR.
Do not enforce unused-member-function in test each commit.
Close#31180
ACKs for top commit:
kevkevinpal:
lgtm ACK [54d07dd](54d07dd37d)
maflcko:
lgtm ACK 54d07dd37d
TheCharlatan:
lgtm ACK 54d07dd37d
Tree-SHA512: 79723ca604d44b065b08793b442de9049a689cc5a4df508299a91966456bf77e24a0f3febaf3114fd23b061716554b70ee866bf594283960d952366ea2096dbc
It prevented the GUI from starting when its settings.json had the -upnp option set. This also doesn't
prevent the node from running, so this error didn't need to be fatal.
Thanks to Sjors for bringing attention to what i broke and to Maflcko for suggestion a simple short
term fix.
bbbbaa0d9a Fix unsigned integer overflows in interpreter (MarcoFalke)
Pull request description:
Unsigned integer overflow is well defined by the language and in some cases even useful or necessary. However, I think that it should be avoided in interpreter, as it makes the code harder to read and requires the whole file to be suppressed in the sanitizer. This puts more burden on reviewers to check that any changes to interpreter that involve unsigned integer overflow are sane.
This patch involves a few changes:
* Evaluate the addition in 64-bit "space". Previously, the first argument was `size_t` (unsigned, 32-bit or 64-bit, depending on platform) and the second was `int` (32-bit on all supported platforms). Thus the addition was done in 32-bit or 64-bit "unsigned space". Now the addition is done in 64-bit "signed space" on all platforms. This is safe because signed integer overflow (UB) isn't expected here with 64-bit integers.
* Clarify that the value passed to the "stack macros" always fits in an `int64_t`. This is done with the C++11 syntax `int64_t{i}`, which fails to compile if `i` needs to be narrowed to fit into `int64_t`.
* Explicitly convert the result of the addition to `size_t`. This isn't needed, because the called function already converts the value (see https://en.cppreference.com/w/cpp/container/vector/operator_at), however I have a slight preference for the explicit cast. (Happy to remove if reviewers prefer without)
The patch does not change the bitcoind binary on my 64-bit system with `clang++ -O2`. However, it does change with gcc.
ACKs for top commit:
achow101:
ACK bbbbaa0d9a
ismaelsadeeq:
Code review ACK bbbbaa0d9a
hebasto:
ACK bbbbaa0d9a, I have reviewed the code and it looks OK.
Tree-SHA512: 0e9cbc6a0afd3db0d1d9489fd5e32ff856217604abde370add1f01c2cae8c526f2afedeb372997217c3a70ab0f8f56442e8230f87456f8e21c9abcb7c6578f7c
2957ca9611 build: have "make test" depend on "make all" (Cory Fields)
Pull request description:
See [Upstream docs](https://cmake.org/cmake/help/latest/variable/CMAKE_SKIP_TEST_ALL_DEPENDENCY.html) for specifics.
Unfortunately, this **seems to have no effect when directly executing `ctest`** :(
This brings the test -> hack -> test cycle more inline with how it worked with autotools.
With `CMAKE_SKIP_TEST_ALL_DEPENDENCY` set to FALSE, `make test` will trigger a rebuild, ensuring that test binaries are current before running them.
To test:
```
cmake -S . -B build
make -C build -j24
touch src/primitives/transaction.cpp
make -C build test ARGS=-j24
```
Without this commit, the above will not rebuild before running tests.
ACKs for top commit:
tdb3:
ACK 2957ca9611
itornaza:
ACK 2957ca9611
laanwj:
ACK 2957ca9611
Tree-SHA512: 46e6d53fd9c28b2f47b34c42ae0b061b01ef7614cdf5ee7d49fb9d3413ca92983d8a996379a9067ac8a8cfc16b230946401e68cf114bd4e3c5c02db2c55ec6bb
e60cecc811 doc: add release note for 31156 (Martin Zumsande)
fc7dfb3df5 test: Don't enforce BIP94 on regtest unless specified by arg (Martin Zumsande)
Pull request description:
The added arg `-test=bip94` is only used in a functional test for BIP94. This is done because the default regtest consensus rules should follow mainnet, not testnet.
Fixes#31137.
ACKs for top commit:
achow101:
ACK e60cecc811
tdb3:
cr and light test ACK e60cecc811
rkrux:
tACK e60cecc811
BrandonOdiwuor:
utACK e60cecc811
laanwj:
Code review ACK e60cecc811
Tree-SHA512: ca2f322f89d8808dfc3565fe020d2615cfcc110e188a02128ad7108fef51c735b33d55b5e6a70c505d78f7291f3c635dc7dfbcd78be1348d4d6e483883be4216
559a8dd9c0 key: clear out secret data in `DecodeExtKey` (Sebastian Falbesoner)
Pull request description:
Same as in `DecodeSecret`, we should also clear out the secret data from the vector resulting from the Base58Check parsing for xprv keys. Note that the if condition is needed in order to avoid UB, see #14242 (commit d855e4cac8).
ACKs for top commit:
davidgumberg:
utACK 559a8dd9c0
achow101:
ACK 559a8dd9c0
tdb3:
cr ACK 559a8dd9c0
laanwj:
Code review ACK 559a8dd9c0
Tree-SHA512: c22499fe2899a9a5a58159ec55e94cf961570d8af06358d4a6d1943d567be9b88657af90d060d3083985ea957886a4f91bb762a2fcf3311007e7a535b42b0fde
57529ac4db test: set P2PConnection.p2p_connected_to_node in peer_connect_helper() (Vasil Dimov)
22cd0e888c test: support WTX INVs from P2PDataStore and fix a comment (Vasil Dimov)
ebe42c00aa test: extend the SOCKS5 Python proxy to actually connect to a destination (Vasil Dimov)
ba621ffb9c test: improve debug log message from P2PConnection::connection_made() (Vasil Dimov)
Pull request description:
If requested, make the SOCKS5 Python proxy redirect connections to a set of given destinations. Actually act as a real proxy, connecting the client to a destination, except that the destination is not what the client asked for.
This would enable us to "connect" to Tor addresses from the functional tests.
Plus a few other minor improvements in the test framework as individual commits.
---
These changes are part of https://github.com/bitcoin/bitcoin/pull/29415 but they make sense on their own and would be good to have them, regardless of the fate of #29415. Also, if this is merged, that would reduce the size of #29415, thus the current standalone PR.
ACKs for top commit:
jonatack:
Approach ACK 57529ac4db
achow101:
ACK 57529ac4db
tdb3:
CR and test ACK 57529ac4db
mzumsande:
Code review / tested ACK 57529ac4db
Tree-SHA512: a2892c97bff2d337b37455c409c6136cb62423ce6cc32b197b36f220c1eec9ca046b599135b9a2603c0eb6c1ac4d9795e73831ef0f04378aeea8b245ea733399
3a4a788ee0 init: Correct coins db cache size setting (TheCharlatan)
Pull request description:
The chainstate caches are currently re-balanced on startup even in the non-assumeutxo case, leading to the database being needlessly re-opened and its cache re-allocated.
Similar to `InitCoinsCache` and `m_coinstip_cache_size_bytes`, the `m_coinsdb_cache_size_bytes` should be set in `InitCoinsDB`.
Together with only conservatively setting the cache values when a assumeutxo chainstate is present, this allows for skipping the cache re-balance during initialization in the normal non-assumeutxo case.
Before:
```
2024-10-09T21:22:17Z Checking all blk files are present...
2024-10-09T21:22:17Z Initializing chainstate Chainstate [ibd] @ height -1 (null)
2024-10-09T21:22:17Z Opening LevelDB in /home/drgrid/.bitcoin/signet/chainstate
2024-10-09T21:22:17Z Opened LevelDB successfully
2024-10-09T21:22:17Z Using obfuscation key for /home/drgrid/.bitcoin/signet/chainstate: b0a6f4e95fd05c92
2024-10-09T21:22:17Z Loaded best chain: hashBestChain=0000000e119967d4937dad58456885ae43fb1761db686947e2f8e168c9a39a4f height=216852 date=2024-10-09T21:06:16Z progress=0.999989
2024-10-09T21:22:17Z Opening LevelDB in /home/drgrid/.bitcoin/signet/chainstate
2024-10-09T21:22:17Z Opened LevelDB successfully
2024-10-09T21:22:17Z Using obfuscation key for /home/drgrid/.bitcoin/signet/chainstate: b0a6f4e95fd05c92
2024-10-09T21:22:17Z [Chainstate [ibd] @ height 216852 (0000000e119967d4937dad58456885ae43fb1761db686947e2f8e168c9a39a4f)] resized coinsdb cache to 8.0 MiB
2024-10-09T21:22:17Z [Chainstate [ibd] @ height 216852 (0000000e119967d4937dad58456885ae43fb1761db686947e2f8e168c9a39a4f)] resized coinstip cache to 440.0 MiB
2024-10-09T21:22:17Z init message: Verifying blocks…
```
After:
```
2024-10-09T21:21:37Z Checking all blk files are present...
2024-10-09T21:21:37Z Initializing chainstate Chainstate [ibd] @ height -1 (null)
2024-10-09T21:21:37Z Opening LevelDB in /home/drgrid/.bitcoin/signet/chainstate
2024-10-09T21:21:37Z Opened LevelDB successfully
2024-10-09T21:21:37Z Using obfuscation key for /home/drgrid/.bitcoin/signet/chainstate: b0a6f4e95fd05c92
2024-10-09T21:21:37Z Loaded best chain: hashBestChain=0000012c12b48011a7d9150ce96ed6a44bbf32b09eeecaff4a667789dda2a566 height=216850 date=2024-10-09T20:37:05Z progress=0.999971
2024-10-09T21:21:37Z init message: Verifying blocks…
```
The change may also be verified by looking at the `feature_assumeutxo.py` functional test debug logs.
ACKs for top commit:
fjahr:
utACK 3a4a788ee0
achow101:
ACK 3a4a788ee0
laanwj:
Code review ACK 3a4a788ee0
BrandonOdiwuor:
Code Review ACK 3a4a788ee0
Tree-SHA512: 87878d0d196bb426370d4b4bd180ca52a34017a0799ecea651c2532461fd2927b0f7cc8182276a7d9bb1fe0ede7d0ad677e3714ca22f321917d711c643acc578
0ea84bc362 test: explicitly check boolean verbosity is disallowed (tdb3)
7a2e6b68cd doc: add rpc guidance for boolean verbosity avoidance (tdb3)
698f302df8 rpc: disallow boolean verbosity in getorphantxs (tdb3)
63f5e6ec79 test: add entry and expiration time checks (tdb3)
808a708107 rpc: add entry time to getorphantxs (tdb3)
56bf302714 refactor: rename rpc_getorphantxs to rpc_orphans (tdb3)
7824f6b077 test: check that getorphantxs is hidden (tdb3)
ac68fcca70 rpc: disallow undefined verbosity in getorphantxs (tdb3)
Pull request description:
Implements follow-up suggestions from #30793.
- Now disallows undefined verbosity levels (below and above valid values) (https://github.com/bitcoin/bitcoin/pull/30793#discussion_r1786093549)
- Disallows boolean verbosity (https://github.com/bitcoin/bitcoin/pull/30793#discussion_r1788273274) and adds guidance to developer-notes
- Checks that `getorphantxs` is a hidden rpc (https://github.com/bitcoin/bitcoin/pull/30793#discussion_r1786107786)
- Adds a test for `expiration` time
- Adds `entry` time to the returned orphan objects (verbosity >=1) to relieve the user from having to calculate it from `expiration`. Also adds associated test. (https://github.com/bitcoin/bitcoin/pull/30793#discussion_r1743687732)
- Minor cleanup (blank line removal and log message move) (https://github.com/bitcoin/bitcoin/pull/30793#discussion_r1786092641)
Included a commit to rename the test to a more generic `get_orphans` to better accommodate future orphanage-related RPCs (e.g. `getorphanangeinfo`). Can drop the refactor commit from this PR if people feel strongly about it.
ACKs for top commit:
achow101:
ACK 0ea84bc362
glozow:
utACK 0ea84bc362
rkrux:
tACK 0ea84bc362
itornaza:
tACK 0ea84bc362
Tree-SHA512: e48a088f333ebde132923072da58e970461e74362d0acebbc799c3043d5727cdf5f28e82b43cb38bbed27c603df6710695dba91ff0695e623ad168e985dce08e
0f4bc63585 [fuzz] txdownloadman and txdownload_impl (glozow)
699643f23a [unit test] MempoolRejectedTx (glozow)
fa584cbe72 [p2p] add TxDownloadOptions bool to make TxRequestTracker deterministic (glozow)
f803c8ce8d [p2p] filter 1p1c for child txid in recent rejects (glozow)
5269d57e6d [p2p] don't process orphan if in recent rejects (glozow)
2266eba43a [p2p] don't find 1p1cs for reconsiderable txns that are AlreadyHaveTx (glozow)
fa7027d0fc [refactor] add CheckIsEmpty and GetOrphanTransactions, remove access to TxDownloadMan internals (glozow)
969b07237b [refactor] wrap {Have,Get}TxToReconsider in txdownload (glozow)
f150fb94e7 [refactor] make AlreadyHaveTx and Find1P1CPackage private to TxDownloadImpl (glozow)
1e08195135 [refactor] move new tx logic to txdownload (glozow)
257568eab5 [refactor] move invalid package processing to TxDownload (glozow)
c4ce0c1218 [refactor] move invalid tx processing to TxDownload (glozow)
c6b21749ca [refactor] move valid tx processing to TxDownload (glozow)
a8cf3b6e84 [refactor] move Find1P1CPackage to txdownload (glozow)
f497414ce7 [refactor] put peerman tasks at the end of ProcessInvalidTx (glozow)
6797bc42a7 [p2p] restrict RecursiveDynamicUsage of orphans added to vExtraTxnForCompact (glozow)
798cc8f5aa [refactor] move Find1P1CPackage into ProcessInvalidTx (glozow)
416fbc952b [refactor] move new orphan handling to ProcessInvalidTx (glozow)
c8e67b9169 [refactor] move ProcessInvalidTx and ProcessValidTx definitions down (glozow)
3a41926d1b [refactor] move notfound processing to txdownload (glozow)
042a97ce7f [refactor] move tx inv/getdata handling to txdownload (glozow)
58e09f244b [p2p] don't log tx invs when in IBD (glozow)
288865338f [refactor] rename maybe_add_extra_compact_tx to first_time_failure (glozow)
f48d36cd97 [refactor] move peer (dis)connection logic to TxDownload (glozow)
f61d9e4b4b [refactor] move AlreadyHaveTx to TxDownload (glozow)
84e4ef843d [txdownload] add read-only reference to mempool (glozow)
af918349de [refactor] move ValidationInterface functions to TxDownloadManager (glozow)
f6c860efb1 [doc] fix typo in m_lazy_recent_confirmed_transactions doc (glozow)
5f9004e155 [refactor] add TxDownloadManager wrapping TxOrphanage, TxRequestTracker, and bloom filters (glozow)
Pull request description:
Part of #27463.
This PR does 3 things:
(1) It modularizes transaction download logic into a `TxDownloadManager`. Transaction download logic refers to the process of deciding what transactions to request, download, and validate.[1] There should be no behavior changes. Using `--color_moved=dimmed_zebra -w` may help.
(2) It adds unit and fuzz (🪄) testing for transaction download.
(3) It makes a few small behavioral changes:
- Stop (debug-only) logging tx invs during IBD
- Just like all other transactions, require orphans have RecursiveDynamicUsage < 100k before adding to vExtraTxnForCompact
- Don't return a 1p1c that contains a parent or child in recent rejects. Don't process any orphan already in recent rejects. These cases should not happen in actual node operation; it's just to allow tighter sanity checks during fuzzing.
There are several benefits to this interface, such as:
- Unit test coverage and fuzzing for logic that currently isn't feasible to test as thoroughly (without lots of overhead) and/or currently only lightly tested through `assert_debug_log` (not good) in functional tests.
- When we add more functionality (e.g. package relay messages, more robust orphan handling), the vast majority of it will be within `TxDownloadManager` instead of `PeerManager`, making it easier to review and test. See #28031 for what this looks like.
- `PeerManager` will no longer know anything about / have access to `TxOrphanage`, `TxRequestTracker` or the rejection caches. Its primary interface with `TxDownloadManager` would be much simpler:
- Passing on `ValidationInterface` callbacks
- Telling `txdownloadman` when a peer {connects, disconnects}
- Telling `txdownloadman`when a {transaction, package} is {accepted, rejected} from mempool
- Telling `txdownloadman` when invs, notfounds, and txs are received.
- Getting instructions on what to download.
- Getting instructions on what {transactions, packages, orphans} to validate.
- Get whether a peer `HaveMoreWork` for the `ProessMessages` loop
- (todo) Thread-safety can be handled internally.
[1]: This module is concerned with tx *download*, not upload. It excludes transaction announcements/gossip which happens after we download/accept a transaction. Txreconciliation (erlay) is excluded from this module, as it only relates to deciding which `inv`s to send or helping the other peer decide which `inv`s to send. It is independent from this logic.
ACKs for top commit:
achow101:
light ACK 0f4bc63585
theStack:
ACK 0f4bc63585
instagibbs:
reACK 0f4bc63585
naumenkogs:
ACK 0f4bc63585
Tree-SHA512: 84ab8ef8a0fc705eb829d7f7d6885f28944aaa42b03172f256a42605677b3e783919bb900d4e3b8589f85a0c387dfbd972bcd61d252d44a88c6aaa90e4bf920f
915640e191 depends: zeromq: don't install .pc files and remove patches for them (Cory Fields)
6b8a74463b cmake: Add `FindZeroMQ` module (Hennadii Stepanov)
Pull request description:
This PR introduces the `FindZeroMQ` module, which first attempts to find the `libzmq` library using CMake's `find_package()` and falls back to `pkg_check_modules()` if unsuccessful.
Addresses https://github.com/bitcoin/bitcoin/issues/30876 for the ZeroMQ package.
ACKs for top commit:
fanquake:
ACK 915640e191
Tree-SHA512: 2f17bae21be5d3f280a13425d22f5d1b2e23837a8aaf5ec89c433767509de030a42d598b261e102bdb5b860d8ede98013c124c3d25e081e956d4ee3a81b2584f
This is needed in the next commit to add compile-time checking to strprintf
calls, because bitcoin-cli.cpp uses dynamic width in many format strings.
This change is easiest to review ignoring whitespace.
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: l0rinc <pap.lorinc@gmail.com>
c4dc81f9c6 test: Remove dead code from interface_zmq (Fabian Jahr)
Pull request description:
The loop removed here appears to be effectively dead code: In case `get_raw_seq` is behind `zmq_mem_seq` the loop runs and tries to get a more recent (higher) number for `get_raw_seq`. However, the exact number of `get_raw_seq` is asserted in the line above: `assert_equal(get_raw_seq, 6)`. If the loop would actually achieve its purpose this assert would need to be racy. This does not seem to be the case and 6 appears to be the final number. `zmq_mem_seq` however does take some time to catch up (if it were continue to be updated). But this is not handled by the loop and does not seem to be relevant at this point in the test. The backlog is consumed a bit later in another loop that handles this correctly already.
ACKs for top commit:
l0rinc:
ACK c4dc81f9c6
tdb3:
CR re ACK c4dc81f9c6
Tree-SHA512: 663a1711ba1ce04a3d2e2916e0df7a7bb51069e28bc2644b816a483628c95b5e6c29fc6eacc31a5f72b7d9af11096f3c437ea1dc57eaa1ee9ddce43cc20bacd3
6c9fe7b73e test: Prevent connection attempts to random IPs in p2p_seednodes.py (Martin Zumsande)
bb97b1ffa9 test: fix intermittent timeout in p2p_seednodes.py (Martin Zumsande)
Pull request description:
Fixes#31103
On some CI runs, the seed node timer in `ThreadOpenConnection` was only started *after* the mocktime was set.
Fix this by waiting for the first connection attempt, which happens after the timer was started.
Also I noticed that the "unreachable" connections are not in fact unreachable, so that the functional test could attempt connections
to random IPs on the internet. This was already noted in https://github.com/bitcoin/bitcoin/pull/29605#discussion_r1701616675 but the suggested fix never made it in, so I added it to this PR.
ACKs for top commit:
sr-gi:
tACK [6c9fe7b](6c9fe7b73e)
laanwj:
Code review ACK 6c9fe7b73e
tdb3:
cr and light test ACK 6c9fe7b73e
Tree-SHA512: 021b6d5325eab85d79708b4b137f61723a36f2b8a1faf681463bad2ea5283ea528b5ff1701467a86b035d3a6972750a61ace5020e58b7aa61ecaad97664488c8
70713303b6 scripted-diff: Rename `PACKAGE_*` variables to `CLIENT_*` (Hennadii Stepanov)
332655cb52 build: Rename `PACKAGE_*` variables to `CLIENT_*` (Hennadii Stepanov)
e6e29e3c94 scripted-diff: Clarify "user agent" variable name (Hennadii Stepanov)
Pull request description:
The use of `PACKAGE_NAME` for the project's variable name is problematic, as this name is commonly used in CMake's [interface variables](https://cmake.org/cmake/help/latest/command/find_package.html#config-mode-version-selection). If third-party CMake code handles with scopes improperly, our `PACKAGE_NAME` variable could end up with an unexpected value.
This PR avoids such conflicts by renaming all `PACKAGE_*` variables to `CLIENT_*`.
The code in the master branch works correctly only incidentally. It is definitely broken in https://github.com/bitcoin/bitcoin/pull/30997.
ACKs for top commit:
TheCharlatan:
Re-ACK 70713303b6
Tree-SHA512: f0992fb9a1ac4a41125b223f97bcaae50f521c813f334e606693dd0113a4732f12451bbcfb158df8bed44f34b37dadac478b2f5661e8b2588b401f43ae4bc1a4
The added regtest option -test=bip94 is only used in the functional
test for BIP94.
This is done because the default regtest consensus rules
should aim to follow to mainnet, not testnet.
BCC needs the PID of a bitcoind process to attach to the tracepoints
(instead of the binary path used before) when the tracepoints have a
semaphore.
For reference, we already use the PID in our tracepoint interface
tests. See 220a5a2841.
Before this commit, we would always prepare tracepoint arguments
regardless of the tracepoint being used or not. While we already made
sure not to include expensive arguments in our tracepoints, this
commit introduces gating to make sure the arguments are only prepared
if the tracepoints are actually used. This is a win-win improvement
to our tracing framework. For users not interested in tracing, the
overhead is reduced to a cheap 'greater than 0' compare. As the
semaphore-gating technique used here is available in bpftrace, bcc,
and libbpf, users interested in tracing don't have to change their
tracing scripts while profiting from potential future tracepoints
passing slightly more expensive arguments. An example are mempool
tracepoints that pass serialized transactions. We've avoided the
serialization in the past as it was too expensive.
Under the hood, the semaphore-gating works by placing a 2-byte
semaphore in the '.probes' ELF section. The address of the semaphore
is contained in the ELF note providing the tracepoint information
(`readelf -n ./src/bitcoind | grep NT_STAPSDT`). Tracing toolkits
like bpftrace, bcc, and libbpf increase the semaphore at the address
upon attaching to the tracepoint. We only prepare the arguments and
reach the tracepoint if the semaphore is greater than zero. The
semaphore is decreased when detaching from the tracepoint.
This also extends the "Adding a new tracepoint" documentation to
include information about the semaphores and updated step-by-step
instructions on how to add a new tracepoint.
The use of `PACKAGE_NAME` for the project's variable name is
problematic, as this name is commonly used in CMake's interface
variables. If third-party CMake code handles with scopes improperly,
our `PACKAGE_NAME` variable could end up with an unexpected value.
This change avoids such conflicts by renaming all `PACKAGE_*` variables
to `CLIENT_*`.
This change allows to the use of the `CLIENT_` namespace without
potential name clashes.
-BEGIN VERIFY SCRIPT-
sed -i "s/\<CLIENT_NAME\>/UA_NAME/g" $( git grep -l "CLIENT_NAME" ./src)
-END VERIFY SCRIPT-
Building packages in depends implies using GNU Make. However, this
assumption can be wrong in environments where the `CMAKE_GENERATOR`
variable is set.
This change explicitly makes CMake use the "Unix Makefiles" generator.
9f243cd7fa Introduce `g_fuzzing` global for fuzzing checks (dergoegge)
Pull request description:
This PR introduces a global `g_fuzzing` that indicates if we are fuzzing.
If `g_fuzzing` is `true` then:
* Assume checks are enabled
* Special fuzzing paths are taken (e.g. pow check is reduced to one bit)
Closes#30950#31057
ACKs for top commit:
maflcko:
review ACK 9f243cd7fa 🗜
brunoerg:
crACK 9f243cd7fa
marcofleon:
Tested ACK 9f243cd7fa
Tree-SHA512: 56e4cad0555dec0c565ea5ecc529628ee4f37d20dc660c647fdc6948fbeed8291e6fe290de514bd4c2c7089654d9ce1add607dc9855462828b62be9ee45e4999
40e5f26a3f mapport: remove dead code in DispatchMapPort (Antoine Poinsot)
38fdf7c1fb mapport: drop outdated comments (Antoine Poinsot)
b7b2435290 doc: add release note for #31130 (Antoine Poinsot)
1b6dec98da depends: drop miniupnpc (Antoine Poinsot)
953533d021 doc: remove mentions of UPnP (Antoine Poinsot)
94ad614482 ci: remove UPnP options (Antoine Poinsot)
a9598e5eaa build: drop miniupnpc dependency (Antoine Poinsot)
a5fcfb7385 interfaces: remove now unused 'use_upnp' arg from 'mapPort' (Antoine Poinsot)
038bbe7b20 daemon: remove UPnP support (Antoine Poinsot)
844770b05e qt: remove UPnP settings (Antoine Poinsot)
Pull request description:
This PR removes UPnP IGD support and drops our [miniupnp](https://github.com/miniupnp/miniupnp) dependency.
Miniupnpc is a C library (somewhat) maintained by a single person which had several vulnerabilities in the past (a couple dozens are listed [here](https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=miniupnp)), some of which directly affected our software ([RCE in 2015](https://bitcoincore.org/en/2024/07/03/disclose_upnp_rce/), [OOM in 2020](https://bitcoincore.org/en/2024/07/31/disclose-upnp-oom/)).
The main purpose of this functionality is to have more (non-data-center) reachable nodes on the network. For a non-technical user running Bitcoin Core at home, the software would automatically open a port on their router to receive incoming connections. This way, users not able to manually open a port on their router would still provide the network with more resources and enhance its diversity.
However, due to past vulnerabilities (and a worry about unknown future ones) in miniupnpc this feature was disabled by default in https://github.com/bitcoin/bitcoin/pull/6795. Having it disabled by default kills (most of?) the purpose of having this functionality in the first place: someone technical enough to understand the `-upnp` startup option or the "enable UPnP" setting is most likely able to open a port on his box in the first place.
In addition, laanwj implemented PCP with a NAT-PMP fallback directly in Bitcoin Core in https://github.com/bitcoin/bitcoin/pull/30043. If we ever want to re-enable automatic NAT traversal by default in Bitcoin Core, this is the best option (and in my opinion the only sane one). The NAT-PMP fallback makes it so compatibility shouldn't be (much of) an issue.
On balance, i believe that keeping this functionality and this barely maintained C dependency has higher costs than benefits. Therefore i propose that we get rid of it.
ACKs for top commit:
jarolrod:
ACK 40e5f26a3f
1440000bytes:
Code Review ACK 40e5f26a3f
laanwj:
Code review ACK 40e5f26a3f
i-am-yuvi:
Tested ACK 40e5f26a3f
Tree-SHA512: 9ea48662775510f5ec6de7af65790f7c8d211603398e9d8c634a86387be81b28081419a95b4d6680d3d7fe6a9f16cec99f16516548201dc7e49781909899a657
Same as in `DecodeSecret`, we should also clear out the secret data from
the vector resulting from the Base58Check parsing for xprv keys. Note
that the if condition is needed in order to avoid UB, see #14242 (commit
d855e4cac8).
f32c34d0c3 functional test: Additional package evaluation coverage (Greg Sanders)
Pull request description:
Current test coverage doesn't ensure that mempool trimming doesn't appear prior to the entire package, and not just the subpackage, is finished being submitted.
Add a scenario that covers this case, where package ancestors can make it in individually, but would be immadiately evicted if not for the package CPFP.
in response to https://github.com/bitcoin/bitcoin/pull/31122#discussion_r1813272637 where if applied onto that PR's old commit, the test fails due to package failure.
ACKs for top commit:
sdaftuar:
re-ACK f32c34d0c3
rkrux:
tACK f32c34d0c3
glozow:
reACK f32c34d0c3
Tree-SHA512: 739fcc5e66878b3def9b25dc588d8cb5349aaaa0901b11475879a413a03f6ea0e87d19de5bc4fb44ddd0436fdc052cdc3ed564f7e2ad510269aab9732d5c24eb
5c299ecafe test: Assert that when we add the max orphan amount that we cannot add anymore and that a random orphan gets dropped (kevkevinpal)
Pull request description:
After joining the bitcoin pr review club about https://github.com/bitcoin/bitcoin/pull/30793
I learned about [`CVE-2012-3789`](https://github.com/bitcoin/bitcoin/blob/master/src/net_processing.cpp#L4693)
So I was motivated to write a functional test that covers this part of the code,
This test should add the max number of orphans to a nodes orphanage and then attempt to add another, then asserts that the number of orphans is still at the max amount
ACKs for top commit:
achow101:
ACK 5c299ecafe
rkrux:
ACK 5c299ecafe
instagibbs:
ACK 5c299ecafe
tdb3:
ACK 5c299ecafe
Tree-SHA512: 687bba337978e0945e94af71632998221e5565a5d83cf5a59ecf2ee52c7262d8ff907b94dceea3b80bed441dd19b24790b2904e88e1da14d30827c5469fcb4d3
Since there is now only two options in the MapPortProtoFlag enum, the
four possible combinations of current and enabled are already covered in
the four `if` branches.
552cae243a fuzz: cover `ASMapHealthCheck` in connman target (brunoerg)
33b0f3ae96 fuzz: use `ConsumeNetGroupManager` in connman target (brunoerg)
18c8a0945b fuzz: move `ConsumeNetGroupManager` to util (brunoerg)
fe624631ae fuzz: fuzz `connman` with a non-empty addrman (brunoerg)
0a12cff2a8 fuzz: move `AddrManDeterministic` to util (brunoerg)
Pull request description:
### Motivation
Currently, we fuzz connman with an addrman from `NodeContext`. However,
fuzzing connman with only empty addrman might not be effective, especially
for functions like `GetAddresses` and other ones that plays with addrman. Also,
we do not fuzz connman with ASMap, what would be good for functions that need
`GetGroup`, or even for addrman. Without it, I do not see how effective would be
fuzzing `ASMapHealthCheck`, for example.
### Changes
- Move `AddrManDeterministic` and `ConsumeNetGroupManager` to util.
- Use `ConsumeNetGroupManager` in connman target to construct a netgroupmanager
and use it for `ConnmanTestMsg`.
- Use `AddrManDeterministic` in connman target to create an addrman. It does
not slow down as "filling" the addrman (e.g. with `FillAddrman`).
- Add coverage for `ASMapHealthCheck`.
ACKs for top commit:
maflcko:
review ACK 552cae243a🏀
dergoegge:
Code review ACK 552cae243a
marcofleon:
Code review ACK 552cae243a. Changes match the PR description.
Tree-SHA512: ba861c839602054077e4bf3649763eeb48357cda83ca3ddd32b02a1b61f4e44a0c5070182f001f9bf531d0d64717876279a7de3ddb9de028b343533b89233851
Current test coverage doesn't ensure that mempool trimming
doesn't appear prior to the entire package, and not just
the subpackage, is finished being submitted.
Add a scenario that covers this case, where package
ancestors can make it in individually, but would be
immadiately evicted if not for the package CPFP.
by passing an additional argument of "outonly" or "o".
This has been requested in order to keep the output within screen limits when running -netinfo
as a live dashboard, i.e. with `watch`.
Also allow passing "h" in addition to "help" to see the help documentation.
fa69a5f4b7 util: Treat Assume as Assert when evaluating at compile-time (MarcoFalke)
Pull request description:
There is no downside or cost of treating an `Assume` at compile-time as an `Assert` and it may even help to find bugs while compiling without `ABORT_ON_FAILED_ASSUME`.
This is also required for https://github.com/bitcoin/bitcoin/pull/31093
ACKs for top commit:
dergoegge:
ACK fa69a5f4b7
brunoerg:
ACK fa69a5f4b7
marcofleon:
ACK fa69a5f4b7
Tree-SHA512: 17604403f841343a6d5b6e5d777e1760d38e0c27dc1fd4479e3741894fba40cdb1fb659cf24519a51d051bd5884a75992d1227ec9fa40fbf53bc619fbfb304ad
fa9747a896 ci: Temporary workaround for old CCACHE_DIR cirrus env (MarcoFalke)
Pull request description:
On a CI re-run, the historic env vars and CI config is used from Cirrus. However, the most recent CI config and CI scripts from this repo are used. This may lead to issues.
For example, `CCACHE_DIR` in the old location may be missing on new CI workers and lead to errors.
Fix it, by falling back to the old logic when the old `CCACHE_DIR` was detected.
ACKs for top commit:
fanquake:
ACK fa9747a896 - have seen this now.
Tree-SHA512: 04f0ca8d09ab0b8216a474fde1e05b79fbc6524884be173e8d728799739b026cda18d1797e0fe53d7e1b0ea69c0485acfe4f8a8f85408ea5bfdcffcf13a7ce55
c495731a31 fuzz: wallet: add target for `CreateTransaction` (brunoerg)
3db68e29ec wallet: move `ImportDescriptors`/`FuzzedWallet` to util (brunoerg)
Pull request description:
This PR adds a fuzz target for the `CreateTransaction` function. It is a regression target for https://github.com/bitcoin/bitcoin/pull/27271 and can be testing by applying:
```diff
@@ -1110,7 +1110,7 @@ static util::Result<CreatedTransactionResult> CreateTransactionInternal(
// This can only happen if feerate is 0, and requested destinations are value of 0 (e.g. OP_RETURN)
// and no pre-selected inputs. This will result in 0-input transaction, which is consensus-invalid anyways
if (selection_target == 0 && !coin_control.HasSelected()) {
- return util::Error{_("Transaction requires one destination of non-0 value, a non-0 feerate, or a pre-selected input")};
+ // return util::Error{_("Transaction requires one destination of non-0 value, a non-0 feerate, or a pre-selected input")};
}
```
Also, it moves `ImportDescriptors` function to `src/wallet/test/util.h` to avoid to duplicate same code.
ACKs for top commit:
marcofleon:
ACK c495731a31
maflcko:
ACK c495731a31 🏻
Tree-SHA512: a439f947b91b01e327e18cd18e63d5ce49f2cb9ca16ca9d56fe337b8cff239b3af4db18fe89478fe5faa5549d37ca935bd321913db7646fbf6818f825cb5d878
The txdownload_impl is similar but allows us to check specific
invariants within its implementation. It will also change a lot more
than the external interface (txdownloadman) will, so we will add more to
this target later.
Avoid the fuzzer situation where:
1. Orphanage has 2 transactions with the same txid, one with witness,
one without witness.
2. The transaction with witness is found to have
`TX_INPUTS_NOT_STANDARD` error. The txid is added to recent rejects
filter, and the tx with witness is deleted from orphanage.
3. A low feerate parent is found. Find1P1CPackage finds the transaction
with no witness in orphanage, and returns the package.
4. net_processing has just been handed a package in which the child is
already in recent rejects.
This is a slight behavior change: if a transaction is in both
reconsiderable rejects and AlreadyHaveTx in another way, we don't try to
return a 1p1c package. This is the correct thing to do, as we don't want
to reconsider transactions that have multiple things wrong with them.
For example, if a transaction is low feerate, and then later found to
have a bad signature, we shouldn't try it again in a package.
The usage of this bool will increase in scope in the next commit.
For this commit, the value of this bool is accurate at each
ProcessInvalidTx callsite:
- ProcessOrphanTx -> this tx is an orphan i.e. has been rejected before
- ProcessPackageResult -> 1p1c only, each transaction is either an
orphan or in m_lazy_recent_rejects_reconsiderable
- ProcessMessage -> tx was received over p2p and validated for the first
time
This will become necessary in later commits that query mempool. We also
introduce the TxDownloadOptions in this commit to make the later diff
easier to review.
This module is going to be responsible for managing everything related
to transaction download, including txrequest, orphan transactions and
package relay. It will be responsible for managing usage of the
TxOrphanage and instructing PeerManager:
- what tx or package-related messages to send to which peer
- whether a tx or package-related message is allowed or useful
- what transactions are available to try accepting to mempool
Future commits will consolidate the interface and re-delegate
interactions from PeerManager to TxDownloadManager.
9bb92c0e7f util: Remove RandAddSeedPerfmon (Hodlinator)
Pull request description:
`RegQueryValueExA(HKEY_PERFORMANCE_DATA, ...)` sometimes hangs *bitcoind.exe* on Windows during startup, at least on CI.
We have other sources of entropy to seed randomness with on Windows, so should be alright removing this. Might resurrect if less drastic fix is found.
Hopefully sufficient to fix#30390.
CI debugged with temporary Windows stack trace dumping + Symbols in #30956.
ACKs for top commit:
achow101:
ACK 9bb92c0e7f
fanquake:
ACK 9bb92c0e7f
hebasto:
ACK 9bb92c0e7f, I have reviewed the code and it looks OK.
laanwj:
Code review ACK 9bb92c0e7f
Tree-SHA512: d3f26b4dd0519ef957f23abaffc6be1fed339eae756aed18042422fc6f0bba4e8fa9a44bf903e54f72747e2d0108146c18fd80576d95fc20690a2daf9c83689d
33a28e252a Change default help arg to `-help` and mention `-h` and `-?` as alternatives (Lőrinc)
f0130ab1a1 doc: replace `-?` with `-h` for bench_bitcoin help (Lőrinc)
Pull request description:
The question mark is interpreted as a wildcard for any single character in Zsh (see https://www.techrepublic.com/article/globbing-wildcard-characters-with-zsh), so `bench_bitcoin -?` will not show the help message on systems using Zsh, such as macOS.
Since `-h` provides equivalent help functionality (as defined in https://github.com/bitcoin/bitcoin/blob/master/src/common/args.cpp#L684-L693), the `benchmarking.md` documentation has been updated to ensure compatibility with macOS.
----
### -?
> % cmake -B build -DBUILD_BENCH=ON && cmake --build build && build/src/bench/bench_bitcoin -?
zsh: no matches found: -?
### -h
> % cmake -B build -DBUILD_BENCH=ON && cmake --build build && build/src/bench/bench_bitcoin -h
Usage: bench_bitcoin [options]
Options:
...
----
Based on the comments the args help default was also changed to `-help`, mentioning `-h` and `-?` (instead of `-?` being the default)
ACKs for top commit:
edilmedeiros:
tACK 33a28e252a
maflcko:
lgtm ACK 33a28e252a
achow101:
ACK 33a28e252a
rkrux:
tACK 33a28e252a
laanwj:
Code review ACK 33a28e252a
Tree-SHA512: 8c6e27488462be9ba9186b34abe6249c1d93026b3963acc0f42c75496f39407563766ae518cf1839156039cc0047e29d91f70d191cfb97e0fbde85665e88c71e
4feaa28728 refactor: Rely on returned value of GetCoin instead of parameter (Lőrinc)
46dfbf169b refactor: Return optional of Coin in GetCoin (Lőrinc)
e31bfb26c2 refactor: Remove unrealistic simulation state (Lőrinc)
Pull request description:
While reviewing [the removal of the unreachable combinations from the Coin cache logic](https://github.com/bitcoin/bitcoin/pull/30673#discussion_r1721727681), we've noticed that the related tests often [reflect impossible states](https://github.com/bitcoin/bitcoin/pull/30673/files#r1740154464).
Browsing the Coin cache refactoring history revealed that migrating `bool GetCoin` to `optional<Coin> GetCoin` was [already proposed a few times before](https://github.com/bitcoin/bitcoin/pull/18746#issuecomment-842393167).
This refactor makes certain invalid states impossible, reducing the possibility of errors and making the code easier to understand. This will let us remove test code that exercises the impossible states as well.
The PR is done in multiple small steps, first swapping the new `optional` return value, slowly strangling out the usages of the return parameter, followed by the removal of the parameter.
Most of the invalid test states were still kept, except for https://github.com/bitcoin/bitcoin/pull/30673/files#r1748087322, where the new design prohibits invalid usage and https://github.com/bitcoin/bitcoin/pull/30673/files#r1749350258 was just marked with a TODO, will be removed in a follow-up PR.
ACKs for top commit:
andrewtoth:
re-ACK 4feaa28728
achow101:
ACK 4feaa28728
laanwj:
Code review ACK 4feaa28728
theStack:
Code-review ACK 4feaa28728
Tree-SHA512: 818d60b2e97f58c489a61120fe761fb67a08dffbefe7a3fce712d362fc9eb8c2cced23074f1bec55fe71c616a3561b5a8737919ad6ffb2635467ec4711683df7
c98fc36d09 wallet: migration, consolidate external wallets db writes (furszy)
7c9076a2d2 wallet: migration, consolidate main wallet db writes (furszy)
9ef20e86d7 wallet: provide WalletBatch to 'SetupDescriptorScriptPubKeyMans' (furszy)
34bf0795fc wallet: refactor ApplyMigrationData to return util::Result<void> (furszy)
aacaaaa0d3 wallet: provide WalletBatch to 'RemoveTxs' (furszy)
57249ff669 wallet: introduce active db txn listeners (furszy)
91e065ec17 wallet: remove post-migration signals connection (furszy)
055c0532fc wallet: provide WalletBatch to 'DeleteRecords' (furszy)
122d103ca2 wallet: introduce 'SetWalletFlagWithDB' (furszy)
6052c7891d wallet: decouple default descriptors creation from external signer setup (furszy)
f2541d09e1 wallet: batch MigrateToDescriptor() db transactions (furszy)
66c9936455 bench: add coverage for wallet migration process (furszy)
Pull request description:
Last step in a chain of PRs (#26836, #28894, #28987, #29403).
#### Detailed Description:
The current wallet migration process performs only individual db writes. Accessing disk to
delete all legacy records, clone and clean each address book entry for every created wallet,
create each new descriptor (with their corresponding master key, caches and key pool), and
also clone and delete each transaction that requires to be transferred to a different wallet.
This work consolidates all individual disk writes into two batch operations. One for the descriptors
creation from the legacy data and a second one for the execution of the migration process itself.
Efficiently dumping all the information to disk at once atomically at the end of each process.
This represent a speed up and also a consistency improvement. During migration, we either
want to succeed or fail. No other outcomes should be accepted. We should never leave a
partially migrated wallet on disk and request the user to manually restore the previous wallet from
a backup (at least not if we can avoid it).
Since the speedup depends on the storage device, benchmark results can vary significantly.
Locally, I have seen a 15% speedup on a USB 3.2 pendrive.
#### Note for Testers:
The first commit introduces a benchmark for the migration process. This one can be
cherry-picked on top of master to compare results pre and post changes.
Please note that the benchmark setup may take some time (~70 seconds here) due to the absence
of a batching mechanism for the address generation process (`GetNewDestination()` calls).
ACKs for top commit:
achow101:
ACK c98fc36d09
theStack:
re-ACK c98fc36d09
pablomartin4btc:
re-ACK c98fc36d09
Tree-SHA512: a52d5f2eef27811045d613637c0a9d0b7e180256ddc1c893749d98ba2882b570c45f28cc7263cadd4710f2c10db1bea33d88051f29c6b789bc6180c85b5fd8f6
Keep the "-upnp" option as a hidden arg for one major version in order
to show a more user friendly error to people who had this option set in
their config file.
8523d8c0fc ci: display logs of failed tests automatically (furszy)
Pull request description:
Saw it here https://github.com/bitcoin/bitcoin/actions/runs/11488618084/job/31975712362?pr=31000.
The 'test-each-commit' and 'win64' CI jobs currently do not display test logs when an error occurs, making it almost impossible to debug issues that don't arise locally. Fix this by setting the CTest `--output-on-failure` flag (per [README](2f40e453cc/src/test/README.md (L130))).
ACKs for top commit:
hebasto:
ACK 8523d8c0fc, I have reviewed the code and it looks OK.
Tree-SHA512: 59c025099fb623e2ed430cfc1ba808e1d3ff72773d021e2280a44423ae53615c16e96a07014eb8581c95aae241b6d2777e388a8931ff0904feb84ca45cb22763
fb46d57d4e cmake, qt, test: Remove problematic code (Hennadii Stepanov)
Pull request description:
Split from https://github.com/bitcoin/bitcoin/pull/30997.
The removed code aimed to make Qt's minimal integration plugin DLL available for `test_bitcoin-qt.exe` on Windows.
However, there are two issues:
1. The code is broken because the destination directory must end with a trailing slash (`/`).
2. It is unnecessary because Qt's minimal integration plugin is not used on Windows. For more details, please refer to the following code:fb46d57d4e/src/qt/test/CMakeLists.txt (L38-L44)
As a side effect, this PR fixes https://github.com/bitcoin-core/gui/issues/842.
ACKs for top commit:
fanquake:
ACK fb46d57d4e
TheCharlatan:
ACK fb46d57d4e
Tree-SHA512: b44d1c5e352e9bbfbba3c263ee03838cd490435da0490d9c8a152e60515520772c8a87aca08d4510f50c2e46b64ac92c666fa81accf43758af2e896693c44ffa
6c6b2442ed build: Replace MAC_OSX macro with existing __APPLE__ (Lőrinc)
Pull request description:
This PR aims to standardize and simplify macOS-specific checks within our codebase by replacing the custom-defined `MAC_OSX` macro with the existing `__APPLE__`macro, defined in e.g. https://sourceforge.net/p/predef/wiki/OperatingSystems/#macos
We already use `__APPLE__` in our own codebase for e.g. https://github.com/bitcoin/bitcoin/blob/master/src/crypto/sha256.cpp#L22
Local Verification confirms that `MAC_OSX` isn't defined, but `__APPLE__` is:
```bash
% echo | cpp -dM | egrep 'MAC_OSX|__MACOS__|__APPLE__'
#define __APPLE__ 1
```
ACKs for top commit:
fanquake:
ACK 6c6b2442ed - at this point it seems unlikely that we'll need to accomodate non-macOS Apple, so consolidating to `__APPLE__` seems ok for now.
Tree-SHA512: dbf87c96211d9d55426ee85d76ef1e05cda3efd1c9248b0974a82834dafc1c1aece3165bd46e4252f0460dc97079bdbcebe98bbd81e9de0d7399c0bc69d5c050
This aims to complete our test framework BDB parser to reflect
our read-only BDB parser in the wallet codebase. This could be
useful both for making review of #26606 easier and to also possibly
improve our functional tests for the BDB parser by comparing with
an alternative implementation.
The removed code aimed to make Qt's minimal integration plugin DLL
available for `test_bitcoin-qt.exe` on Windows.
However, there are two issues:
1. The code is broken because the destination directory must end with a
trailing slash (`/`).
2. It is unnecessary because Qt's minimal integration plugin is not
used on Windows. For more details, please refer to the following
code.
fa1c5cc9df fees: Log non-fatal errors as [warning], instead of info-level (MarcoFalke)
ddddbac9c1 fees: Pin required version to 149900 (MarcoFalke)
fa5126adcb fees: Pin "version that wrote" to 0 (MarcoFalke)
Pull request description:
Coupling the fees serialization with CLIENT_VERSION is problematic, because:
* `CLIENT_VERSION` may change, even though the serialization format does not change. This is harmless, but still confusing.
* If a serialization format change was backported (unlikely), it may lead to incorrect results.
* `CLIENT_VERSION` is changed at a different time during the release process than any serialization format change. This is harmless for releases of Bitcoin Core, but may be confusing when using the development branch.
* It is harder to reason about a global `CLIENT_VERSION` when changing the format, than to reason about a versioning local to the module.
Fix all issues by pinning the current version number in the module locally. In the future it can then be modified locally to the module, if needed.
ACKs for top commit:
hodlinator:
re-ACK fa1c5cc9df
TheCharlatan:
Re-ACK fa1c5cc9df
Tree-SHA512: 93870176ed50cc5a734576d66398a6036b31632228a9e05db1fa5452229e35ba4126f003e7db246aeb9891764ed47bde4470c674ec2bce7fd3ddd97e43944627
82e16e6983 cmake: Refactor install kernel dependencies (Hennadii Stepanov)
42e6277987 build: Add static libraries to Kernel install component (TheCharlatan)
Pull request description:
Fixes the installation of the pkgconfig file and the static library when installing only the `Kernel` component.
This is a followup to fix#30835 and #30814, which were merged shortly after one another, but are interrelated. Can be tested with:
```
cmake -B build -DBUILD_SHARED_LIBS=OFF -DBUILD_KERNEL_LIB=ON
cmake --build build --target bitcoinkernel
cmake --install build --component Kernel
```
ACKs for top commit:
hebasto:
ACK 82e16e6983, tested on Ubuntu 23.10.
fanquake:
ACK 82e16e6983
Tree-SHA512: 07c18a341d4464e489c28fb262600338f1711248309ffb2af0ef3ab1abf06f10873c435895b63010e0be8e44af77046324896dfd872479792aa049831606dc45
a0c9595810 doc: Make list of targets in depends README consistent (laanwj)
Pull request description:
The description of `i686-pc-linux-gnu` and `x86_64-pc-linux-gnu` is incomplete and inconsistent with the others. Fix this. Also use "64 bit" consistently instead of "64-bit".
ACKs for top commit:
maflcko:
lgtm ACK a0c9595810
hebasto:
ACK a0c9595810.
jarolrod:
ACK a0c9595810
rkrux:
ACK a0c9595810
Tree-SHA512: eedefb19639dd08f25627ceaab0d2c3745b256e561e55f8506d14721d0236978f1b1bef79f9fe126b7f42d869887ca988d04b3536d98a27e0eb182f0a7f64183
31cc5006c3 init: Return fatal failure on snapshot validation failure (Martin Zumsande)
8f1246e833 init: Improve chainstate init db error messages (TheCharlatan)
cd093049dd init: Remove incorrect comment about shutdown condition (MarcoFalke)
635e9f85d7 init: Remove misleading log line when user chooses not to retry (TheCharlatan)
720ce880a3 init: Improve comment describing chainstate load retry behaviour (Martin Zumsande)
baea842ff1 init: Remove unneeded argument for mempool_opts checks (stickies-v)
Pull request description:
These are mostly followups from #30968, making the code, log lines, error messages, and comments more consistent.
The last commit is an attempt at improving the error reporting when loading the chainstate. It aims to more cleanly distinguish between errors arising from a specific database, and errors where the culprit may be less clear.
ACKs for top commit:
achow101:
ACK 31cc5006c3
mzumsande:
Code Review / lightly tested ACK 31cc5006c3
BrandonOdiwuor:
Code Review ACK 31cc5006c3.
stickies-v:
ACK 31cc5006c3
Tree-SHA512: 59fba4845ee45a3d91bf55807ae6b1c81458463b96bf664c8b1badfac503f6b01efd52a915fc399294e68a3f69985362a5a10a3844fa23f7707145ebe9ad349b
04e4d52420 test: add test for specifying custom pidfile via `-pid` (Sebastian Falbesoner)
b832ffe044 refactor: introduce default pid file name constant in tests (tdb3)
Pull request description:
This small PR adds test coverage for the `-pid` command line option, which allows to overrule the pid filename (`bitcoind.pid` by default). One can specify either a relative path (within the datadir) or an absolute one; the latter is tested using `self.options.tmpdir`. Note that the functional test file `feature_init.py` so far only contained a stress test; with this new sub-test added, both the description and the test name are adapted to be more generic.
ACKs for top commit:
achow101:
ACK 04e4d52420
tdb3:
ACK 04e4d52420
ryanofsky:
Code review ACK 04e4d52420
naiyoma:
Tested ACK [04e4d52420)
Tree-SHA512: b2bc8a790e5d187e2c84345f344f65a176b62caecd9797c3b9edf10294c741c33a24e535be640b56444b91dcf9c65c7dd152cdffd8b1c1d9ca68e5e3c6ad1e99
These addrs aren't unreachable as the test claims.
Specify a (non-working) proxy to make sure the connections fails
even if the addr was reachable.
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
On some CI runs, the timer in ThreadOpenConnection was only started *after*
the mocktime was set. Fix this by waiting for the first connection attempt,
which happens after the timer was started.
Also convert some comments into log messages/add a log, so that the test
isn't completely silent.
The description of `i686-pc-linux-gnu` and `x86_64-pc-linux-gnu` is
incomplete and inconsistent with the rest. Fix this. Also use "64 bit"
consistently instead of "64-bit".
Also, remove not needed and possibly redundant function name and class
names from the log string. Also, minimally reword the log messages.
Also, remove redundant trailing newlines from log messages, while
touching.
fa71bedf86 ci: Approximate MAKEJOBS in image build phase (MarcoFalke)
Pull request description:
The `MAKEJOBS` env var is the default in image builds, which is fine, because it is only relevant when building msan (or iwyu) and only differs when setting MAKEJOBS to something other than `nproc` (currently used as an approximation).
So the normal workflow of `MAKEJOBS="-j$(nproc)" FILE_ENV="./ci/test/00_setup_env_native_msan.sh" ./ci/test_run_all.sh` already works today.
However, `MAKEJOBS="-j1" FILE_ENV="./ci/test/00_setup_env_native_msan.sh" ./ci/test_run_all.sh` does not.
This is hard to fix, because making the env var a build arg means that changing it (and only it) requires a new (expensive and redundant) build.
So add an option `HAVE_CGROUP_CPUSET`, which can be set to approximate `MAKEJOBS` a bit. Can be tested via:
`HAVE_CGROUP_CPUSET=yo MAKEJOBS="-j_something" FILE_ENV="./ci/test/00_setup_env_native_msan.sh" ./ci/test_run_all.sh`
ACKs for top commit:
fanquake:
ACK fa71bedf86
Tree-SHA512: 43ef194c71d726f4cfa3fe08a5894c7872150f37da7e4fa0c2d89e4572bc63acadb5dae3286a5e5cc14a8ce3e1ebcc14571f1a3541e8db2d18d2f7503764a2f3
Set `P2PConnection.p2p_connected_to_node` in
`P2PConnection.peer_connect_helper()` instead of
`TestNode.add_p2p_connection()` and
`TestNode.add_outbound_p2p_connection()`.
This way tests can create an instance of `P2PConnection` and use
`P2PConnection.peer_connect_helper()` directly.
If requested, make the SOCKS5 Python proxy redirect each connection to a
given destination. Actually act as a real proxy, connecting the
client to a destination, except that the destination is not what the
client asked for.
This would enable us to "connect" to Tor addresses from the functional
tests.
a16917fb59 rpc, net: improve `mapped_as` doc for getrawaddrman/getpeerinfo (brunoerg)
bdad0243be rpc, net: getrawaddrman "mapped_as" follow-ups (brunoerg)
Pull request description:
- Change `addrman` to reference to const since it isn't modified (https://github.com/bitcoin/bitcoin/pull/30062#discussion_r1612272793).
- Improve documentation of `mapped_as`/`source_mapped_as` in `getrawaddrman` RPC by mentioning that both fields will be only available if asmap flag is set. It is the same message for `mapped_as` field in `getpeerinfo`.
ACKs for top commit:
fjahr:
re-ACK a16917fb59
0xB10C:
re-ACK a16917fb59
laanwj:
re-ACK a16917fb59
Tree-SHA512: c66b2ee9d24da93d443be83f6ef3b2d39fd5bf3f73e2974574cad238ffb82035704cf4fbf1bac22a63734948e285e8e091c2884bb640202efdb473315e770233
RegQueryValueExA(HKEY_PERFORMANCE_DATA, ...) sometimes hangs bitcoind.exe on Windows during startup, at least on CI.
We have other sources of entropy to seed randomness with on Windows, so should be alright removing this. Might resurrect if less drastic fix is found.
4d3da08d1b guix: Enable CET for `glibc` package (Hennadii Stepanov)
Pull request description:
Pulled from #30685. This doesn't need to wait for anything.
ACKs for top commit:
laanwj:
ACK 4d3da08d1b
TheCharlatan:
ACK 4d3da08d1b
Tree-SHA512: 1f4645971381fd342adec52c826fc0023722519a3e28043c9fe8b64bbc1abad822fcc25a64f3f959e3f3a10f5c119029f4cae13c22bac6badcbec9ae8b501dfc
86e2a6b749 [test] A non-standard transaction which is also consensus-invalid should return the consensus error (Antoine Poinsot)
f859ff8a4e [validation] Improve script check error reporting (dergoegge)
Pull request description:
An input script might be invalid for multiple reasons. For example, it might fail both a standardness check and a consensus check, which can lead to a `mandatory-script-verify-flag-failed` error being reported that includes the script error string from the standardness failure (e.g. `mandatory-script-verify-flag-failed (Using OP_CODESEPARATOR in non-witness script)`), which is confusing.
ACKs for top commit:
darosior:
re-ACK 86e2a6b749
ariard:
Re-Code Review ACK 86e2a6b7
instagibbs:
ACK 86e2a6b749
Tree-SHA512: 053939107c0bcd6643e9006b2518ddc3a6de47d2c6c66af71a04e8af5cf9ec207f19e54583b7a056efd77571edf5fd4f36c31ebe80d1f0777219c756c055eb42
The `addPackageTxs` method of the `BlockAssembler` currently has access
to two mempool variables, as an argument and as a member. Clean this up
and clarify that they both are the same mempool instance by removing the
argument and instead only using the member variable in the method.
Co-Authored-By: Anthony Towns <aj@erisian.com.au>
Co-authored-by: stickies-v <stickies-v@protonmail.com>
Before, we did not explicity say that both fields
`{source_}mapped_as` (that are optional in getrawaddrman)
will be only available if the asmap config flag is set.
Co-authored-by: Jon Atack <jon@atack.com>
The wallet is isolated during migration and reloaded at the end
of the process. There is no benefit on connecting the signals
few lines before unloading the wallet.
fac6cfe5ac lint: commit-script-check.sh: echo to stderr (MarcoFalke)
Pull request description:
This makes it easier to redirect the produced `git diff` on failure. On success, it shouldn't hurt, because the same output is still present, just on stderr.
Can be tested by introducing a fault in any scripted diff and then calling `commit-script-check.sh HEAD~..HEAD > any_file.txt`. Previously the file contained the full output, now it contains just the diff.
ACKs for top commit:
TheCharlatan:
ACK fac6cfe5ac
Tree-SHA512: b4dfad10a4a902729a7ad7533ed0ef86b9e79761083f2ec623d448a551462b268fe04bdba387ca62160dae9ef7b1781e005dec60f18b111d9bfa6b97357108e6
Also separated the roundtrip testing from the random string decoding for clarity
Note that while BIP 173 claims:
```
The human-readable part, which is intended to convey the type of data, or anything else that is relevant to the reader. This part MUST contain 1 to 83 US-ASCII characters, with each character having a value in the range [33-126]. HRP validity may be further restricted by specific applications.
```
bech32::Encode rejects uppercase letters.
cd0edf26c0 tracing: cast block_connected duration to nanoseconds (0xb10c)
Pull request description:
When the `validation:block_connected` tracepoint was introduced in 8f37f5c2a5, the connect block duration was passed in microseconds `µs`. By starting to use steady clock in fabf1cdb20 this changed to nanoseconds `ns`. As the test only checked if the duration value is `> 0` as a plausibility check, this went unnoticed. This was detected this when setting up monitoring for block validation time as part of the Great Consensus Cleanup Revival discussion.
This change casts the duration explicitly to nanoseconds, updates the documentation, and adds a check for an upper bound to the tracepoint interface tests. The upper bound is quite lax as mining the block takes much longer than connecting the empty test block. It's however able to detect a duration passed in an incorrect unit (1000x off).
A previous version of this PR casted the duration to microseconds `µs` - however, as the last three major releases have had the duration as nanoseconds (and this went unnoticed), we assume that this is the API now and changeing it back to microseconds would break the API again. See also https://github.com/bitcoin/bitcoin/pull/29877#issuecomment-2067867597
ACKs for top commit:
maflcko:
re-lgtm ACK cd0edf26c0
laanwj:
re-ACK cd0edf26c0
Tree-SHA512: 54a1eea0297e01c07c2d071ffafbf97dbd080f763e1dc0014ff086a913b739637c1634b1cf87c90b94a3c2f66006acfaada0414a15769cac761e03bc4aab2a77
184f12c154 doc: remove dependency install instructions from win docs (fanquake)
Pull request description:
This duplicates what is in depends, and is outdated.
Closes#31090.
ACKs for top commit:
maflcko:
lgtm ACK 184f12c154
jarolrod:
ACK 184f12c154
BrandonOdiwuor:
ACK 184f12c154
Tree-SHA512: 089c9ff91c501c22ec1b9d5925a2b8c6cd1ea9ac2b75dd6a8c5fe75cf2f0090d808842cb321017894d2da70a30a87dbc1c4c481771d3c4aba13ce44244fcf392
a647d4400d doc: update signet documentation related to build directories (Torkel Rogstad)
Pull request description:
While setting up my own signet I noticed that the binary paths in the documentation for this is out of date, after build artifacts moved to the `build` directory. This PR mimics what happened in #30741
ACKs for top commit:
maflcko:
lgtm ACK a647d4400d
pablomartin4btc:
ACK a647d4400d
tdb3:
Code review and light test ACK a647d4400d
Tree-SHA512: ac7c3806e0ff65860c41d7b7bdad538368d8a6d8d289c10f9714804f963bafd3a9658301b6697f110f5462a92826b62770963508d5eebf88bf9a0a8442d9f72d
To detect cross-compiling, the host and build platforms are compared.
The `build` variable is always an output of `config.sub`, but the `host`
is not. This can lead to false results. For example, on OpenBSD:
- host=amd64-unknown-openbsd7.5
- build=x86_64-unknown-openbsd7.5
This change sets the default value of the `host` variable to the value
of `build`, ensuring cross-compiling won't be triggered when the `HOST`
variable is not set.
fa43c4f93c test: Print CompletedProcess object on error (MarcoFalke)
Pull request description:
It would be good to know the output on `Error parsing command output`. Otherwise test failures are meaningless: https://github.com/bitcoin/bitcoin/issues/30792#issuecomment-2325911157
Fix it by just printing the full `CompletedProcess` object.
Also, use the modern `subprocess.run` to simplify the code.
ACKs for top commit:
BrandonOdiwuor:
Code Review ACK fa43c4f93c
laanwj:
This contains some useful information, so ACK fa43c4f93c
Tree-SHA512: ae7c1cb1f48af2a6feae6d1a5a967c0720f6c6675c1ce20ace7cac18c00f3d4069b8abcc58204855e92ff5303158b9a942bab3b71acae0737768d941a5773c91
ec585f11c3 Reserve space for transaction inputs in CreateTransactionInternal (Lőrinc)
c76aaaf900 Reserve space for transaction outputs in CreateTransactionInternal (Lőrinc)
Pull request description:
Reserved memory for the transaction inputs and outputs.
Split out of https://github.com/bitcoin/bitcoin/pull/30050/files#r1597631104
ACKs for top commit:
achow101:
ACK ec585f11c3
TheCharlatan:
ACK ec585f11c3
stickies-v:
ACK ec585f11c3
Tree-SHA512: de399fb19824423467f48af64aa57f41a23cdd00eb17461e0131e4deafdd15e0d2daebf6a0a7ac7728b2fb486b2a54f1a7ef26bbe823c56b2a09f892f6b9a581
fa1b139d17 Bump python minimum supported version to 3.10 (MarcoFalke)
Pull request description:
All supported operating systems ship with python 3.10 (or later), so bumping the minimum should not cause any issues. A bump will allow new code to use new python features.
For reference:
* https://packages.debian.org/bookworm/python3
* https://packages.ubuntu.com/jammy/python3
* FreeBSD 13/14 ships with 3.11
* CentOS-like 8/9 ships with 3.11/3.12 (via `appstream`)
* OpenSuse Tumbleweed ships with all python versions, e.g. https://software.opensuse.org/package/python312-base
This is for Bitcoin Core 29.0 in 2025 (next year), not the soon upcoming 28.0 this fall.
ACKs for top commit:
achow101:
ACK fa1b139d17
AngusP:
ACK fa1b139d17
l0rinc:
ACK fa1b139d17
stickies-v:
ACK fa1b139d17
Tree-SHA512: 910b202ff2374bb21c93e5249a151fd2c8f63759bed5659676b0e467afa6e8e977be797c3fccceca303c82575e11ec236a8d7c5880910e4314b3875b820e7e8a
a0e089a71d build: Bump minimum supported macOS to 13.0 (Hennadii Stepanov)
Pull request description:
Running Bitcoin Core on unsupported OSes may expose users to security issues.
macOS Monterey 12 received its final security update ([12.7.6](https://support.apple.com/en-us/100100)) on July 2024. Apple classifies the hardware that can run macOS 12 at most as ["obsolete worldwide"](https://support.apple.com/en-us/102772).
ACKs for top commit:
maflcko:
lgtm ACK a0e089a71d
m3dwards:
ACK a0e089a71d
itornaza:
reACK a0e089a71d
Tree-SHA512: b219730de87bcb2bcb40a972e910f516c739a538b0741fc245d23df04650f7e2f5774c38c1d1c9c053ed9e2a377488002feb708e8c7cba9c0070b81169719b10
Due to a bug in earlier versions, some wallets without private keys may
have an encryption key. This encryption key is unused and can lead to
confusing behavior elsewhere. When such wallets are detected, those
encryption keys will now be deleted from the wallet. For safety, we only
do this to wallets which have private keys disabled, have encryption keys,
and definitely do not have encrypted keys.
There is no need to compare the field to CLIENT_VERSION. Either the
format remains compatible and the value can be left unchanged, or it is
incompatible and the value needs to be increased to at least 289900+1.
d823ba6e20 doc: fuzz: remove Honggfuzz NetDriver instructions (brunoerg)
Pull request description:
Remove Honggfuzz NetDriver instructions from the documentation since it has not been useful for us. See https://github.com/bitcoin/bitcoin/issues/30957 and https://github.com/bitcoin/bitcoin/pull/31012.
ACKs for top commit:
maflcko:
lgtm ACK d823ba6e20
marcofleon:
ACK d823ba6e20
Tree-SHA512: f63fde1076d523dc5e511ef868ca3c1ea2e38fe7df56ae275f33209581f96452d86effedb54d9b0ee8b7a1d492b610799807a727d8bd81e2286d31db4aa68731
e64b2f1a16 doc: cmake: prepend and explain "build/" where needed (Larry Ruane)
Pull request description:
This is a small follow-on to #30741, prepend `build/` to the path for `test_runner.py`.
ACKs for top commit:
jonatack:
ACK e64b2f1a16
maflcko:
lgtm ACK e64b2f1a16
tdb3:
re ACK e64b2f1a16
Tree-SHA512: 80943d4f342987bf060adacb1c7db2e9ff8de5a6da592846ba23f230281d3a5b306162c4c86e61739a29323eaa4abf09f69f41302996d5809f448e5788a74a87
1fe1b3ba8e doc: doxygen comment for m_args usage in tests (willcl-ark)
Pull request description:
Closes: #25055
Add a doxygen comment to the `m_args` member in the unit test framework, clarifying its purpose and providing usage guidelines.
ACKs for top commit:
maflcko:
lgtm ACK 1fe1b3ba8e
brunoerg:
ACK 1fe1b3ba8e
Tree-SHA512: 9b8dc30e3b0d26c0cecec4599dc5addca519965603073d02f37fa0a46c488659958e327d9c25da8acdb4bb9b082a64455baaffb406ac11827d7f56a094522fce
Running Bitcoin Core on unsupported OSes may expose users to security
issues.
macOS Monterey 12 received its final security update (12.7.6) on July
2024. Apple classifies the hardware that can run macOS 12 at most as
"obsolete worldwide".
ccd10fdb97 build: Add missing USDT header dependency to kernel (Cory Fields)
Pull request description:
Noticed while testing a branch that replaces `boost::multi_index` with a custom replacement.
Currently depends builds pick up usdt and boost from the same path, and because boost always exists, the usdt path is implicitly included. So without boost, USDT isn't found.
An alternative to this would be to disable USDT for the kernel. I'd be open to either approach.
ACKs for top commit:
hebasto:
ACK ccd10fdb97, the diff looks correct.
fanquake:
ACK ccd10fdb97
Tree-SHA512: 2f91b8d5c8b169f7b72323d9163b5201f606ccdab95de7085847d2a672d10f940f69642c2528226a5efa4c589af24ca3bb9dd909eed0993e4cecd9689b4bed2f
0b3ec8c59b clusterlin: remove Cluster type (Pieter Wuille)
1c24c62510 clusterlin: merge two DepGraph fuzz tests into simulation test (Pieter Wuille)
0606e66fdb clusterlin: add DepGraph::RemoveTransactions and support for holes in DepGraph (Pieter Wuille)
75b5d42419 clusterlin: make DepGraph::AddDependency support multiple dependencies at once (Pieter Wuille)
abf50649d1 clusterlin: simplify DepGraphFormatter::Ser (Pieter Wuille)
eaab55ffc8 clusterlin: rework DepGraphFormatter::Unser (Pieter Wuille)
5901cf7100 clusterlin: abstract out DepGraph::GetReduced{Parents,Children} (Pieter Wuille)
Pull request description:
Part of cluster mempool: #30289
This adds:
* `DepGraph::AddDependencies` to add 0 or more dependencies to a single transaction at once (identical to calling `DepGraph::AddDependency` once for each, but more efficient).
* `DepGraph::RemoveTransactions` to remove 0 or more transactions from a depgraph.
* `DepGraph::GetReducedParents` (and `DepGraph::GetReducedChildren`) to get the (reduced) direct parents and children of a transaction in a depgraph.
After which, the `Cluster` type is removed.
This is the result of fleshing out the design for the "intermediate layer" ("TxGraph", no PR yet) between the cluster linearization layer and the mempool layer. My earlier thinking was that TxGraph would store `Cluster` objects (vectors of pairs of `FeeFrac`s and sets of parents), and convert them to `DepGraph` on the fly whenever needed. However, after more consideration, it seems better to have TxGraph store `DepGraph` objects, and manipulate them directly without constantly re-creating them. This requires `DepGraph` to have some additional functionality.
The bulk of the complexity here is the addition of `DepGraph::RemoveTransactions`, which leaves the remaining transactions' positions within the `DepGraph` untouched (we want existing identifiers to remain valid), so this implies that graphs can now have "holes" (positions that are unused, but followed by positions that are used). To enable that, an extension of the fuzz/test serialization format `DepGraphFormatter` is included to deal with such holes.
ACKs for top commit:
sdaftuar:
reACK 0b3ec8c59b
instagibbs:
reACK 0b3ec8c59b
ismaelsadeeq:
reACK 0b3ec8c59b
glozow:
ACK 0b3ec8c59b, reviewed range-diff from aab53ddcd8fcbc3c0be0da9383f8e06abe5badda and `clusterlin_depgraph_sim`
Tree-SHA512: a804b7f26d544c5cb0847322e235c810525cb0607737be6116c3156d582da3ba3352af8ea48e74eed5268f9c3eca63b30181d01b23a6dd0be1b99191f81cceb0
882f736d0a doc: lint: correct outdated comment (s/Makefile.am/CMakeLists.txt/) (Sebastian Falbesoner)
1786be7b4a scripted-diff: drop config/ subdir for bitcoin-config.h, rename to bitcoin-build-config.h (Sebastian Falbesoner)
Pull request description:
This PR is a follow-up to #30856, as suggested in comment https://github.com/bitcoin/bitcoin/pull/30856#issuecomment-2356804690. With the scripted diff, review should be fairly trivial, but it could still be seen as controversial due to the large number of files (78 in total) being touched.
ACKs for top commit:
fanquake:
ACK 882f736d0a
Tree-SHA512: 2e6cae4590f660e741edf84df456168b8b1f3861d381cfebf6647bb0a303c26bf7b969a837e0058e59bf852d220990dd8f5f400d8975fd0fab106d0507a70c9b
The chainstate caches are currently re-balanced on startup
even in the non-assumeutxo case, leading to the database being
needlessly re-opened and its cache re-allocated.
Similar to `InitCoinsCache` and `m_coinstip_cache_size_bytes` the
`m_coinsdb_cache_size_bytes` should be set in `InitCoinsDB`.
Together with only conservatively setting the cache values when a
assumeutxo chainstate is present, this allows for skipping the cache
re-balance during initialization in the normal non-assumeutxo case.
525e9dcba0 Add submitSolution to BlockTemplate interface (Sjors Provoost)
47b4875ef0 Add getCoinbaseMerklePath() to Mining interface (Sjors Provoost)
63d6ad7c89 Move BlockMerkleBranch back to merkle.{h,cpp} (Sjors Provoost)
Pull request description:
The new `BlockTemplate` interface introduced in #30440 allows for a more efficient way for a miner to submit the block solution. Instead of having the send the full block, it only needs to provide the nonce, timestamp, version fields and coinbase transaction.
This PR introduces `submitSolution()` for that. It's currently unused.
#29432 and https://github.com/Sjors/bitcoin/pull/48 use it to process the Stratum v2 message [SubmitSolution](https://github.com/stratum-mining/sv2-spec/blob/main/07-Template-Distribution-Protocol.md#77-submitsolution-client---server). The method should be sufficiently generic to work with alternative mining protocols (none exist that I'm aware off).
This PR also introduces `getCoinbaseMerklePath()`, which is needed in Stratum v2 to construct the `merkle_path` field of the `NewTemplate` message (see [spec](https://github.com/stratum-mining/sv2-spec/blob/main/07-Template-Distribution-Protocol.md#72-newtemplate-server---client)). The coinbase merkle path is also used in Stratum "v1", see e.g. https://bitcoin.stackexchange.com/questions/109820/questions-on-merkle-root-hashing-for-stratum-pools
This last function uses `BlockMerkleBranch` which was moved to the test code in #13191. The reason back then for moving it was that it was no longer used. This PR moves it back.
This PR does not change behaviour since both methods are unused.
ACKs for top commit:
achow101:
ACK 525e9dcba0
itornaza:
Code review ACK 525e9dcba0
tdb3:
Code review and light test ACK 525e9dcba0
ryanofsky:
Code review ACK 525e9dcba0. Left minor suggestions but none are important, and looks like this could be merged as-is
Tree-SHA512: 2a6a8f5d409ff4926643193cb67702240c7c687615414371e53383d2c13c485807f65e21e8ed98515b5456eca3d9fca13cec04675814a4081467d88b849c5653
36a6d4b007 doc: update IBD requirements in doc/README.md (Mackain)
Pull request description:
A small change to the first paragraph of the Setup part of the README that has been bugging me for a while.
The disk space required for the Bitcoin transactions can no longer be described as "a few" hundred gigabytes.
So I thought it was time it was changed to "several" instead.
<!--
*** Please remove the following help text before submitting: ***
Pull requests without a rationale and clear improvement may be closed
immediately.
GUI-related pull requests should be opened against
https://github.com/bitcoin-core/gui
first. See CONTRIBUTING.md
-->
<!--
Please provide clear motivation for your patch and explain how it improves
Bitcoin Core user experience or Bitcoin Core developer experience
significantly:
* Any test improvements or new tests that improve coverage are always welcome.
* All other changes should have accompanying unit tests (see `src/test/`) or
functional tests (see `test/`). Contributors should note which tests cover
modified code. If no tests exist for a region of modified code, new tests
should accompany the change.
* Bug fixes are most welcome when they come with steps to reproduce or an
explanation of the potential issue as well as reasoning for the way the bug
was fixed.
* Features are welcome, but might be rejected due to design or scope issues.
If a feature is based on a lot of dependencies, contributors should first
consider building the system outside of Bitcoin Core, if possible.
* Refactoring changes are only accepted if they are required for a feature or
bug fix or otherwise improve developer experience significantly. For example,
most "code style" refactoring changes require a thorough explanation why they
are useful, what downsides they have and why they *significantly* improve
developer experience or avoid serious programming bugs. Note that code style
is often a subjective matter. Unless they are explicitly mentioned to be
preferred in the [developer notes](/doc/developer-notes.md), stylistic code
changes are usually rejected.
-->
<!--
Bitcoin Core has a thorough review process and even the most trivial change
needs to pass a lot of eyes and requires non-zero or even substantial time
effort to review. There is a huge lack of active reviewers on the project, so
patches often sit for a long time.
-->
ACKs for top commit:
achow101:
ACK 36a6d4b007
danielabrozzoni:
ACK 36a6d4b007
jonatack:
ACK 36a6d4b007
ismaelsadeeq:
ACK 36a6d4b007
tdb3:
ACK 36a6d4b007
itornaza:
ACK 36a6d4b007
Tree-SHA512: c5b21aca526c0ebe5f3234bd72e4080dc64cbba0ccd2306397aafe8349bc3573773ee64ff31fafcf59ea1afc7527caaf6d7cd8fe798311d9dc11ad0cd539e21e
ca2e4ba352 refactor: include the proper header rather than forward-declaring RemovalReasonToString (Cory Fields)
Pull request description:
Trivial no-op fixup.
This was pointed out by #31053, which causes the include order to be shuffled around:
```
[21:49:26.130] /ci_container_base/src/validationinterface.cpp:22:13: error: redundant 'RemovalReasonToString' declaration [readability-redundant-declaration,-warnings-as-errors]
[21:49:26.130] 22 | std::string RemovalReasonToString(const MemPoolRemovalReason& r) noexcept;
[21:49:26.130] | ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[21:49:26.130] /ci_container_base/src/kernel/mempool_removal_reason.h:22:13: note: previously declared here
[21:49:26.130] 22 | std::string RemovalReasonToString(const MemPoolRemovalReason& r) noexcept;
[21:49:26.130] | ^
```
I don't see any reason why the include shouldn't just be used.
ACKs for top commit:
maflcko:
lgtm ACK ca2e4ba352
hebasto:
ACK ca2e4ba352, IWYU seems [agree](https://cirrus-ci.com/task/6170839912022016):
TheCharlatan:
ACK ca2e4ba352
Tree-SHA512: e3584cae4f50bf2bc6c824bfaddfe683ef6a17d16138d0cbcc544b98bd64d5d7353b0826b1e8cf16e12410e27b0fcedde27100d4241b7cc194cd4465c8175a5b
33381ea530 scripted-diff: Modernize nLocalServices to m_local_services (Fabian Jahr)
Pull request description:
The type of the `nLocalServices` variable was changed to `std::atomic<ServiceFlags>` in #30807 and I suggested the variable name to get updated with a scripted diff along with it. It wasn't included in the PR but I am still suggesting to do it as a follow-up since I had already prepared the commit.
ACKs for top commit:
sipa:
utACK 33381ea530
achow101:
ACK 33381ea530
furszy:
utACK 33381ea530
jonatack:
ACK 33381ea530
theStack:
ACK 33381ea530
Tree-SHA512: 407ea9eac694f079aa5b5c1611b5874d7a0897ba6bc3aa0570be94afe1bf3a826657b6890b6597c03c063e95b9dc868f0bdfbfc41e77ec7e06f5b045bf065c71
fa22e5c430 refactor: Remove dead code that assumed tip == nullptr (MarcoFalke)
fa2e443965 refactor: Replace g_genesis_wait_cv with m_tip_block_cv (MarcoFalke)
fa7f52af1a refactor: Use wait_for predicate to check for interrupt (MarcoFalke)
5ca28ef28b refactor: Split up NodeContext shutdown_signal and shutdown_request (Ryan Ofsky)
fad8e7fba7 bugfix: Mark m_tip_block_cv as guarded by m_tip_block_mutex (MarcoFalke)
fa18586c29 refactor: Add missing GUARDED_BY(m_tip_block_mutex) (MarcoFalke)
fa4c075033 doc: Clarify waitTipChanged docs (MarcoFalke)
Pull request description:
`g_genesis_wait_cv` is similar to `m_tip_block_cv` but shuffling everything through a redundant `boost::signals2`.
So remove it, along with some other dead code, as well as minor fixups.
ACKs for top commit:
ryanofsky:
Code review ACK fa22e5c430 (just rebased since last review)
Sjors:
ACK fa22e5c430
TheCharlatan:
ACK fa22e5c430
Tree-SHA512: a2cb59b651aaf85a3574723adfe403487566788ad945933b0458816ccc841fce08ca77b31afbd2d6adb5bf1deed7229c028bee74fb4bbaf6576e9edcfa0ad817
fae44c83da test: Remove 0.16.3 test from wallet_backwards_compatibility.py (MarcoFalke)
Pull request description:
The test checks that any wallet created with current master can not be loaded with `v0.16.3`. This is interesting documentation, however it is probably not something to keep as a test, because:
* It seems like an extremely unlikely (and unsupported) edge case that someone creates a wallet with master and then goes ahead to open it with a long EOL software version.
* A better test would be the inverse: Create a wallet with `v0.16.3` and open it with current master. This is already tested in `wallet_upgradewallet.py`, where I've added an additional balance check before upgrading the `v0.16.3` wallet.
* The test is intermittently failing when shutting down the `v0.16.3` node, for example in https://github.com/bitcoin/bitcoin/pull/30875#issuecomment-2357565564. The exact cause is unclear, but given that the test isn't worthy to keep, removing it will ensure that the error disappears.
ACKs for top commit:
Sjors:
utACK fae44c83da
fanquake:
ACK fae44c83da - I agree that test seems to have past it's usefulness, and the fact that it otherwise causes intemittent issues is further reason to remove it.
Tree-SHA512: 85bf428e616e0880198c1a7529936520505d7fa87c2eeb87a0457f13b50a163accaf5f80f9364dea978f6bd14b0b5350cda88f49aa7584682c8b5e0b0b117724
fa1cffacae ci: Install missing nproc in macos task (MarcoFalke)
faf7a2bccc ci: Add missing -DWERROR=ON to test-each-commit (MarcoFalke)
Pull request description:
Found by in Sjors in https://github.com/bitcoin/bitcoin/pull/27260#discussion_r1785860610 (Thanks!)
Also, includes an unrelated commit to simplify the ci scripts, by assuming nproc exists on macos as well. (Having more than one commit is also required to trigger the `test-each-commit` task)
ACKs for top commit:
theuni:
utACK fa1cffacae.
Tree-SHA512: 881df6964da31c3f7161b763bdf74029743567272c7a4db24ded10909a8cedad39b494a9f486f0b059bbeb0beb190dcd91369650d014e1fc3b5f7ae07cd54771
fa6d14eacb test: Treat exclude list warning as failure in CI (MarcoFalke)
Pull request description:
An outdated exclude list or otherwise an error in the exclude list handling is usually a bug.
So make it fatal in the CI, instead of silently ignoring it.
Fixes https://github.com/bitcoin/bitcoin/pull/30872/files#r1757015334
Can be tested with something like (with and without `--ci`):
```
./bld-cmake/test/functional/test_runner.py wallet_disable -x wallet_disablee
ACKs for top commit:
tdb3:
ACK fa6d14eacb
ismaelsadeeq:
utACK fa6d14eacb
Tree-SHA512: 03a70dff9d1272d982591d60ab764f9233d4802488bc1bad305a2755e2d7ed86e691ee94767a3bc5f68321b63214aba44e6f9edd1543dfad7a20f9397cf78734
fa5ebc9920 ci: Double ctest timeout (MarcoFalke)
Pull request description:
It looks like msan sometimes hits the timeout. So double it, which should still be useful to catch real timeouts in the wine windows-cross unit tests.
Example: https://cirrus-ci.com/task/5141974322249728
ACKs for top commit:
fanquake:
ACK fa5ebc9920
Tree-SHA512: de346f34c4d157be515706fd71c2072fb5859d50d0e8cd7e6c4637578e4dc54aa8c75b2521e7aecf679b486c4e20fd184e259ffc701b8b668d7f61abff883875
ae56b3230b depends: For mingw cross compile use -gcc-posix to prevent library conflict (laanwj)
Pull request description:
CMake parses some paths from the spec of the C compiler, assuming it will be the linker, resulting in the link to end up with `-L/usr/lib/gcc/x86_64-w64-mingw32/12-win32` on debian bookworm if both `-win32` and `-posix` variants are installed, and `-win32` is the default alternative.
This results in the wrong C++ library being linked, missing std::threads::hardware_concurrency and other threading functions.
To fix this, use the `-posix` variant of gcc as well when available. This fixes a regression compared to autotools, where this scenario worked.
ACKs for top commit:
theuni:
utACK ae56b3230b.
hebasto:
ACK ae56b3230b. I've tested on both Debian Bookworm and Ubuntu 24.04 with the `g++-mingw-w64-x86-64` package installed. The resulting CMake internal configuration appears more accurate. For instance, on Ubuntu 24.04, for the `bitcoin-tx` target, the diff in `build/src/CMakeFiles/bitcoin-tx.dir/linkLibs.rsp` looks as follows:
Tree-SHA512: f36fae50f91a29f565940494af9e46f47e219b99e329c0714ace47c516ac524602d5b6538a07488157bc2a71be7bac72176097fff3178129c5084bf6cc823167
e0287bc4b2 test: remove unused code from script_tests (fanquake)
Pull request description:
This has been unused since #29648. Noticed while running a newer version of clang-tidy (19.1.1):
```bash
[127/391][6.2s] /opt/homebrew/opt/llvm/bin/clang-tidy -p=build -quiet --config-file=/bitcoin/src/.clang-tidy /bitcoin/src/test/script_tests.cpp
bitcoin/src/test/script_tests.cpp:126:25: error: local copy 'tx2' of the variable 'tx' is never modified and never used; consider removing the statement [performance-unnecessary-copy-initialization,-warnings-as-errors]
126 | CMutableTransaction tx2 = tx;
| ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~
127 | BOOST_CHECK_MESSAGE(VerifyScript(scriptSig, scriptPubKey, &scriptWitness, flags, MutableTransactionSignatureChecker(&tx, 0, txCredit.vout[0].nValue, MissingDataBehavior::ASSERT_FAIL), &err) == expect, message);
512 warnings generated.
```
ACKs for top commit:
maflcko:
review ACK e0287bc4b2
BrandonOdiwuor:
ACK e0287bc4b2
Tree-SHA512: f69513d0b898e0e9afad047bcec200707b057e3718a3d35bd479a788a1973e49ee7e5f48feadb8731ab5fdbd12a2b53b0bcf65296701e2296c3fdb67cdcabfb5
605926da0a depends: Print ready-to-use `--toolchain` option for CMake invocation (Hennadii Stepanov)
Pull request description:
Requested in https://github.com/bitcoin/bitcoin/pull/30997#issuecomment-2385057317:
> P.S. it would be nice if `make` in `depends` provides the incantation needed for the configure stage.
An example of a build log with this PR:
```
$ make --no-print-directory -C depends -j16 NO_QT=1 NO_WALLET=1 NO_UPNP=1 NO_NATPMP=1 NO_ZMQ=1 NO_USDT=1 LOG=1
Extracting boost...
/home/hebasto/git/bitcoin/depends/sources/boost_1_81_0.tar.gz: OK
Preprocessing boost...
Configuring boost...
Building boost...
Staging boost...
Postprocessing boost...
Caching boost...
Extracting libevent...
/home/hebasto/git/bitcoin/depends/sources/libevent-2.1.12-stable.tar.gz: OK
Preprocessing libevent...
Configuring libevent...
Building libevent...
Staging libevent...
Postprocessing libevent...
Caching libevent...
copying packages: boost libevent
to: /home/hebasto/git/bitcoin/depends/x86_64-pc-linux-gnu
To build Bitcoin Core with these packages, pass '--toolchain /home/hebasto/git/bitcoin/depends/x86_64-pc-linux-gnu/toolchain.cmake' to the first CMake invocation.
```
ACKs for top commit:
Sjors:
tACK 605926da0a
theuni:
ACK 605926da0a
jarolrod:
ACK 605926da0a
pablomartin4btc:
ACK 605926da0a
Tree-SHA512: 658358cb12a2409486962273d418f8cfa9609c84c6e70b426250173925c0938a6eba0dca72f43d78cc63c9cb927514c959473da04bdcda714228c5d8c248c1fe
This combines the clusterlin_add_dependency and clusterlin_cluster_serialization
fuzz tests into a single clusterlin_depgraph_sim fuzz test. This tests starts
from an empty DepGraph and performs a arbitrary number of AddTransaction,
AddDependencies, and RemoveTransactions operations on it, and compares the
resulting state with a naive reimplementation.
This commits introduces support in DepGraph for the transaction positions to be
non-continuous. Specifically, it adds:
* DepGraph::RemoveTransactions which removes 0 or more positions from a DepGraph.
* DepGraph::Positions() to get a set of which positions are in use.
* DepGraph::PositionRange() to get the highest used position in a DepGraph + 1.
In addition, it extends the DepGraphFormatter format to support holes in a
compatible way (it serializes non-holey DepGraphs identically to the old code,
and deserializes them the same way)
This changes DepGraph::AddDependency into DepGraph::AddDependencies, which takes
in a single child, but a set of parent transactions, making them all dependencies
at once.
This is important for performance. N transactions can have O(N^2) parents combined,
so constructing a full DepGraph using just AddDependency (which is O(N) on its own)
could take O(N^3) time, while doing the same with AddDependencies (also O(N) on its
own) only takes O(N^2).
Notably, this matters for DepGraphFormatter::Unser, which goes from O(N^3) to O(N^2).
Co-Authored-By: Greg Sanders <gsanders87@gmail.com>
This does not change the serialization format.
It turns out that it is unnecessary to keep track of the order of transactions
in the so-far reconstructed DepGraph to decide how far from the end to insert
a new transaction.
This commit does not change the serialization format. Its purpose is making a
few changes already in order to reduce the diff size of the later commit that
introduces support for holes in DepGraph.
The previous approach was to immediately construct a transaction as soon as its
feerate was known in a preliminary position, and then undo that, and place it
in the correct position once the position information is known (such that a
deserialization error in between would not result in an inconsistent state).
The new approach is to delay the actual transaction creation until all its
information is known, avoiding the need to undo and redo. This requires a
different means of determining whether dependencies are redundant, but that has
the advantage that a later commit can apply all dependencies at once, reducing
the complexity of deserialization.
A fuzz test already relies on these operations, and a future commit will need
the same logic too. Therefore, abstract them out into proper member functions,
with proper testing.
This has been unused since #29648.
Noticed while running a newer version of clang-tidy (19.1.1):
```bash
[127/391][6.2s] /opt/homebrew/opt/llvm/bin/clang-tidy -p=build -quiet --config-file=/bitcoin/src/.clang-tidy /bitcoin/src/test/script_tests.cpp
bitcoin/src/test/script_tests.cpp:126:25: error: local copy 'tx2' of the variable 'tx' is never modified and never used; consider removing the statement [performance-unnecessary-copy-initialization,-warnings-as-errors]
126 | CMutableTransaction tx2 = tx;
| ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~
127 | BOOST_CHECK_MESSAGE(VerifyScript(scriptSig, scriptPubKey, &scriptWitness, flags, MutableTransactionSignatureChecker(&tx, 0, txCredit.vout[0].nValue, MissingDataBehavior::ASSERT_FAIL), &err) == expect, message);
512 warnings generated.
```
56aad83307 ci: set a ctest timeout of 1200 (20 minutes) (fanquake)
Pull request description:
This should be long enough (with headroom) for our longest running tests, which even under MSAN, TSAN, Valgrind, etc max out at about 800s.
i.e under Valgrind I see the longer runtimes as:
```bash
135/136 Test #8: bench_sanity_check_high_priority ..... Passed 371.19 sec
136/136 Test #122: coinselector_tests ................... Passed 343.39 sec
```
In the CI `tests` [under TSAN](https://cirrus-ci.com/task/6321297691508736?logs=ci#L2520):
```bash
tests ................................ Passed 795.20 sec
```
[and MSAN](https://cirrus-ci.com/task/4913922807955456?logs=ci#L2226):
```bash
tests ................................ Passed 658.48 sec
```
This will also prevent the current issue we are seeing of `ctest` running until it reaches the CI timeout, see #30969.
We still need to figure out what underlying issue is causing the tests to (sometimes) run for so long, but in the mean time, this will stop `ctest` wasting our CI CPU. It should also make it more clear in the logs, exactly which test is the one that is hitting the timeout.
ACKs for top commit:
maflcko:
review ACK 56aad83307
tdb3:
re ACK 56aad83307
Tree-SHA512: 43c0dc12b8b12b1d9804751a9816935e2abbe962b451e12a268f2d2c430bc568b83995dbc405f100b596dfb0f1e9f65b78074de98916592d3ae4ebc2126e3a6c
It is bad, because it is both printed for non-GUI users and does not
convey additional information.
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
This should be long enough (with headroom) for our longest running tests,
which even under MSAN, TSAN, Valgrind, etc max out at about 800s.
i.e under Valgrind I see the longer runtimes as:
```bash
135/136 Test #8: bench_sanity_check_high_priority ..... Passed 371.19 sec
136/136 Test #122: coinselector_tests ................... Passed 343.39 sec
```
In the CI `tests` under TSAN:
```bash
tests ................................ Passed 795.20 sec
```
and MSAN:
```bash
tests ................................ Passed 658.48 sec
```
This will also prevent the current issue we are seeing of `ctest`
running until it reaches the CI timeout, see #30969.
However, we still need to figure out what underlying issue is causing
the tests to (sometimes) run for so long, but in the mean time, this
will stop `ctest` wasting our CI CPU.
80761afced qt6: Handle different signatures of `QANEF::nativeEventFilter` (Hennadii Stepanov)
Pull request description:
Split from https://github.com/bitcoin/bitcoin/pull/30997.
This PR ensures compatibility across all supported Qt versions.
For more details, please refer to 3b38c73c7f.
No behaviour change.
ACKs for top commit:
maflcko:
lgtm ACK 80761afced
promag:
Code review ACK 80761afced.
Tree-SHA512: a265e1c33cc7da37003bb0e6fd40950acb5e948ca9ec63a59a79c5e2a1894334f48d5565539c91d4d777b48a589366958df1498eaa6935e3b7fb534493adb51a
deacf3c7cd cmake: Avoid hardcoding Qt's major version in Find module (Hennadii Stepanov)
Pull request description:
This PR facilitates future migration to Qt 6 and is a prerequisite for https://github.com/bitcoin/bitcoin/pull/30997.
No behaviour change.
ACKs for top commit:
l0rinc:
utACK deacf3c7cd
promag:
Code review ACK deacf3c7cd.
maflcko:
lgtm ACK deacf3c7cd
Tree-SHA512: 6991f30e9cf6a7103bfe91f8958246b17360210bf695ab620ca1c8b59565bf6192cc366036cf54f860ccc4d782b9c28899970978de79392c88a45ef149c06a79
fee4cba484 gui: Fix proxy details display in Options Dialog (pablomartin4btc)
Pull request description:
Currently, setting up a proxy (whether SOCKS5 or Tor) with an IPv6 address works correctly via the command line or configuration file in both `bitcoind` and `bitcoin-qt` (also from the UI the ipv6 address gets saved properly in `settings.json`). However, the UI does not reflect this properly, which can create confusion. Since some ISPs and VPNs still experience issues with IPv6, users may mistakenly think there is a problem with Bitcoin Core, when in fact the proxy setup is functioning as expected.
So this PR ensures that the proxy IP is displayed correctly in the UI when using an IPv6 address.
No functionality impact; changes only affect UI display.
<details>
<summary>Click her to see <b>before</b> and <b>after</b> screenshots.</summary>
- Before:

- After:

</details>
---
<details>
<summary>Test instructions</summary>
(Ubuntu 22.04)
1. Start ssh service on localhost.
`ssh -D [::1]:1080 -f -C -q -N localhost`
2. Check that the service is up and running.
```
ps aux | grep ssh
pepe 2860289 0.0 0.0 20456 5576 ? Ss 06:59 0:00 ssh -D [::1]:1080 -f -C -q -N localhost
```
3. Check with `bitcoind` if it works correctly.
`bitcoind -onlynet=ipv6 -proxy=[::1]:1080`
4. Check for established connections.
```
netstat -natl |grep 1080
tcp6 0 0 ::1:1080 :::* LISTEN
tcp6 0 0 ::1:47610 ::1:1080 ESTABLISHED
tcp6 0 0 ::1:1080 ::1:47610 ESTABLISHED
tcp6 0 0 ::1:1080 ::1:47606 TIME_WAIT
```
```./build/src/bitcoin-cli getpeerinfo
[
{
"id": 0,
"addr": "[2a01:4f9:4a:2a07::2]:8333",
"addrbind": "[::1]:47638",
"network": "ipv6",
...
```
5. Stop `bitcoind` and run `bitcoin-qt` adding the corresponding configuration in `settings.json`.
```
{
"onlynet": "ipv6",
"proxy": "[::1]:1080",
}
```
6. Open the Peers window to check available connections or run `getpeerinfo` on the rpc-console window.
7. Same can be done for Tor setting up `tor` service (I'll add instructions later) and configuring on its default port 9050 and forcing `"onlynet": "onion"` to verify easily the net traffic.
</details>
---
Thanks jarolrod and vasild for your help on validating ipv6 was not broken.
ACKs for top commit:
vasild:
ACK fee4cba484
promag:
Code review ACK fee4cba484.
hebasto:
ACK fee4cba484, I have reviewed the code and it looks OK.
Tree-SHA512: 4be9052569ccb1e17ce94fb15691debf0651fa172ed1a83d60696d10f20d469b19d70a979b65322951f5783cd7582d55b39b669edb588e20404d8d10e767c49a
98c1536852 test: add getorphantxs tests (tdb3)
93f48fceb7 test: add tx_in_orphanage() (tdb3)
34a9c10e8c rpc: add getorphantxs (tdb3)
f511ff3654 refactor: move verbosity parsing to rpc/util (tdb3)
532491faf1 net: add GetOrphanTransactions() to PeerManager (tdb3)
91b65adff2 refactor: add OrphanTxBase for external use (tdb3)
Pull request description:
This PR adds a new hidden rpc, `getorphantxs`, that provides the caller with a list of orphan transactions. This rpc may be helpful when checking orphan behavior/scenarios (e.g. in tests like `p2p_orphan_handling`) or providing additional data for statistics/visualization.
```
getorphantxs ( verbosity )
Shows transactions in the tx orphanage.
EXPERIMENTAL warning: this call may be changed in future releases.
Arguments:
1. verbosity (numeric, optional, default=0) 0 for an array of txids (may contain duplicates), 1 for an array of objects with tx details, and 2 for details from (1) and tx hex
Result (for verbose = 0):
[ (json array)
"hex", (string) The transaction hash in hex
...
]
Result (for verbose = 1):
[ (json array)
{ (json object)
"txid" : "hex", (string) The transaction hash in hex
"wtxid" : "hex", (string) The transaction witness hash in hex
"bytes" : n, (numeric) The serialized transaction size in bytes
"vsize" : n, (numeric) The virtual transaction size as defined in BIP 141. This is different from actual serialized size for witness transactions as witness data is discounted.
"weight" : n, (numeric) The transaction weight as defined in BIP 141.
"expiration" : xxx, (numeric) The orphan expiration time expressed in UNIX epoch time
"from" : [ (json array)
n, (numeric) Peer ID
...
]
},
...
]
Result (for verbose = 2):
[ (json array)
{ (json object)
"txid" : "hex", (string) The transaction hash in hex
"wtxid" : "hex", (string) The transaction witness hash in hex
"bytes" : n, (numeric) The serialized transaction size in bytes
"vsize" : n, (numeric) The virtual transaction size as defined in BIP 141. This is different from actual serialized size for witness transactions as witness data is discounted.
"weight" : n, (numeric) The transaction weight as defined in BIP 141.
"expiration" : xxx, (numeric) The orphan expiration time expressed in UNIX epoch time
"from" : [ (json array)
n, (numeric) Peer ID
...
],
"hex" : "hex" (string) The serialized, hex-encoded transaction data
},
...
]
Examples:
> bitcoin-cli getorphantxs 2
> curl --user myusername --data-binary '{"jsonrpc": "2.0", "id": "curltest", "method": "getorphantxs", "params": [2]}' -H 'content-type: application/json' http://127.0.0.1:8332/
```
```
$ build/src/bitcoin-cli getorphantxs 2
[
{
"txid": "50128aac5deab548228d74d846675ad4def91cd92453d81a2daa778df12a63f2",
"wtxid": "bb61659336f59fcf23acb47c05dc4bbea63ab533a98c412f3a12cb813308d52c",
"bytes": 133,
"vsize": 104,
"weight": 415,
"expiration": 1725663854,
"from": [
1
],
"hex": "020000000001010b992959eaa2018bbf31a4a3f9aa30896a8144dbd5cfaf263bf07c0845a3a6620000000000000000000140fe042a010000002251202913b252fe537830f843bfdc5fa7d20ba48639a87c86ff837b92d083c55ad7c102015121c0000000000000000000000000000000000000000000000000000000000000000100000000"
},
{
"txid": "330bb7f701604a40ade20aa129e9a3eb8a7bf024e599084ca1026d3222b9f8a1",
"wtxid": "b7651f7d4c1a40c4d01f6a1e43a121967091fa0f56bb460146c1c5c068e824f6",
"bytes": 133,
"vsize": 104,
"weight": 415,
"expiration": 1725663854,
"from": [
2
],
"hex": "020000000001013600adfe41e0ebd2454838963d270916d2b47239c9eebb93a992b720d3589a080000000000000000000140fe042a010000002251202913b252fe537830f843bfdc5fa7d20ba48639a87c86ff837b92d083c55ad7c102015121c0000000000000000000000000000000000000000000000000000000000000000100000000"
}
]
```
ACKs for top commit:
glozow:
reACK 98c1536852
hodlinator:
re-ACK 98c1536852
danielabrozzoni:
ACK 98c1536852
pablomartin4btc:
tACK 98c1536852
itornaza:
reACK 98c1536852
Tree-SHA512: 66075f9faa83748350b87397302100d08af92cbef5fadb27f2b4903f028c08020bf34a23e17262b41abb3f379ca9f46cf6cd5459b8681f2b83bffbbaf3c03ff9
Since the `WalletModel` class is unavailable when compiling
without wallet support `(-DENABLE_WALLET=0)`, the RPC executor
class should not be coupled to it. This decoupling ensures GUI
compatibility with builds that omit wallet support.
5be34bacf6 qt: Fix linking when configured with `-DENABLE_WALLET=OFF` (Hennadii Stepanov)
Pull request description:
Split from https://github.com/bitcoin/bitcoin/pull/30997.
When building with Qt 6 in my dev branch, I encountered a linker error when configured with `-DENABLE_WALLET=OFF`:
```
$ cmake -B build -DENABLE_WALLET=OFF -DBUILD_GUI=ON
$ cmake --build build -t bitcoin-qt
<snip>
[100%] Linking CXX executable bitcoin-qt
/usr/bin/ld: libbitcoinqt.a(rpcconsole.cpp.o): in function `QtPrivate::MetaObjectForType<WalletModel const*, void>::metaObjectFunction(QtPrivate::QMetaTypeInterface const*)':
/usr/include/x86_64-linux-gnu/qt6/QtCore/qmetatype.h:903:(.text._ZN9QtPrivate17MetaObjectForTypeIPK11WalletModelvE18metaObjectFunctionEPKNS_18QMetaTypeInterfaceE[_ZN9QtPrivate17MetaObjectForTypeIPK11WalletModelvE18metaObjectFunctionEPKNS_18QMetaTypeInterfaceE]+0x2b): undefined reference to `WalletModel::staticMetaObject'
/usr/bin/ld: libbitcoinqt.a(rpcconsole.cpp.o): in function `QMetaTypeIdQObject<WalletModel const*, 8>::qt_metatype_id()':
/usr/include/x86_64-linux-gnu/qt6/QtCore/qmetatype.h:1313:(.text._ZZN9QtPrivate16QMetaTypeForTypeIPK11WalletModelE17getLegacyRegisterEvENUlvE_4_FUNEv[_ZZN9QtPrivate16QMetaTypeForTypeIPK11WalletModelE17getLegacyRegisterEvENUlvE_4_FUNEv]+0x53): undefined reference to `WalletModel::staticMetaObject'
collect2: error: ld returned 1 exit status
gmake[3]: *** [src/qt/CMakeFiles/bitcoin-qt.dir/build.make:154: src/qt/bitcoin-qt] Error 1
gmake[2]: *** [CMakeFiles/Makefile2:2107: src/qt/CMakeFiles/bitcoin-qt.dir/all] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:2114: src/qt/CMakeFiles/bitcoin-qt.dir/rule] Error 2
gmake: *** [Makefile:998: bitcoin-qt] Error 2
```
This PR resolves the issue.
ACKs for top commit:
promag:
ACK 5be34bacf6, all other changes in 33657e1c958146312e4c68765a92871920401396 are not required, makes the code harder to read.
pablomartin4btc:
tACK 5be34bacf6
Tree-SHA512: d10da665384e6539b8e9106dc905ba30e9e57cd4603fc7e5eb893fc899f55f26889c570687fa5daf55c6fc5bf4fcfcfcd9b70822cfe637f31f9151bb653b7941
5625840c11 qt6, test: Handle deprecated `QVERIFY_EXCEPTION_THROWN` (Hennadii Stepanov)
cb750b4b40 qt6, test: Use `qWarning()` instead of `QWARN()` macro (Hennadii Stepanov)
Pull request description:
Split from https://github.com/bitcoin/bitcoin/pull/30997.
This PR ensures compatibility across all supported Qt versions.
---
This PR can be tested on macOS using the first commit from https://github.com/bitcoin/bitcoin/pull/30997 and Homebrew's `qt` package.
ACKs for top commit:
promag:
Code review ACK 5625840c11.
Sjors:
tACK 5625840c11
Tree-SHA512: e7307eaf0027c6addc9481ba91ed31b81554ffb0d2ba77938e68915c9d490a7962e55a330f97ea31d49bbfb30f92773c3af3afc867a4215d00752405d7e3bb6d
27709f51ee docs: Add instructions on how to self-sign bitcoin-core binaries for macOS (Chris Stewart)
Pull request description:
Related to #15774
This PR adds instructions to the release notes to tell users how to self sign bitcoin core binaries so they are executable on macOS.
Tested on
```
Darwin Chriss-MacBook-Pro.local 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:46 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6031 arm64
```
These commands do not appear to require 'phoning home'. I tested these commands when disconnected from a network connection and things worked.
ACKs for top commit:
andrewtoth:
reACK 27709f51ee
achow101:
ACK 27709f51ee
Tree-SHA512: db19c61577bb774420a2506d3f06bc0193116117f09ebd2d022a4524e8ca32d2cf9277a2997744ddfe8844600a569176e194aafc252dd31b48fc6e74db3c74d0
a7498cc7e2 Fix bug in p2p_headers_presync harness (marcofleon)
Pull request description:
The calculation for the test chain's work (`total_work`) should be outside of the loop. Previously, `total_work` was being miscalculated due to multiple additions of work from the same headers. Now, each header's work is only counted once, providing an accurate total.
https://github.com/bitcoin/bitcoin/pull/30918 followup
ACKs for top commit:
dergoegge:
utACK a7498cc7e2
instagibbs:
ACK a7498cc7e2
glozow:
makes sense, utACK a7498cc7e2
mzumsande:
ACK a7498cc7e2
Tree-SHA512: b95f25dcf7ace220e30f1d72f50d85ee18777467927c0cc1ed8582b390cb7185ffc0e2f127309eb083044fb41f5a13fce5ebb15b7952718a899bafff26921be8
fa2b7d8d6b Remove redundant unterminated-logprintf tidy check (MarcoFalke)
bbbb2e43ee log: Enforce trailing newline, Remove redundant m_started_new_line (MarcoFalke)
Pull request description:
There are many problems around missing a trailing newline while logging:
* All log lines are currently terminated by a trailing newline. This means any runtime code trying to handle a "missing" newline is currently dead code.
* Leaving a line unterminated is racy and can cause content corruption by mixing log lines from different sources.
* It requires extra code like `m_started_new_line` to keep track of, which is annoying and pointless to maintain, because it is currently dead code, see https://github.com/bitcoin/bitcoin/pull/30386#discussion_r1684380835.
* It requires a standalone `unterminated-logprintf` clang-tidy plugin, which is unmaintained (no one updated it for the new log function names), probably harder to maintain than normal C++ code (because it requires clang AST matcher knowledge), brittle (it can fail to detect issues at any time, if it goes out-of-sync, or be explicitly disabled via `NOLINT`), and annoying for devs (it is slow and intricate to run locally and thus only effectively run on CI or via the CI scripts).
Fix all issues by enforcing the trailing newline in logs directly in the code. Then remove all the other stuff.
This refactor does not change behavior.
ACKs for top commit:
stickies-v:
re-ACK fa2b7d8d6b
achow101:
ACK fa2b7d8d6b
ryanofsky:
Code review ACK fa2b7d8d6b. Just comment and test cleanup since last review
Tree-SHA512: 10ed420f6c2fdb0f491d6c880be8dd2e8beef628f510adebadf4c3849d9f5e28906519d5cbaeb295f4c7c1b07c4c88a9905b3cfe30fee3a2c91ac9fd24ae6755
fd38711217 ci: make CI job fail when check-deps.sh script fails (Ryan Ofsky)
d51edecddc common: move pcp.cpp and netif.cpp files from util to common library since they depend on netaddress.cpp (Ryan Ofsky)
Pull request description:
Move util/pcp.cpp and util/netif.cpp to common/ because they depend on netaddress.cpp which is part of the common library. This was causing check-deps.sh script to fail as reported by _fanquake_ in https://github.com/bitcoin/bitcoin/pull/30415#issuecomment-2385475097.
Also make CI fail when the `check-deps.sh` script fails. Previously it would output errors but not cause the job to fail (which was not intended).
ACKs for top commit:
Sjors:
utACK fd38711217
laanwj:
Untested ACK fd38711217
achow101:
ACK fd38711217
tdb3:
ACK fd38711217
Tree-SHA512: 06316e68617ded7d96d540c9934b08cf9fbba5ff5e7f54d7a0c0a9087a26bf8adc97e9e8c39a2bfd3da34e27f3652b1531ec6136a2c69393ae0b26585abadb6b
a1576edab3 test: add missing sync to feature_fee_estimation.py (Martin Zumsande)
Pull request description:
This fixes a race:
- In the `test_estimate_dat_is_flushed_periodically` subtest, node 0 is isolated and creates 10 blocks (no sync).
- In `clear_estimates` the nodes are reconnected (but we don't wait for them to sync!)
- In the `sanity_check_rbf_estimates` subtest, node 1 generates another block and syncs with the other nodes. The sync fails if the generated block is at the same height as the tip of node 0.
Fix this by adding a sync to `clear_estimates`.
Fixes#30990Fixes#30640
ACKs for top commit:
maflcko:
lgtm ACK a1576edab3
tdb3:
code review ACK a1576edab3
Tree-SHA512: 608ba619cacb4ff3a1ea934e03286f18c96afeebf06439334d40bff72025bd7bcc2c1093dae1824b30a37d3ac3ea569bc3118c33c0ca51610592aa1b4f420840
CMake parses some paths from the spec of the C compiler, assuming it
will be the linker, resulting in the link to end up with
`-L/usr/lib/gcc/x86_64-w64-mingw32/12-win32` on debian bookworm if both
-win32 and -posix variants are installed, and -win32 is the default
alternative.
This results in the wrong C++ library being linked, missing
std::threads::hardware_concurrency and other threading functions.
To fix this, use the -posix variant of gcc as well when available. This
fixes a regression compared to autotools, where this scenario worked.
Previously the check-deps.sh would write information about unexpected
dependencies to stderr, but return exit code 0, so the error would be ignored
by CI. Now it will return code 1 and cause CI to fail if unexpected
dependencies are detected.
Instead of having a single NodeContext::shutdown member that is used both to
request shutdowns and check if they have been requested, use separate members
for each. Benefits of this change:
1. Should make code a little clearer and easier to search because it is easier
to see which parts of code are triggering shutdowns and which parts are just
checking to see if they were triggered.
2. Makes it possible for init.cpp to specify additional code to run when a
shutdown is requested, like signalling the m_tip_block_cv condition variable.
Motivation for this change was to remove hacky NodeContext argument and
m_tip_block_cv access from the StopRPC function, so StopRPC can just be
concerned with RPC functionality, not other node functionality.
This is not strictly required, but all places using m_tip_block_cv
(except shutdown) already take the lock. The annotation makes it easier
to catch potential deadlocks before review.
Adding the missing lock to the shutdown sequence is a bugfix.
An alternative would be to take the lock and release it before
notifying, see
https://github.com/bitcoin/bitcoin/pull/30967#discussion_r1778899716
It should be obvious that a wait is not needed if the tip does not
match.
Also, remove a comment that the blockTip notification was only meant for
the "UI". (It is used by other stuff for a long time)
940edd6ac2 test: refactor: introduce and use `TRUC_CHILD_MAX_VSIZE` constant (Sebastian Falbesoner)
c16ae71768 test: switch MiniWallet padding unit from weight to vsize (Sebastian Falbesoner)
Pull request description:
This PR is a late follow-up for #30162, where I retrospectively consider the padding unit of choice as a mistake. The weight unit is merely a consensus rule detail and is largely irrelevant from a user's perspective w.r.t. fee-rate calculations and mempool policy rules (e.g. for package relay and TRUC limits), so there doesn't seem to be any value of using a granularity that we can't even guarantee to reach exactly anyway.
Switch to the more natural unit of vsize instead, which simplifies both the padding implementation (no "round up to the next multiple of 4" anymore) and the current tests that take use of this padding. The rather annoying multiplications by `WITNESS_SCALE_FACTOR` can then be removed and weird-looking magic numbers like `4004` can be replaced by numbers that are more connected to actual policy limit constants from the codebase, e.g. `1001` for exceeding `TRUC_CHILD_MAX_VSIZE` by one. The second commits introduces a constant for that.
ACKs for top commit:
glozow:
reACK 940edd6 via range-diff
instagibbs:
reACK 940edd6ac2
maflcko:
re-ACK 940edd6ac2🍷
achow101:
ACK 940edd6ac2
Tree-SHA512: 35325f22bbe548664273051b705059b8f2f4316215be116c71b8c21dc87d190b3e8fcc4a48f04deaba2f3632a9c809d272b0bae654cf74d7492759554c0f0d14
e9d60af988 refactor: Replace init retry for loop with if statement (TheCharlatan)
c1d8870ea4 refactor: Move most of init retry for loop to a function (TheCharlatan)
781c01f580 init: Check mempool arguments in AppInitParameterInteractions (TheCharlatan)
Pull request description:
The for loop around the chain loading logic in `init.cpp` allows users of the GUI to retry once on failure with reindexing without having to manually set the reindex flag on startup. However this current mechanism has problems:
* It is badly documented and has led to confusion among developers and bugs making it into master. Examples:
* https://github.com/bitcoin/bitcoin/pull/28830/files#r1598392660
* https://github.com/bitcoin/bitcoin/pull/30132#issuecomment-2120741121
* It can only ever iterate once, making the choice of a for loop questionable.
* With its large scope it is easy for re-entry bugs to sneak in. Example:
* https://github.com/bitcoin/bitcoin/pull/28830#discussion_r1601589963
Attempt to fix this by moving the bulk of the logic into a separate function and replacing the for loop with a simpler `if` statement.
The diff's in this pull request are best reviewed with `--color-moved-ws=ignore-all-space --color-moved=dimmed-zebra`. The error behaviour can be tested by either manually making `LoadChainstate` return a failure, or deleting some of the block index database files.
ACKs for top commit:
maflcko:
review ACK e9d60af988🚸
josibake:
crACK e9d60af988
achow101:
ACK e9d60af988
ryanofsky:
Code review ACK e9d60af988. Nice change to make AppInitMain shorter and more understandable.
Tree-SHA512: 5e5c0a5fd1b32225346450f8482f0ae8792e1557cdab1518112c1a3ec3a4400b64f5796692245cc5bf2f9010bb97b3a9558f07626a285ccd6ae525dd671ead13
5c7cacf649 ci: Remove natpmp build option and libnatpmp dependency (laanwj)
7e7ec984da doc: Remove mention of natpmp build options (laanwj)
061c3e32a2 depends: Drop natpmp and associated option from depends (laanwj)
20a18bf6aa build: Drop libnatpmp from build system (laanwj)
7b04709862 qt: Changes for built-in PCP+NAT-PMP (laanwj)
52f8ef66c6 net: Replace libnatpmp with built-in NATPMP+PCP implementation in mapport (laanwj)
97c97177cd net: Add PCP and NATPMP implementation (laanwj)
d72df63d16 net: Use GetLocalAddresses in Discover (laanwj)
e02030432b net: Add netif utility (laanwj)
754e425438 crypto: Add missing WriteBE16 function (laanwj)
Pull request description:
Continues #30005. Closes #17012..
This PR adds PCP (Port Control Protocol) from [RFC6887](https://datatracker.ietf.org/doc/html/rfc6887). This adds, in addition to the existing IPv4 port mapping (which now uses PCP, with fallback to NAT-PMP), support for IPv6 pinholing-that is, opening a port on the firewall to make it reachable.
PCP, like NAT-PMP is a simple UDP-based protocol, and the implementation is self-contained, so this gets rid of lthe libnatpnp dependency without adding a new one. It should otherwise be a drop-in replacement. NAT-PMP fallback is implemented so this will not make router support worse.
For now it is disabled by default, though in the future (not in this PR) we could consider enable it by default to increase the number of connectable nodes without adding significant attack surface.
To test:
```bash
bitcoind -regtest -natpmp=1 -debug=net
```
(most of the changes in this PR are, ironically, removing the libnatpmp dependency and associated build system and build docs)
## TODO
- [x] Default gateway discovery on Linux / FreeBSD
- [x] Default gateway discovery on Windows
- [x] Default gateway discovery on MacOS
- [x] Either solve FreeBSD compile issue (probably upstream issue) or remove FreeBSD support
## Things to consider for follow-up PRs
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1658764974 avoid unreachable nets (not given to -onlynet=)
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1658949236 could announce an addr:port where we do not listen (no -bind)
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1684368824 could announce the wrong port because it uses GetListenPort()
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1679709347 if we requested one port but another was assigned, then which one to use in the renewal?
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1772017020 Use `GetAdapterAddresses` to discover local addresses for Windows
ACKs for top commit:
Sjors:
ACK 5c7cacf649
achow101:
ACK 5c7cacf649
vasild:
ACK 5c7cacf649
Tree-SHA512: e35b69e56d5f5449a3d48a799f9b7b65107c65eeb3e245c2c1e9d42221e469ca5ead90afae423160601cd664dd553a51c859e04f4492f335b064aae3bf23e3bc
f1daa80521 guix: Drop no longer needed `PATH` modification (Hennadii Stepanov)
Pull request description:
I don't see any reason why this would be necessary in the master branch @ d812cf1189.
Additionally, from https://github.com/bitcoin/bitcoin/pull/30940#pullrequestreview-2322355196:
> 2. I don't understand why "In the Guix environment, `${BASEPREFIX}/${HOST}/native/bin` is added to the `PATH` environment variable," according to the description. Setting this seems indiscriminate, like a sledgehammer approach, something that would cause the guix build to behave differently from normal depends builds and lead to confusing issues like this one.
My Guix build:
```
aarch64
015b853d60c742120b88f1501ce241c8b7b3e874eca9ab150ba2ec282ecb9572 guix-build-f1daa80521ec/output/aarch64-linux-gnu/SHA256SUMS.part
2a8ed51f02046a73dc9a391b8939528c2e506d545274c934202a5643f26b102b guix-build-f1daa80521ec/output/aarch64-linux-gnu/bitcoin-f1daa80521ec-aarch64-linux-gnu-debug.tar.gz
0ce7a6c81b657cfcbd2edf1e18cca8f66bd7bbe15a12b90dd60ddb1218b72254 guix-build-f1daa80521ec/output/aarch64-linux-gnu/bitcoin-f1daa80521ec-aarch64-linux-gnu.tar.gz
de6cb71e37a1c2e9a9a9952d4456a7fde407b38f95a1447928ded3f592b2e47f guix-build-f1daa80521ec/output/arm-linux-gnueabihf/SHA256SUMS.part
c91be594ad4d02a2cb4cea2f57e91ebeae9a1cda66ec49e05ecc3a793e767f24 guix-build-f1daa80521ec/output/arm-linux-gnueabihf/bitcoin-f1daa80521ec-arm-linux-gnueabihf-debug.tar.gz
eb8ea448df1734009129d88cdf28a1ae5918bff19a58fa9525c0b1dde0dfd987 guix-build-f1daa80521ec/output/arm-linux-gnueabihf/bitcoin-f1daa80521ec-arm-linux-gnueabihf.tar.gz
6d558c036b66c81fb5843b1918f24fec6bd901098a0dfb15100b497e12e8fdc3 guix-build-f1daa80521ec/output/arm64-apple-darwin/SHA256SUMS.part
23691ecaf5d23c72f06fe81054a84e2549d8e89582317b6d3e14276aeba0b07f guix-build-f1daa80521ec/output/arm64-apple-darwin/bitcoin-f1daa80521ec-arm64-apple-darwin-unsigned.tar.gz
8965a32937894d6dd75e6b04809bdc925187967c2547a795dec2e11a75262624 guix-build-f1daa80521ec/output/arm64-apple-darwin/bitcoin-f1daa80521ec-arm64-apple-darwin-unsigned.zip
ec0b2f35f498537ca6eb8b306a1e26cf97b7f1bdf140f3c4ca8b18c643fc4599 guix-build-f1daa80521ec/output/arm64-apple-darwin/bitcoin-f1daa80521ec-arm64-apple-darwin.tar.gz
d46d8117efdbfe90be13bcf36ba2ddcfa7c53ba01762a53c72a1b48f2cac569c guix-build-f1daa80521ec/output/dist-archive/bitcoin-f1daa80521ec.tar.gz
facf7bbec0e9324e9ed58b8da07c5b1df2f120bd9090f7d124613ed62092dd46 guix-build-f1daa80521ec/output/powerpc64-linux-gnu/SHA256SUMS.part
c065b222f60ec19b7585daf197dadcb529fa588de1b26e767e4ebd43d6345562 guix-build-f1daa80521ec/output/powerpc64-linux-gnu/bitcoin-f1daa80521ec-powerpc64-linux-gnu-debug.tar.gz
4e837a86ce6adbd595dc31d2b584c3322acd30b6f18b57144f18fc09289fec65 guix-build-f1daa80521ec/output/powerpc64-linux-gnu/bitcoin-f1daa80521ec-powerpc64-linux-gnu.tar.gz
f4362984a846e97c6a388366bb2922294c66bb3f78adf71064f97ab5a346e4ed guix-build-f1daa80521ec/output/riscv64-linux-gnu/SHA256SUMS.part
427fc7fdac244c6dd4fdf0312486e3bcf8372c68fd3570bdb815734544b8369e guix-build-f1daa80521ec/output/riscv64-linux-gnu/bitcoin-f1daa80521ec-riscv64-linux-gnu-debug.tar.gz
ae9a07f7e2e656efbba99246be5767798028c13fcf5d172a595b734f5e1241c4 guix-build-f1daa80521ec/output/riscv64-linux-gnu/bitcoin-f1daa80521ec-riscv64-linux-gnu.tar.gz
8ff7494e648fe5744efd4522a003d94b531dcab28cb8c2fea05a09897be111ce guix-build-f1daa80521ec/output/x86_64-apple-darwin/SHA256SUMS.part
9845e894fc6b0dd339dc4f62f3bc4e37f76935f309887798ca488fb5465b2b6c guix-build-f1daa80521ec/output/x86_64-apple-darwin/bitcoin-f1daa80521ec-x86_64-apple-darwin-unsigned.tar.gz
fa0e07573ae977ef6bb3ecaa07b1e434c52041865e2def9de6a041fb3749d27d guix-build-f1daa80521ec/output/x86_64-apple-darwin/bitcoin-f1daa80521ec-x86_64-apple-darwin-unsigned.zip
cd99cda53a8fbcc5380333058426055977cd39d3bdc0da571b3f64d293787719 guix-build-f1daa80521ec/output/x86_64-apple-darwin/bitcoin-f1daa80521ec-x86_64-apple-darwin.tar.gz
a6beac93eb8f9516a13ab7451b0c45b2898fd56315a066cc6470ba84226bba27 guix-build-f1daa80521ec/output/x86_64-linux-gnu/SHA256SUMS.part
f50e03971274371ef0ec5710de4879670f75cb29a8eacd5c02f0d622740d026a guix-build-f1daa80521ec/output/x86_64-linux-gnu/bitcoin-f1daa80521ec-x86_64-linux-gnu-debug.tar.gz
97dcd833014cccaac1b228f438ac49aec94603f0317c606e9344d9709302dbbd guix-build-f1daa80521ec/output/x86_64-linux-gnu/bitcoin-f1daa80521ec-x86_64-linux-gnu.tar.gz
7c2ea5572f9f137523b88f6a0f1ac711abd6a7ef8aa361ceea35d01e700a3778 guix-build-f1daa80521ec/output/x86_64-w64-mingw32/SHA256SUMS.part
c64d33e04dfc8adfe5a48d6ed17579a69e0b8938e2973bd1810bcaefe5dc9506 guix-build-f1daa80521ec/output/x86_64-w64-mingw32/bitcoin-f1daa80521ec-win64-debug.zip
87d81e11510ffef0082e3be80ce3f8f5e7d9f5c3cdb1dae887d4341cf678af31 guix-build-f1daa80521ec/output/x86_64-w64-mingw32/bitcoin-f1daa80521ec-win64-setup-unsigned.exe
d46887ef5d23fe19ce23dd356dc3a1c03a1164778f78466b4ef415038b42e3eb guix-build-f1daa80521ec/output/x86_64-w64-mingw32/bitcoin-f1daa80521ec-win64-unsigned.tar.gz
c1a54433d0849548734e8962590e3a33b529665cd610f2ed5acbb1a52c02ae23 guix-build-f1daa80521ec/output/x86_64-w64-mingw32/bitcoin-f1daa80521ec-win64.zip
```
ACKs for top commit:
theuni:
utACK f1daa80521 since guix is happy.
fanquake:
ACK f1daa80521
Tree-SHA512: 50fd8fb01727a462e3935ad840de465acee9520eb5e9cfd972476960e6f738a8fd7e9cb62f27cdad643d013e5b487df1671c45f46af2476aaeeec21cfa60e6c1
The `QWARN()` macro internally uses `QTest::qWarn()`, which has been
deprecated since Qt 6.3. Replacing it with `qWarning()` ensures
compatibility across all Qt versions.
The weight unit is merely a consensus rule detail and is largely
irrelevant for fee-rate calculations and mempool policy rules (e.g. for
package relay and TRUC limits), so there doesn't seem to be any value of
using a granularity that we can't even guarantee to reach exactly
anyway.
Switch to the more natural unit of vsize instead, which simplifies both
the padding implementation and the current tests that take use of this
padding. The rather annoying multiplications by `WITNESS_SCALE_FACTOR`
can then be removed and weird-looking magic numbers like `4004` can be
replaced by numbers that are more connected to actual policy limit
constants from the codebase, e.g. `1001` for exceeding
`TRUC_CHILD_MAX_VSIZE` by one.
f5a2000579 test: re-bucket long-running tests (willcl-ark)
Pull request description:
Re-bucket:
- `p2p_node_network_limited -v*transport`
- `feature_assume_utxo`
On CI runners these tests are taking longer than their current bucket suggests, often being among the last to finish.
Re-bucket them to improve CI efficiency.
ACKs for top commit:
maflcko:
review ACK f5a2000579
Tree-SHA512: 3da5c888db64a311276338270ba1dcad3eb2a24e205f6bb86fc92f767ecfa63682f13fafffff569fa0cfaea607ccb538f31e3934a086d482c3fe1be5d39f8791
6c3c619b35 test: generalize HasReason and use it in FailFmtWithError (Lőrinc)
Pull request description:
Standardized boost exception checking in recent tests introduced in https://github.com/bitcoin/bitcoin/pull/30546#discussion_r1756493521 by extending `HasReason` to accept `const char*` through `string_view` in `operator()`.
Note that `HasReason` only checks partial matches - but since we're specifying the whole error string, it doesn't affect us in this case.
ACKs for top commit:
maflcko:
review ACK 6c3c619b35
hodlinator:
ACK 6c3c619b35
Tree-SHA512: 740fb18b8fea78e4eb9740ceb0fe75d37246c28cfa2638b9d093e9514dd6d7926cc5be9ec57f8027cca3aa9d616e8c54322d2401cfa67fd25282f7816e63532d
fafd1a0f64 ci: Inline PACKAGE_MANAGER_INSTALL (MarcoFalke)
Pull request description:
The fallback `bash -c "$PACKAGE_MANAGER_INSTALL git"` is only needed by the `lint` task, so simplify it and inline `PACKAGE_MANAGER_INSTALL` once. Also, fixup the docs to add some other packages which are needed by podman in user-mode.
ACKs for top commit:
fanquake:
ACK fafd1a0f64
Tree-SHA512: e1665446d0fb5a2e8f2cb912117c7a42838c840199e7504a859b9155a13a2ff2e8606ac56689446f973fce02b00240041f071ebf00520778ed81eb1a01de6663
f158993fd5 contrib: Update asmap link in seeds readme (Fabian Jahr)
Pull request description:
I am moving all my ASMap related repositories to an asmap org. While there is a redirect in place that works for now, GitHub doesn't guarantee that it will keep working in the long term. So we should still fix the links.
ACKs for top commit:
laanwj:
ACK f158993fd5
jarolrod:
ACK f158993fd5
brunoerg:
ACK f158993fd5
Tree-SHA512: c6f831b6ed1b520a38623a7d29c01677a45bf06fea77f4da6b421dd16e630836c9abf8191330e3887232fd688edca534067142d37d7e504a1a785ae451b11721
f951f1fab2 ci: add timestamps to cirrus jobs (willcl-ark)
Pull request description:
Currently, debugging where time is spent in the cirrus jobs feels annoying, e.g. trying to see where time may be spent in https://github.com/bitcoin/bitcoin/issues/30969
Enable timestamps in the logs for more information.
ACKs for top commit:
maflcko:
review ACK f951f1fab2
tdb3:
Code Review ACK f951f1fab2
hebasto:
ACK f951f1fab2.
Tree-SHA512: 4ca6c3722013323e794339c39af3041a519e3ecf75216093c4ca99668c0932fe41cc547819f742a3e8a47709ec6baec976c542eb3848c0248f439841110a4e85
faf801515f test: Add missing sync_mempools() to fill_mempool() (MarcoFalke)
fa48be6f02 test: Refactor fill_mempool to extract send_batch helper (MarcoFalke)
Pull request description:
Not doing the sync will lead to (intermittent) issues, as explained in https://github.com/bitcoin/bitcoin/issues/30922#issuecomment-2364529013.
Fix all issues by doing the sync by default and disable it in places that do not need the sync.
Fixes#30922
ACKs for top commit:
mzumsande:
Tested ACK faf801515f
ismaelsadeeq:
Tested ACK faf801515f
marcofleon:
Tested ACK faf801515f
Tree-SHA512: 2de62d168cbb6857a9fb8bc12c42a9093fedf5e9beb6f83a32b3fa72a5ba3cf03631055fd25ef553399a27a6fe0d71c44cfe60660a4d31986debd13b8ab00228
4cf84b344d depends, doc: No need to specify general requirement (Hennadii Stepanov)
Pull request description:
CMake is no longer required solely for `libmultiprocess`.
ACKs for top commit:
maflcko:
review ACK 4cf84b344d
Tree-SHA512: 014808ef523d6f0101aaa52a1858bebc16da9eb44664beb204e271b033c4dfa48115ce85c4ce5f151e82f90f83c64aaa25af73ef5a9b9b64953d2e07ff19633f
The current order is incorrect:
```bash
./build/src/bitcoin-cli loadtxoutset -rpcclienttimeout=0 utxo-840000.dat
error code: -1
error message:
loadtxoutset "path"
```
The Mining interface uses this function in the next commit
to calculate the coinbase merkle path. Stratum v2 uses
this to send a compact work template.
This partially undoes the change in 4defdfab94,
but is not a revert, because the implementation changed in the meantime.
This commit also documents the function.
1a33281766 doc: multiprocess documentation improvements (Ryan Ofsky)
d043950ba2 multiprocess: Add serialization code for BlockValidationState (Ryan Ofsky)
33c2eee285 multiprocess: Add IPC wrapper for Mining interface (Ryan Ofsky)
06882f8401 multiprocess: Add serialization code for vector<char> (Russell Yanofsky)
095286f790 multiprocess: Add serialization code for CTransaction (Russell Yanofsky)
69dfeb1876 multiprocess: update common-types.h to use C++20 concepts (Ryan Ofsky)
206c6e78ee build: Make bitcoin_ipc_test depend on bitcoin_ipc (Ryan Ofsky)
070e6a32d5 depends: Update libmultiprocess library for cmake headers target (Ryan Ofsky)
Pull request description:
Add Cap'n Proto wrapper for the Mining interface introduced in #30200, and its associated types.
This PR combined with #30509 will allow a separate mining process, like the one being implemented in https://github.com/Sjors/bitcoin/pull/48, to connect to the node over IPC, and create, manage, and submit block templates. (#30437 shows another simpler demo of a process using the Mining interface.)
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
achow101:
ACK 1a33281766
TheCharlatan:
ACK 1a33281766
itornaza:
ACK 1a33281766
Tree-SHA512: 0791078dd6885dbd81e3d14c75fffff3da8d1277873af379ea6f9633e910c11485bb324e4cde3d936d50d343b16a10b0e8fc1e0fc6d7bdca7f522211da50c01e
c0a0c72b4d validation: Disable CheckForkWarningConditions for background chainstate (Martin Zumsande)
Pull request description:
The comparison of `m_best_invalid` with the tip of the chainstate makes no sense for the background chainstate and can lead to incorrect log messages.
Fixes#30958
ACKs for top commit:
fjahr:
utACK c0a0c72b4d
achow101:
ACK c0a0c72b4d
TheCharlatan:
ACK c0a0c72b4d
tdb3:
CR ACK c0a0c72b4d
Tree-SHA512: 316811b1ade8521681f3a105c7db59dfdb27b1e13efeba95cd6183d750db2e07f48968e8fc54fe3834d3311519cf8c6b77d7e0fd61facbf2b07598e79110ff85
The for loop has been a long standing source of confusion and bugs, both
because its purpose is not clearly documented and because the body of
the for loop contains a lot of logic.
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
This makes it clearer which state is being mutated by the function and
facilitates getting rid of the for loop in the following commit. Move
creation of the required options into the function too, such that the
function takes fewer arguments and is more self-contained.
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
06e7e83632 doc: correct the zmq automatic build info (tdb3)
Pull request description:
The current documentation states that the ZMQ feature is automatically included when building.
Noticed this was no longer the case when building to review PR #30942.
ACKs for top commit:
l0rinc:
ACK 06e7e83632
hebasto:
re-ACK 06e7e83632.
Tree-SHA512: 24269c29415e54815b105e3f830f31f055e9bdcb3bf4a7cede068651875190adfc4e72ad5448b3c30a23b0db024f743d9f18576a6fda46d87deb5ea39352e626
fa6c1946d2 doc: Adjust links in OSS-Fuzz section (MarcoFalke)
Pull request description:
Adjust the links after the google issue tracker migration and replace the remaining paragraph with a link to https://bitcoincore.org/en/security-advisories/
ACKs for top commit:
dergoegge:
ACK fa6c1946d2
Tree-SHA512: 4c3aba8bb5641756be5c66c2ee21b927a675622ddac021303f8196a1b2d8eb891406f6dedab0f04e12312339953c87302891df4ad5f5956c0b2ddd453452e1c9
The comparison of m_best_invalid with the tip of the respective chainstate
makes no sense for the background chainstate, and can lead to incorrect
error messages.
7bd3ee62f6 test: Use shell builtins in run_command test case (Ava Chow)
Pull request description:
Uses the [suggested command](https://github.com/bitcoin/bitcoin/issues/30938#issuecomment-2363906135)
Fixes#30938
ACKs for top commit:
maflcko:
review ACK 7bd3ee62f6
hebasto:
ACK 7bd3ee62f6.
Tree-SHA512: 683b15cafaf0103eeadf872ea6ce9a7d884b2605d3dcf4e66b0173cdb149c24965e7c5fa62aaddf2ac55df3f449aeb787176992c96cfee5d0b86621259e1dfe9
f20fe33e94 test: Add basic balance coverage to wallet_assumeutxo.py (Fabian Jahr)
037b101e80 test: Add coverage for best block locator write in wallet_backup (Fabian Jahr)
31c0df0389 wallet: migration, write best locator before unloading wallet (furszy)
7e3dbe4180 wallet: Write best block to disk before backup (Fabian Jahr)
Pull request description:
I discovered that we don't write the best block to disk when trying to explain the behavior described here: https://github.com/bitcoin/bitcoin/pull/30455#discussion_r1719951882
In the context of that test, the behavior is confusing and I think it also shows that one of the already existing tests in `wallet_assumeutxo.py` doesn't actually test what it says. It only fails because the best block isn't written and actually, the height of the backup that is loaded is at the snapshot height during backup. So it really shouldn't fail since it's past the background sync blocks already.
I'm not sure if this is super relevant in practice though so I am first looking for concept ACKs on the `BackupWallet` code change. Either way, I think this behavior should be documented better if it is left as is and the test should be changed.
ACKs for top commit:
achow101:
ACK f20fe33e94
furszy:
ACK f20fe33
Tree-SHA512: bb384a940df5c942fffe2eb06314ade4fc5d9b924012bfef3b1c456c4182a30825d1e137d8ae561d93d3a8a2f4d1c1ffe568132d20fa7d04844f1e289ab4a28b
Add support for passing CTransaction and CTransactionRef types to IPC
functions.
These types can't be passed currently because IPC serialization code currently
only supports deserializing types that have an Unserialize() method, which
CTransaction does not, because it is supposed to represent immutable
transactions. Work around this by adding a CustomReadField overload that will
call CTransaction's deserialize_type constructor.
These types also can't be passed currently because serializing transactions
requires TransactionSerParams to be set. Fix this by setting TX_WITH_WITNESS as
default serialization parameters for IPC code.
7942951e3f Remove unused g_best_block (Ryan Ofsky)
e3a560ca68 rpc: use waitTipChanged for longpoll (Ryan Ofsky)
460687a09c Remove unused CRPCSignals (Sjors Provoost)
dca923150e Replace RPCNotifyBlockChange with waitTipChanged() (Sjors Provoost)
2a40ee1121 rpc: check for negative timeout arg in waitfor* (Sjors Provoost)
de7c855b3a rpc: recommend -rpcclienttimeout=0 for waitfor* (Sjors Provoost)
77ec072925 rpc: fix waitfornewblock description (Sjors Provoost)
285fe9fb51 rpc: add test for waitforblock and waitfornewblock (Sjors Provoost)
b94b27cf05 Add waitTipChanged to Mining interface (Sjors Provoost)
7eccdaf160 node: Track last block that received a blockTip notification (Sjors Provoost)
ebb8215f23 Rename getTipHash() to getTip() and return BlockRef (Sjors Provoost)
89a8f74bbb refactor: rename BlockKey to BlockRef (Sjors Provoost)
Pull request description:
This continues the work in #30200 so that a future Stratum v2 Template Provider (see #29432) can avoid accessing node internals. It needs to know when a new block arrives in order to push new templates to connected clients.
`waitTipChanged()` uses a new kernel notification `notifications().m_tip_block_mutex`, which this PR also introduces (a previous version used `g_best_block`).
In order to ensure the new method works as intended, the `waitfornewblock`, `waitforblock` and `waitforblockheight` RPC methods are refactored to use it. This allows removing `RPCNotifyBlockChange`.
There's a commit to add (direct) tests for the methods that are about to be refactored:
- `waitfornewblock` was already implicitly tested by `feature_shutdown.py`.
- `waitforblockheight` by `feature_coinstatsindex.py` and `example_test.py`
This PR renames `getTipHash()` to `getTip()` and returns a `BlockRef` (renamed from `BlockKey`) so that callers can use either the height or hash.
The later commits make trivial improvements to the `waitfor*` RPC calls (not needed for this PR).
The `waitTipChanged()` method could probably also be used for the longpoll functionality in `getblocktemplate`, but I'm a bit reluctant to touch that.
`RPCServer::OnStarted` no longer does anything and `RPCServer::OnStopped` merely prints a log statement. They were added in #5711 as a refactor. This PR drops them entirely.
Finally `g_best_block` is also dropped.
ACKs for top commit:
achow101:
ACK 7942951e3f
ryanofsky:
Code review ACK 7942951e3f. Just rebased since last review
TheCharlatan:
Re-ACK 7942951e3f
Tree-SHA512: a5559446b4000c95e07aad33284b7ee2e57aafd87e1ae778b3825d59689566d047a8047e47a10f76e6e341e7dc72fd265a65afbc0a9c011d17c4cafd55031837
- p2p_node_network_limited -v*transport
- feature_assume_utxo
On CI runners these tests are taking longer than their current bucket
suggests, often being among the last to finish.
Re-bucket them to improve CI efficiency.
In the Guix environment, `${BASEPREFIX}/${HOST}/native/bin` is added to
the `PATH` environment variable, causing CMake to search for package
configurations in the `native` subdirectory first.
Explicitly specifying the top-priority search prefixes for the
`Libmultiprocess` and `LibmultiprocessNative` packages resolves the
issue.
This adds an utility header with two functions that will be needed for
PCP, `QueryDefaultGateway` and `GetLocalAddresses`.
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
5e190cd11f Replace CScript _hex_v_u8 appends with _hex (Lőrinc)
cac846c2fb Allow CScript's operator<< to accept spans, not just vectors (Lőrinc)
c78d8ff4cb prevector: avoid GCC bogus warnings in insert method (Lőrinc)
Pull request description:
Split out of https://github.com/bitcoin/bitcoin/pull/30377#discussion_r1722326803.
Replace `_hex_v_u8` for `CScript` appends to `_hex`, to skip vector conversion before serializing to the `prevector` in `CScript`.
To enable both `unsigned char` and `std::byte` values, I've extracted the existing serialization to append the size & data in separate private methods to clarify that it does more than just a simple data insertion.
There were also discussion on eliminating the operators here completely to obviate when we're serializing fixed-size collections as raw bytes, and when we're prefixing them with their size - should also be done in a separate PR.
ACKs for top commit:
achow101:
ACK 5e190cd11f
ryanofsky:
Code review ACK 5e190cd11f. Looks good!
hodlinator:
re-ACK 5e190cd11f
Tree-SHA512: 27a646629e017b2a05416d5eb964dda8b25b900d466221eff7bfa1339ded443e1c5c4cf8ff20cb3bba915a2603787a9fa6f6ec12bc0b9415d9eb07b57289192b
f482d0e366 fuzz: reduce number of iterations in `crypto_aeadchacha20poly1305` target (brunoerg)
Pull request description:
By reducing the number of iterations we improve the performance of this target and may increase coverage.
Running with `-runs=100000` from qa-assets I noticed a significant performance improvement and an increase on cov:
master:
```
#100000 DONE cov: 567 ft: 4078 corp: 124/33Kb lim: 4096 exec/s: 793 rss: 499Mb
```
PR:
```
#100000 DONE cov: 568 ft: 3833 corp: 113/15188b lim: 1746 exec/s: 1250 rss: 544Mb
```
ACKs for top commit:
achow101:
ACK f482d0e366
marcofleon:
Tested ACK f482d0e366. Saw the same slight increase in coverage. Executed 100,000 runs several times and total time went from 30-35 sec to 20-25 sec.
stratospher:
ACK f482d0e. saw similar coverage stats
Tree-SHA512: 1a96dbc22a0aed396b7f8cc9b13534b7f20a461f64f167c69c650529d535e360499f1a501abc1f957f7541ee1860b36a5580aa488a1edbfa0270c9ed83ef741d
bc52cda1f3 fix use int32_t instead of int type for risczero compile with (-march=rv32i, -mabi=ilp32) (Simon)
Pull request description:
When compile bitcoin by the toolchain(`riscv32-unknown-elf-g++`) from risc0 , the compiler argument is `-march=rv32i, -mabi=ilp32`, which will get the error which due to not serialize the value of type int .
```
blockbody-guest: cargo:warning=In file included from depend/bitcoin/src/hash.h:14,
blockbody-guest: cargo:warning= from depend/bitcoin/src/script/interpreter.h:9,
blockbody-guest: cargo:warning= from depend/bitcoin/src/script/interpreter.cpp:6:
blockbody-guest: cargo:warning=depend/bitcoin/src/serialize.h: In instantiation of 'void Serialize(Stream&, const T&) [with Stream = HashWriter; T = int]':
blockbody-guest: cargo:warning=depend/bitcoin/src/hash.h:144:20: required from 'HashWriter& HashWriter::operator<<(const T&) [with T = int]'
blockbody-guest: cargo:warning=depend/bitcoin/src/script/interpreter.cpp:1613:12: required from 'uint256 SignatureHash(const CScript&, const T&, unsigned int, int, const CAmount&, SigVersion, const PrecomputedTransactionData*) [with T = CTransaction; CAmount = long long int]'
blockbody-guest: cargo:warning=depend/bitcoin/src/script/interpreter.cpp:1664:36: required from 'bool GenericTransactionSignatureChecker<T>::CheckECDSASignature(const std::vector<unsigned char>&, const std::vector<unsigned char>&, const CScript&, SigVersion) const [with T = CTransaction]'
blockbody-guest: cargo:warning=depend/bitcoin/src/script/interpreter.cpp:1785:16: required from here
blockbody-guest: cargo:warning=depend/bitcoin/src/serialize.h:776:7: error: request for member 'Serialize' in 'a', which is of non-class type 'const int'
blockbody-guest: cargo:warning= 776 | a.Serialize(os);
```
--------------
### Reason
"The toolchain from RISC Zero defines int and int32_t as different types, although they have the same width. This means that `src/compat/assumptions.h` compiles fine; however, the templated serialization code cannot accept values of type int. Fix the compilation on RISC Zero by serializing int32_t instead of int values.
This patch will explicitly use the `int32_t` type instead of `int` to avoid errors when compiling with the risc0 toolchain. Additionally, this patch will not change any behavior on platforms where compilation was previously successful.
Fixes#30747
ACKs for top commit:
maflcko:
review-only ACK bc52cda1f3
achow101:
ACK bc52cda1f3
TheCharlatan:
ACK bc52cda1f3
Tree-SHA512: ef880e7dfa1335bf2704ab17c0f506f17390b8259755674dfcd57131736492b2f4cfc36babda6902202b7c55a7513991e21f6634b0cd9b2b03baf4f1c0f8d78b
e6994efe08 fix: increase rpcbind check robustness (tdb3)
d38e3aed89 fix: handle invalid rpcbind port earlier (tdb3)
83b67f2e6d refactor: move host/port checking (tdb3)
73c243965a test: add tests for invalid rpcbind ports (tdb3)
Pull request description:
Previously, when an invalid port was specified in `-rpcbind`, the `SplitHostPort()` return value in `HTTPBindAddresses()` was ignored and attempt would be made to bind to the default rpcbind port (with the host/port treated as a host).
This rearranges port checking code in `AppInitMain()` to handle the invalid port before reaching `HTTPBindAddresses()`. Also adds a check in `HTTPBindAddresses()` as a defensive measure for future changes.
Adds then updates associated functional tests as well.
ACKs for top commit:
achow101:
ACK e6994efe08
ryanofsky:
Code review ACK e6994efe08
zaidmstrr:
Code review ACK [e6994ef](e6994efe08)
Tree-SHA512: bcc3e5ceef21963821cd16ce6ecb83d5c5657755882c05872a7cfe661a1492b1d631f54de22f41fdd173512d62dd15dc37e394fe1a7abe4de484b82cd2438b92
8466329127 chain: simplify `deleteRwSettings` code and improve it's doc (ismaelsadeeq)
f8d91f49c7 chain: dont check for null settings value in `overwriteRwSetting` (ismaelsadeeq)
df601993f2 chain: ensure `updateRwSetting` doesn't update to a null settings (ismaelsadeeq)
c8e2eeeffb chain: uniformly use `SettingsAction` enum in settings methods (ismaelsadeeq)
1e9e735670 chain: move new settings safely in `overwriteRwSetting` (ismaelsadeeq)
1c409004c8 test: remove wallet context from `write_wallet_settings_concurrently` (ismaelsadeeq)
Pull request description:
This PR addresses the remaining review comments from #30697
1. Disallowed overwriting settings values with a `null` value.
2. Uniformly used the `SettingsAction` enum in all settings methods instead of a boolean parameter.
3. Updated `overwriteRwSetting` to receive the `common::SettingsValue` parameter by value, enabling it to be moved safely.
4. Removed wallet context from the `write_wallet_settings_concurrently` unit test, as it is not needed.
ACKs for top commit:
achow101:
ACK 8466329127
ryanofsky:
Code review ACK 8466329127. Looks good, thanks for taking suggestions and applying them to the right commits. Only changes since last review were documentation improvements and simplifying delete method.
furszy:
Code review ACK 8466329127
Tree-SHA512: baf2f59ed5aac4a4bda0c84fb6554a466a40d1f7b52b61dc2ff293d83ae60e82b925b7003237b633fecb65eba3a4c108e69166046895d1295809fbe0de67b052
51f7668d31 addrman: change nid_type from int to int64_t (Martin Zumsande)
051ba3290e addrman, refactor: introduce user-defined type for internal nId (Martin Zumsande)
Pull request description:
With `nIdCount` being incremented for each addr received, an attacker could cause an overflow in the past, see https://bitcoincore.org/en/2024/07/31/disclose-addrman-int-overflow/
Even though that attack was made infeasible indirectly by addr rate-limiting (PR #22387), to be on the safe side and prevent any regressions change the `nId`s used internally to `int64_t`.
This is being done by first introducing a user-defined type for `nId`s in the first commit, and then updating it to `int64_t` (thanks sipa for help with this!).
Note that `nId` is only used internally, it is not part of the serialization, so `peers.dat` should not be affected by this.
I assume that the only reason this was not done in the past is to not draw attention to this previously undisclosed issue.
ACKs for top commit:
naumenkogs:
ACK 51f7668d31
stratospher:
ACK 51f7668d31. I think it's a good change to make the nId space large(64 bits) so that the nId values are distinct.
achow101:
ACK 51f7668d31
Tree-SHA512: 68d4b8b0269a01a9544bedfa7c1348ffde00a288537e4c8bf2b88372ac7d96c4566a44dd6b06285f2fcf31b4f9336761e3bca7253fbc20db5e0d04e887156224
54227e681a rpc, cli: improve error message on multiwallet mode (pablomartin4btc)
Pull request description:
Running a CLI command when multiple wallets are loaded and `-rpcwallet` is not specified, should return a clearer error.
Currently in `master`:
```
$ bitcoin-cli -regtest -generate 1
error code: -19
error message:
Wallet file not specified (must request wallet RPC through /wallet/<filename> uri-path).
Try adding "-rpcwallet=<filename>" option to bitcoin-cli command line.
```
With this change:
```
$ bitcoin-cli -regtest -generate 1
error code: -19
error message:
Multiple wallets are loaded. Please select which wallet to use by requesting the RPC through the /wallet/<walletname> URI path. Or for the CLI, specify the "-rpcwallet=<walletname>" option before the command (run "bitcoin-cli -h" for help or "bitcoin-cli listwallets" to see which wallets are currently loaded).
```
ACKs for top commit:
maflcko:
review ACK 54227e681a
achow101:
ACK 54227e681a
furszy:
utACK 54227e681a
mzumsande:
Code Review ACK 54227e681a
jonatack:
ACK 54227e681a
Tree-SHA512: 51ff24f64858aa6be6adf6f20105c9f076ebea743780bf2a4399f7fe8b5239cbb1ea06d32b2ef5e850da2369abb0ef7a52c50c2b8f31f4ca90d3a486abc9b77e
58499b00d0 refactor: move `SignSignature` helpers to test utils (Sebastian Falbesoner)
Pull request description:
These helpers haven't been used in production code since segwit was merged more than eight years ago (see commit 605e8473, PR #8149), so it seems appropriate to move them to the test utils module. As suggested by instagibbs, see https://github.com/bitcoin/bitcoin/pull/30352#discussion_r1697515508.
ACKs for top commit:
instagibbs:
ACK 58499b00d0
pablomartin4btc:
ACK 58499b00d0
Tree-SHA512: a52d3b92b477246f2ceb57c3690d0229a492b65a15dae331faeae9d96e5907f7fe1176edc1530243e0f088586984fd7ba435a0a2d2f2531c04d076fdf3f4095f
7025942687 build: drop superfluous `HAVE_BUILD_INFO` define (Sebastian Falbesoner)
0dd662510c build: drop obj/ subdir for generated build.h, rename to bitcoin-build-info.h (Sebastian Falbesoner)
Pull request description:
As indicated by the TODO, the obj subdirectory is not needed anymore now for the generated build.h header, since autotools are gone and we don't have in-source builds anymore (see #30454, #30664). In the second commit the superflous `HAVE_BUILD_INFO` macro is dropped, as suggested in https://github.com/bitcoin/bitcoin/pull/30856#pullrequestreview-2292424496.
ACKs for top commit:
theuni:
utACK 7025942687
Tree-SHA512: 0a3b2cbbcf638344ceb74e5ba5a0fe2b1718427b23a18c8890258db36ce7177006a146178ed88d9c5ae956a5426f3844e86c1f4cca7c40946359742bffda983b
bitcoin-build-info.h should always be generated before clientversion.cpp
is compiled due to the following explicit dependency in src/CMakeLists.txt:
add_dependencies(bitcoin_clientversion generate_build_info)
Hence there is no need to gate the inclusion of that header with an
extra define.
Now that this file is not in a subfolder anymore, prefix it with
"bitcoin-" to avoid potential collisions. Also add "info" for a more
descriptive name.
caac06f784 streams: reorder/document functions (Pieter Wuille)
67a3d59076 streams: remove unused code (Pieter Wuille)
Pull request description:
This is a follow-up to #30884.
Remove a number of dead code paths, and improve the code organization and documentation, in `AutoFile`.
ACKs for top commit:
maflcko:
re-ACK caac06f784
theStack:
Code-review ACK caac06f784
l0rinc:
ACK caac06f784
tdb3:
CR ACK caac06f784
Tree-SHA512: 297791f093e0142730f815c11dd3466b98f7e7edea86094a815dae989ef40d8056db10e0fed6e575d530903c18e80c08d36d3f1e6b828f2d955528f365b22008
facbcd4cef log: Use ConstevalFormatString (MarcoFalke)
fae9b60c4f test: Use LogPrintStr to test m_log_sourcelocations (MarcoFalke)
fa39b1ca63 doc: move-only logging warning (MarcoFalke)
Pull request description:
This changes all logging (including the wallet logging) to produce a
`ConstevalFormatString` at compile time, so that the format string can be
validated at compile-time.
I tested with `clang` and found that the compiler will use less than 1% more of time and memory.
When an error is found, the compile-time error depends on the compiler, but it may look similar to:
```
src/util/string.h: In function ‘int main(int, char**)’:
src/bitcoind.cpp:265:5: in ‘constexpr’ expansion of ‘util::ConstevalFormatString<1>(((const char*)"Hi %s %s"))’
src/util/string.h:38:98: in ‘constexpr’ expansion of ‘util::ConstevalFormatString<1>::Detail_CheckNumFormatSpecifiers(std::basic_string_view<char>(((const char*)((util::ConstevalFormatString<1>*)this)->util::ConstevalFormatString<1>::fmt)))’
src/util/string.h:78:34: error: expression ‘<throw-expression>’ is not a constant expression
78 | if (num_params != count) throw "Format specifier count must match the argument count!";
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
This refactor does not change behavior of the compiled executables.
ACKs for top commit:
hodlinator:
re-ACK facbcd4cef
l0rinc:
ACK facbcd4cef
ryanofsky:
Code review ACK facbcd4cef
pablomartin4btc:
re-ACK facbcd4cef
stickies-v:
Approach ACK and code LGTM facbcd4cef modulo a `tinyformat::format_error` concern.
Tree-SHA512: 852f74d360897020f0d0f6e5064edc5e7f7dacc2bec1d5feff22c634a2fcd2eb535aa75be0b7191d9053728be6108484c737154b02d68ad3186a2e5544ba0db8
fa99e4521b ci: Allow CCACHE_DIR bind mount (MarcoFalke)
fa252da0b9 ci: Remove hardcoded CCACHE_DIR in cirrus (MarcoFalke)
fa146904e1 ci: Bump default CCACHE_MAXSIZE to 500M (MarcoFalke)
aaaa7cf8ba cirrus: Drop CCACHE_NOHASHDIR (MarcoFalke)
fa7ca182a9 ci: Print inner env (MarcoFalke)
Pull request description:
The outer env is printed when the `/tmp/env-$USER-$CONTAINER_NAME` is created. There is also a separate env printed when building the container image (usually with default values).
To confirm that the inner container env is correctly derived from the outer env, and not from the default build env, print it a third time.
ACKs for top commit:
l0rinc:
utACK fa99e4521b
willcl-ark:
ACK fa99e4521b
Tree-SHA512: 59fd3262d551e09224866e31c14ca865461e81abbe00b83391fe3a9c7ada30fd2fd0272e4aa812df2712433ac7594d1a55cf674248b341359cec09c8d3f0c58b
a9964c0444 doc: Updating docs from autotools to cmake (kevkevinpal)
Pull request description:
A bit of a followup from https://github.com/bitcoin/bitcoin/pull/30840
- In this change the documentation where we refer to the `./configure` script which is now gone and have converted the configure params to use the `cmake` equivalent.
ACKs for top commit:
maflcko:
ACK a9964c0444
jonatack:
utACK a9964c0444
jarolrod:
ACK a9964c0444
tdb3:
ACK a9964c0444
pablomartin4btc:
re-ACK a9964c0444
Tree-SHA512: f7ed20b8ad61f028c0d242b9cc70650d8da63057d4a8f7da88f0117a8d3241c5fe8fcf19d56ec82088160b9fee9b175fe9f64e5a845260d3696dc7e94bfdd0bd
ccccb67851 ci: Use clang-19 in msan tasks (MarcoFalke)
Pull request description:
A new clang version generally comes with bugfixes, new sanitizer features, deprecations, as well as new features.
Upgrade the memory sanitizer tasks to use the new version.
(Ref https://github.com/bitcoin/bitcoin/pull/30634)
ACKs for top commit:
fanquake:
ACK ccccb67851 Tested both jobs on aarch64, and one on x86_64 with `mmap_rnd_bits`.
Tree-SHA512: a42bf2da7c08aa54c0c5ab3811ff51b98b80b276be135eed32395a55ae93a42d41d7cd32c307062dcca711a892958ea141168c2a06025560074f8c5d20190946
replaced --enable-debug with -DCMAKE_BUILD_TYPE=Debug in developer-notes
replaced --enable-multiprocess with -DWITH_MULTIPROCESS=ON
replaced --disable-zmq with -DWITH_ZMQ=OFF
735436df8c Remove outdated Eclipser fuzzing documentation (Jon Atack)
Pull request description:
Remove the Eclipser fuzzing documentation from `doc/fuzzing.md`, as that repository (https://github.com/SoftSec-KAIST/Eclipser) hasn't been updated in several years, appears possibly unmaintained, and likely isn't being actively used for fuzzing Bitcoin Core.
These docs were originally added in https://github.com/bitcoin/bitcoin/pull/22585.
ACKs for top commit:
maflcko:
review ACK 735436df8c
brunoerg:
ACK 735436df8c
Tree-SHA512: 7ccbf93c10add53e92edf67a622722935029add63f8fbb6e733b96e9d155faeb8d5d3678adb0e7f2ce8ccbdffd2a34c3dc93adbcf4e3ce0cdd03e20ad3e6bbd6
d01b85bfec ci: Use `ninja` to build in macOS native CI job (Hennadii Stepanov)
Pull request description:
This PR addresses [this](https://github.com/bitcoin/bitcoin/pull/30911#issuecomment-2354922939) comment:
> I wonder if one CI task should be using Ninja (and cmake >= 3.27), if it isn't too hard to implement. Otherwise this config will remain untested and errors may sneak in to the master branch, only being detected after merge.
ACKs for top commit:
maflcko:
review ACK d01b85bfec
theuni:
ACK d01b85bfec.
jonatack:
ACK d01b85bfec
jarolrod:
ACK d01b85bfec
Tree-SHA512: 5cbbc87f0e48512441a4f0cf10af2f6d73f24d3e8667b338b176fd1667fd5d7739349bcede3aeef973497ff67d33cb8f7d7f3681c3ede8e8b2f673b853d5bc63
Adds invalid rpcbind port checking to
`HTTPBindAddresses()`. While movement of
`CheckHostPortOptions()` in the previous
commit handles rcpbind port errors, updating
`HTTPBindAddresses()` port checking adds
a defensive measure for potential future
changes.
Previously, when an invalid port was specified
in `-rpcbind`, the `SplitHostPort()` return value
in `HTTPBindAddresses()` was ignored and attempt
would be made to bind to the default rpcbind port
(with the host/port treated as a host).
This rearranges port checking code in
`AppInitMain()` to handle the invalid
port before reaching `HTTPBindAddresses()`.
Also adjusts associated functional tests.
The primary objective is to provide users with clearer
and more informative error messages when encountering
the RPC_WALLET_NOT_SPECIFIED error, which occurs when
multiple wallets are loadad.
This commit also rectifies the error message consistency
by bringing the error message in line with the definition
established in protocol.h ("error when there are multiple
wallets loaded").
This changes all logging (including the wallet logging) to produce a
ConstevalFormatString at compile time, so that the format string can be
validated at compile-time.
Also, while touching the wallet logging, avoid a copy of the template
Params by using const Params&.
This test would fail to mark the chain tip as "invalid" instead
of "headers-only" without the previous commit marking the headers
as BLOCK_FAILED_CHILD.
Without doing so, header-only chains building on a chain that
will be marked as invalid would still be eligible for m_best_header.
This improves both getblockchaininfo and getchaintips behavior.
While this adds an iteration over the entire block index, it can only be
triggered by the user (invalidateblock) or by others at a cost (the
header needs to be accepted in the first place, so it needs valid PoW).
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
This means that it is being called in two situations:
1.) As part of the invalidateblock rpc
2.) When we receive a block for which we have a valid
header in our block index, but the block turns out to be invalid
It recalculates m_best_header by looping over the entire
block index. Even though this is not very performant, it
will only be used in rare situations that cannot be
triggered by others without a cost:
As part of to invalidateblock / reconsiderblock rpcs, or when a
block with an accepted header with valid PoW turns out to be invalid
later during full validation.
af9f987893 doc: update NeedsRedownload() comment (Sjors Provoost)
Pull request description:
Noticed two outdated comments while reviewing #29370.
Since #21009 we no longer roll back the chain, when a user updates a pre-segwit node to a modern node. In this unlikely scenario we tell the user to `-reindex`.
This PR updates a comment in `PopulateAndValidateSnapshot` to reflect that change. Ditto for the description of `nStatus` in `chain.h`.
ACKs for top commit:
maflcko:
re-ACK af9f987893
fjahr:
ACK af9f987893
Tree-SHA512: d590f1cff6823297764c863753ed5478b8933d503c43933902d50b449dfd852a02aeb318c072ad25d02e4c2583d7026cd176a10b0584292d6bbe381a063f5c45
2a581144f2 build: Minimize I/O operations in GenerateHeaderFromJson.cmake (Lőrinc)
aa003d1568 build: Minimize I/O operations in GenerateHeaderFromRaw.cmake (Lőrinc)
Pull request description:
Follow up of the https://github.com/bitcoin/bitcoin/pull/30883 revert.
Replaced multiple file writes with a single string template write.
The raw content is first grouped into 8 byte chunks, followed by another regex replace which wraps them in `std::byte` or just the raw bytes, prefixed with `0x`.
Tested the output with `diff -w` and they're the same - only whitespace differences because slightly different source formatting.
----
Tested the `Raw` performance with:
```bash
time cmake -DRAW_SOURCE_PATH=src/bench/data/block413567.raw -DHEADER_PATH=build/after/block413567.raw.h -DRAW_NAMESPACE=benchmark::data -P cmake/script/GenerateHeaderFromRaw.cmake
```
Before:
> 15.41s user 23.06s system 97% cpu 39.593 total
After:
> 0.77s user 0.06s system 97% cpu 0.849 total
----
Tested the `Json` performance with:
```bash
time cmake -DJSON_SOURCE_PATH=src/secp256k1/src/wycheproof/ecdsa_secp256k1_sha256_bitcoin_test.json -DHEADER_PATH=build/after/ecdsa_secp256k1_sha256_bitcoin_test.json -P cmake/script/GenerateHeaderFromJson.cmake
````
Before:
> 3.57s user 6.01s system 94% cpu 10.136 total
After:
> 0.17s user 0.01s system 98% cpu 0.187 total
ACKs for top commit:
maflcko:
review ACK 2a581144f2👒
hebasto:
ACK 2a581144f2.
willcl-ark:
tACK 2a581144f2
Tree-SHA512: 5e44f79d1c0dbb61d8b64f28d4c3c87a176981f72104b28800eef2037b0728076cbcf14ff07b05ff94d4e8800605586cfd5df00519db9027933c5943348c01d2
fab932b421 ci: Remove incorrectly hardcoded HOST in mac_native task (MarcoFalke)
fa8f35d786 ci: Use macos-14 GHA image (MarcoFalke)
Pull request description:
There shouldn't be any downside, because XCode remains pinned to the same version.
However, builds are expected to be a bit faster with M1, which seems nice.
ACKs for top commit:
hebasto:
ACK fab932b421.
willcl-ark:
ACK fab932b421
Tree-SHA512: 9719e05c67b8b5f3d59bd1a38eef00407b1ae5e123b18151c494b6d2dbf55bd2b0b5bb6c1a0469635c7b3bb5f23990d3bb2f339f56ce3955e8a1b97ac9f295d4
89bf11b807 guix: build Linux GCC with --enable-cet (fanquake)
Pull request description:
Similar to #29695, and in the same vein of explicitly configuring hardening options in our release toolchain.
See https://gcc.gnu.org/install/configure.html:
>` --enable-cet`
> Enable building target run-time libraries with control-flow instrumentation, see `-fcf-protection option`. When --enable-cet is specified target libraries are configured to add `-fcf-protection` and, if needed, other target specific options to a set of building options.
> `--enable-cet=auto` is default. CET is enabled on Linux/x86 if target binutils supports Intel CET instructions and disabled otherwise. In this case, the target libraries are configured to get additional `-fcf-protection` option.
ACKs for top commit:
TheCharlatan:
ACK 89bf11b807
Tree-SHA512: 772d8529713a31e5db42be4e053582bb9ba6f26079ae136c6bf8303c4992a90d61159dbb0fde7a4b4cb7b4bf5024d5397a78004e6188b36e1c36dd5e5cdc49ad
The waitforblock RPC method takes a hash argument and waits for that specific block. The waitfornewblock waits for any new block. This commit fixes the documentation.
Also signal m_tip_block_cv when StopRPC is called, for
consistency with g_best_block_cv. This is handled in
StopRPC instead of OnRPCStopped() because the latter
is deleted in a later commit.
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
a240e150e8 streams: remove AutoFile::Get() entirely (Pieter Wuille)
e624a9bef1 streams: cache file position within AutoFile (Pieter Wuille)
Pull request description:
Fixes#30833.
Instead of relying on frequent `ftell` calls (which appear to cause a significant slowdown on some systems) in XOR-enabled `AutoFile`s, cache the file position within `AutoFile` itself.
ACKs for top commit:
achow101:
ACK a240e150e8
davidgumberg:
untested reACK a240e150e8
theStack:
Code-review ACK a240e150e8
Tree-SHA512: fd3681edc018afaf955dc7a41a0c953ca80d46c1129e3c5b306c87c95aae93b2fe7b900794eb8b6f10491f9211645e7939918a28838295e6873eb226fca7006f
e4e3b44e9c net: call `Select` with reachable networks in `ThreadOpenConnections` (brunoerg)
829becd990 addrman: change `Select` to support multiple networks (brunoerg)
f698636ec8 net: add `All()` in `ReachableNets` (brunoerg)
Pull request description:
This PR changes addrman's `Select` to support multiple networks and change `ThreadOpenConnections` to call it with reachable networks. It can avoid unnecessary `Select` calls and avoid exceeding the max number of tries (100), especially when turning a clearnet + Tor/I2P/CJDNS node to Tor/I2P/CJDNS. Compared to #29330, this approach is "less aggresive". It does not add a new init flag and does not impact address relay.
I did an experiment of calling `Select` without passing a network until it finds an address from a network that compose 20% ~ 25% of the addrman (limited to 100 tries).

ACKs for top commit:
achow101:
ACK e4e3b44e9c
vasild:
ACK e4e3b44e9c
naumenkogs:
ACK e4e3b44e9c
Tree-SHA512: e8466b72b85bbc2ad8bfb14471eb27d2c50d4e84218f5ede2c15a6fa3653af61b488cde492dbd398f7502bd847e95bfee1abb7e01092daba2236d3ce3d6d2268
6a1aa510e3 rpc: check block index before reading block / undo data (Martin Zumsande)
6cbf2e5f81 rpc: Improve gettxoutproof error when only header is available. (Martin Zumsande)
69fc867ea1 test: add coverage to getblock and getblockstats (Martin Zumsande)
5290cbd585 rpc: Improve getblock / getblockstats error when only header is available. (Martin Zumsande)
e5b537bbdf rest: improve error when only header of a block is available. (Martin Zumsande)
Pull request description:
Fixes#20978
If a block was pruned, `getblock` already returns a specific error: "Block not available (pruned data)".
But if we haven't received the full block yet (e.g. in a race with block downloading after a new block was received headers-first, or during IBD) we just return an unspecific "Block not found on disk" error and log
`ERROR: ReadBlockFromDisk: OpenBlockFile failed for FlatFilePos(nFile=-1, nPos=0) `
which suggest something went wrong even though this is a completely normal and expected situation.
This PR improves the error message and stops calling `ReadRawBlockFromDisk()`, when we already know from the header that the block is not available on disk.
Similarly, it prevents all other rpcs from calling blockstorage read functions unless we expect the data to be there, so that `LogError()` will only be thrown when there is an actual file system problem.
I'm not completely sure if the cause is important enough to change the wording of the rpc error, that some scripts may rely on.
If reviewers prefer it, an alternative solution would be to keep returning the current "Block not found on disk" error, but return it immediately instead of calling `ReadRawBlockFromDisk`, which would at least prevent the log error and also be an improvement in my opinion.
ACKs for top commit:
fjahr:
re-ACK 6a1aa510e3
achow101:
ACK 6a1aa510e3
andrewtoth:
re-ACK 6a1aa510e3
Tree-SHA512: 491aef880e8298a05841c4bf8eb913ef84820d1ad5415fd17d9b441bff181959ebfdd432b5eb8347dc9c568433f9a2384ca9d84cd72c79d8a58323ca117538fe
bb3b980dfd validation: drop maximum -dbcache (Sjors Provoost)
Pull request description:
Due to recent UTXO set growth, the current maximum value for `-dbcache` of 16GB is ~just months away from being~ insufficient (for those who wish to complete IBD with the UTXO set held in RAM).
This drops the limit. It also adds a warning that it's up to users to check that they have enough RAM.
Fixes#28249.
---
A previous version of this PR increased the maximum to 64GB. It also made startup abort if the value provided is too high, rather than quietly round it down. But this didn't get much support.
ACKs for top commit:
achow101:
ACK bb3b980dfd
tdb3:
ACK bb3b980dfd
BenWestgate:
crACK bb3b980dfd.
Tree-SHA512: 8515fff468c2387a0b04bd9523ab1df46d6325738588b7550fabddbb8624817a583d95b95ea246407f9f0ff3e43e760cf7334621bec6af79710176328528a3ef
fc7b507e9a tidy: add clang-tidy `modernize-use-starts-ends-with` check (Roman Zeyde)
Pull request description:
ACKs for top commit:
jonatack:
re-ACK fc7b507e9a only change since my previous ACK is the commit message
achow101:
ACK fc7b507e9a
stickies-v:
ACK fc7b507e9a
hebasto:
ACK fc7b507e9a, I have reviewed the code and it looks OK.
Tree-SHA512: 334e0ff91b9b108a57cdfc12ee53685b792d377e11124c7c394b8f681a8168a8d65a56c7f884555238e65e97e9ad62ede52b79219ce258979e54abdd76721df1
a93c171faa Drop unneeded nullptr check from CreateNewBlock() (Sjors Provoost)
dd87b6dff3 Have createNewBlock return BlockTemplate interface (Sjors Provoost)
Pull request description:
Suggested in https://github.com/bitcoin/bitcoin/pull/29432#issuecomment-2225337100
An external program that uses the Mining interface may need quick access to some information in the block template, while it can wait a bit longer for the full raw transaction data.
This would be the case for a Stratum v2 Template Provider which needs to send a [NewTemplate](https://github.com/stratum-mining/sv2-spec/blob/main/07-Template-Distribution-Protocol.md#72-newtemplate-server---client) message message (which doesn't include transactions) as quickly as possible. It does not include the serialized block transactions.
ACKs for top commit:
achow101:
ACK a93c171faa
ryanofsky:
Code review ACK a93c171faa. Since last review, just rebased with no changes or conflicts
itornaza:
Code review ACK a93c171faa
TheCharlatan:
Re-ACK a93c171faa
Tree-SHA512: 17cb61eb5548e9d4a69e34dd732f68a06cde2ad3d82c8339efee704c7860d5de144d93b23d6ecd6ee4ec205844e5560ad0f6d3917822fa75bb8e640c5f51af9a
a97f43d63a fuzz: Add harness for p2p headers sync (marcofleon)
a0eaa4749f Add FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION in PoW check (marcofleon)
a3f6f5acd8 build: Automatically define FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION for fuzz builds (marcofleon)
0c02d4b2bd net_processing: Make MAX_HEADERS_RESULTS a PeerManager option (marcofleon)
Pull request description:
This PR reopens https://github.com/bitcoin/bitcoin/pull/28043. It's a regression fuzz test for https://github.com/bitcoin/bitcoin/pull/26355 and [a couple bugs](ed6cddd98e) that were addressed in https://github.com/bitcoin/bitcoin/pull/25717. This should help us move forward with the [removal of mainnet checkpoints](https://github.com/bitcoin/bitcoin/pull/25725).
It seems like the main concern in https://github.com/bitcoin/bitcoin/pull/28043 was the global mock function for proof of work. This PR aims to be an improvement by replacing the previous approach with a fuzz build configured using `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION`. This ensures that the simplified test code will never be in a release binary. If we agree this is the way to go, there are some other places (for future targets) where this method could be used.
In this target, PoW isn't being tested, so the goal is to bypass the check and let the fuzzer do its thing. In the other harnesses where PoW is actually being fuzzed, `CheckProofOfWork` is now `CheckProofOfWorkImpl`. So, the only change to that function is in the name.
More about `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` can be found at https://llvm.org/docs/LibFuzzer.html#fuzzer-friendly-build-mode and https://github.com/AFLplusplus/AFLplusplus/blob/stable/docs/fuzzing_in_depth.md#d-modifying-the-target.
ACKs for top commit:
naumenkogs:
ACK a97f43d63a
dergoegge:
reACK a97f43d63a
instagibbs:
tested ACK a97f43d63a
brunoerg:
ACK a97f43d63a
Tree-SHA512: 60b0bc6aadd8ca4c39db9cbba2da2debaaf68afcb6a8dd75c1ce48ca9e3996948fda8020930b6771a424e0f7c41b0b1068db4aa7dbe517f8fc152f1f712058ad
9ad2fe7e69 clusterlin: only start/use search when enough iterations left (Pieter Wuille)
bd044356ed clusterlin: improve heuristic to decide split transaction (optimization) (Pieter Wuille)
71f2629398 clusterlin: include topological pot subsets automatically (optimization) (Pieter Wuille)
e20fda77a2 clusterlin: reduce computation of unnecessary pot sets (optimization) (Pieter Wuille)
6060a948ca clusterlin bench: add example hard cluster benchmarks (Pieter Wuille)
2965fbf203 clusterlin: track upper bound potential set for work items (optimization) (Pieter Wuille)
9e43e4ce10 clusterlin: use feerate-sorted depgraph in SearchCandidateFinder (Pieter Wuille)
b80e6dfe78 clusterlin: add reordering support for DepGraph (Pieter Wuille)
85a285a306 clusterlin: separate initial search entries per component (optimization) (Pieter Wuille)
e4faea9ca7 clusterlin bench: have low/high iter benchmarks instead of per-iter (Pieter Wuille)
Pull request description:
Part of cluster mempool: #30289
Depends on #30126, and was split off from it.
This improves the candidate search algorithm introduced in the previous PR with a variety of optimizations.
The resulting search algorithm largely follows Section 2 of [How to linearize your cluster](https://delvingbitcoin.org/t/how-to-linearize-your-cluster/303#h-2-finding-high-feerate-subsets-5), though with a few changes:
* Connected component analysis is performed inside the search algorithm (creating initial work items per component for each candidate), rather than once at a higher level. This duplicates some work but is significantly simpler in implementation.
* No ancestor-set based presplitting inside the search is performed; instead, the `best` value is initialized with the best topologically valid set known to the LIMO algorithm before search starts: the better one out of the highest-feerate remaining ancestor set, and the highest-feerate prefix of remaining transactions in `old_linearization`.
* Work items are represented using an included set *inc* and an undefined set *und*, rather than included and excluded.
* Potential sets *pot* are not computed for work items with empty *inc*.
At a high level, the only missing optimization from that post is bottleneck analysis; my thinking is that it only really helps with clusters that are already relatively cheap to linearize (doing so would need to be done at a higher level, not inside the search algorithm).
---
Overview of the impact of each commit here on linearize performance:
* **[clusterlin bench: have low/high iter benchmarks instead of per-iter](21a184db63)**: no impact
* **[separate initial search entries per component (optimization)](c84c5c86ba)**: reduce iterations, increase start-up cost
* **[add reordering support for DepGraph](019ff29609)**: no impact
* **[use feerate-sorted depgraph in SearchCandidateFinder](8e27dd5a22)**: typically reduce iterations, increase start-up cost
* **[track upper bound potential set for work items](781e0fb3aa)**: reduce iterations, increase cost per iteration
* **[reduce computation of unnecessary pot sets](9fe834fa97)**: reduce cost per iteration
* **[include topological pot subsets automatically](30612710a4)**: reduce iterations, increase cost per iteration
* **[improve heuristic to decide split transaction](1880c00ab1)**: typically reduce iterations, increase cost per iteration
* **[only start/use search when enough iterations left](12760a57b3)**: just account for start-up cost as equivalent iterations
ACKs for top commit:
sdaftuar:
ACK 9ad2fe7e69
instagibbs:
reACK 9ad2fe7e69
glozow:
reACK 9ad2fe7e69, just have a question about the docs
Tree-SHA512: 108bcbb0676f36071eb83954059b5f3d6646c745015b644a2a5d7f5a8ac9424c2d01d339fa6318a3aff4cf313308e85bb80b0090899720a3fcba027b8025590a
bc7900f33d kernel: Move background load thread to node context (TheCharlatan)
Pull request description:
The thread handle is never used by the ChainstateManager, so move it out and into the node context. Users of the kernel library now no longer have to manually join the thread when destructing the ChainstateManager.
ACKs for top commit:
maflcko:
ACK bc7900f33d🔄
achow101:
ACK bc7900f33d
ryanofsky:
Code review ACK bc7900f33d. Nice cleanup
jonatack:
Light ACK bc7900f33d
stickies-v:
ACK bc7900f33d
Tree-SHA512: add9c4823731324e3db50f95e023e99d55db7cc75c69083ae7c9c2157e5540968caa6cf10674aa4901f91366b02ebb1ff18bb977fec0a46431e2196448958b9d
72c9a1fe94 test: Check that network stays suspended after dumptxoutset if it was off before (Fabian Jahr)
Pull request description:
Follow-up to #30817 which covered the robustness of `dumptxoutset`: network is deactivated during the run but re-activated even when an issue was encountered. But it did not cover the case if the user had deactivated the network themselves before. In that case the user may want the network to stay off so the network is not reactivated after `dumptxoutset` finishes. A test for this behavior is added here.
ACKs for top commit:
achow101:
ACK 72c9a1fe94
pablomartin4btc:
ACK 72c9a1fe94
theStack:
utACK 72c9a1fe94
tdb3:
tested ACK 72c9a1fe94
Tree-SHA512: 18a57c5782e99a018414db0597e88debeb32666712c2a75dddbb55135d8f1ddd1eeacba8bbbd35fc03b6c4ab0522fe074ec08edea729560b018f51efabf00e89
07f4cebe57 refactor: move m_is_inbound out of CNodeState (Sergi Delgado Segura)
Pull request description:
`m_is_inbound` cannot be changed throughout the life of a `Peer`. However, we are currently storing it in `CNodeState`, which requires locking `cs_main` in order to access it. This can be moved to the outside scope and only require `m_peer_mutex`.
This is a refactor in preparation for Erlay reworks.
ACKs for top commit:
maflcko:
ACK 07f4cebe57 🗞
achow101:
ACK 07f4cebe57
marcofleon:
ACK 07f4cebe57
naumenkogs:
ACK 07f4cebe57
Tree-SHA512: bcc77135646c697204a4605971774cb3ccdf11b3e363a7339bfb4d1678de1681d6ca642dc467f7d71834a94dd56e05367e7fd32a3e6dc6be30c89342f39bf695
95560616fb code style: update .editorconfig file (Sebastian Falbesoner)
Pull request description:
Updates the .editorconfig file, first introduced in 2021 (see PR #21123, commit 7a135d57) w.r.t. following changes:
- consider Rust .rs files (relevant since #28076, commit bbbbdb0c)
- reflect build system change to CMake (#30454, #30664)
- add setting for bare Makefile still used for depends builds
Can be tested e.g. by using the editorconfig-vim plugin (https://github.com/editorconfig/editorconfig-vim). The PR is made under the assumption that the file is still considered useful, especially for new contributors. If people feel like that's not the case anymore, the alternative is to delete it, obviously.
Top commit has no ACKs.
Tree-SHA512: 8406b1caf31e310f7e17c607d97beac583481e71b4425e0be2bbd8207096aa374a70151b58aae5fdb648ef5ff5c7e1d0a2949e6de3355bdd2009d8353ee24af0
282f0e9255 Unit test runner documentation fix and improvements (Jon Atack)
Pull request description:
Running `test_bitcoin --help` prints the list of arguments that may be passed, not the list of tests, so fix that.
Improve the content and order of the unit test documentation.
ACKs for top commit:
pablomartin4btc:
re-ACK 282f0e9255
tdb3:
re ACK 282f0e9255
Tree-SHA512: 0d25108ab641bcd9b53f99d139afeec90a34f44d5b00c3c677f7539d87782440a28fadc348663b8c28ace43552834737b9c1e8f5517c68edc8547695a9cb5063
Updates the .editorconfig file, first introduced in 2021
(see PR #21123, commit 7a135d57) w.r.t. following changes:
- consider Rust .rs files (relevant since #28076, commit bbbbdb0c)
- reflect build system change to CMake (#30454, #30664)
- add setting for the bare Makefile still used for depends builds
Can be tested e.g. by using the editorconfig-vim plugin
(https://github.com/editorconfig/editorconfig-vim).
- Running `test_bitcoin --help` prints the list of arguments that may be passed,
not the list of tests, so fix that.
- Improve the content and order of the unit test documentation.
This avoids low-level log errors that are supposed to only occur when
there is an actual problem with the block on disk missing unexpectedly,
but not in the case where the block and/or undo data are expected not to be there.
It changes behavior such that in the first case (block index indicates
data is available but retrieving it fails) an error is thrown.
It also adjusts a functional tests that tried to simulate not
having undo data (but having block data) by deleting the undo file.
This situation should occur reality because block and undo data are pruned together.
Instead, test this situation with a block that hasn't been connected.
This improves the error message of the getblock and getblockstats rpc and prevents calls to
ReadRawBlockFromDisk(), which are unnecessary if we know
from the header nStatus field that the block is not available.
This avoids calling ReadRawBlockFromDisk() when the block is expected
not to be available because we haven't downloaded it yet and only know
the header.
001b1cf010 build: use standard branch-protection for aarch64-linux (fanquake)
Pull request description:
Use `-mbranch-protection=standard` when targetting `*aarch64-*-linux*`.
Part of #24123, but this flag can already be used on a best effort basis.
Note that this flag is also already used by default, in the toolchain, on various distros (i.e Fedora).
ACKs for top commit:
hebasto:
ACK 001b1cf010.
TheCharlatan:
ACK 001b1cf010
Tree-SHA512: 2d7ae60f59921a62d51139cb0fd5cecbed4f63266564b2623b7d160f5b0c2c42c78ef8aeff787f485eccc46a9ffd5da70023ec093df6add7c982e0d48a1601b5
The thread handle is never used by the ChainstateManager, so move it out
and into the node context. Users of the kernel library now no longer
have to manually join the thread when destructing the ChainstateManager.
a965f2bc07 gui: fix crash when closing wallet (furszy)
Pull request description:
The crash occurs because `WalletController::removeAndDeleteWallet` is called twice for the
same wallet model: first in the GUI's button connected function `WalletController::closeWallet`,
and then again when the backend emits the `WalletModel::unload` signal.
This causes the issue because `removeAndDeleteWallet` inlines an `erase(std::remove())`.
So, if `std::remove` returns an iterator to the end (indicating the element wasn't found
because it was already erased), the subsequent call to `erase` leads to an undefined behavior.
Test Notes:
Try closing any wallet using the toolbar button in the GUI. It will crash in master, but not here.
Fixes https://github.com/bitcoin/bitcoin/issues/30887.
ACKs for top commit:
pablomartin4btc:
tACK a965f2bc07
jarolrod:
ACK a965f2bc07
hebasto:
ACK a965f2bc07.
Tree-SHA512: c94681b95cb566f7aabd0d4fb10f797c2cea6ac569abc265e918f08e6abae3335432a0b0879372b54b2c109798ed0a4a249bf162c34add59cbd18d38a2d9660e
Similar to #29695, and in the same vein of explicitly configuring
hardening options in our release toolchain.
See https://gcc.gnu.org/install/configure.html:
> Enable building target run-time libraries with control-flow instrumentation,
> see `-fcf-protection option`. When --enable-cet is specified target
> libraries are configured to add `-fcf-protection` and, if needed,
> other target specific options to a set of building options.
> `--enable-cet=auto` is default. CET is enabled on Linux/x86 if target
> binutils supports Intel CET instructions and disabled otherwise.
> In this case, the target libraries are configured to get additional
> `-fcf-protection` option.
An external program that uses the Mining interface may need quick access to some information in the block template, while it can wait a bit longer for the full raw transaction data.
This would be the case for a Stratum v2 Template Provider which needs to send a NewTemplate message (which doesn't include transactions) as quickly as possible.
The crash occurs because 'WalletController::removeAndDeleteWallet' is called
twice for the same wallet model: first in the GUI's button connected function
'WalletController::closeWallet', and then again when the backend emits the
'WalletModel::unload' signal.
This causes the issue because 'removeAndDeleteWallet' inlines an
erase(std::remove()). So, if 'std::remove' returns an iterator to the end
(indicating the element wasn't found because it was already erased), the
subsequent call to 'erase' leads to an undefined behavior.
This test checks m_log_sourcelocations, not the formatting with format
specifiers. Those are tested in logging_LogPrintMacros below.
So just use LogPrintStr directly in this test, without format specifiers
and format args.
This is required for a follow-up commit.
Tested the performance with:
> time cmake -DJSON_SOURCE_PATH=src/secp256k1/src/wycheproof/ecdsa_secp256k1_sha256_bitcoin_test.json -DHEADER_PATH=build/after/ecdsa_secp256k1_sha256_bitcoin_test.json -P cmake/script/GenerateHeaderFromJson.cmake
Before:
> 3.57s user 6.01s system 94% cpu 10.136 total
After:
> 0.17s user 0.01s system 98% cpu 0.187 total
Replaced multiple file writes with a single string template write.
The raw content is first grouped into 8 byte chunks, followed by another regex replace which wraps them in `std::byte`.
Tested the output with `diff -w` and they're the same - only whitespace differences because slightly different source formatting.
Tested the performance with:
> time cmake -DRAW_SOURCE_PATH=src/bench/data/block413567.raw -DHEADER_PATH=build/after/block413567.raw.h -DRAW_NAMESPACE=benchmark::data -P cmake/script/GenerateHeaderFromRaw.cmake
Before:
> 15.41s user 23.06s system 97% cpu 39.593 total
After:
> 0.77s user 0.06s system 97% cpu 0.849 total
Empirically, this approach seems to be more efficient in common real-life
clusters, and does not change the worst case.
Co-Authored-By: Suhas Daftuar <sdaftuar@gmail.com>
Automatically add topologically-valid subsets of the potential set pot
to inc. It can be proven that these must be part of the best reachable
topologically-valid set from that work item.
This is a crucial optimization that (apparently) reduces the maximum
number of iterations from ~2^(N-1) to ~sqrt(2^N).
Co-Authored-By: Suhas Daftuar <sdaftuar@gmail.com>
Keep track of which transactions in the graph have an individual
feerate that is better than the best included set so far. Others do not
need to be added to the pot set, as they cannot possibly help beating
best.
In each work item, keep track of a conservative overestimate of the best
possible feerate that can be reached from it, and then use these to avoid
exploring hopeless work items.
Add a DepGraph(depgraph, reordering) function that constructs a new DepGraph
corresponding to an old one, but with its transactions is a modified order
(given as a vector from old to new positions).
Also use this reordering feature inside DepGraphFormatter::Unser, which needs
a small modification so that its reordering mapping is old-to-new (rather than
the new-to-old it used before).
Before this commit, the worst case for linearization involves clusters which
break apart in several smaller components after the first candidate is
included in the output linearization.
Address this by never considering work items that span multiple components
of what remains of the cluster.
19f4a7c95a test: Wait for local services to update in feature_assumeutxo (Fabian Jahr)
Pull request description:
Closes#30878
It seems like there is a race where the block is stored locally and `getblock` does not error anymore, but `ActivateBestChain` has not finished yet, so the local services are not updated yet either. Fix this by waiting for the local services to update.
Can be reproduced locally by adding the sleep here:
```cpp
──────────────────────────────────────────────────────────────────────────────────────────────────────────┐
src/validation.cpp:3567: bool Chainstate::ActivateBestChain(BlockValidationState& state, std::shared_ptr< │
──────────────────────────────────────────────────────────────────────────────────────────────────────────┘
}
if (WITH_LOCK(::cs_main, return m_disabled)) {
std::this_thread::sleep_for(std::chrono::seconds(10));
// Background chainstate has reached the snapshot base block, so exit.
// Restart indexes to resume indexing for all blocks unique to the snapshot
```
ACKs for top commit:
maflcko:
review-only ACK 19f4a7c95a
achow101:
ACK 19f4a7c95a
pablomartin4btc:
tACK 19f4a7c95a
furszy:
Code review ACK [19f4a7c](19f4a7c95a).
Tree-SHA512: 70dad3795988956c5e20f2d2d895fb56c5e3ce257c7547d3fd729c88949f0e24cb34594da1537bce8794ad02b2db44e7e46e995aa32539cd4dd84c4f1d4bb1c4
Put the warning closer to where it is relevant. That is, put it close to
the functions that actually do unconditional logging.
Also, remove a stray empty line.
fa5bc450d5 util: Use compile-time check for LogConnectFailure (MarcoFalke)
fa7087b896 util: Use compile-time check for FatalErrorf (MarcoFalke)
faa62c0112 util: Add ConstevalFormatString (MarcoFalke)
fae7b83eb5 lint: Remove forbidden functions from lint-format-strings.py (MarcoFalke)
Pull request description:
The `test/lint/lint-format-strings.py` was designed to count the number of format specifiers and assert that they are equal to the number of parameters passed to the format function. The goal seems reasonable, but the implementation has many problems:
* It is written in Python, meaning that C++ code can not be parsed correctly. Currently it relies on brittle regex and string parsing.
* Apart from the parsing errors, there are also many logic errors. For example, `count_format_specifiers` allows a mix of positional specifiers and non-positional specifiers, which can lead to runtime format bugs. Also, `count_format_specifiers` silently skipped over "special" format specifiers, which are valid in tinyformat, which again can lead to runtime format bugs being undetected.
* The brittle logic has a history of breaking in pull requests that are otherwise fine. This causes the CI to fail and the pull request being blocked from progress until the bug in the linter is fixed, or the code is rewritten to work around the bug.
* It is only run in the CI, or when the developer invokes the script. It would be better if the developer got the error message at compile-time, directly when writing the code.
Fix all issues by using a `consteval` checked format string in `FatalErrorf` and `LogConnectFailure`.
This is the first step toward https://github.com/bitcoin/bitcoin/issues/30530 and a follow-up will apply the approach to the other places.
ACKs for top commit:
stickies-v:
re-ACK fa5bc450d5
l0rinc:
ACK fa5bc450d5
hodlinator:
ACK fa5bc450d5
ryanofsky:
Code review ACK fa5bc450d5
Tree-SHA512: d6189096b16083143687ed1b1559cf4f92f97dd87bc5d00673e44f4fb9fce7bb7b215cfdfc39b6e6a24f0b75a79a03ededce966639e554f7172e1fc22cf015ae
1eac96a503 Compare FromUserHex result against other hex validators and parsers (Lőrinc)
19947863e1 Use BOOST_CHECK_EQUAL for optional, arith_uint256, uint256, uint160 (Lőrinc)
743ac30e34 Add std::optional support to Boost's equality check (Lőrinc)
Pull request description:
Enhanced `FromUserHex` coverage by:
* Added `std::optional` support to `BOOST_CHECK_EQUAL`, allowing direct comparisons of `std::optional<T>` with other `T` expected values.
* Increased fuzz testing for hex parsing to validate against other hex validators and parsers.
----
* Use BOOST_CHECK_EQUAL for https://github.com/bitcoin/bitcoin/pull/30569#discussion_r1706637780 arith_uint256, uint256, uint160
Example error before:
> unknown location:0: fatal error: in "validation_chainstatemanager_tests/chainstatemanager_args": std::bad_optional_access: bad_optional_access
test/validation_chainstatemanager_tests.cpp:781: last checkpoint
after:
> test/validation_chainstatemanager_tests.cpp:801: error: in "validation_chainstatemanager_tests/chainstatemanager_args": check set_opts({"-assumevalid=0"}).assumed_valid_block == uint256::ZERO has failed [std::nullopt != 0000000000000000000000000000000000000000000000000000000000000000]
ACKs for top commit:
stickies-v:
re-ACK 1eac96a503
ryanofsky:
Code review ACK 1eac96a503. Only changes since last review were auto type and fuzz test tweaks.
hodlinator:
ACK 1eac96a503
Tree-SHA512: f1d2c65f0ee4e97830700be5b330189207b11ed0c89a8cebf0f97d43308402a6b3732e10130c79a0c044f7d2eeabfb5359990825aadf02c4ec19428dcd982b00
0dd16d7118 build: Add a pkg-config file for libbitcoinkernel (TheCharlatan)
45be32f838 build: Produce a usable static kernel library (TheCharlatan)
Pull request description:
Since the move to cmake, the kernel static library that is installed after a cmake --install build is unusable. It lacks symbols for the internal libraries, besides those defined in the kernel library target.
Fix this by explicitly installing all the required internal static libraries. To make usage of these installed libraries easy, add a pkg-config file that can be used during linking.
This patch can be tested with:
```
cmake -B build -DBUILD_SHARED_LIBS=OFF -DBUILD_KERNEL_LIB=ON
cmake --build build
cmake --install build
g++ -std=c++20 -o test_chainstate src/bitcoin-chainstate.cpp -I/home/drgrid/bitcoin/src $(pkg-config --libs --static libbitcoinkernel)
```
Attempts to solve #30801
ACKs for top commit:
hebasto:
ACK 0dd16d7118.
fanquake:
ACK 0dd16d7118 - this looks like a good place to start.
ryanofsky:
Code review ACK 0dd16d7118
Tree-SHA512: 92f7bc959584bdc595f4aa6d0ab133355481075fe8564224fd7ac122fd7bdd75f98cf26ef0a6a7d84fd552d2258ddca1b674eca91122469a58bacc5f0a0ec2ef
`m_is_inbound` cannot be changed throughout the life of a `Peer`. However, we
are currently storing it in `CNodeState`, which requires locking `cs_main` in
order to access it. This can be moved to the outside scope and only require
`m_peer_mutex`.
This is a refactor in preparation for Erlay reworks.
72b46f28bf test: fix exclude parsing for functional runner (Max Edwards)
Pull request description:
This restores previous behaviour of being able to exclude a test by name without having to specify .py extension.
It was noticed in https://github.com/bitcoin/bitcoin/issues/30851 that tests were no longer being excluded.
PR https://github.com/bitcoin/bitcoin/pull/30244 introduced being able to exclude a specific tests based on args (such as `--exclude "rpc_bind.py --ipv6`) but it made the wrong assumption that test names intended to be excluded would include the .py extension.
The following https://github.com/bitcoin/bitcoin/pull/30244#issuecomment-2344009687 shows that this is not how the `--exclude` flag was used in CI.
https://github.com/bitcoin/bitcoin/pull/30244#issuecomment-2344009687 gave three examples of `--exclude` being used in CI so I compared the number of tests that the runner would run for these three examples in three situations, before #30244 was introduced, in master today and with this PR applied.
Example:
`--previous-releases --coverage --extended --exclude feature_dbcrash`
Test count:
Before #30244 introduced: 314
Master: 315
With this PR: 314
Example:
`--exclude feature_init,rpc_bind,feature_bind_extra`
Test count:
Before #30244 introduced: 306
Master 311
With this PR: 306
Example:
`--exclude rpc_bind,feature_bind_extra`
Before #30244 introduced: 307
Master 311
With this PR: 307
I've also tested that the functionality introduced with #30244 remains and we can still exclude specific tests by argument.
ACKs for top commit:
maflcko:
review ACK 72b46f28bf
willcl-ark:
ACK 72b46f28bf
Tree-SHA512: 37c0e3115f4e3efdf9705f4ff8cd86a5cc906aacc1ab26b0f767f5fb6a953034332b29b0667073f8382a48a2fe9d649b7e60493daf04061260adaa421419d8c8
23eedc5d1e build: Skip secp256k1 ctime tests when tests are not being built (Hennadii Stepanov)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/pull/30791#issuecomment-2340860619:
> Building with a fuzz engine fails, because the ctime tests are auto-detected in cmake, based on whether or not valgrind-devel is installed or not.
ACKs for top commit:
maflcko:
re-review ACK 23eedc5d1e
fanquake:
ACK 23eedc5d1e
Tree-SHA512: bfc0f2798acd36be9c52073d578b42c002606c60ef3fe8ef633eaea4f5382a3e9765d31637e4c25d8b71fd70473b29c24af4732e55e5183f27b48725b61fa15b
2ad560139b Remove unused src_dir param from run_tests (Lőrinc)
Pull request description:
The `src_dir` usage was removed in a8a2e364ac (diff-437d7f6e9f2229879b60aae574a8217f14c643bbf3cfa9225d8011d6d52df00cL598), making the parameter unused.
Top commit has no ACKs.
Tree-SHA512: 1fd8b93811b4ab467ba5a160a4fe204e9606e1bf237c7595ed6f8b7821cf59d2a776c0e1e154852a45b2a35e5bdbd8996314e4f63a9c750f21b9a17875cb636a
This makes it easier to overwrite the value.
Also, drop the dot in the CCACHE_DIR fallback value, because the folder
in the scratch dir does not need and probably should not be hidden.
f15e817811 build: add more CMake presets (dev-mode, libfuzzer, libfuzzer-nosan) (Pieter Wuille)
Pull request description:
Add three more cmake presets to the project-wide `CMakePresets.json` file:
* `dev-mode`: enables all features and dependencies
* `libfuzzer`: builds for fuzzing with libfuzzer and the typical sanitizers (but not the optional ones that require suppressions) enabled.
* `libfuzzer-nosan`: builds for fuzzing with libfuzzer and no (other) sanitizers
... and then uses these in some documentation.
ACKs for top commit:
ryanofsky:
Code review ACK f15e817811. This change is much needed to simplify my command line.
TheCharlatan:
ACK f15e817811
Tree-SHA512: a5f67bb7119fd36832ca5eb7189db04bfaf88f954aa461bfb2aeb866469057b0d0272835c418bc3a264c30dd8fba6d2e2cc8a6741a889f28f52c1c09b3ba9704
be4f78275f contrib: test for FORTIFY_SOURCE in security-check.py (fanquake)
Pull request description:
Test for the existence of fortified functions in the ELF release binaries.
Currently skips `bitcoin-util` and checks for RISC-V.
ACKs for top commit:
TheCharlatan:
ACK be4f78275f
Tree-SHA512: decea5f359f1e673aa0119916f674f409a13b69db7da366cd95c1540201e117ff5a979da67bc2517fe786c2ac23d1006a9aaf662d7eadeec35da6aae4998c065
7b04fabe2d build: Introduce "Kernel" installation component (Hennadii Stepanov)
Pull request description:
This PR enables building and installing only `libbitcoinkernel`, without the need to disable other targets during the project build system generation:
```
$ rm -rf build && cmake -B build -DBUILD_KERNEL_LIB=ON
$ cmake --build build --target bitcoinkernel
$ cmake --install build --component Kernel --prefix /home/hebasto/INSTALL
-- Install configuration: "RelWithDebInfo"
-- Installing: /home/hebasto/INSTALL/lib/libbitcoinkernel.so
```
Please note, that only the `bitcoinkernel` target is being built.
Related to https://github.com/bitcoin/bitcoin/issues/30801 and https://github.com/bitcoin/bitcoin/pull/30814.
ACKs for top commit:
TheCharlatan:
ACK 7b04fabe2d
ryanofsky:
Code review ACK 7b04fabe2d
Tree-SHA512: eac114dde059e47c91938a4a9108fc0fc693b5342ed3b6ecb971615be8ad3225b9985aae12d6ad18e673edf1bd39a5ecf259c1b61734f221669091bf2ce93a67
5ba03e7d35 build: Use CMake's default permissions in macOS `deploy` target (Hennadii Stepanov)
Pull request description:
This PR ensures that the file permissions in macOS `zip` archives are independent of the user's `umask` value.
Fixes https://github.com/bitcoin/bitcoin/issues/30815.
ACKs for top commit:
fanquake:
ACK 5ba03e7d35 - I'm going to merge this now so we return to usable (comparable) guix builds.
Tree-SHA512: 78f724cd3ffd5c1fd5fc1b4832f1e8154c62723f3de5ac9599f44715cbd08a3dfbb806801411c55069773d2e34c9f8cab25585dbad2f032c36b68dd83cb51847
c45186ca54 ci: Switch from `make` to `cmake --build` (Hennadii Stepanov)
6e5f33af58 ci: Handle log files regardless of CMake's version (Hennadii Stepanov)
Pull request description:
This PR addresses the change in logging that [happened](https://cmake.org/cmake/help/latest/release/3.26.html#configure-log) in CMake 3.26.
Additionally, the `make` invocation replaced with `cmake --build`.
Here are examples of the CI logs:
- for a an error during the build system generation: https://cirrus-ci.com/build/5210987156996096
- for a compiler error: https://cirrus-ci.com/build/4617660913156096
ACKs for top commit:
maflcko:
review ACK c45186ca54
fanquake:
ACK c45186ca54
Tree-SHA512: 2096f08c482ab9e10056cd4ec694ce40996243e2a1af2212dfff8cccbf0f51391d9a3dc396f7bba4f2877072a13a42bf667a02a44eab44e917aafb14d04e8e39
b07fe666f2 build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake` (Hennadii Stepanov)
Pull request description:
This PR aims to reduce build time by replacing multiple `file(WRITE|APPEND ...)` commands with a single `file(WRITE ...)` command.
Due to differences in implementation (e.g., filesystem design, system calls, caching), a noticeable improvement in build time is observed only on Windows.
Additionally, the code has been refactored to remove the `remainder` local variables.
ACKs for top commit:
sipsorcery:
tACK b07fe666f2
maflcko:
review ACK b07fe666f2
TheCharlatan:
ACK b07fe666f2
Tree-SHA512: 6ed3ae8fe7d8859af38d83918eddf7cb318607787863b95589f4a7a45a36f8c4bd1c01e366078d0515115c121bc857dc63471e52ff26fc49edbc8bb69875e947
0037d53d1a build: Fix `ENABLE_WALLET` option (Hennadii Stepanov)
Pull request description:
The removed commands were left over from the transition from autodetection to explicit options in the CMake staging branch. These commands prevented the `-DENABLE_WALLET=OFF` option from being work properly when building with depends.
How to test:
```
$ make -C depends NO_QT=1
```
On the master branch @ c66c68345e:
```
$ rm -rf build && cmake -B build --toolchain depends/x86_64-pc-linux-gnu/toolchain.cmake -DENABLE_WALLET=OFF
< snip >
Optional features:
wallet support ...................... ON
- descriptor wallets (SQLite) ...... ON
- legacy wallets (Berkeley DB) ..... ON
external signer ..................... ON
< snip >
```
With this PR:
```
$ rm -rf build && cmake -B build --toolchain depends/x86_64-pc-linux-gnu/toolchain.cmake -DENABLE_WALLET=OFF
< snip >
Optional features:
wallet support ...................... OFF
external signer ..................... ON
< snip >
ACKs for top commit:
maflcko:
review ACK 0037d53d1a
kevkevinpal:
ACK [0037d53](0037d53d1a)
pablomartin4btc:
tACK 0037d53d1a
Tree-SHA512: 0eb14ef104f12a4205172d646c2af820e04514286b5b9a4ceb59c248ce880198dd4051d669098c46c0c0dce069bb60899d90509bbcae65cbeb958e52564fe920
992f83bb6f test: add coverage for assumeUTXO honest peers disconnection (furszy)
6d5812e5c8 assumeUTXO: fix peers disconnection during sync (furszy)
Pull request description:
Because AssumeUTXO nodes prioritize tip synchronization, they relay their local
address through the network before completing the background chain sync.
This, combined with the advertising of full-node service (`NODE_NETWORK`), can
result in an honest peer in IBD connecting to the AssumeUTXO node (while syncing)
and requesting an historical block the node does not have. This behavior leads to
an abrupt disconnection due to perceived unresponsiveness from the AssumeUTXO
node.
This lack of response occurs because nodes ignore `getdata` requests when they do
not have the block data available (further discussion can be found in #30385).
Fix this by refraining from signaling full-node service support while the
background chain is being synced. During this period, the node will only
signal `NODE_NETWORK_LIMITED` support. Then, full-node (`NODE_NETWORK`)
support will be re-enabled once the background chain sync is completed.
Thanks mzumsande for a post-#30385 convo too.
Testing notes:
Just cherry-pick the second commit (bb08c22) on master.
It will fail there, due to the IBD node requesting historical blocks to the snapshot
node - which is bad because the snapshot node will ignore the requests and
stall + disconnect after some time.
ACKs for top commit:
achow101:
ACK 992f83bb6f
naumenkogs:
ACK 992f83bb6f
mzumsande:
ACK 992f83bb6f
Tree-SHA512: fef525d1cf3200c2dd89a346be9c82d77f2e28ddaaea1f490a435e180d1a47a371cadea508349777d740ab56e94be536ad8f7d61cc81f6550c58b609b3779ed3
4b1ce3cac8 docs: updated developer notes for --with-sanitizers to -DSANITIZERS and removed resource for -fsanitze flags (kevkevinpal)
Pull request description:
In the developer notes we are incorrectly using the Autotools `--with-sanitizers` configure flag which we should now be using `cmake -B build -DSANITIZERS=<values>` instead now
ACKs for top commit:
maflcko:
review ACK 4b1ce3cac8
achow101:
ACK 4b1ce3cac8
pablomartin4btc:
ACK 4b1ce3cac8
Tree-SHA512: 029d55d802f6b4a85f3541f9a23e9ac85e6c590e91081204bfa737169138f61877883db51ad99cd8b802b0737eec35df5a33a5351b3e6b2f180f3ad0282d3616
Extracted existing serialization to append size & data in separate private methods to clarify that it does more than just a simple data insertion.
* the C style casts were changed to static_cast
* `unsigned char` and `uint8_t` were changed to value_type for forward compatibility
* `data + sizeof(data)` was changed to `std::cend`
* data insertion (in AppendData) relies on pointer arithmetic now to enable both `std::span<const value_type>` and `std::span<const std::byte>` operators
* use uint32_t for data size instead of size_t
* used span instead of raw pointers in the new methods
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
When compiling with GCC 12.2, both `-Warray-bounds` and `-Wstringop-overflow` warnings were triggered in the `prevector::insert` method during CScript prevector operations.
GCC incorrectly assumed that operator new could modify the state of class members, leading to false positives during the memmove operation.
Following the approach in https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;h=cca06f0d6d76b0, we introduced local copies for the destination pointer in memmove operations. This prevents GCC from misinterpreting memory manipulation as unsafe.
A minimal reproducer triggering this issue in GCC 12.2 and passing in GCC 12.3 can be found at https://godbolt.org/z/8r9TKKoxv.
-------
Full error (with changes from the next commit as well):
```
In file included from /ci_container_base/src/script/script.h:11,
from /ci_container_base/src/primitives/transaction.h:11,
from /ci_container_base/src/primitives/block.h:9,
from /ci_container_base/src/kernel/chainparams.h:11,
from /ci_container_base/src/kernel/chainparams.cpp:6:
In member function ‘void prevector<N, T, Size, Diff>::fill(T*, InputIterator, InputIterator) [with InputIterator = const unsigned char*; unsigned int N = 28; T = unsigned char; Size = unsigned int; Diff = int]’,
inlined from ‘void prevector<N, T, Size, Diff>::insert(iterator, InputIterator, InputIterator) [with InputIterator = const unsigned char*; unsigned int N = 28; T = unsigned char; Size = unsigned int; Diff = int]’ at /ci_container_base/src/prevector.h:395:13,
inlined from ‘void CScript::AppendData(const prevector<28, unsigned char>::value_type*, size_t)’ at /ci_container_base/src/script/script.h:439:15,
inlined from ‘CScript& CScript::operator<<(std::span<const std::byte>)’ at /ci_container_base/src/script/script.h:496:17,
inlined from ‘CBlock CreateGenesisBlock(uint32_t, uint32_t, uint32_t, int32_t, const CAmount&)’ at /ci_container_base/src/kernel/chainparams.cpp:76:54:
/ci_container_base/src/prevector.h:216:13: error: writing 65 bytes into a region of size 32 [-Werror=stringop-overflow=]
216 | new(static_cast<void*>(dst)) T(*first);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/ci_container_base/src/kernel/chainparams.cpp: In function ‘CBlock CreateGenesisBlock(uint32_t, uint32_t, uint32_t, int32_t, const CAmount&)’:
/ci_container_base/src/kernel/chainparams.cpp:76:49: note: destination object ‘<anonymous>’ of size 32
76 | const CScript genesisOutputScript = CScript() << "04678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5f"_hex << OP_CHECKSIG;
| ^
In file included from /usr/lib/gcc/x86_64-w64-mingw32/12-posix/include/c++/cstring:42,
from /ci_container_base/src/crypto/common.h:11,
from /ci_container_base/src/uint256.h:9,
from /ci_container_base/src/consensus/params.h:9,
from /ci_container_base/src/kernel/chainparams.h:9:
In function ‘void* memmove(void*, const void*, size_t)’,
inlined from ‘void prevector<N, T, Size, Diff>::insert(iterator, InputIterator, InputIterator) [with InputIterator = const unsigned char*; unsigned int N = 28; T = unsigned char; Size = unsigned int; Diff = int]’ at /ci_container_base/src/prevector.h:393:16,
inlined from ‘void CScript::AppendData(const prevector<28, unsigned char>::value_type*, size_t)’ at /ci_container_base/src/script/script.h:439:15,
inlined from ‘CScript& CScript::operator<<(std::span<const std::byte>)’ at /ci_container_base/src/script/script.h:496:17,
inlined from ‘CBlock CreateGenesisBlock(uint32_t, uint32_t, uint32_t, int32_t, const CAmount&)’ at /ci_container_base/src/kernel/chainparams.cpp:76:54:
/usr/share/mingw-w64/include/string.h:214:33: warning: ‘void* __builtin_memmove(void*, const void*, long long unsigned int)’ offset [65, 35] is out of the bounds [0, 32] of object ‘<anonymous>’ with type ‘CScript’ [-Warray-bounds]
214 | return __builtin___memmove_chk(__dst, __src, __n, __mingw_bos(__dst, 0));
| ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/ci_container_base/src/kernel/chainparams.cpp: In function ‘CBlock CreateGenesisBlock(uint32_t, uint32_t, uint32_t, int32_t, const CAmount&)’:
/ci_container_base/src/kernel/chainparams.cpp:76:49: note: ‘<anonymous>’ declared here
76 | const CScript genesisOutputScript = CScript() << "04678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5f"_hex << OP_CHECKSIG;
| ^
```
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Example error before:
> unknown location:0: fatal error: in "validation_chainstatemanager_tests/chainstatemanager_args": std::bad_optional_access: bad_optional_access
test/validation_chainstatemanager_tests.cpp:781: last checkpoint
after:
> test/validation_chainstatemanager_tests.cpp:801: error: in "validation_chainstatemanager_tests/chainstatemanager_args": check set_opts({"-assumevalid=0"}).assumed_valid_block == uint256::ZERO has failed [std::nullopt != 0000000000000000000000000000000000000000000000000000000000000000]
Also added extra minimum_chainwork test to make it symmetric with assumevalid
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
082779d606 test: Add explicit onion bind to p2p_permissions (Ava Chow)
Pull request description:
When the bind option is replaced in the bitcoin.conf, bitcoind will attempd to bind to the default tor listening port. If another bitcoind is running that is already bound to that port, the bind will fail which, since #22729, causes the test to fail.
This failure can be avoided by explicitly binding the tor port when the bind is removed.
ACKs for top commit:
tdb3:
ACK 082779d606
theStack:
re-ACK 082779d606
glozow:
ACK 082779d606
Tree-SHA512: 4acb69ea2e00aeacf9e7c9ab9595ceaf0e0d2adbd795602034b2184197d9bad54c7bc9f3da43ef9c52a71869fe96ba8c87fc5b7c37880f258f5a2aaab2b4046c
The removed commands were left over from the transition from
autodetection to explicit options. These commands prevented the
`-DENABLE_WALLET=OFF` option from being work properly when building with
depends.
Exercising and verifying the following points:
1. An IBD node can sync headers from an AssumeUTXO node at
any time.
2. IBD nodes do not request historical blocks from AssumeUTXO
nodes while they are syncing the background-chain.
3. The assumeUTXO node dynamically adjusts the network services
it offers according to its state.
4. IBD nodes can fully sync from AssumeUTXO nodes after they
finish the background-chain sync.
Because AssumeUTXO nodes prioritize tip synchronization, they relay their local
address through the network before completing the background chain sync.
This, combined with the advertising of full-node service (NODE_NETWORK), can
result in an honest peer in IBD connecting to the AssumeUTXO node (while syncing)
and requesting an historical block the node does not have. This behavior leads to
an abrupt disconnection due to perceived unresponsiveness (lack of response)
from the AssumeUTXO node.
This lack of response occurs because nodes ignore getdata requests when they do
not have the block data available (further discussion can be found in PR 30385).
Fix this by refraining from signaling full-node service support while the
background chain is being synced. During this period, the node will only
signal 'NODE_NETWORK_LIMITED' support. Then, full-node ('NODE_NETWORK')
support will be re-enabled once the background chain sync is completed.
When the bind option is replaced in the bitcoin.conf, bitcoind will
attempd to bind to the default tor listening port. If another bitcoind
is running that is already bound to that port, the bind will fail which,
since #22729, causes the test to fail.
This failure can be avoided by explicitly binding the tor port when the
bind is removed.
43cd83b0c7 test: move uint256_tests/operator_with_self to arith_uint256_tests (stickies-v)
c6c994cb2b test: remove test-only uint160S (stickies-v)
62cc4656e2 test: remove test-only uint256S (stickies-v)
adc00ad728 test: remove test-only arith_uint256S (stickies-v)
f51b237723 refactor: rpc: use uint256::FromHex for ParseHashV (stickies-v)
Pull request description:
_Continuation of #30569._
Since fad2991ba0, `uint256S()` has been [deprecated](fad2991ba0 (diff-800776e2dda39116e889839f69409571a5d397de048a141da7e4003bc099e3e2R138)) because it is less robust than the `base_blob::FromHex()` introduced in https://github.com/bitcoin/bitcoin/pull/30482. Specifically, it tries to recover from length-mismatches, recover from untrimmed whitespace, 0x-prefix and garbage at the end, instead of simply requiring exactly 64 hex-only characters. (see also https://github.com/bitcoin/bitcoin/pull/30532)
This PR removes `uint256S()` (and `uint160S()`) completely, with no non-test behaviour change.
Specifically, the main changes in this PR are:
- the (minimal) last non-test usage of `uint256S()` in `ParseHashV()` is removed without behaviour change, which can partially be verified by cherry-picking and/or modifying [this test commit](1f2b0fa86d)).
- the test usage of `uint{160,256}S()` is removed, largely replacing it with `uint{160,256}::FromHex()` where applicable, potentially modifying the test by removing non-hex characters or dropping the test entirely if removing non-hex characters makes it redundant
- the now unused `uint{160,256}S()` functions are removed completely.
- unit test coverage on converting `uint256` <-> `arith_uint256` through `UintToArith256()` and `ArithToUint256()` is beefed up, and `arith_uint256` tests are moved to `arith_uint256_tests.cpp`, removing the `uint256_tests.cpp` dependency on `uint256h`, mirroring how the code is structured.
_Note: `uint256::FromUserHex()` exists to more leniently construct uint256 from user input, allowing "0x" prefixes and too-short-input, as safer alternative to `uint256S()` where necessary._
ACKs for top commit:
l0rinc:
reACK 43cd83b0c7
hodlinator:
re-ACK 43cd83b0c7
ryanofsky:
Code review ACK 43cd83b0c7. Only code change is a small refactoring which looks good. The rest of the PR is all test changes, which I only lightly reviewed, but seem to be positive and do what's described
Tree-SHA512: 48147a4c6af671597df0f72c1b477ae4631cd2cae4645ec54d0e327611ff302c9899e344518c81242cdde82930f6ad23a3a7e6e0b80671816e9f457b9de90a5c
d4c7c4009d init: error out if -maxconnections is negative (Sergi Delgado Segura)
c773649481 init: improves file descriptors accounting and docs (Sergi Delgado Segura)
29008a7ff4 init: fixes fd accounting regarding poll/select (Sergi Delgado Segura)
Pull request description:
The current logic for file descriptor accounting is pretty convoluted and hard to follow. This is partially caused by the lack of documentation plus non-intuitive variable naming (which was more intuitive when fewer things were accounted for, but
hasn't aged well). This has led to this accounting being error-prone and hard to maintain (as shown in the first commit of this PR).
Redefine some of the constants, plus document what are we accounting for so this can be extended more easily
Fixes#18911
ACKs for top commit:
sr-gi:
> ACK [d4c7c40](d4c7c4009d)
naumenkogs:
ACK d4c7c4009d
vasild:
ACK d4c7c4009d
TheCharlatan:
ACK d4c7c4009d
Tree-SHA512: 1524d10c8ad8f354f6ab9c244699adbcdae2dd7aba37de5b8f9e177c629e8a2cce0f6e8117e076dde3a592f5283bd30a4201db96a3c011e335c02d1fde7414bc
30073e6b3a multiprocess: Add -ipcbind option to bitcoin-node (Russell Yanofsky)
73fe7d7230 multiprocess: Add unit tests for connect, serve, and listen functions (Ryan Ofsky)
955d4077aa multiprocess: Add IPC connectAddress and listenAddress methods (Russell Yanofsky)
4da20434d4 depends: Update libmultiprocess library for CustomMessage function and ThreadContext bugfix (Ryan Ofsky)
Pull request description:
Add `-ipcbind` option to `bitcoin-node` to make it listen on a unix socket and accept connections from other processes. The default socket path is `<datadir>/node.sock`, but this can be customized.
This option lets potential wallet, gui, index, and mining processes connect to the node and control it. See examples in #19460, #19461, and #30437.
Motivation for this PR, in combination with #30510, is be able to release a bitcoin core node binary that can generate block templates for a separate Stratum v2 mining service, like the one being implemented in https://github.com/Sjors/bitcoin/pull/48, that connects over IPC.
Other things to know about this PR:
- While the `-ipcbind` option lets other processes to connect to the `bitcoin-node` process, the only thing they can actually do after connecting is call methods on the [`Init`](https://github.com/bitcoin/bitcoin/blob/master/src/ipc/capnp/init.capnp#L17-L20) interface which is currently very limited and doesn't do much. But PRs [#30510](https://github.com/bitcoin/bitcoin/pull/30510), [#29409](https://github.com/bitcoin/bitcoin/pull/29409), and [#10102](https://github.com/bitcoin/bitcoin/pull/10102) expand the `Init` interface to expose mining, wallet, and gui functionality respectively.
- This PR is not needed for [#10102](https://github.com/bitcoin/bitcoin/pull/10102), which runs GUI, node, and wallet code in different processes, because [#10102](https://github.com/bitcoin/bitcoin/pull/10102) does not use unix sockets or allow outside processes to connect to existing processes. [#10102](https://github.com/bitcoin/bitcoin/pull/10102) lets parent and child processes communicate over internal socketpairs, not externally accessible sockets.
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
achow101:
ACK 30073e6b3a
TheCharlatan:
Re-ACK 30073e6b3a
itornaza:
Code review ACK 30073e6b3a
Tree-SHA512: 2b766e60535f57352e8afda9c3748a32acb5a57b2827371b48ba865fa9aa1df00f340732654f2e300c6823dbc6f3e14377fca87e4e959e613fe85a6d2312d9c8
c2b779da4e refactor: Manage dumptxoutset RAII classes with std::optional (Fabian Jahr)
4b5bf335ad test: Add coverage for failing dumptxoutset behavior (Fabian Jahr)
Pull request description:
This adds a test that checks that network activity is not suspended if dumptxoutset fails in the middle of its process which is implemented with the `NetworkDisable` RAII class. I would have liked to add coverage for the `TemporaryRollback` RAII class but that seems a lot more tricky since the failure needs to happen at some point after the rollback and on the scale of our test chain here I couldn't find a way to do it yet. This was requested by pablomartin4btc here: https://github.com/bitcoin/bitcoin/pull/30808#pullrequestreview-2280450117. To test the test you can comment out the content of the destructor of `NetworkDisable`.
It also addresses the feedback by ryanofsky to use `std::optional` instead of `std::unique_ptr` for the management of the RAII object: https://github.com/bitcoin/bitcoin/pull/30808#discussion_r1744149228
ACKs for top commit:
achow101:
ACK c2b779da4e
pablomartin4btc:
cr & tACK c2b779da4e
tdb3:
ACK c2b779da4e
BrandonOdiwuor:
Code Review ACK c2b779da4e
theStack:
ACK c2b779da4e
Tree-SHA512: 9556e75014a2599bb870b70faf887608b332f2312626333f771d4ec11c04f863a2cf17e223ec473d4e8b0c9e8008394a4e0c321561f7ef3a2eec713dcfaea58a
ee47ca29d6 init: fix fatal error on '-wallet' negated option value (furszy)
Pull request description:
Currently, if users provide a double negated value such as '-nowallet=0' or a non-boolean
convertible value to a negated option such as '-nowallet=not_a_boolean', the initialization
process results in a fatal error, causing an unclean shutdown and displaying a poorly
descriptive error message:
"JSON value of type bool is not of expected type string." (On bitcoind. The GUI
does not display any error msg - upcoming PR -).
This PR fixes the issue by ensuring that only string values are returned in the
the "wallet" settings list, failing otherwise. It also improves the clarity of the
returned error message.
Note:
This bug was introduced in https://github.com/bitcoin/bitcoin/pull/22217. Where the `GetArgs("-wallet")` call was
replaced by `GetSettingsList("-wallet")`.
ACKs for top commit:
achow101:
ACK ee47ca29d6
ryanofsky:
Code review ACK ee47ca29d6, just adding the suggested test since last review
TheCharlatan:
ACK ee47ca29d6
ismaelsadeeq:
Tested ACK ee47ca29d6
Tree-SHA512: 5f01076f74a048019bb70791160f0accc2db7a457d969cb23687bed81ccbbdec1dda68311e7c6e2dd56250e23e8d926d4066e5014b2a99a2fc202e24ed264fbd
27c976d11a fix: increase consistency of rpcauth parsing (tdb3)
2ad3689512 test: add norpcauth test (tdb3)
67df0dec1a test: blank rpcauth CLI interaction (tdb3)
ecc98ccff2 test: add cases for blank rpcauth (tdb3)
Pull request description:
The current `rpcauth` parsing behavior is inconsistent and unintuitive (see https://github.com/bitcoin/bitcoin/pull/29141#issuecomment-1972085251 and additional details below).
The current behavior inconsistently treats empty `rpcauth` as an error (or not) depending on the location within CLI/bitcoin.conf and the location of adjacent valid `rpcauth` params.
Empty `rpcauth` is now consistently treated as an error and prevents bitcoind from starting.
Continuation of the upforgrabs PR #29141.
### Additional details:
Current `rpcauth` behavior is nonsensical:
- If an empty `rpcauth` argument was specified as the last command line argument, it would cause all other `rpcauth` arguments to be ignored.
- If an empty `rpcauth` argument was specified on the command line followed by any nonempty `rpcauth` argument, it would cause an error.
- If an empty `rpcauth=` line was specified after non-empty rpcauth line in the config file it would cause an error.
- If an empty `rpcauth=` line in a config file was first it would cause other rpcauth entries in the config file to be ignored, unless there were `-rpcauth` command line arguments and the last one was nonempty, in which case it would cause an error.
New behavior is simple:
- If an empty rpcauth config line or command line argument is used it will cause an error
ACKs for top commit:
naiyoma:
Tested ACK [27c976d11a)
achow101:
ACK 27c976d11a
ryanofsky:
Code review ACK 27c976d11a. Since last review commit message was just tweaked to clarify previous behavior.
Tree-SHA512: af2e9dd60d1ad030409ae2c3805ab139c7435327823d9f8bbeede815f376cb696a5929b08a6e8c8b5f7278ed49cfb231789f9041bd57f1f03ec96501b669da5b
30803a35d5 cmake: decouple FORTIFY_SOURCE check from Debug build type (fanquake)
Pull request description:
`FORTIFY_SOURCE` should be used if `ENABLE_HARDENING=ON` and optimisations are being used. This should not be coupled to any particular build type, because even if the build type is `Debug`, optimisations might still be in use.
Fixes: #30800.
Also somewhat of a followup to https://github.com/bitcoin/bitcoin/pull/30778#discussion_r1742257436.
ACKs for top commit:
ryanofsky:
Code review ACK 30803a35d5
TheCharlatan:
ACK 30803a35d5
Tree-SHA512: 298f8805a5bb2f1ff54e51ea31324d712c2070cc3eba26561c31001ace4bfa37ae6d18531cbd45e2faf610a0a1b83b420fcde6e329e17f02b021d26563583913
1f054eca4e cmake: add USE_SOURCE_PERMISSIONS to all configure_file usage (fanquake)
Pull request description:
`USE_SOURCE_PERMISSIONS` is the default, so this should not change behaviour. However, being explicit makes it clear what we are doing.
Related to #30815.
See https://cmake.org/cmake/help/latest/command/configure_file.html#options.
ACKs for top commit:
hebasto:
ACK 1f054eca4e.
TheCharlatan:
ACK 1f054eca4e
Tree-SHA512: efed91b8aa0813100304ee58e169bbf5cfbb7db465ec4f7e6cbbae6053f09a36757bf96b4d1cb9ddf4c1cab0ceb3ab18805ebefa122535518ffb501c9b489d3d
7a669fde18 docs: Fix minor typo (Gutflo)
Pull request description:
Fix typo in doc/build-windows-msvc.md:
- "Micsrosoft" -> Microsoft
No test required.
ACKs for top commit:
l0rinc:
ACK 7a669fde18
Tree-SHA512: fd3815ebf449885e8a27d4f21e61a4482a7983ccfe40b13a4d658a304845775acd2f4f8acec2e85c24b6179223bb21baaecf1dd0a4d2921427686148ac1ed208
Following changes were made:
1) Catch and signal error for duplicate string destinations.
2) Catch and signal error for invalid value type.
3) Catch and signal error for string destination not found in tx outputs.
4) Improved 'InterpretSubtractFeeFromOutputInstructions()' code organization.
5) Added test coverage for all possible error failures.
Also, fixed two PEP 8 warnings at the 'wallet_sendmany.py' file:
- PEP 8: E302 expected 2 blank lines, found 1 at the SendmanyTest class declaration.
- PEP 8: E303 too many blank lines (2) at skip_test_if_missing_module() and set_test_params()
Since the move to cmake, the kernel static library that is installed
after a cmake --install build is unusable. It lacks symbols for the
internal libraries, besides those defined in the kernel library target.
This is because cmake, unlike the libtool archiver, does not combine
multiple static libraries into a single static library on installation.
This is likely an intentional design choice, since there were a bunch of
caveats to the way libtool calculated these libraries.
Fix this problem by installing all the required libraries. The user must
then link all of them along with the bitcoin kernel library.
move/formatting-only change.
These tests do not cover uint256, so move them to the appropriate
test suite. Additionally, apply clang-format suggestions.
uint160S is a test-only function, and testing input that
is not allowed in uint160::FromHex() is superfluous.
Tests that can't use uint160::FromHex() because they use input
with non-hex digit characters are
a) modified by dropping the non-hex digit characters if that
provides useful test coverage.
b) dropped if the test without non-hex digit characters does
not provide useful test coverage, e.g. because it is now
duplicated.
uint256S was previously deprecated for being unsafe. All non-test
usage has already been removed in earlier commits.
1. Tests now use uint256::FromHex() or other constructors wherever
possible without further modification.
2. Tests that can't use uint256::FromHex() because they use input
with non-hex digit characters are
a) modified by dropping the non-hex digit characters if that
provides useful test coverage.
b) dropped if the test without non-hex digit characters does
not provide useful test coverage, e.g. because it is now
duplicated.
Additionally, use BOOST_CHECK_EQUAL where relevant on touched lines
to make error messages more readable.
Tests that are solely testing constructing from a hex string
are dropped, others are modified to use a uint256 constructor
or the arith_uint256 uint64_t constructor.
Since an arith_uint256 can not be constructed from a string
directly, we need to ensure that test coverage on
UintToArith256(uint256::FromHex()) is not reduced.
uint256::FromHex() already has good test coverage, but
the test coverage on UintToArith256() and ArithToUint256()
is increased in this commit by upgrading the `conversion`
test case.
Moreover, since `uint256.h` does not have any dependencies
on `arith_uint256.h`, the conversion tests are moved to
`arith_uint256_tests.cpp` so the dependency can be cleaned
up entirely in a future commit.
fa9d7d5d20 test: Work around boost compilation error (MarcoFalke)
fa3ecdf778 Revert "build: work around issue with Boost <= 1.80 and Clang >= 18" (MarcoFalke)
Pull request description:
There seems to be an issue compiling the `chainstatemanager_rebalance_caches` test case with some specific versions of Boost in combination with some specific versions of Clang. For example, Boost 1.74 may fail in combination with Clang 18. [1]
The error stems from a mixed-type closeness comparison. Given that the comparison is using floating point, and isn't meant to be exact, work around the compile error by ensuring both sides of the comparison are using the same type (`double`).
This also allows to drop a previous workaround.
[1] Error:
```
In file included from /usr/include/boost/mpl/integral_c.hpp:32:
/usr/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for the enumeration type 'udt_builtin_mixture_enum' [-Wenum-constexpr-conversion]
73 | typedef AUX_WRAPPER_INST( BOOST_MPL_AUX_STATIC_CAST(AUX_WRAPPER_VALUE_TYPE, (value - 1)) ) prior;
| ^
/usr/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
24 | # define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
| ^
In file included from ../../../src/test/validation_chainstatemanager_tests.cpp:8:
In file included from ../../../src/node/chainstatemanager_args.h:9:
In file included from ../../../src/validation.h:28:
In file included from ../../../src/txmempool.h:26:
In file included from /usr/include/boost/multi_index/hashed_index.hpp:38:
In file included from /usr/include/boost/multi_index/detail/node_handle.hpp:22:
In file included from /usr/include/boost/multi_index_container_fwd.hpp:18:
In file included from /usr/include/boost/multi_index/indexed_by.hpp:17:
In file included from /usr/include/boost/mpl/vector.hpp:36:
In file included from /usr/include/boost/mpl/vector/vector20.hpp:18:
In file included from /usr/include/boost/mpl/vector/vector10.hpp:18:
In file included from /usr/include/boost/mpl/vector/vector0.hpp:24:
In file included from /usr/include/boost/mpl/vector/aux_/clear.hpp:18:
In file included from /usr/include/boost/mpl/vector/aux_/vector0.hpp:22:
In file included from /usr/include/boost/mpl/vector/aux_/iterator.hpp:19:
In file included from /usr/include/boost/mpl/plus.hpp:19:
In file included from /usr/include/boost/mpl/aux_/arithmetic_op.hpp:17:
In file included from /usr/include/boost/mpl/integral_c.hpp:32:
/usr/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for the enumeration type 'int_float_mixture_enum' [-Wenum-constexpr-conversion]
/usr/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
24 | # define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
| ^
2 errors generated.
ACKs for top commit:
hebasto:
ACK fa9d7d5d20.
fanquake:
ACK fa9d7d5d20
Tree-SHA512: 4964b23162f2351c7d3cf7e9efa7860d62f3b6717c3cc5be967d286f1ddb3539c2637247c79aa83123d36ff111ba77df22be2a25487ddd94dc1321d5e751dc70
This change enables building and installing only `libbitcoinkernel`,
without the need to disable other targets during the project build
system generation.
Add `-ipcbind` option to `bitcoin-node` to listen on an IPC socket and accept
connections from other processes. In the future, there will be an `-ipcconnect`
option added to `bitcoin-wallet` and `bitcoin-node` to allow wallet and gui
processes to connect to the node and access it.
Example usage:
src/bitcoin-node -regtest -debug -ipcbind=unix
src/bitcoin-wallet -regtest -ipcconnect=unix info
src/bitcoin-gui -regtest -ipcconnect=unix
src/bitcoin-mine -regtest -ipcconnect=unix
3ae35b427f ci: run check-deps.sh as part of clang-tidy job (Ryan Ofsky)
0aaa1298a0 contrib: fix check-deps.sh when libraries do not import symbols (Ryan Ofsky)
3c99f5a38a contrib: fix check-deps.sh to check for weak symbols (Ryan Ofsky)
86c80e9cf2 contrib: make check-deps.sh script work with cmake (Ryan Ofsky)
Pull request description:
Fix check-deps.sh to check for weak symbols so it can detect when an exported template function like is used from another library.
Also update the script to work with cmake and configure it to run as part of CI.
Problem was reported by hebasto in https://github.com/bitcoin/bitcoin/pull/29015#issuecomment-2209258843
ACKs for top commit:
TheCharlatan:
Re-ACK 3ae35b427f
hebasto:
ACK 3ae35b427f, I have reviewed the code and it looks OK. Also I've tested it locally.
Tree-SHA512: c3b58175450b675e6e848549b81bcfe42930ea9bcd693063867ce3f0ac3999c98cd2c3e961f163ff06641e8288f3a4e81530936a296a83d45d33364f27489521
fadbcd51fc bench: Remove redundant logging benchmarks (MarcoFalke)
fa8dd952e2 bench: Use LogInfo instead of the deprecated alias LogPrintf (MarcoFalke)
Pull request description:
`LogPrint*ThreadNames` is redundant with `LogWith(out)ThreadNames`,
because they all measure toggling the thread names (and check that it
has no effect on performance).
Fix it by removing the redundant ones. This also allows to drop a deprecated logging alias.
ACKs for top commit:
stickies-v:
ACK fadbcd51fc
Tree-SHA512: 4fe137f374aa4ee1aa0e1da4a1f9839c0e52c23dbb93198ecafee98de39d311cc47304bba4191f3807aa00c51b1eae543e3f270f03d341c84910e5e341a1d475
fa84f9decd test: Pin and document TEST_DIR_PATH_ELEMENT (MarcoFalke)
2222f7a874 test: Rename SeedRand::SEED to FIXED_SEED for clarity (MarcoFalke)
Pull request description:
Two small test changes:
* A refactor to update the name and documentation around `SeedRand::FIXED_SEED`.
* A change to extract and document `TEST_DIR_PATH_ELEMENT`, and to change its value to better match the `TMPDIR_PREFIX` in functional tests. The value previously included `PACKAGE_NAME`, which is cute, but doesn't explain why it was used (to include a space). So just use `test_common bitcoin` to achieve the same with less effort.
ACKs for top commit:
hodlinator:
ACK fa84f9decd
ryanofsky:
Code review ACK fa84f9decd
Tree-SHA512: eb35d6598bb08f9b996e3a4762d8f26b2441c0ca00780798e473015af735dfc9997120895a922b94d4b6ada45adadba4a686e9cf9c285ddf688848e764c64840
a7a4e11db8 cmake: scope Boost Test check to vcpkg (fanquake)
Pull request description:
This check was added for `vcpkg`, given how it packages Boost. However, we don't need to run the check for other platforms, and it's quite slow. So, scope it to just `vcpkg`.
On my machine, this reduces the time to run `time cmake -B build` from ~12 seconds, to ~6 seconds.
Fixes: #30787.
ACKs for top commit:
kevkevinpal:
lgtm ACK [a7a4e11](a7a4e11db8)
maflcko:
review ACK a7a4e11db8
davidgumberg:
Tested ACK a7a4e11db8
hebasto:
re-ACK a7a4e11db8.
Tree-SHA512: 67cf3908a5381e21aeaa168a6f76b6e066d64a8ad2127d5ae9fe71a0f04bccf58a400726d9d4e228b3bdb6fca799034fd05a38388278fea30a1a841f6adac017
cd062d6684 build: work around issue with Boost <= 1.80 and Clang >= 18 (fanquake)
Pull request description:
Our current minimum supported Boost is `1.73.0`. However, when compiling with Boost `1.74.0` (Debian Stable), using Clang `18`, compilation fails with:
```bash
In file included from /usr/include/boost/mpl/integral_c.hpp:32:
/usr/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for the enumeration type 'udt_builtin_mixture_enum' [-Wenum-constexpr-conversion]
73 | typedef AUX_WRAPPER_INST( BOOST_MPL_AUX_STATIC_CAST(AUX_WRAPPER_VALUE_TYPE, (value - 1)) ) prior;
| ^
/usr/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
24 | # define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
| ^
In file included from ../../../src/test/validation_chainstatemanager_tests.cpp:8:
In file included from ../../../src/node/chainstatemanager_args.h:9:
In file included from ../../../src/validation.h:28:
In file included from ../../../src/txmempool.h:26:
In file included from /usr/include/boost/multi_index/hashed_index.hpp:38:
In file included from /usr/include/boost/multi_index/detail/node_handle.hpp:22:
In file included from /usr/include/boost/multi_index_container_fwd.hpp:18:
In file included from /usr/include/boost/multi_index/indexed_by.hpp:17:
In file included from /usr/include/boost/mpl/vector.hpp:36:
In file included from /usr/include/boost/mpl/vector/vector20.hpp:18:
In file included from /usr/include/boost/mpl/vector/vector10.hpp:18:
In file included from /usr/include/boost/mpl/vector/vector0.hpp:24:
In file included from /usr/include/boost/mpl/vector/aux_/clear.hpp:18:
In file included from /usr/include/boost/mpl/vector/aux_/vector0.hpp:22:
In file included from /usr/include/boost/mpl/vector/aux_/iterator.hpp:19:
In file included from /usr/include/boost/mpl/plus.hpp:19:
In file included from /usr/include/boost/mpl/aux_/arithmetic_op.hpp:17:
In file included from /usr/include/boost/mpl/integral_c.hpp:32:
/usr/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for the enumeration type 'int_float_mixture_enum' [-Wenum-constexpr-conversion]
/usr/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
24 | # define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
| ^
2 errors generated.
```
Work around this issue by ignoring this diagnostic for this include. I did attempt to just downgrade the error into a warning, but that did not seem to work. Not a huge fan of inline warning/issue suppression, but this seems like the cleanest thing to do here (and easy to backport to `28.x`).
Can be tested with something like:
```bash
docker pull debian:bookworm
docker run -it debian:bookworm /bin/bash
apt update && apt install ccache cmake git pkg-config libboost-dev libevent-dev python3 libsqlite3-dev lsb-release wget software-properties-common gnupg
git clone https://github.com/bitcoin/bitcoin
wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
./llvm.sh 18
cd bitcoin
cmake -B build -DCMAKE_C_COMPILER=clang-18 -DCMAKE_CXX_COMPILER=clang++-18
cmake --build build -j17
<snip>
In file included from /usr/include/boost/mpl/integral_c.hpp:32:
/usr/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for the enumeration type 'int_float_mixture_enum' [-Wenum-constexpr-conversion]
/usr/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
24 | # define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
| ^
2 errors generated.
Apply the patch
cmake --build build -j17
ctest --test-dir build -j17
```
Fixes#30751.
ACKs for top commit:
achow101:
ACK cd062d6684
hebasto:
ACK cd062d6684, tested on Fedora 40 using the downloaded [Boost 1.74](https://archives.boost.io/release/1.74.0/source/) and commands as follows:
Tree-SHA512: 13e5b3a544496ed2a6529ad45d03a2d872ebf41caaa06d0eec23a639d678ae1c55d73f2d4b164a4cc9e2c163264e736cd85eae90fde8089ca999cd810b16ecb5
The current logic for file descriptor accounting is pretty convoluted and hard
to follow. This is partially caused by the lack of documentation plus non-intuitive
variable naming (which was more intuitive when fewer things were accounted for, but
hasn't aged well). This has led to this accounting being error-prone and hard to maintain
(as shown in he previous commit).
Redefine some of the constants, plus document what are we accounting for so this can be
extended more easily
Remove FreeBSD workaround to #2695
We are computing our file descriptors limits based on whether we use
poll or select. However, we are taking that into account only partially
(subtracting from fd_max in one case, but from nFD later on). Moreover,
nBind is also only accounted for partially.
Simplify and fix this logic
`FORTIFY_SOURCE` should be used if `ENABLE_HARDENING=ON` and optimisations
are being used. This should not be coupled to any particular build type,
because even if the build type is `Debug`, optimisations might still
be in use.
Fixes: #30800.
This check was added for vcpkg, given how it packages Boost. However, we
don't need to run the check for other platforms, and it's quite slow.
So, scope it to VCPKG. On my machine, this reduces the time to run
`cmake -B build` from ~12 seconds, to ~6 seconds.
Fixes: #30787
fa3a7ebe5b lint: Check for release note snippets in the wrong folder (MarcoFalke)
Pull request description:
It is a common mistake to place the snippets in the wrong folder, where they could be missed. For example https://github.com/bitcoin/bitcoin/pull/30719#pullrequestreview-2262535007 or commit 84900ac34f.
Fix all issues by adding a simple lint check.
Can be tested by reverting a prior commit that violated the rule and then running the new check:
```
git revert 35ef34eab7
( cd ./test/lint/test_runner/ && RUST_BACKTRACE=1 cargo run -- --lint=doc_release_note_snippets )
ACKs for top commit:
l0rinc:
ACK fa3a7ebe5b
TheCharlatan:
Re-ACK fa3a7ebe5b
Tree-SHA512: 65a13696178aa8f94daa12a767cc74861293c631c19da9ca23c0fd43cedd47e7928d0ef14ad9ad83a434c1ac0e006f5a632ba9679756e071dea65b3cbf927c2d
Our current minimum supported Boost is `1.73.0`. However, when compiling
with Boost `1.74.0` (Debian Stable), using Clang `18`, compilation fails
with:
```bash
In file included from /usr/include/boost/mpl/integral_c.hpp:32:
/usr/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for the enumeration type 'udt_builtin_mixture_enum' [-Wenum-constexpr-conversion]
73 | typedef AUX_WRAPPER_INST( BOOST_MPL_AUX_STATIC_CAST(AUX_WRAPPER_VALUE_TYPE, (value - 1)) ) prior;
| ^
/usr/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
24 | # define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
| ^
In file included from ../../../src/test/validation_chainstatemanager_tests.cpp:8:
In file included from ../../../src/node/chainstatemanager_args.h:9:
In file included from ../../../src/validation.h:28:
In file included from ../../../src/txmempool.h:26:
In file included from /usr/include/boost/multi_index/hashed_index.hpp:38:
In file included from /usr/include/boost/multi_index/detail/node_handle.hpp:22:
In file included from /usr/include/boost/multi_index_container_fwd.hpp:18:
In file included from /usr/include/boost/multi_index/indexed_by.hpp:17:
In file included from /usr/include/boost/mpl/vector.hpp:36:
In file included from /usr/include/boost/mpl/vector/vector20.hpp:18:
In file included from /usr/include/boost/mpl/vector/vector10.hpp:18:
In file included from /usr/include/boost/mpl/vector/vector0.hpp:24:
In file included from /usr/include/boost/mpl/vector/aux_/clear.hpp:18:
In file included from /usr/include/boost/mpl/vector/aux_/vector0.hpp:22:
In file included from /usr/include/boost/mpl/vector/aux_/iterator.hpp:19:
In file included from /usr/include/boost/mpl/plus.hpp:19:
In file included from /usr/include/boost/mpl/aux_/arithmetic_op.hpp:17:
In file included from /usr/include/boost/mpl/integral_c.hpp:32:
/usr/include/boost/mpl/aux_/integral_wrapper.hpp:73:31: error: integer value -1 is outside the valid range of values [0, 3] for the enumeration type 'int_float_mixture_enum' [-Wenum-constexpr-conversion]
/usr/include/boost/mpl/aux_/static_cast.hpp:24:47: note: expanded from macro 'BOOST_MPL_AUX_STATIC_CAST'
24 | # define BOOST_MPL_AUX_STATIC_CAST(T, expr) static_cast<T>(expr)
| ^
2 errors generated.
```
Work around this issue by ignoring this diagnostic for this include.
I did attempt to just downgrade the error into a warning, but that did
not seem to work.
See https://github.com/bitcoin/bitcoin/issues/30751 for further
discussion.
7346b01092 qt, build: remove unneeded `Q_IMPORT_PLUGIN` macro calls (Sebastian Falbesoner)
Pull request description:
After the recent full removal of Autotools (PR [#30664](https://github.com/bitcoin/bitcoin/pull/30664)), these macros are not needed anymore in the .cpp files according to the TODO in qt's CMakeLists.txt. Tested building on OpenBSD 7.5, where the XCB plugin was still imported according to the debug log:
```
2024-09-02T21:13:27Z Bitcoin Core version v28.99.0-7346b0109208 (release build)
2024-09-02T21:13:27Z Qt 5.15.12 (dynamic), plugin=xcb
2024-09-02T21:13:27Z No static plugins.
2024-09-02T21:13:27Z Style: fusion / QFusionStyle
2024-09-02T21:13:27Z System: OpenBSD 7.5, x86_64-little_endian-lp64
```
ACKs for top commit:
hebasto:
ACK 7346b01092.
Tree-SHA512: ffa033fc6e0412a99d2c167044cc7af89512a731172d6911db71434f5353e811802c268d853a76d3732e9da954444cf6c39a852aeb25938c435826e117a16fca
faecca9a85 test: Use span for raw data (MarcoFalke)
fac973647d test: Use string_view for json_tests (MarcoFalke)
Pull request description:
The build system converts raw data into a C++ header file for tests.
This change modernizes the code to use the convenience wrappers `std::span` and `std::string_view`, so that redundant copies can be avoided.
ACKs for top commit:
fjahr:
re-ACK faecca9a85
TheCharlatan:
ACK faecca9a85
stickies-v:
ACK faecca9a85
hebasto:
ACK faecca9a85, I have reviewed the code and the generated headers.
Tree-SHA512: 1f4951c54aff11ba27c41fb70f2821bdb79e06ca0abae734b970bd0d64dda9d8cced824a891fd51b3e9d4e5715ee9eb49ed5d369010a45eca7c3bec9f8641235
d9fcbfc372 build: Add `JOBS` variable support to `CoverageFuzz.cmake` script (Hennadii Stepanov)
e7cf4a6f27 build: Add missed `-g` for "Coverage" build configuration (Hennadii Stepanov)
fe2003ab12 build: Add `COMMAND_ERROR_IS_FATAL` to every process in coverage scrips (Hennadii Stepanov)
Pull request description:
The first commit ensures early error catching.
The second commit adds the `-g` flag that was missed during the migration from Autotools.
This PR is intended to be tested with GCC compiler (as clang support is still under [scrutiny](https://github.com/hebasto/bitcoin/issues/341)). Depending on the `lcov` version, additional flags `-DCMAKE_C_FLAGS="-fprofile-update=atomic" -DCMAKE_CXX_FLAGS="-fprofile-update=atomic"` may be required.
ACKs for top commit:
maflcko:
review ACK d9fcbfc372
tdb3:
cr re ACK d9fcbfc372
Tree-SHA512: 0998411dc1ccd60d7bd6b36f4e2881f699202c65dcc8c177b46380d0f255d291d9537f1dc6fb35478b632f3515d3484d8e7d2877126c57e3f02b21f90160f1eb
5567754087 depends: build libevent with -D_GNU_SOURCE (fanquake)
Pull request description:
Currently, builds of libevent in depends, using CMake, fail on some systems, like Alpine, with the following:
```bash
/bitcoin/depends/work/build/aarch64-unknown-linux-musl/libevent/2.1.12-stable-1516ed47ea8/evmap.c: In function 'evmap_signal_add_':
/bitcoin/depends/work/build/aarch64-unknown-linux-musl/libevent/2.1.12-stable-1516ed47ea8/evmap.c:456:31: error: 'NSIG' undeclared (first use in this function)
456 | if (sig < 0 || sig >= NSIG)
```
From what I can tell the `GNU_SOURCE` "detection" in libevents CMake build system, never? really worked, primarily relies on looking for a deprecated define, and it's not clear what a nice fix is. For now, always build with `_GNU_SOURCE`, to match the autotools behaviour.
ACKs for top commit:
TheCharlatan:
ACK 5567754087
Tree-SHA512: 4552b4a92867e8fa2af0ffa39b2be6c994bf739de7ce6a7c581590be486da81f7d93fca816854548c1e912347d33a35218c441b5058c3cbd3e82c74a9b7c78d9
This change allows to drop brittle sizeof calls in favor of the
std::span::size method.
Other improvements include:
* Use of a namespace to mark test and bench data
* Use of the modern std::byte
* Drop of a no longer used std::vector copy and the bench/data module
In case of a failure to create the dump, the node should not be left in an inconsistent state like deactivated network activity or an invalidated blockchain.
e5f7272ad3 doc: fix assumeutxo design doc link (marcofleon)
Pull request description:
A correction to a link as I was exploring Assumeutxo stuff.
ACKs for top commit:
fjahr:
ACK e5f7272ad3
tdb3:
ACK e5f7272ad3
MarnixCroes:
ACK e5f7272ad3
Tree-SHA512: b7380d884a196b89eed32bc14af8ca11191c9f2bcb1c5c163bb627fd87a6231dbd86da6e659baddb8c652961b5e44f80509606fee9ae38a53e90ebb05d082404
LogPrint*ThreadNames is redundant with LogWith(out)ThreadNames, because
they all measure toggling the thread names (and check that it has no
effect on performance).
This also allows to remove unused and deprecated macros.
8131bf7483 ci: parse TEST_RUNNER_EXTRA into an array (Max Edwards)
c4762b0aa0 test: allow excluding func test by name and arg (Max Edwards)
Pull request description:
While working on CI I wanted to disable some functional tests so I used the `TEST_RUNNER_EXTRA` var. The problem I had was tests that have flags such as `rpc_bind.py --ipv6` must be passed in quotes otherwise the `--ipv6` portion will be considered an argument to `test_runner.py` rather than a test name.
This change allows proper parsing of quotes and complex values such as:
```shell
TEST_RUNNER_EXTRA='--exclude "rpc_bind.py --ipv6,feature_proxy.py"'
```
Update:
While testing this it was noticed that `test_runner.py` when given `--exclude "rpc_bind.py --ipv6"` will exclude all `rpc_bind.py` tests so this PR has been updated to include a change to the test runner to only exclude the specific test if you pass an arg or exclude all tests of that name if you do not pass an arg. `--exclude rpc_bind.py` will exclude all three variants and `--exclude rpc_bind --ipv6` will only exclude the IPV6 variant.
ACKs for top commit:
maflcko:
ACK 8131bf7483
achow101:
ACK 8131bf7483
hebasto:
ACK 8131bf7483, tested on Ubuntu 23.10 and Windows 11.
Tree-SHA512: 82b73f12d627f533d8e5be4a518d455ef4427a755bbe03ccd11d0bb70c7ff3cee76220b0264fcfb236661c4cf5deba034cbfc2372b96d5861f3436c21eae8264
fafdb7df34 lint: Speed up flake8 checks (MarcoFalke)
faf17df7fb lint: Document missing py_lint dependency (MarcoFalke)
faebeb828f lint: Remove python whitespace and shadowing lint rules (MarcoFalke)
7777047835 lint: Remove python lint rules that are SyntaxError (MarcoFalke)
faaf3e53f0 test: [refactor] Fix F841 flake8 (MarcoFalke)
444421db69 test: [refactor] Fix E714 pycodestyle (MarcoFalke)
Pull request description:
The checks have many issues:
* Some checks that could in theory hide bugs are not applied -> Fix them and apply them going forward
* Some checks are redundant Python 2 checks, or of low value -> Remove them
* The checks are slow -> Speed them up from ~10 seconds to about ~20 milliseconds
ACKs for top commit:
davidgumberg:
review and tested reACK fafdb7df34
kevkevinpal:
ACK [fafdb7d](fafdb7df34)
achow101:
ACK fafdb7df34
Tree-SHA512: a0488b722cfaf7071bd6848cd3be002e0b6c38af80d8b5cbb08613c0b174ef63277289f960db8ac31adb09fe563a4973203b8fb10b83cbcfdc6f0ef39bd04410
Script was failing when called on libraries that do not import symbols, because
bash pipefail option was specified, and grep was used in some pipelines to
filter symbols, and grep returns status 1 when it doesn't match any lines. This
could cause the script to fail on some systems and configurations, such as the
clang-tidy CI configuration
https://cirrus-ci.com/task/4801670352207872?logs=ci#L6191 where the
libbitcoin_crypto_x86_shani.a library does not import symbols.
Fix check-deps.sh to check for weak symbols so it can detect when an exported
template function is used from another library.
In a previous version of this commit, this change caused an invalid dependency
in the consensus library on the TryParseHex template function from the util
library to be detected, and a suppression was added here. But #30377 removed
the invalid dependency so the suppression is no longer needed.
The invalid dependency and problem detecting weak symbol usage was originally
reported by Hennadii Stepanov in
https://github.com/bitcoin/bitcoin/pull/29015#issuecomment-2209258843
ae48a22a3d test: fixing failing system_tests/run_command under some Locales (Jadi)
Pull request description:
the run_command test under system_tests fails if the locale is anything
other than English ones because results such as "No such file or directory"
will be different under Non-English locales.
On the old version, a `ls nonexistingfile` was used to generate the error
output which is not ideal. In the current version we are using a Python one-liner
to generate a non 0 zero return value and "err" on stderr and check the
expected value against this.
fixes https://github.com/bitcoin/bitcoin/issues/30608
ACKs for top commit:
maflcko:
review ACK ae48a22a3d
achow101:
ACK ae48a22a3d
hebasto:
ACK ae48a22a3d, tested on Ubuntu 24.04 by switching locale.
Tree-SHA512: af7522ddcd786fa4a6832c8336ca89d8ff05f49ff963cbe1a96653b0edf29e0f950a032f23d742b16b3895e90cf5117b5f6a95464268dec67039df166d7d8639
These helpers haven't been used in production code since segwit was
merged more than eight years ago (see commit 605e8473, PR #8149),
so it seems appropriate to move them to the test utils module.
Can be reviewed via `--color-moved=dimmed-zebra`.
01960c53c7 fuzz: make FuzzedDataProvider usage deterministic (Martin Leitner-Ankerl)
Pull request description:
There exist many usages of `fuzzed_data_provider` where it is evaluated directly in the function call.
Unfortunately, [the order of evaluation of function arguments is unspecified](https://en.cppreference.com/w/cpp/language/eval_order), and a simple example shows that it can differ e.g. between clang++ and g++: https://godbolt.org/z/jooMezWWY
When the evaluation order is not consistent, the same fuzzing/random input will produce different output, which is bad for coverage/reproducibility. This PR fixes all these cases I have found where unspecified evaluation order could be a problem.
Finding these has been manual work; I grepped the sourcecode for these patterns, and looked at each usage individually. So there is a chance I missed some.
* `fuzzed_data_provider`
* `.Consume`
* `>Consume`
* `.rand`
I first discovered this in https://github.com/bitcoin/bitcoin/pull/29013#discussion_r1420236394. Note that there is a possibility that due to this fix the evaluation order is now different in many cases than when the fuzzing corpus has been created. If that is the case, the fuzzing corpus will have worse coverage than before.
Update: In list-initialization the order of evaluation is well defined, so e.g. usages in `initializer_list` or constructors that use `{...}` is ok.
ACKs for top commit:
achow101:
ACK 01960c53c7
vasild:
ACK 01960c53c7
ismaelsadeeq:
ACK 01960c53c7
Tree-SHA512: e56d087f6f4bf79c90b972a5f0c6908d1784b3cfbb8130b6b450d5ca7d116c5a791df506b869a23bce930b2a6977558e1fb5115bb4e061969cc40f568077a1ad
c8e6771af0 test: restrict multiple CLI arguments (naiyoma)
8838c4f171 common/args.h: automate check for multiple cli commands (naiyoma)
Pull request description:
This PR is a continuation of the validation suggested [here](https://github.com/bitcoin/bitcoin/pull/27815) to ensure that only one Request Handler can be specified at a time.
ACKs for top commit:
stratospher:
reACK c8e6771.
achow101:
ACK c8e6771af0
tdb3:
cr re ACK c8e6771af0
Tree-SHA512: f4fe036fee342a54f1a7ac702ac35c40bf3d420fde6ab16313a75327292d5ee5c8ece1be9f852a13fcf73da1148b143b37b4894e294052abdde6eefb2e8c6f3f
ec317bc44b test: update satoshi_round function (naiyoma)
Pull request description:
This PR refactors `satoshi_round` to accept different rounding modes and make rounding a required argument.
Continuation of https://github.com/bitcoin/bitcoin/pull/23225
ACKs for top commit:
maflcko:
review ACK ec317bc44b
achow101:
ACK ec317bc44b
AngusP:
ACK ec317bc44b
Tree-SHA512: 070f0aa6f66e58bff7412cae6b71f5f4ab8c718c7b5eeba4bb604fe22c6416a1ced0474294f504e1c28943ddc073104466b5944b43bae27e42bee5ca85afa468
fb6d51eb25 signet/miner: Use argparse exclusive groups (Anthony Towns)
338a266a9a signet/miner: add support for a poolnum/poolid tag in mined blocks (Anthony Towns)
409ab7d35b signet/miner: add Generate.mine function (Anthony Towns)
7b31332370 signet/miner: add Generate.gbt function (Anthony Towns)
85c5c0bea9 signet/miner: add Generate.next_block_time function (Anthony Towns)
5540e6ca49 signet/miner: move next_block_* functions into new Generator class (Anthony Towns)
35f4631196 signet/miner: rename do_decode_psbt to decode_psbt (Anthony Towns)
aac040b439 signet/miner: drop create_coinbase function (Anthony Towns)
16951f549e signet/miner: drop do_createpsbt function (Anthony Towns)
3aed0a4284 signet/miner: drop get_reward_address function (Anthony Towns)
Pull request description:
Refactors the code a bunch, and adds `--poolnum` / `--poolid` options so that signers can tag their coinbases in a way that explorers can recognise (see also https://github.com/bitcoin-data/mining-pools/pull/82 and https://github.com/mempool/mempool/issues/2903).
The refactoring in particular helps enable the "try using inquisition's getblocktemplate, and if that doesn't work fall back to core's getblocktemplate" logic, as described/implemented in https://github.com/bitcoin-inquisition/bitcoin/pull/7
ACKs for top commit:
achow101:
ACK fb6d51eb25
danielabrozzoni:
Code review ACK fb6d51eb25
Tree-SHA512: d84095c4045ab196685b847e04ce2cdaedf387bc2527430ede918318dc5b70bf3d87b754264016f895f506fac70d4fdea5ef3cd8c3c375fd586afeae01e045e5
6eeb188d40 test: adds seednode functional tests (Sergi Delgado Segura)
3270f0adad net: Favor peers from addrman over fetching seednodes (Sergi Delgado Segura)
Pull request description:
This is a follow-up of #28016 motivated by https://github.com/bitcoin/bitcoin/pull/28016#pullrequestreview-1913140932 and https://github.com/bitcoin/bitcoin/pull/28016#issuecomment-1984448937.
The current behavior of seednode fetching is pretty eager: we do it as the first step under `ThreadOpenNetworkConnections` even if some peers may be queryable from our addrman. This poses two potential issues:
- First, if permanently set (e.g. running with seednode in a config file) we'd be signaling such seed every time we restart our node
- Second, we will be giving the seed node way too much influence over our addrman, populating the latter with data from the former even when unnecessary
This changes the behavior to only add seednodes to `m_addr_fetch` if our addrman is empty, or little by little after we've spent some time trying addresses from our addrman. Also, seednodes are added to `m_addr_fetch` in random order, to avoid signaling the same node in case more than one seed is added and we happen to try them over multiple restarts
ACKs for top commit:
achow101:
ACK 6eeb188d40
cbergqvist:
ACK 6eeb188d40
itornaza:
Tested ACK 6eeb188d40
tdb3:
ACK 6eeb188d40
Tree-SHA512: b04445412f22018852d6bef4d3f1e88425ee6ddb434f61dcffa9e0c41b8e31f8c56f83858d5c7686289c86dc4c9476c437df15ea61a47082e2bb2e073cc62f15
a3108a7c56 rpc: Manage dumptxoutset rollback with RAII class (Fabian Jahr)
c5eaae3b89 doc: Add -rpcclienttimeout=0 to loadtxoutset examples (Fabian Jahr)
598b9bba5a rpc: Don't re-enable previously disabled network after dumptxoutset (Fabian Jahr)
Pull request description:
First, this addresses two left-over comments in #29553:
- When running `dumptxoutset` the network gets disabled in the beginning and then re-enabled at the end. The network would be re-enabled even if the user had already disabled the network themself before running `dumptxoutset`. The network is now not re-enabled anymore since that might not be what the user wants.
- The `-rpcclienttimeout=0` option is added to `loadtxoutset` examples in documentation
Additionally, pablomartin4btc notified me that he found his node stuck at the invalidated height after some late testing after #29553 was merged. We could not find the actual source of the issue since his logs got lost. However, it seems likely that some kind of disruption stopped the process before the node could roll forward again. We fixed this issue for network disablement with a RAII class previously and it seems logical that this can happen the same way for the rollback part so I suggest to also fix it the same way.
An example to reproduce the issue described above as I think it happened: Remove the `!` in the following line in `PrepareUTXOSnapshot()` to simulate an issue occurring during `GetUTXOStats()`.
```
if (!maybe_stats) {
```
This leaves the node in the following state on master:
```
$ build/src/bitcoin-cli -rpcclienttimeout=0 -named dumptxoutset utxo-859750.dat rollback=859750
error code: -32603
error message:
Unable to read UTXO set
$ build/src/bitcoin-cli getchaintips
[
{
"height": 859762,
"hash": "00000000000000000002ec7a0fcca3aeca5b35545b52eb925766670aacc704ad",
"branchlen": 12,
"status": "headers-only"
},
{
"height": 859750,
"hash": "0000000000000000000010897b6b88a18f9478050200d8d048013c58bfd6229e",
"branchlen": 0,
"status": "active"
},
```
(Note that the first tip is `headers-only` and not `invalid` only because I started `dumptxoutset` before my node had fully synced to the tip. pablomartin4btc saw it as `invalid`.)
ACKs for top commit:
maflcko:
re-ACK a3108a7c56🐸
achow101:
ACK a3108a7c56
pablomartin4btc:
cr ACK a3108a7c56
Tree-SHA512: d2ab32f62de2253312e27d7d753ec0995da3fe7a22ffc3d6c7cfa3b68a4a144c59210aa82b7a704c2a29c3b2aad6ea74972e3e8bb979ee4b7082a20f4bfddc9c
66d13c8702 test: add check that large txs aren't put into orphanage (Sebastian Falbesoner)
ed7d224666 test: add `BulkTransaction` helper to unit test transaction utils (Sebastian Falbesoner)
Pull request description:
This PR adds test coverage for the following check in `TxOrphanage::AddTx`, where large orphan txs are ignored in order to avoid memory exhaustion attacks:
5abb9b1af4/src/txorphanage.cpp (L22-L34)
Note that this code-path isn't reachable under normal circumstances, as txs larger than `MAX_STANDARD_TX_WEIGHT` are already rejected earlier in the course of doing the mempool standardness checks (see `MemPoolAccept::PreChecks` -> `IsStandardTx` -> `reason = "tx-size";`), so this is only relevant if tx standardness rules are disabled via `-acceptnonstdtxns=1`. The ignore path is checked ~~by asserting the debug log, which is ugly, but as far as I know there is currently no way to access the orphanage entries from the outside~~ via unit test that checks the return value of `AddTx`. As an alternative to adding test coverage, one might consider removing this check altogether (or replacing it with an `Assume`), as it's redundant as explained above.
ACKs for top commit:
maflcko:
review ACK 66d13c8702
glozow:
ACK 66d13c8702
tdb3:
re-ACK 66d13c8702
Tree-SHA512: 88e8254ab5fca70c387a5992649ea6a704a65162999be972cc86bd74fc26c5f0f1e13e04856708d07ad5524cb77c0918e19663db92b3593e842469dfe04af6a1
8888beea8d scripted-diff: fuzz: Rename fuzz_seed_corpus to fuzz_corpora (MarcoFalke)
Pull request description:
Now that cmake was a breaking change for all fuzz scripts, it seems fine to bundle it with another breaking change to rename the fuzz corpora directory, as discussed and approved in https://github.com/bitcoin-core/qa-assets/issues/200:
* The word "seed" in the old name doesn't really apply. In reality it is a collection of fuzz input seeds, as well as fuzz inputs.
* The rename will also allow in the future (when there is a need and desire) to provide a minimal set of possibly hand-crafted or otherwise non-fuzz-generated fuzz seed inputs to some fuzz targets (and possibly store them in a separate folder and validate that their format is still accurate and matches the fuzz target code).
* Finally, "corpus" is renamed to corpora, to clarify that the folder holds the fuzz inputs for several fuzz targets.
ACKs for top commit:
brunoerg:
ACK 8888beea8d
marcofleon:
ACK 8888beea8d
Tree-SHA512: abc693ca5d946850f04b6349e2a98f8fbc2ba9991be5a025bc0f357e341cbe7510f2f5f0e47b997d07136736d818df361270f372b8fb70860995a0605ca81e4d
fa78ed83be doc: Clarify libbitcoin_consensus in design/libraries.md (MarcoFalke)
Pull request description:
Now that the shared library has been removed in commit 80f8b92f4f, update the documentation to drop the no-longer applicable prefix "Stable...".
ACKs for top commit:
hebasto:
ACK fa78ed83be.
fanquake:
ACK fa78ed83be
Tree-SHA512: d7b946d50f734c0474ff6155a655a2bb873f76e071bfeeca1dd42ea5fdd32bc1e45129826bb54e3f111265d19c2aba2d02cb77ad7663f9fc40c8c875e5fddda2
the run_command test under system_tests fails if the locale is anything
other than English ones because results such as "No such file or directory"
will be different under Non-English locales.
On the old version, a `ls nonexistingfile` was used to generate the error
output which is not ideal. In the current version we are using a Python one-liner
to generate a non 0 zero return value and "err" on stderr and check the
expected value against this.
fixes#30608
03d49d0f25 http: set TCP_NODELAY when creating HTTP server (Roman Zeyde)
Pull request description:
Otherwise, the default HTTP server config may result in high latency, due to Nagle's algorithm (on the server) and delayed ACK (on the client):
[1] https://www.extrahop.com/blog/tcp-nodelay-nagle-quickack-best-practices
[2] https://eklitzke.org/the-caveats-of-tcp-nodelay
Without the fix, fetching a small block takes ~40ms (when connection keep-alive is enabled):
```
$ ab -k -c 1 -n 100 http://localhost:8332/rest/block/00000000000002b5898f7cdc80d9c84e9747bc6b9388cc989971d443f05713ee.bin
Server Software:
Server Hostname: localhost
Server Port: 8332
Document Path: /rest/block/00000000000002b5898f7cdc80d9c84e9747bc6b9388cc989971d443f05713ee.bin
Document Length: 25086 bytes
Concurrency Level: 1
Time taken for tests: 4.075 seconds
Complete requests: 100
Failed requests: 0
Keep-Alive requests: 100
Total transferred: 2519200 bytes
HTML transferred: 2508600 bytes
Requests per second: 24.54 [#/sec] (mean)
Time per request: 40.747 [ms] (mean)
Time per request: 40.747 [ms] (mean, across all concurrent requests)
Transfer rate: 603.76 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 0 41 4.1 41 42
Waiting: 0 0 0.1 0 1
Total: 0 41 4.1 41 42
Percentage of the requests served within a certain time (ms)
50% 41
66% 41
75% 41
80% 41
90% 42
95% 42
98% 42
99% 42
100% 42 (longest request)
```
With the fix, it takes ~0.2ms:
```
$ ab -k -c 1 -n 1000 http://localhost:8332/rest/block/00000000000002b5898f7cdc80d9c84e9747bc6b9388cc989971d443f05713ee.bin
Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software:
Server Hostname: localhost
Server Port: 8332
Document Path: /rest/block/00000000000002b5898f7cdc80d9c84e9747bc6b9388cc989971d443f05713ee.bin
Document Length: 25086 bytes
Concurrency Level: 1
Time taken for tests: 0.194 seconds
Complete requests: 1000
Failed requests: 0
Keep-Alive requests: 1000
Total transferred: 25192000 bytes
HTML transferred: 25086000 bytes
Requests per second: 5147.05 [#/sec] (mean)
Time per request: 0.194 [ms] (mean)
Time per request: 0.194 [ms] (mean, across all concurrent requests)
Transfer rate: 126625.50 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 0 0 0.0 0 0
Waiting: 0 0 0.0 0 0
Total: 0 0 0.0 0 0
Percentage of the requests served within a certain time (ms)
50% 0
66% 0
75% 0
80% 0
90% 0
95% 0
98% 0
99% 0
100% 0 (longest request)
```
ACKs for top commit:
achow101:
ACK 03d49d0f25
theStack:
re-ACK 03d49d0f25
tdb3:
ACK 03d49d0f25
Tree-SHA512: bbf3d78b8521f569430850ec4315a75711303547df1a3de213a4ad34c9700105e374e0a649352fd05f8e4badb5b59debd3720e1c5d392c5113d7816648f7fcaa
b31a0cd037 log: expand BCLog::LogFlags (categories) to 64 bits (Larry Ruane)
Pull request description:
Increase the maximum number of logging categories from 32 to 64.
We're currently using 29 of the 32 available logging categories (there are only 3 remaining). It would be good to increase the limit soon; the fourth PR to be merged that adds a new logging category will be blocked until something like this is done.
This PR also adds a `TEST` category that uses the new range (`1ULL << 63`) in case there's a hidden assumption somewhere that the `BCLog::LogFlags` type is 32 bits. (Also added a test for this test category.) It also provides an example showing that the expression must be `1ULL << <shift>` for shift value 31 and beyond.
ACKs for top commit:
achow101:
ACK b31a0cd037
vasild:
ACK b31a0cd037
ryanofsky:
Code review ACK b31a0cd037, just dropping mask_bit constant since last review. I still think
theStack:
Code-review ACK b31a0cd037
Tree-SHA512: de422dbeb479848d370aed42d415f42461457ab0eda62b245dc7ff9f0e111626e7d4c0d62ff13082ec664d05fbb0db04c71eb4b6f22eb8f19198826a67c4035e
a2955f0979 validation: Use span for ImportBlocks paths (TheCharlatan)
20515ea3f5 validation: Use span for CalculateClaimedHeadersWork (TheCharlatan)
52575e96e7 validation: Use span for ProcessNewBlockHeaders (TheCharlatan)
Pull request description:
Makes it friendlier for potential future users of the kernel library if they do not store the headers in a std::vector, but can guarantee contiguous memory.
Take this opportunity to also change the argument of ImportBlocks previously taking a `std::vector` to a `std::span`.
ACKs for top commit:
stickies-v:
re-ACK a2955f0979 - no changes except further walking the ~file~ path of modernizing variable names.
maflcko:
ACK a2955f0979🕑
achow101:
ACK a2955f0979
danielabrozzoni:
ACK a2955f0979
Tree-SHA512: 8b07f4ad26e270b65600d1968cd78847b85caca5bfbb83fd9860389f26656b1d9a40b85e0990339f50403d18cedcd2456990054f3b8b0bedce943e50222d2709
94b0adcc37 rpc, refactor: Prevent potential race conditions in dumptxoutset (Fabian Jahr)
e868a6e070 doc: Improve assumeutxo guide and add more docs/comments (Fabian Jahr)
b29c21fc92 assumeutxo: Remove devtools/utxo_snapshot.sh (Fabian Jahr)
20a1c77aa7 contrib: Remove test_utxo_snapshots.sh (Fabian Jahr)
8426850352 test: Test for dumptxoutset at specific height (Fabian Jahr)
993cafe7e4 RPC: Add type parameter to dumptxoutset (Fabian Jahr)
fccf4f91d2 RPC: Extract ReconsiderBlock helper (Fabian Jahr)
446ce51c21 RPC: Extract InvalidateBlock helper (Fabian Jahr)
Pull request description:
This adds a height parameter to the `dumptxoutset` RPC. This internalizes the workflow that was previously done by scripts: roll back the chain to the height we actually want the snapshot from, create the snapshot, roll forward to the real tip again.
The nice thing about internalizing this functionality is that we can write tests for the code and it gives us more options to make the functionality robust. The shell scripts we have so far will be more cumbersome to maintain in the long run, especially since we will only notice later when we have broken them. I think it's safe to remove these `test_utxo_snapshots.sh` as well when we have this option in `dumptxoutset` because we have also added some good additional functional test coverage for this functionality.
ACKs for top commit:
Sjors:
re-utACK 94b0adcc37
achow101:
ACK 94b0adcc37
mzumsande:
ACK 94b0adcc37
pablomartin4btc:
re-ACK 94b0adcc37
Tree-SHA512: a4c9af5f687d1ca7bfb579a36f363882823386b5fa80c05de531b05a2782b5da6ff5baf3ada4bca8f32f63975d86f1948175abed9affe51fc958472b5f838dab
When the tracepoint was introduced in 8f37f5c2a5,
the connect_block duration was passed in microseconds `µs`.
By starting to use steady clock in fabf1cdb20
this changed to nanoseconds `ns`. As the test only checked if the
duration value is `> 0` as a plausibility check, this went unnoticed.
I detected this when setting up monitoring for block validation time
as part of the Great Consensus Cleanup Revival discussion.
This change casts the duration explicitly to nanoseconds (as it has been
nanoseconds for the last three releases; switching back now would 'break'
the broken API again; there don't seem to be many users affected), updates
the documentation and adds a check for an upper bound to the tracepoint
interface tests. The upper bound is quite lax as mining the block takes
much longer than connecting the empty test block. It's however able to
detect incorrect duration units passed.
787dfaf084 ci: Do not override `-g -O1` set in `MSAN_FLAGS` (Hennadii Stepanov)
26c460aa8b build: Fix linking for `fuzz` target when building with MSan (Hennadii Stepanov)
Pull request description:
The first commit fixes https://github.com/bitcoin/bitcoin/issues/30760.
The second commit:
1. Preserves `-g -O1` set in `MSAN_FLAGS`. Since configuration-specific flags override general flags, these are set to empty strings. A similar approach is used in the OSS-Fuzz repository.
2. Sets the "Debug" build configuration when depends are built with `DEBUG=1`, ensuring that `linux_debug_CPPFLAGS` from depends are passed to the main build system.
ACKs for top commit:
maflcko:
review-only ACK 787dfaf084
fanquake:
ACK 787dfaf084 - as a follow up it would be good to:
Tree-SHA512: c324390d1dbda30f82025d8482ddb0cfa0395f9ba225a2ddce05a123c65e0622a6a1d5f0fa03f09e21d62792431cf3da5c49e41a3ac7f7a958d0392a0430f29c
This is used in both cases - TCP server (accept) and TCP client (connect).
The message "Connected & Listening address:port" is confusing.
Print both ends of the TCP connection.
6a68343ffb doc: Prepend 'build/' to binary paths under 'src/' in docs (Lőrinc)
91b3bc2b9c doc: Update documentation generation example in developer-notes.md (Lőrinc)
Pull request description:
In [the other readmes](6ce50fd9d0/src/test/README.md (L19)) we've provided a default build directory instead, unified the `developer-notes.md` to specify it explicitly.
In the next commit I've used this default to go over each reference to our binaries and changed their in-source references to the build directory.
Some of these changes were in example outputs - I haven't validated that the outputs are still the same.
I haven't modified the build folders in the devtools.
ACKs for top commit:
maflcko:
review ACK 6a68343ffb
pablomartin4btc:
ACK 6a68343ffb
fanquake:
ACK 6a68343ffb - we still need to followup with other scripts/devtools, and likely unify what we are doing in some way, but this is an improvement.
Tree-SHA512: 905d9c68cafe1e405e98d6aa089d7a36a34c9e03403df5c67ac2c9a98cfa54a0305b647cb92247dcb9f49e9b509a8ba88367392b95618c67059684c67b6c36fb
Currently, builds of libevent in depends, using CMake, fail on some
systems, like Alpine, with the following:
```bash
/bitcoin/depends/work/build/aarch64-unknown-linux-musl/libevent/2.1.12-stable-1516ed47ea8/evmap.c: In function 'evmap_signal_add_':
/bitcoin/depends/work/build/aarch64-unknown-linux-musl/libevent/2.1.12-stable-1516ed47ea8/evmap.c:456:31: error: 'NSIG' undeclared (first use in this function)
456 | if (sig < 0 || sig >= NSIG)
```
From what I can tell the `_GNU_SOURCE` "detection" in libevents CMake build
system, never? really worked, and it's not clear what a nice fix is.
For now, always use `_GNU_SOURCE` when building libevent in depends.
cd src/qt/res/icons
convert bitcoin.png -modulate 100,87,119.4 -define icon:auto-resize="256,48,32,16" bitcoin_signet.ico
This commit also removes the 64-bit mention from testnet.
After the recent full removal of Autotools (PR #30664), these
macros are not needed anymore in the .cpp files according to the
TODO in qt's CMakeLists.txt. Tested building on OpenBSD 7.5, where
the XCB plugin was still imported according to the debug log.
To avoid PoW being a blocker for fuzz tests,
`FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is used in fuzz builds to
bypass the actual PoW validation in `CheckProofOfWork`. It's
replaced with a check on the last byte of the hash, which allows the
fuzzer to quickly generate (in)valid blocks by checking a single bit,
rather than performing the full PoW computation.
If PoW is the target of a fuzz test, then it should call
`CheckProofOfWorkImpl`.
fa247e6e8c test: Avoid intermittent timeout in p2p_headers_sync_with_minchainwork.py (MarcoFalke)
Pull request description:
Similar to https://github.com/bitcoin/bitcoin/pull/30705:
The goal of this test case is to check that the sync works at all, not to check any timeout.
On extremely slow hardware (for example qemu virtual hardware), downloading the 4110 BLOCKS_TO_MINE may take longer than the block download timeout.
Fix it by pinning the time using mocktime temporarily, and advance it immediately after the sync.
ACKs for top commit:
stratospher:
ACK fa247e6. Checked the timeout downloading block logs before/after using `setmocktime`.
tdb3:
ACK fa247e6e8c
Tree-SHA512: f61632a8d9e484f1b888aafbf87f7adf71b8692387bd77f603cdbc0de49f30d42e654741d46ae1ff8b9706a5559ee0faabdb192ed0db7449010b68bfcdbaa42d
ddef914bbb doc: remove extraneous install statement (tdb3)
bc532c915e doc: add with_bdb to unix build docs (tdb3)
Pull request description:
Existing instructions for building legacy wallet support omit `-DWITH_BDB=ON`, which results in:
```
CMake Warning:
Manually-specified variables were not used by the project:
BerkeleyDB_INCLUDE_DIR
```
and a build without BDB support.
This PR updates the docs to include `-DWITH_BDB=ON`.
Also adds a minor correction to the OpenBSD build doc.
Checked by building on Linux (Debian 12.7), FreeBSD 14.1, and OpenBSD 7.5 and attempting to create a legacy wallet with the `createwallet` rpc (with `-deprecatedrpc=create_bdb`).
ACKs for top commit:
l0rinc:
utACK ddef914bbb
fanquake:
ACK ddef914bbb
Tree-SHA512: 261568700b95fc073e03db6ca64a5f0544d5aed337aee4275575c1d0d1373c2a96911947abd202da3ed7c3b7a662b700b0596c0dabefe4b50900a798eed7e118
8d7f8fabae doc: fix compiler flags for macOS configuration (marcofleon)
Pull request description:
Small CMake correction in the macOS build docs. My Mac will actually use AppleClang if I don't specify the compiler flags.
```bash
% cmake -B testbuild
-- The CXX compiler identification is AppleClang 15.0.0.15000309
```
```bash
% cmake -B testbuild -DCMAKE_C_COMPILER="$(brew --prefix llvm)/bin/clang" -DCMAKE_CXX_COMPILER="$(brew --prefix llvm)/bin/clang++"
-- The CXX compiler identification is Clang 18.1.8
```
ACKs for top commit:
maflcko:
review ACK 8d7f8fabae
brunoerg:
ACK 8d7f8fabae
Tree-SHA512: f25f0b38d5868d3182b79b8d0ff75ba7041e6282a6f4c83ed08ba7254ec9d8ae2553df36a4654a91ede5777e4c727dbdce6dcee7a5fe35bdd403d17e249ddd18
fa09cb41f5 refactor: Remove unused LogPrint (MarcoFalke)
3333415890 scripted-diff: LogPrint -> LogDebug (MarcoFalke)
Pull request description:
`LogPrint` has many issues:
* It seems to indicate that something is being "printed", however config options such as `-printtoconsole` actually control what and where something is logged.
* It does not mention the log severity (debug).
* It is a deprecated alias for `LogDebug`, according to the dev notes.
* It wastes review cycles, because reviewers sometimes point out that it is deprecated.
* It makes the code inconsistent, when both are used, possibly even in lines right next to each other (like in `InitHTTPServer`)
Fix all issues by removing the deprecated alias.
I checked all conflicting pull requests and at the time of writing there are no conflicts, except in pull requests that are marked as draft, are yet unreviewed, or are blocked on feedback for other reasons. So I think it is fine to do now.
ACKs for top commit:
stickies-v:
ACK fa09cb41f5
danielabrozzoni:
utACK fa09cb41f5
TheCharlatan:
ACK fa09cb41f5
Tree-SHA512: 14270f4cfa3906025a0b994cbb5b2e3c8c2427c0beb19c717a505a2ccbfb1fd1ecf2fd03f6c52d22cde69a8d057e50d2207119fab2c2bc8228db3f10d4288d0f
faa382ae76 ci, doc: Drop reference to `src/.bear-tidy-config` (Hennadii Stepanov)
d71ac76842 build: Remove Autotools-based build system (Hennadii Stepanov)
e268b48419 doc: Adjust `doc/design/libraries.md` (Hennadii Stepanov)
d209e4f156 doc: Drop mentions of `share/genbuild.sh` (Hennadii Stepanov)
Pull request description:
This PR deletes the Autotools-based build system.
The MSVC build system is deleted in https://github.com/bitcoin/bitcoin/pull/30731.
ACKs for top commit:
maflcko:
re-ACK faa382ae76🍦
TheCharlatan:
ACK faa382ae76
fanquake:
ACK faa382ae76
Tree-SHA512: 53df977b5b199a1c38f7f61a042a62b24831c559ba65a461b4ac1c96a1a56e2dfd676df79f1358fd1cc1749ff27e7b548086157f337d4f596c1054cb3d2d5739
8756ccd712 scripted-diff: Replace ParseHex[<std::byte>]("str") -> "str"_hex[_u8] (Hodlinator)
9cb687351f refactor: Prepare for ParseHex -> ""_hex scripted-diff (Hodlinator)
50bc017040 refactor: Hand-replace some ParseHex -> ""_hex (Hodlinator)
5b74a849cf util: Add consteval ""_hex[_v][_u8] literals (l0rinc)
dc5f6f6812 test refactor: util_tests - parse_hex clean up (Hodlinator)
2b5e6eff36 refactor: Make XOnlyPubKey tolerate constexpr std::arrays (Hodlinator)
403d86f1cc refactor: vector -> span in CCrypter (Hodlinator)
bd0830bbd4 refactor: de-Hungarianize CCrypter (Hodlinator)
d99c816971 refactor: Improve CCrypter related lines (Hodlinator)
7e1d9a8468 refactor: Enforce lowercase hex digits for consteval uint256 (Hodlinator)
Pull request description:
Motivation:
* Validates and converts the hex string into bytes at compile time instead of at runtime like `ParseHex()`.
* Eliminates runtime dependencies: https://github.com/bitcoin/bitcoin/pull/30377#issuecomment-2214432177, https://github.com/bitcoin/bitcoin/pull/30048#discussion_r1592108480
* Has stricter requirements than `ParseHex()` (disallows whitespace and uppercase hex digits) and replaces it in a bunch of places.
* Makes it possible to derive other compile time constants.
* Minor: should shave off a few runtime CPU cycles.
`""_hex` produces `std::array<std::byte>` as the momentum in the codebase is to use `std::byte` over `uint8_t`.
Also makes `uint256` hex string constructor disallow uppercase hex digits. Discussed: https://github.com/bitcoin/bitcoin/pull/30560#discussion_r1701323070
Surprisingly does not change the size of the Guix **bitcoind** binary (on x86_64-linux-gnu) by 1 single byte.
Spawned already merged PRs: #30436, #30482, #30532, #30560.
ACKs for top commit:
l0rinc:
ACK 8756ccd712
stickies-v:
Rebase re-ACK 8756ccd712, no changes since a096215c9c71a2ec03e76f1fd0bcdda0727996e0
ryanofsky:
Code review ACK 8756ccd712, just rebasing since last review and taking advantage of CScript constructors in #29369, also tweaking a code comment
Tree-SHA512: 9b2011b7c37e0ef004c669f8601270a214b388916316458370f5902c79c2856790b1b2c7c123efa65decad04886ab5eff95644301e0d84358bb265cf1f8ec195
74da8cb286 ci: Delete no longer needed workaround (Hennadii Stepanov)
Pull request description:
This PR removes a workaround that was necessary at some point during the development of the CMake staging branch.
ACKs for top commit:
fanquake:
ACK 74da8cb286
Tree-SHA512: 619a513efe86af8e24fc3b6e4124df8f3ff3699216a3f87a4385aeb5e3c605f2b035d1594604cd3efe66281ac879d954d412ee4ae8423408e46ebd32956883a5
With nId being incremented for each addr received,
an attacker could cause an overflow in the past.
(https://bitcoincore.org/en/2024/07/31/disclose-addrman-int-overflow/)
Even though that attack was made infeasible by
rate-limiting (PR #22387), to be on the safe side change the
type to an int64_t.
This makes it easier to track which spots refer to an nId
(as opposed to, for example, bucket index etc. which also use int)
Co-authored-by: Pieter Wuille <pieter@wuille.net>
66dd1b4e58 build: Drop no longer needed workaround (Hennadii Stepanov)
Pull request description:
This PR deletes a workaround that is no longer needed since https://github.com/bitcoin/bitcoin/pull/30508 was merged.
ACKs for top commit:
fanquake:
ACK 66dd1b4e58
Tree-SHA512: abb8e79b525989afe88f94899e4dc29c80d4593ea23f44c6b3d08710e6ddd1619e748798534973fa4ee9f48d9fad7226445b7a2cb4aec0bdb5d1b7ff2f6689ea
Makes it friendlier for potential future users of the kernel library if
they do not store the headers in a std::vector, but can guarantee
contiguous memory.
Makes it friendlier for potential future users of the kernel library if
they do not store the headers in a std::vector, but can guarantee
contiguous memory.
Makes it friendlier for potential future users of the kernel library if
they do not store the headers in a std::vector, but can guarantee
contiguous memory.
a865494dee lint: remove autotools packages (fanquake)
b02f29e7ef doc: replace Autotools with CMake (fanquake)
Pull request description:
These don't seem to be included in, i.e #30664.
ACKs for top commit:
maflcko:
lgtm ACK a865494dee
hebasto:
re-ACK a865494dee.
Tree-SHA512: bafa2675d7c819478fb9b3f44f557ec767acb8fa3c4a191b1b8a1e47352a4cb6cebbb3138d961058d846926359f5451241a8badcbe3edd7e067d69ecfc45df93
bd7ce05f9d test: fix `TestShell` initialization (late follow-up for #30463) (Sebastian Falbesoner)
Pull request description:
Creating a `TestShell` instance as stated in the [docs](https://github.com/bitcoin/bitcoin/blob/master/test/functional/test-shell.md) currently fails on master:
```
$ python3
Python 3.10.13 (main, Mar 15 2024, 07:36:23) [Clang 16.0.6 ] on openbsd7
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.insert(0, "/home/thestack/bitcoin/test/functional")
>>> from test_framework.test_shell import TestShell
>>> test = TestShell().setup(num_nodes=2, setup_clean_chain=True)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/thestack/bitcoin/test/functional/test_framework/test_shell.py", line 70, in __new__
TestShell.instance = TestShell.__TestShell()
TypeError: BitcoinTestFramework.__init__() missing 1 required positional argument: 'test_file'
```
Since #30463, BitcoinTestFramework instances expect the path of the calling test at construction, in order to find shared data like the configuration (config.ini) and the cache. Note that in contrast to actual functional tests, we can't simply pass `__file__` here, as the test shell module sits within the `test_framework` subfolder, so we have to navigate up to the parent directory and append some dummy test file name.
On the long-term we should probably add some TestShell instantation smoke-test to detect issues like this early. As I'm not too familiar with the CI I'm not sure what is a good way to achieve this (a functional test obviously can't be used, as that's already a BitcoinTestFramework test in itself), but happy to take suggestions.
ACKs for top commit:
ismaelsadeeq:
Tested ACK bd7ce05f9d
danielabrozzoni:
tACK bd7ce05f9d
brunoerg:
ACK bd7ce05f9d
Tree-SHA512: c3a2365e2cda48a233ee724673c490787981354914f33e10eadbbad9c68e8403d84c5551229a611401e743886539de380ba4bfcb77032b6c85731e3bbe962dc1
a563f41232 Remove second node since only 1 is needed for the test (Martin Saposnic)
1f4cdb3d69 Replace custom funding tx creation with MiniWallet. (Martin Saposnic)
Pull request description:
In response to issue https://github.com/bitcoin/bitcoin/issues/30600, optimizations have been implemented to enhance test efficiency and readability:
This PR refactors the `rpc_signrawtransactionwithkey.py` functional test to use MiniWallet for creating funding transactions. This simplifies the test code and improves performance by eliminating the need to mine new blocks for each funding transaction.
Key changes:
- Replaced custom `send_to_address` method with MiniWallet's `send_to` method
- Removed unnecessary setup of a clean chain and second node
- Simplified transaction creation and signing process
ACKs for top commit:
glozow:
ACK a563f41232
ismaelsadeeq:
code review ACK a563f41232
theStack:
ACK a563f41232
Tree-SHA512: 318959f89702b169453d537dafb822f5ef1921db1088941d8bbdb3171dd7a6ecad590e57a3802bc37bcf8992267ed6ffa7f156b229d9817ebf812bd35df509b5
78358ce09d ci: add libzmq3-dev to test-each-commit job (fanquake)
a2b1d2c5ec doc: remove bsdmainutils (fanquake)
36ff336d2e ci: remove bsdmainutils (fanquake)
Pull request description:
This was previously used to install `hexdump` (for the tests). However that isn't used by CMake. I'm not aware of any other tools from this package being used.
ACKs for top commit:
maflcko:
ACK 78358ce09d
hebasto:
ACK 78358ce09d.
Tree-SHA512: 01c1be81feba03a9645e3d382067df4cd7c64de184871c8d9691053a8f871fcedf48d298303554560df2cb1949fb35d5ce9ff20c751ff35789b7689d656c0287
fac587ea07 ci: Use C++23 once for testing (MarcoFalke)
fa053ab7c0 build: Add Centos Stream 9 EOL URL (MarcoFalke)
Pull request description:
There are no plans to switch to C++23 anytime soon in the next couple of years. The only place right now that is known to benefit is `src/compat/byteswap.h`.
However, it is still useful to test with the option, because deprecated, removed or changed language features, as well as compiler changes that are guarded by the language version will be tested and developers can learn about them upfront.
Also includes a minor doc fixup commit.
ACKs for top commit:
davidgumberg:
ACK fac587ea07
TheCharlatan:
ACK fac587ea07
Tree-SHA512: 1b81788eb5b4da77715d8b047279de65ae6b8920d5a21fd8cc94c3b0edb588ab8ffb7eaffb2f8b7806045de1d47ca85ca629f49038eca762f3136bf380cf3c87
7de0c99804 doc: update dev note examples for CMake (fanquake)
Pull request description:
Update the examples in the developer notes to work with CMake.
Also added an explicit `-DCMAKE_EXPORT_COMPILE_COMMANDS=ON` for clarity.
ACKs for top commit:
davidgumberg:
Tested ACK 7de0c99804
TheCharlatan:
ACK 7de0c99804
jonatack:
Tested ACK 7de0c99804 on arm64 macOS 14.6.1
Tree-SHA512: 561fe5e777c5b29a4f26309700c03a730c5bbb2f838630abfaa4174112ced66e733c2109cb429a1927f1f3692bf1945f6386bcaffe604a76ea24633932d39171
fa80d39d82 ci: Re-add configs removed in cmake migration (MarcoFalke)
Pull request description:
In commit 9730288a0c many configs were removed from the CI without explanation.
Fix it by adding them back.
Can be reviewed by looking at:
* the parity table https://gist.github.com/hebasto/2ef97d3a726bfce08ded9df07f7dab5e
* the installed packages
* the CI logs from before the cmake migration and the CI logs of this pull request
ACKs for top commit:
fanquake:
ACK fa80d39d82
Tree-SHA512: a33335e117750e6c2e1490bb621f67c466f901793e43abe1bd0e263ef16fdcbc9e88be55c206167f3a5ddb39c1df6989c0fb7a96d9240243c000ba2e7f5e2747
Previously they may have taken more than 10 seconds. Now they should
finish in less than one second.
This also allows to drop one dependency to be installed.
The rules have many issues:
* Most are redundant, because Python already has a built-in
IndentationError, a subclass of SyntaxError, to enforce whitespace.
* They are not enforced consistently anyway, see for examples [1][2]
below.
* They are stylistic rules where the author intentionally formatted the
code to be easier to read. Starting to enforce them now would make the
code harder to read and create frustration in the future.
Fix all issues by removing them.
[1]:
test/functional/feature_cltv.py:63:35: E272 [*] Multiple spaces before keyword
|
61 | # | Script to prepend to scriptSig | nSequence | nLockTime |
62 | # +-------------------------------------------------+------------+--------------+
63 | [[OP_CHECKLOCKTIMEVERIFY], None, None],
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ E272
[2]:
contrib/asmap/asmap.py:395:13: E306 [*] Expected 1 blank line before a nested definition, found 0
|
393 | prefix.pop()
394 | hole = not fill and (lhole or rhole)
395 | def candidate(ctx: Optional[int], res0: Optional[list[ASNEntry]],
| ^^^ E306
Any kind of syntax error is already reported, so there is no need to
enumerate all possible types of syntax errors of ancient versions of
Python 2 or 3.
Ideally all call sites should accept std::byte instead of uint8_t but those transformations are left to future PRs.
-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended 's/\bParseHex\(("[^"]*")\)/\1_hex_u8/g' $(git grep -l ParseHex -- :src ':(exclude)src/test/util_tests.cpp')
sed -i --regexp-extended 's/\bParseHex<std::byte>\(("[^"]*")\)/\1_hex/g' $(git grep -l ParseHex -- :src ':(exclude)src/test/util_tests.cpp')
sed -i --regexp-extended 's/\bScriptFromHex\(("[^"]*")\)/ToScript(\1_hex)/g' src/test/script_tests.cpp
-END VERIFY SCRIPT-
Co-Authored-By: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
- Adds using namespace.
- Extracts ToScript helper function from ScriptFromHex, to be used heavily in the next commit.
- Changes ScriptFromHex from using ParseHex to TryParseHex, now asserting the string is valid.
- Use even number of hex digits in comment (and apply replacement from next commit to only touch line once).
The following scripted-diff commit will replace ParseHex("...") with "..."_hex_u8, but this replacement will not work in cases where vectors are needed instead of arrays, and is not ideal in cases where std::byte is accepted.
For example, it is currently necessary to use _hex_v_u8 when calling CScript operator<< because that operator does not currently support std::array or std::byte.
Conversely, it is incorrect to use _hex_v instead of _hex in net_processing.cpp for the MakeAndPushMessage argument, because if the argument is a std::vector it is considered variable-length and serialized with a size prefix, but if the argument is a std::array or Span is it considered fixed length and serialized without a prefix.
By the same logic, it is also safe to change the NUMS_H constant in pubkey.cpp from a std::vector to std::array because it is never serialized.
* Use BOOST_CHECK_EQUAL_COLLECTIONS and BOOST_CHECK_EQUAL instead of deprecated BOOST_CHECK.
* Avoid repeating expected values.
* Break out repeated HEX_PARSE_INPUT and rename ParseHex_expected to HEX_PARSE_OUTPUT.
Done in preparation for adding a couple more tests in the next commit.
Co-Authored-By: l0rinc <pap.lorinc@gmail.com>
Beyond renaming it also adjusts whitespace and adds braces to conform to current doc/developer-notes.md.
TestEncrypt: Change iterator type to auto in ahead of vector -> span conversion.
Only touches functions that will be modified in next commit.
7ee5c3c5b2 Fix a few likely documentation typos (Lőrinc)
Pull request description:
Found them during CMake migration - and ran a quick spellcheck for the rest to cover any remaining ones
ACKs for top commit:
maflcko:
lgtm ACK 7ee5c3c5b2
Tree-SHA512: c6e7aa1e952e0d093745c4e6004c3907b7a215c6f998cc205307c0c68abcc067bf3f56e22af0deb1710186e8a871306f4bae8a35c74581e5299abcbbcddfaa75
948238a683 test: Remove FastRandomContext global (Ryan Ofsky)
fa0fe08eca scripted-diff: [test] Use g_rng/m_rng directly (MarcoFalke)
fa54cab473 test: refactor: Accept any RandomNumberGenerator in RandMoney (MarcoFalke)
68f77dd21e test: refactor: Pass rng parameters to test functions (Ryan Ofsky)
fa19af555d test: refactor: Move g_insecure_rand_ctx.Reseed out of the helper that calls MakeRandDeterministicDANGEROUS (MarcoFalke)
3dc527f460 test: refactor: Give unit test functions access to test state (Ryan Ofsky)
fab023e177 test: refactor: Make unsigned promotion explicit (MarcoFalke)
fa2cb654ec test: Add m_rng alias for the global random context (MarcoFalke)
fae7e3791c test: Correct the random seed log on a prevector test failure (MarcoFalke)
Pull request description:
This is mostly a style-cleanup for the tests' random generation:
1) `g_insecure_rand_ctx` in the tests is problematic, because the name is a leftover when the generator was indeed insecure. However, now the generator is *deterministic*, because the seed is either passed in or printed (c.f. RANDOM_CTX_SEED). Stating that deterministic randomness is insecure in the tests seems redundant at best. Fix it by just using `m_rng` for the name.
2) The global random context has many one-line aliases, such as `InsecureRand32`. This is problematic, because the same line of code may use the context directly and through a wrapper at the same time. For example in net_tests (see below). This inconsistency is harmless, but confusing. Fix it by just removing the one-line aliases.
```
src/test/net_tests.cpp: auto msg_data_1 = g_insecure_rand_ctx.randbytes<uint8_t>(InsecureRandRange(100000));
````
3) The wrapper for randmoney has the same problem that the same unit test uses the context directly and through a wrapper at the same time. Also, it has a single type of Rng hardcoded. Fix it by accepting any type.
ACKs for top commit:
hodlinator:
ACK 948238a683
ryanofsky:
Code review ACK 948238a683. Only changes since last review were changing a comments a little bit.
marcofleon:
Code review ACK 948238a683. Only changes since my last review are the improvements in `prevector_tests`.
Tree-SHA512: 69c6b46a42cb743138ee8c87ff26a588dbe083e3efb3dca49b8a133ba5d3b09e8bf01c590ec7e121a7d77cb1fd7dcacd927a9ca139ac65e1f7c6d1ec46f93b57
a0abcbd382 doc: Mention multipath specifier (Ava Chow)
0019f61fc5 tests: Test importing of multipath descriptors (Ava Chow)
f97d5c137d wallet, rpc: Allow importdescriptors to import multipath descriptors (Ava Chow)
32dcbca3fb rpc: Allow importmulti to import multipath descriptors correctly (Ava Chow)
64dfe3ce4b wallet: Move internal to be per key when importing (Ava Chow)
1692245525 tests: Multipath descriptors for scantxoutset and deriveaddresses (Ava Chow)
cddc0ba9a9 rpc: Have deriveaddresses derive receiving and change (Ava Chow)
360456cd22 tests: Multipath descriptors for getdescriptorinfo (Ava Chow)
a90eee444c tests: Add unit tests for multipath descriptors (Ava Chow)
1bbf46e2da descriptors: Change Parse to return vector of descriptors (Ava Chow)
0d640c6f02 descriptors: Have ParseKeypath handle multipath specifiers (Ava Chow)
a5f39b1034 descriptors: Change ParseScript to return vector of descriptors (Ava Chow)
0d55deae15 descriptors: Add DescriptorImpl::Clone (Ava Chow)
7e86541f72 descriptors: Add PubkeyProvider::Clone (Ava Chow)
Pull request description:
It is convenient to have a descriptor which specifies both receiving and change addresses in a single string. However, as discussed in https://github.com/bitcoin/bitcoin/issues/17190#issuecomment-895515768, it is not feasible to use a generic multipath specification like BIP 88 due to combinatorial blow up and that it would result in unexpected descriptors.
To resolve that problem, this PR proposes a targeted solution which allows only a single pair of 2 derivation indexes to be inserted in the place of a single derivation index. So instead of two descriptor `wpkh(xpub.../0/0/*)` and `wpkh(xpub.../0/1/*)` to represent receive and change addresses, this could be written as `wpkh(xpub.../0/<0;1>/*)`. The multipath specifier is of the form `<NUM;NUM>`. Each `NUM` can have its own hardened specifier, e.g. `<0;1h>` is valid. The multipath specifier can also only appear in one path index in the derivation path.
This results in the parser returning two descriptors. The first descriptor uses the first `NUM` in all pairs present, and the second uses the second `NUM`. In our implementation, if a multipath descriptor is not provided, a pair is still returned, but the second element is just `nullptr`.
The wallet will not output the multipath descriptors (yet). Furthermore, when a multipath descriptor is imported, it is expanded to the two descriptors and each imported on its own, with the second descriptor being implicitly for internal (change) addresses. There is no change to how the wallet stores or outputs descriptors (yet).
Note that the path specifier is different from what was proposed. It uses angle brackets and the semicolon because these are unused characters available in the character set and I wanted to avoid conflicts with characters already in use in descriptors.
Closes#17190
ACKs for top commit:
darosior:
re-ACK a0abcbd382
mjdietzx:
reACK a0abcbd382
pythcoiner:
reACK a0abcbd
furszy:
Code review ACK a0abcbd
glozow:
light code review ACK a0abcbd382
Tree-SHA512: 84ea40b3fd1b762194acd021cae018c2f09b98e595f5e87de5c832c265cfe8a6d0bc4dae25785392fa90db0f6301ddf9aea787980a29c74f81d04b711ac446c2
04fb085f6b build: remove old MSVC build system (fanquake)
Pull request description:
This has been superseded by CMake, and should not be blocked on anything for removal.
ACKs for top commit:
laanwj:
ACK 04fb085f6b
hebasto:
ACK 04fb085f6b.
TheCharlatan:
ACK 04fb085f6b
vasild:
ACK 04fb085f6b, except:
Tree-SHA512: 219005bd0de1a9e37dabe45c979601df3083368f4c5ea224f70cf4e6f52e49d8f25713015ce005be3b34b56377eaafa339be7c50c63ce7f3ce7841ae55f3221f
680f05ee98 build: fix version number post CMake (fanquake)
Pull request description:
CMake was merged after branching-off for `28.x`.
ACKs for top commit:
hebasto:
ACK 680f05ee98
TheCharlatan:
ACK 680f05ee98
vasild:
ACK 680f05ee98
Tree-SHA512: 18e10a49830b529f54f7c9ed81ff0489f9aac9779f34935ebe0051c742f3b7862ca5d3409a6d6347c2d9c6439dcdd1394333278308012fed473f5b00d80afdd5
41051290ab cmake: Ignore build subdirectories within source directory (Hennadii Stepanov)
6ce50fd9d0 doc: Update for CMake-based build system (Hennadii Stepanov)
9730288a0c ci: Migrate CI scripts to CMake (Hennadii Stepanov)
c360837ca5 cmake, lint: Adjust `lint_includes_build_config` (Hennadii Stepanov)
3885441ee0 cmake: Add presets for native Windows builds (Hennadii Stepanov)
7681746b20 cmake: Add vcpkg manifest file (Hennadii Stepanov)
8b6f1c4353 cmake: Add `Coverage` and `CoverageFuzz` scripts (Hennadii Stepanov)
65bdbc1ff2 cmake: Add `docs` build target (Hennadii Stepanov)
fb75ebbc33 cmake: Add compiler diagnostic flags (Hennadii Stepanov)
e821f0a37a cmake: Migrate Guix build scripts to CMake (Hennadii Stepanov)
747adb6ffe cmake: Add `Maintenance` module (Hennadii Stepanov)
1f60b30df0 cmake: Add `APPEND_{CPP,C,CXX,LD}FLAGS` cache variables (Hennadii Stepanov)
2b43c45b13 cmake: Add `AddWindowsResources` module (Hennadii Stepanov)
973a3b0c5d cmake: Implement `install` build target (Hennadii Stepanov)
84ac35cfd4 cmake: Add cross-compiling support (Hennadii Stepanov)
0d01c228a7 build: Generate `toolchain.cmake` in depends (Hennadii Stepanov)
91a799247d depends: Add host-specific `cmake_system_version` variables (Hennadii Stepanov)
9b31209b4c depends: Rename `cmake_system` -> `cmake_system_name` (Hennadii Stepanov)
4a5208a81d Revert "build, qt: Do not install *.prl files" (Hennadii Stepanov)
6522af62af depends: Amend handling flags environment variables (Hennadii Stepanov)
90cec4d251 cmake: Add `MULTIPROCESS` option (Hennadii Stepanov)
bb1a450dcb cmake: Build `bitcoin-chainstate` executable (Hennadii Stepanov)
aed38ea58c cmake: Build `bitcoinkernel` library (Hennadii Stepanov)
975d67369b cmake: Build `test_bitcoin-qt` executable (Hennadii Stepanov)
10fcc668a3 cmake: Add `WITH_DBUS` option (Hennadii Stepanov)
5bb5a4bc75 cmake: Add `libqrencode` optional package support (Hennadii Stepanov)
57a6e2ef4a cmake: Build `bitcoin-qt` executable (Hennadii Stepanov)
30f642952c cmake: Add `WERROR` option (Hennadii Stepanov)
c98d4a4c34 cmake: Add `REDUCE_EXPORTS` option (Hennadii Stepanov)
a01cb6e63f cmake: Add `HARDENING` option (Hennadii Stepanov)
a8a2e364ac cmake: Add Python-based tests (Hennadii Stepanov)
3d85379570 cmake: Add fuzzing options (Hennadii Stepanov)
908530e312 cmake: Add `SANITIZERS` option (Hennadii Stepanov)
8bb0e85631 cmake: Build `bench_bitcoin` executable (Hennadii Stepanov)
801735163a cmake: Add external signer support (Hennadii Stepanov)
353e0c9e96 cmake: Add `systemtap-sdt` optional package support (Hennadii Stepanov)
d2fda82b49 cmake: Add `libzmq` optional package support (Hennadii Stepanov)
ae7b39a0e1 cmake: Add `libminiupnpc` optional package support (Hennadii Stepanov)
6480e1dcdb cmake: Add `libnatpmp` optional package support (Hennadii Stepanov)
e73e9304a1 cmake: Build `bitcoin-util` executable (Hennadii Stepanov)
027c6d7caa cmake: Build `bitcoin-tx` executable (Hennadii Stepanov)
d10c5c34c3 cmake: Add wallet functionality (Hennadii Stepanov)
ab2e99b0d9 cmake: Create test suite for `ctest` (Hennadii Stepanov)
959370bd76 cmake: Build `test_bitcoin` executable (Hennadii Stepanov)
b27bf9700d cmake: Build `bitcoin-cli` executable (Hennadii Stepanov)
a9813df826 cmake: Build `bitcoind` executable (Hennadii Stepanov)
97829ce2d5 cmake: Add `FindLibevent` module (Hennadii Stepanov)
3118e40c61 cmake: Build `bitcoin_consensus` library (Hennadii Stepanov)
809a2f1929 cmake: Build `bitcoin_util` static library (Hennadii Stepanov)
0a9a521a70 cmake: Build `bitcoin_crypto` library (Hennadii Stepanov)
958971f476 cmake: Build `univalue` static library (Hennadii Stepanov)
752747fda8 cmake: Generate `obj/build.h` header (Hennadii Stepanov)
1f0a78edf3 cmake: Build `minisketch` static library (Hennadii Stepanov)
12bfbc8154 cmake: Build `leveldb` static library (Hennadii Stepanov)
51985c5304 cmake: Build `crc32c` static library (Hennadii Stepanov)
db7a198f29 cmake: Build `secp256k1` subtree (Hennadii Stepanov)
dbb7ed14e8 cmake: Add `ccache` support (Hennadii Stepanov)
cedfdf6c72 cmake: Redefine/adjust per-configuration flags (Hennadii Stepanov)
b6b5e732c8 cmake: Add global compiler and linker flags (Hennadii Stepanov)
f98327931b cmake: Add `TryAppendLinkerFlag` module (Hennadii Stepanov)
4a0af29697 cmake: Add `TryAppendCXXFlags` module (Hennadii Stepanov)
35cffc497d cmake: Add POSIX threads support (Hennadii Stepanov)
fd72d00ffe cmake: Add position independent code support (Hennadii Stepanov)
07069e2bb0 cmake: Add introspection module (Hennadii Stepanov)
27d687fc1f cmake: Add `config/bitcoin-config.h` support (Hennadii Stepanov)
fe5cdace5f cmake: Print compiler and linker flags in summary (Hennadii Stepanov)
70683884c5 cmake: Introduce interface libraries to encapsulate common flags (Hennadii Stepanov)
a2317e27b7 cmake: Add root `CMakeLists.txt` file (Hennadii Stepanov)
Pull request description:
This PR introduces a new CMake-based build system, which is a drop-in replacement for the current Autotools-based build system.
ML announcement: https://groups.google.com/g/bitcoindev/c/hgKkfQWzrTo
As discussed during the recent CoreDev meetup in April, the switch from Autotools to CMake is intended to happen as soon as possible after branching 28.x off, which means that 29.0 will be built using CMake.
This PR branch is essentially the [staging branch](https://github.com/hebasto/bitcoin/tree/cmake-staging), with every change reviewed and tested by a group of contributors, including (in alphabetical order):
- [**achow101**](https://github.com/achow101)
- [**fanquake**](https://github.com/fanquake)
- [**maflcko**](https://github.com/maflcko)
- [**m3dwards**](https://github.com/m3dwards)
- [**pablomartin4btc**](https://github.com/pablomartin4btc)
- [**real-or-random**](https://github.com/real-or-random)
- [**ryanofsky**](https://github.com/ryanofsky)
- [**sipsorcery**](https://github.com/sipsorcery)
- [**TheCharlatan**](https://github.com/TheCharlatan)
- [**theStack**](https://github.com/theStack)
- [**theuni**](https://github.com/theuni)
- [**vasild**](https://github.com/vasild)
Reviewing in a separate staging repo was suggested in https://github.com/bitcoin/bitcoin/pull/27060#issuecomment-1431798320.
The accompanying changes to the OSS-Fuzz project are available in https://github.com/hebasto/oss-fuzz/pull/8.
Please refer to the [build options parity table](https://gist.github.com/hebasto/2ef97d3a726bfce08ded9df07f7dab5e). The "auto" value is no longer available; non-default values must be specified explicitly. Additionally, the new default values have been chosen to suit the everyday build experience for the majority of developers.
System requirements for using the CMake-based build system:
- CMake >= 3.22 (if not available in your system's repository, it can be downloaded from https://cmake.org/download/)
- a build tool of your choice:
- any Make (GNU Make is no longer a requirement); GNU Make is still required to build depends
- Ninja (https://ninja-build.org/)
- MSBuild
- Xcode
A note for Windows users: The default installation of the latest version of MSVC 17.10.4 includes both CMake 3.28.3 and the vcpkg package manager).
---
We, the build system developers, kindly ask reviewers to refrain from making suggestions that are not directly related to the migration process or can be implemented separately. Bugs in the scripts and errors in the updated documentation should be the focus of this PR. Please be advised that comments not aligned with this PR's goal may be ignored.
Thank you all for your understanding.
ACKs for top commit:
maflcko:
review ACK 41051290ab🐥
sipsorcery:
ACK 41051290ab.
vasild:
ACK 41051290ab
TheCharlatan:
ACK 41051290ab
pablomartin4btc:
tACK 41051290ab
i-am-yuvi:
tACK [`4105129`](41051290ab)
theuni:
ACK 41051290ab.
fanquake:
ACK 41051290ab
Tree-SHA512: 6c1445054436c6c00ad63bfa0f19d64091a2b25c9bd694f85bf2218ac358ffb774d6c000685b3ca1e9b50401babed989fa2a0694b774c211d226bfd1944c9b39
fad0cf6f26 refactor: Use std::ranges::equal in GetNetworkForMagic (MarcoFalke)
fadf0a7e15 refactor: Remove Span operator==, Use std::ranges::equal (MarcoFalke)
Pull request description:
`std::span` removed the comparison operators, so it makes sense to remove them for the `Span` "backport" as well. Using `std::ranges::equal` also has the benefit that some `Span` temporary constructions can now be dropped.
This is required to move from `Span` toward `std::span`.
ACKs for top commit:
hodlinator:
Untested Code Review re-ACK fad0cf6
stickies-v:
ACK fad0cf6f26
TheCharlatan:
ACK fad0cf6f26
Tree-SHA512: 5b9d1826ceac2aabae2295bc89893dd23ac3a1cc0d41988331cdbdc21be531aa91795d5273819f349f79648c6c4f30ed31af6e7a3816153e92080061b92ffe00
8dec4e1294 fuzz: fix timeout in `crypto_fschacha20poly1305` (brunoerg)
Pull request description:
Fixes#30505
This PR fixes a timeout in `crypto_fschacha20poly1305` by reducing the number of iterations. I left it running for a while and noticed it speeds up the target and do not impact coverage.
ACKs for top commit:
maflcko:
lgtm ACK 8dec4e1294
stratospher:
ACK 8dec4e1. saw similar coverage stats (these are from different machines, saw more similar from same machine).
Tree-SHA512: 66cd689f10f09cf880c1f2c77aa481336a95a7e10b655d156b2dc9cd38165670000019a6fb3775485ccb5cce248939a52a31ebd755a6befed6274cc841d21d97
49f9b645ea refactor: Testnet4 - Replace uint256S("str") -> uint256{"str"} (Hodlinator)
Pull request description:
Ran scripted-diff from 2d9d752e4f:
```
sed -i --regexp-extended -e 's/\buint256S\("(0x)?([^"]{64})"\)/uint256{"\2"}/g' $(git grep -l uint256S)
```
Follow-up to Testnet4 introduction #29775 which overlapped with work on `uint256` `consteval` ctor #30560 (the latter includes the scripted-diff commit).
Going forward `uint256{}` should be used for constants instead of `uint256S()`.
ACKs for top commit:
maflcko:
review-ACK 49f9b645ea🐮
fjahr:
ACK 49f9b645ea
Tree-SHA512: 94fe5d9f1fb85e9ce5c3c4c5e4c31667e8cbb55ee691a4b5b3ae4172ccac38230281071023663965917f188b4c19bdf67afd4e3cdf69d89e97c65faea88af833
setup_clean_chain=True is deleted so it uses the default.
Also, vout is now returned from send_to_address,
so now there is no need to fetch it manually
Also remove not-needed code that was used with the old
transaction handling.
18d65d2772 test: use uint256::FromUserHex for RANDOM_CTX_SEED (stickies-v)
6819e5a329 node: use uint256::FromUserHex for -assumevalid parsing (stickies-v)
2e58fdb544 util: remove unused IsHexNumber (stickies-v)
8a44d7d3c1 node: use uint256::FromUserHex for -minimumchainwork parsing (stickies-v)
70e2c87737 refactor: add uint256::FromUserHex helper (stickies-v)
85b7cbfcbe test: unittest chainstatemanager_args (stickies-v)
Pull request description:
Since fad2991ba0, `uint256S` has been [deprecated](fad2991ba0 (diff-800776e2dda39116e889839f69409571a5d397de048a141da7e4003bc099e3e2R138)) because it is less robust than the `base_blob::FromHex()` introduced in [the same PR](https://github.com/bitcoin/bitcoin/pull/30482). Specifically, it tries to recover from length-mismatches, recover from untrimmed whitespace, 0x-prefix and garbage at the end, instead of simply requiring exactly 64 hex-only characters. _(see also #30532)_
This PR carves out the few `uint256S` callsites that may potentially prove a bit more controversial to change because they deal with user input and backwards incompatible behaviour change.
The main behaviour change introduced in this PR is:
- `-minimumchainwork` will raise an error when input is longer than 64 hex digits
- `-assumevalid` will raise an error when input contains invalid hex characters, or when it is longer than 64 hex digits
- test: the optional RANDOM_CTX_SEED env var will now cause tests to abort when it contains invalid hex characters, or when it is longer than 64 hex digits
After this PR, the remaining work to remove `uint256S` completely is almost entirely mechanical and/or test related. I will open that PR once #30560 is merged because it builds on that.
ACKs for top commit:
hodlinator:
re-ACK 18d65d2772
l0rinc:
ACK 18d65d2772
achow101:
ACK 18d65d2772
ryanofsky:
Code review ACK 18d65d2772. Very nice change that cleans up the API, adds checking for invalid values, makes parsing of values more consistent, and adds test coverage.
Tree-SHA512: ec118ea3d56e1dfbc4c79acdbfc797f65c4d2107b0ca9577c848b4ab9b7cb8d05db9f3c7fe8441a09291aca41bf671572431f4eddc59be8fb53abbea76bbbf86
fa7b9b99a2 refactor: Require std::input_iterator for all InputIterator in prevector (MarcoFalke)
d444441900 refactor: Allow CScript construction from any std::input_iterator (MarcoFalke)
Pull request description:
Currently only (pre)vector iterators and raw pointers are accepted. However, this makes it harder to construct from input iterators provided by other classes, such as `std::span`.
Fix that.
ACKs for top commit:
delta1:
reACK fa7b9b9
achow101:
ACK fa7b9b99a2
hodlinator:
ACK fa7b9b99a2
ryanofsky:
Code review ACK fa7b9b99a2
Tree-SHA512: 2760861f8bce42fb27dc0825e61621cb157f1ac3619a0834df38eb8319b6dcf9de43d90397a4c160f43340880c1553df638848e9057a27c792214331243ef4a5
fab0e834b8 bench: [refactor] iwyu (MarcoFalke)
Pull request description:
Missing includes are problematic, because:
* Upcoming releases of a C++ standard library implementation often minimize their internal header dependencies. For example, `_LIBCPP_REMOVE_TRANSITIVE_INCLUDES` (https://libcxx.llvm.org/DesignDocs/HeaderRemovalPolicy.html). This can lead to compile failures, which are easy to fix for developers, but may not be for users. For example, commit 138f867156 had to add missing includes to accommodate GCC 15 (and the commit had to be backported).
* A Bitcoin Core developer removing a feature from a module and wanting to drop the now unused includes may not be able to do so without touching other unrelated files, because those files rely on the transitive includes.
Moreover, missing or extraneous includes are problematic, because they may be confusing the code reader as to what the real dependencies are.
Finally, extraneous includes may slow down the build.
Fix all issues in `bench`, by applying the rule include-what-you-use (iwyu).
Follow-up pull requests will handle the other places.
ACKs for top commit:
hodlinator:
ACK fab0e834b8
achow101:
ACK fab0e834b8
TheCharlatan:
ACK fab0e834b8
hebasto:
ACK fab0e834b8.
brunoerg:
crACK fab0e834b8
stickies-v:
ACK fab0e834b8
Tree-SHA512: f079c05d3ddebafabbd5a6c76d43d17337d1a962b97ba0ee27612f91c58491e7ce4e4229be54bd6e75a15512798c6f59925d0a076a37c050f8b9ef455ae5c9a2
1bf9b70658 docs: Add 379 and 387 to bips.md (Ava Chow)
35ef34eab7 docs: Remove release 28.0 release notes fragments (Ava Chow)
f5cf43bb91 build: Bump to 28.99 (Ava Chow)
Pull request description:
* Bump to 28.99 in preparation for the 28.x branching
* Remove current release note fragments. They've been moved to [draft release notes]((https://github.com/bitcoin-core/bitcoin-devwiki/wiki/28.0-Release-Notes-Draft).) in the wiki.
* Updated bips.md with missing BIPs that were implemented a while ago.
ACKs for top commit:
maflcko:
lgtm ACK 1bf9b70658
tdb3:
re ACK 1bf9b70658
Tree-SHA512: 9d5b05330268bb233369d78f06011dbdcdcb87770c499ae743ac4833735ba657b6611ff1e332c9d4added854b694c5a3a961d911247303f7ddaaaa72149d5574
b061b35105 seeds: Regenerate mainnet seeds (virtu)
02dc45c506 seeds: Pull nodes from Luke's seeder (virtu)
7a2068a0ff seeds: Pull nodes from virtu's crawler (virtu)
Pull request description:
This builds on #30008 and adds data [exported](https://github.com/virtu/seed-exporter) by [my crawler](https://github.com/virtu/p2p-crawler) an additional source for seed nodes. Data covers all supported network types.
[edit: Added Luke's seeder as input as well.]
### Motivation
- Further decentralizes the seed node selection process (in the long term potentially enabling an _n_-source threshold for nodes to prevent a single source from entering malicious nodes)
- No longer need to manually curate seed node list for any network type: See last paragraph of OP in #30008. My crawler has been [discovering the handful of available cjdns nodes](https://21.ninja/reachable-nodes/nodes-by-net-type/) for around two months, all but one of which meet the reliability criteria.
- Alignment of uptime requirements for Onion and I2P nodes with those of clearnet nodes to 50%: If I'm reading the code correctly, seeders appear to optimize for up-to-dateness by using [lower connection timeouts](3c1a63c672/src/crawl.rs (L349)) than [Bitcoin Core](bc87ad9854/src/netbase.cpp (L40C27-L40C48)) to maximize throughput. Since my crawler does not have the same timeliness requirements, it opts for accuracy by using generous timeouts. As a result, its data contains additional eligible Onion (and other darknet nodes), as is shown in the histogram below. Around 4500 Onion nodes are discovered so far (blue); my data adds ~6400 more (orange); ~ 1500 nodes take longer than the default 20-second Bitcoin Core timeout and won't qualify as "good".

Here's the current results with 512 nodes for all networks except cjdns:
<details>
<summary>Using the extra data</summary>
```
IPv4 IPv6 Onion I2P CJDNS Pass
10335 2531 11545 1589 10 Initial
10335 2531 11545 1589 10 Skip entries with invalid address
5639 1431 11163 1589 8 After removing duplicates
5606 1417 11163 1589 8 Enforce minimal number of blocks
5606 1417 11163 1589 8 Require service bit 1
4873 1228 11163 1589 8 Require minimum uptime
4846 1225 11161 1588 8 Require a known and recent user agent
4846 1225 11161 1588 8 Filter out hosts with multiple bitcoin ports
512 512 512 512 8 Look up ASNs and limit results per ASN and per net
```
</details>
<details>
<summary>Before</summary>
```
IPv4 IPv6 Onion I2P CJDNS Pass
5772 1323 443 0 2 Initial
5772 1323 443 0 2 Skip entries with invalid address
4758 1110 443 0 2 After removing duplicates
4723 1094 443 0 2 Enforce minimal number of blocks
4723 1094 443 0 2 Require service bit 1
3732 867 443 0 2 Require minimum uptime
3718 864 443 0 2 Require a known and recent user agent
3718 864 443 0 2 Filter out hosts with multiple bitcoin ports
512 409 443 0 2 Look up ASNs and limit results per ASN and per net
```
</details>
### To dos
- [x] Remove manual nodes and update README
- [x] Mark nodes with connection times exceeding Bitcoin Core's default as bad in [exporter](https://github.com/virtu/seed-exporter): [done](https://github.com/virtu/seed-exporter/pull/12)
- [x] Regenerate mainnet seeds
- [x] Rebase, then remove WIP label once #30008 gets merged
ACKs for top commit:
achow101:
ACK b061b35105
fjahr:
utACK b061b35105
Tree-SHA512: 63e86220787251c7e8d2d5957bad69352e19ae17d7b9b2d27d8acddfec5bdafe588edb68d77d19c57f25f149de723e2eeadded0c8cf13eaca22dc33bd8cf92a0
1b41d45d46 wallet: bugfix: ensure atomicity in settings updates (ismaelsadeeq)
Pull request description:
This PR fixes#30620.
As outlined in the issue, creating two wallets with `load_on_startup=true` simultaneously results in only one wallet being added to the startup file.
The current issue arises because the wallet settings update process involves:
1. Obtaining the settings value while acquiring the settings lock.
2. Modifying the settings value.
3. Overwriting the settings value while acquiring the settings lock again.
This sequence is not thread-safe. Different threads could modify the same base value simultaneously, overwriting data from other workers without realizing it.
The PR attempts to fix this by modifying the chain interface's `updateRwSetting` method to accept a function that will be called with the settings reference. This function will either update or delete the setting and return an enum indicating whether the settings need to be overwritten in this or not.
Additionally, this PR introduces two new methods to the chain interface:
- `overwriteRwSetting`: This method replaces the setting with a new value.
Used in `VerifyWallets`
- `deleteRwSettings`: This method completely erases a specified setting.
This method is currently used only in `overwriteRwSetting`.
These changes ensure that updates are race-free across all clients.
ACKs for top commit:
achow101:
ACK 1b41d45d46
furszy:
self-code-ACK 1b41d45d46
Tree-SHA512: 50cda612b782aeb5e03e2cf63cc44779a013de1c535b883b57af4de22f24b0de80b4edecbcda235413baec0a12bdf0e5750fb6731c9e67d32e742d8c63f08c13
fa5b58ea01 test: Avoid intermittent block download timeout in p2p_ibd_stalling (MarcoFalke)
Pull request description:
Fixes#30704
The goal of the test is to check the stalling timeout, not the block download timeout.
On extremely slow hardware (for example qemu virtual hardware), downloading the 1023 blocks may take longer than the block download timeout.
Fix it by pinning the time using mocktime, and only advance it when testing the stalling timeout.
ACKs for top commit:
tdb3:
CR ACK fa5b58ea01
brunoerg:
utACK fa5b58ea01
Tree-SHA512: 9a9221f264bea52be5e9fe81fd319f5a6970cd315cc5e9f5e2e049c5d84619b19b9f6f075cda8d34565c2d6c17a88fb57e195c66c271e40f73119a77caecb6d7
Regenerate mainnet seeds from new sources without the need for hardcoded
data. Result has 512 nodes from each network type except cjdns, for
which only eight nodes were found that match the seed node criteria.
Pull additional nodes from virtu's crawler. Data includes sufficient
Onion and I2P nodes to align the uptime requirements for these networks
to that of clearnet nodes (i.e., 50%). Data also includes more than
three times the number of CJDNS nodes currently hardcoded into
nodes_main_manual.txt, so hardcoded nodes becomes obsolete.
Because we don't have type checking for command-line/settings/config
args, strings are interpreted as 'false' for non-boolean args.
By convention, this "forces" us to interpret negated strings as 'true',
which conflicts with the negated option definition in all the settings
classes (they expect negated options to always be false and ignore any
other value preceding them). Consequently, when retrieving all "wallet"
values from the command-line/settings/config, we also fetch the negated
string boolean value, which is not of the expected 'string' type.
This mismatch leads to an internal fatal error, resulting in an unclean
shutdown during initialization. Furthermore, this error displays a poorly
descriptive error message:
"JSON value of type bool is not of expected type string"
This commit fixes the fatal error by ensuring that only string values are
returned in the "wallet" settings list, failing otherwise. It also improves
the clarity of the returned error message.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
41ad84a00c seeds: Use fjahr's more up to date asmap (Ava Chow)
d8fd1e0faf seeds: Fixed seeds update (Ava Chow)
f1f24d7214 seeds: Add testnet4 fixed seeds file (Ava Chow)
8ace71c737 seeds: Remove manual onion and i2p seeds (Ava Chow)
ed5b86cbe4 seeds: Add testnet instructions (Ava Chow)
0676515397 seeds: Also pull from achow101 seeder (Ava Chow)
5bab3175a6 makeseeds: Configurable minimum blocks for testnet4's smaller chain (Ava Chow)
d2465dfac6 makeseeds: Shuffle ips after parsing (Ava Chow)
af550b3a0f makeseeds: Support CJDNS (Ava Chow)
d5a8c4c4bd makeseeds: Update user agent regex (Ava Chow)
Pull request description:
The [DNS seeder](https://github.com/achow101/dnsseedrs) that I wrote collects statistics on node reliability in the same way that sipa's seeder does, and also outputs this information in the same file format. Thus it can also be used in our fixed seeds update scripts. My seeder additionally crawls onion v3, i2p, and cjdns, so will now be able to set those fixed seeds automatically rather than curating manual lists.
In doing this update, I've found that `makeseeds.py` is missing newer versions from the regex as well as cjdns support; both of these have been updated.
I also noticed that the testnet fixed seeds are all manually curated and sipa's seeder does not appear to publish any testnet data. Since I am also running the seeder for testnet, I've added the commands to generate testnet fixed seeds from my seeder's data too.
Lastly, I've updated all of the fixed seeds. However, since my seeder has not found any cjdns nodes that met the reliability criteria (possibly due to connectivity issues present in those networks), I've left the previous manual seeds for that network.
ACKs for top commit:
fjahr:
re-ACK 41ad84a00c
virtu:
ACK [41ad84a](41ad84a00c)
Tree-SHA512: 6ba0141f053d9d6ae7d8c9574f061be38f3e65b28de1d6660c1885ab942623b5a0ec70754b4fcfc5d98fe970f5f179a940d5880b5061ed698f7932500e01d3ee
5b4f34006d devtools, utxo-snapshot: Fix block height out of range (pablomartin4btc)
Pull request description:
<details>
<summary>Fixing a <a href="https://github.com/bitcoin/bitcoin/pull/28553#pullrequestreview-2251032570">bug</a> in <code>utxo_snapshot.sh</code>.</summary>
```
/contrib/devtools/utxo_snapshot.sh 840000 snapshot2.dat ./src/bitcoin-cli -datadir=${AU_DATADIR}
Do you want to disable network activity (setnetworkactive false) before running invalidateblock? (Y/n):
Disabling network activity
false
error code: -8
error message:
Block height out of range
```
And the user will see the following in the node and it would stay there if not reset:
```
2024-08-21T14:44:13Z UpdateTip: new best=00000000000000afa0cd000a16e244f56032735d41acd32ac00337aceb2a5240 height=235382 version=0x00000002 log2_work=69.987697 tx=17492185 date='2013-05-09T23:54:32Z' progress=0.016219 cache=71.0MiB(571085txo)
2024-08-21T14:44:13Z UpdateTip: new best=0000000000000087c5e0b820afff496b95ba44ad64640c73b234d3261d3f99d2 height=235383 version=0x00000002 log2_work=69.987750 tx=17492341 date='2013-05-09T23:54:47Z' progress=0.016219 cache=71.0MiB(571291txo)
2024-08-21T14:44:13Z UpdateTip: new best=000000000000014a4b5fddf3c8abb6209247255ca9e8df786b271dd1b2ac82a6 height=235384 version=0x00000002 log2_work=69.987804 tx=17492344 date='2013-05-10T00:20:18Z' progress=0.016219 cache=71.0MiB(571297txo)
2024-08-21T14:44:13Z SetNetworkActive: false
```
</details>
This is a "temporary" fix until #29553 gets merged, which will remove the script entirely.
Handle the "Block height out of range" error gracefully by checking if the node has synchronized to or beyond the required block height, otherwise without this validation the node would keep the network disabled if the user selected that option.
<details>
<summary>Provide a user-friendly message if the block height is out of range and exit the script cleanly.</summary>
```
/contrib/devtools/utxo_snapshot.sh 840000 snapshot2.dat ./src/bitcoin-cli -datadir=${AU_DATADIR}
Error: The node has not yet synchronized to block height 840001.
Please wait until the node has synchronized past this block height and try again.
```
</details>
ACKs for top commit:
achow101:
ACK 5b4f34006d
fjahr:
tACK 5b4f34006d
Tree-SHA512: 2b71286b627872d7cfdb367e29361afa3806a7ef9d65075b93892b735ff2ab729069e2f7259d30262909e73cef17fb7dca231615cc1863968cd042f4a2a4f901
e1d5dd732d test: check xor.dat recreated when missing (tdb3)
d1610962bf test: add null block xor key (tdb3)
1ad999b9da refactor: lift NUM_XOR_BYTES (tdb3)
d8399584dd refactor: move read_xor_key() to TestNode (tdb3)
d43948c3ef refactor: use unlink rather than os.remove (tdb3)
c8176f758b test: add blocks_key_path (tdb3)
Pull request description:
Builds on PR #30657.
Refactors `read_xor_key()` from `util.py` to `test_node.py` (comment https://github.com/bitcoin/bitcoin/pull/30657#discussion_r1723358327)
Adds a check that `xor.dat` is created when missing (comment https://github.com/bitcoin/bitcoin/pull/30657#discussion_r1717724161)
Help states:
```
-blocksxor
Whether an XOR-key applies to blocksdir *.dat files. The created XOR-key
will be zeros for an existing blocksdir or when `-blocksxor=0` is
set, and random for a freshly initialized blocksdir. (default: 1)
```
ACKs for top commit:
maflcko:
ACK e1d5dd732d
achow101:
ACK e1d5dd732d
theStack:
re-ACK e1d5dd732d
brunoerg:
reACK e1d5dd732d
Tree-SHA512: 325912ef646ec88e0a58e1ece263a2b04cbc06497e8fe5fcd603e509e80c6bcf84b09dd52dfac60e23013f07fc2b2f6db851ed0598649c3593f452c4a1424bd9
- Settings updates were not thread-safe, as they were executed in
three separate steps:
1) Obtain settings value while acquiring the settings lock.
2) Modify settings value.
3) Overwrite settings value while acquiring the settings lock.
This approach allowed concurrent threads to modify the same base value
simultaneously, leading to data loss. When this occurred, the final
settings state would only reflect the changes from the last thread
that completed the operation, overwriting updates from other threads.
Fix this by making the settings update operation atomic.
- Add test coverage for this behavior.
Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
Accepting any Rng in RandMoney makes tests more flexible to use a
different Rng. Also, passing in the Rng clarifies the call sites, so
that they all use g_rand_ctx explicitly and consistently.
The global g_insecure_rand_ctx will be removed in the future, so
removing it from this helper is useful.
Also, tying the two concepts of the global internal RNGState and the
global test-only rng context is a bit confusing, because tests can
simply use the m_rng, if it exists. Also, tests may seed more than one
random context, or none at all, or a random context of a different type.
Fix all issues by moving the Reseed call to the two places where it is
used.
60055f1abc test: replace deprecated secp256k1 context flags usage (Sebastian Falbesoner)
Pull request description:
The flags `SECP256K1_CONTEXT_{SIGN,VERIFY}` have been marked as deprecated since libsecp256k1 version 0.2 (released in December 2022), with the recommendation to use SECP256K1_CONTEXT_NONE instead, see https://github.com/bitcoin-core/secp256k1/pull/1126 and 1988855079/CHANGELOG.md (L132). Note that in contrast to other deprecated functions/variables, these defines don't have a deprecated attribute and hence don't lead to a compiler warning (see https://github.com/bitcoin-core/secp256k1/pull/1126#discussion_r922105271), so they are not easily detected.
ACKs for top commit:
TheCharlatan:
ACK 60055f1abc
ismaelsadeeq:
utACK 60055f1abc
tdb3:
light CR and test ACK 60055f1abc
Tree-SHA512: d93cf49e018a58469620c0d2f50242141f22dabc70afb2a7cd64e416f4f55588714510ae5a877376dd1e6b6f7494261969489af4b18a1c9dff0d0dfdf93f1fa8
fa5aeab3cb test: Avoid duplicate curl call in get_previous_releases.py (MarcoFalke)
Pull request description:
Seems odd having to translate `404` to "Binary tag was not found". Also, it seems odd to write a for-loop over a list with one item.
Fix both issues by just using a single call to `curl --fail ...`.
Can be tested with: `test/get_previous_releases.py -b v99.99.99`
Before:
```
Releases directory: releases
Fetching: https://bitcoincore.org/bin/bitcoin-core-99.99.99/bitcoin-99.99.99-x86_64-linux-gnu.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 286k 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
Binary tag was not found
```
After:
```
Releases directory: releases
Fetching: https://bitcoincore.org/bin/bitcoin-core-99.99.99/bitcoin-99.99.99-x86_64-linux-gnu.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 286k 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
curl: (22) The requested URL returned error: 404
ACKs for top commit:
fanquake:
ACK fa5aeab3cb
brunoerg:
utACK fa5aeab3cb
tdb3:
tested ACK fa5aeab3cb
Tree-SHA512: d5d31e0bccdd9de9b4a8ecf2e69348f4e8cee773050c8259b61db1ce5de73f6fbfffbe8c4d2571f7bef2de29cb42fd244573deebfbec614e487e76ef41681b9c
Removes dependency on unsafe and deprecated uint256S.
This makes parsing more strict, by requiring RANDOM_CTX_SEED
to be a string of up to 64 hex digits (optionally prefixed with
"0x"), whereas previously any string would be accepted, with
non-hex characters silently ignored and input longer than
64 characters (ignoring "0x" prefix) silently trimmed.
Can be tested with:
$ RANDOM_CTX_SEED=z ./src/test/test_bitcoin --log_level=all --run_test=timeoffsets_tests/timeoffsets_warning -- -printtoconsole=1 | grep RANDOM_CTX_SEED
RANDOM_CTX_SEED must consist of up to 64 hex digits ("0x" prefix allowed), it was set to: 'z'.
Co-Authored-By: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Removes dependency on unsafe and deprecated uint256S.
This makes parsing more strict, by returning an error
when the input contains non-hex characters, or when it
contains more than 64 hex digits.
Also make feature_assumevalid.py more robust by using CBlock.hash
which is guaranteed to be 64 characters long, as opposed to the
variable-length hex(CBlock.sha256)
Removes dependency on unsafe and deprecated uint256S.
This makes parsing more strict, by returning an error
when the input contains more than 64 hex digits.
FromUserHex will be used in future commits to construct
uint256 instances from user hex input without being
unnecessarily restrictive on formatting by allowing
0x-prefixed input that is shorter than 64 characters.
cccc5bfd35 test: Enable detect_leaks=1 in ASAN_OPTIONS explicitly (MarcoFalke)
Pull request description:
It should be enabled by default, but being explicit can't hurt.
ACKs for top commit:
fanquake:
ACK cccc5bfd35
Tree-SHA512: ed284abd05c7a99c30b509844aa75785a5ccb506d8296a71347b4c328750a6a4ed1f87e7a3ec36ab17f27b467c033cc8ca5eb5e2b951f2ae7473327c5eb1ddae
59ff17e5af miner: adjust clock to timewarp rule (Sjors Provoost)
e929054e12 Add timewarp attack mitigation test (Sjors Provoost)
e85f386c4b consensus: enable BIP94 on regtest (Sjors Provoost)
dd154b0568 consensus: lower regtest nPowTargetTimespan to 144 (Sjors Provoost)
Pull request description:
Because #30647 reduced the timewarp attack threshold from 7200s to 600s, our miner code will fail to propose a block template (on testnet4) if the last block of the previous period has a timestamp two hours in the future. This PR fixes that and also adds a test.
The non-test changes in the last commit should be in v28, otherwise miners have to patch it themselves. If necessary I can split that out into a separate PR, but I prefer to get the tests in as well.
In order to add the test, we activate BIP94 on regtest.
In order for the test to run faster, we reduce its difficulty retarget period to 144, the same number that's already used for softfork activation logic. Regtest does not actually adjust its difficulty, so this change has no effect (except for `getnetworkhashps`, see commit).
An alternative approach would be to run this test on testnet4, by hardcoding its first 2015 in the test suite. But since the timewarp mitigation is a serious candidate for a future mainnet softfork, it seems better to just deploy it on regtest.
The next commits add a test and fix the miner code.
The `MAX_TIMEWARP` constant is moved to `consensus.h` so both validation and miner code have access to it.
ACKs for top commit:
achow101:
ACK 59ff17e5af
fjahr:
ACK 59ff17e5af
glozow:
ACK 59ff17e5af
Tree-SHA512: 50af9fdcba9b0d5c57e1efd5feffd870bd11b5318f1f8b0aabf684657f2d33ab108d5f00b1475fe0d38e8e0badc97249ef8dda20c7f47fcc1698bc1008798830
Integer promotion will already turn the `signed` into `unsigned` in
those lines. However, make the `unsigned` explicit so that the code is
clearer and a compiler warning is avoided when switching to m_rng:
| test/validation_block_tests.cpp: warning: comparison of integers of different signs: 'int' and 'const unsigned int' [-Wsign-compare]
| 136 | bool gen_invalid = m_rng.randrange(100) < invalid_rate;
| | ~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~
| 137 | bool gen_fork = m_rng.randrange(100) < branch_rate;
| | ~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~
| 2 warnings generated.
rand_cache is unused since commit
16329224e7, so it can be removed
rand_seed is wrong since commit
022cf47dd7, because it is no longer
printing the seed that was used to seed the global random context in
tests. Instead, it prints a (random-ish) value derived from the global
random context via InsecureRand256().
Finally, the for loop creating new prevector_tester objects will always
use the same seed since commit fae43a97ca,
because repeated calls to SeedInsecureRand/SeedRandomForTest will always
reseed the global with the same "static const" seed.
Fix all issues by
* removing the unused rand_cache,
* removing the call to SeedRandomForTest which restored the same seed on
every call in the process, and
* Reseeding the global random context with the (random-ish) rand_seed.
917e70a620 test: assumeutxo: check that UTXO-querying RPCs operate on snapshot chainstate (Sebastian Falbesoner)
Pull request description:
Inspired by some manual testing I did for #28553, this PR checks that RPCs which explicitly query the UTXO set database (i.e. `gettxoutsetinfo`, `scantxoutset` and `gettxout`) operate on the snapshot chainstate as expected.
ACKs for top commit:
fjahr:
utACK 917e70a620
achow101:
ACK 917e70a620
tdb3:
ACK 917e70a620
Tree-SHA512: 40ecd1c5dd879234df1667fa5444a1fbbee9b7c456f597dc982d1a2bce46fe9107711b005ab829e570ef919a4914792f72f342d71d92bad2ae9434b5e68d5bd3
Handle the Block height out of range error gracefully by checking if
the node has synchronized to or beyond the required block height,
otherwise without this validation the node would keep the network
disabled if the user selected that option.
Provide a user-friendly message if the block height is out of range
and exit the script cleanly.
fa899fb7aa fuzz: Speed up utxo_snapshot fuzz target (MarcoFalke)
fa386642b4 fuzz: Speed up utxo_snapshot by lazy re-init (MarcoFalke)
fa645c7a86 fuzz: Remove unused DataStream object (MarcoFalke)
fae8c73d9e test: Disallow fee_estimator construction in ChainTestingSetup (MarcoFalke)
Pull request description:
Two commits to speed up unit and fuzz tests.
Can be tested by running the fuzz target and looking at the time it took, or by looking at the flamegraph. For example:
```
FUZZ=utxo_snapshot perf record -g --call-graph dwarf ./src/test/fuzz/fuzz -runs=100
hotspot ./perf.data
ACKs for top commit:
TheCharlatan:
Re-ACK fa899fb7aa
marcofleon:
Re ACK fa899fb7aa
brunoerg:
ACK fa899fb7aa
Tree-SHA512: d3a771bb12d7ef491eee61ca47325dd1cea5c20b6ad42554babf13ec98d03bef8e7786159d077e59cc7ab8112495037b0f6e55edae65b871c7cf1708687cf717
The flags SECP256K1_CONTEXT_{SIGN,VERIFY} have been deprecated since
libsecp256k1 version 0.2 (released in December 2022), with the
recommendation to use SECP256K1_CONTEXT_NONE instead.
This currently has no effect due to fPowNoRetargeting,
except for the getnetworkhashps when called with -1.
It will when the next commit enforces the timewarp attack mitigation on regtest.
16e95bda86 Move maximum timewarp attack threshold back to 600s from 7200s (Matt Corallo)
Pull request description:
In 6bfa26048d the testnet4 timewarp attack fix block time variation was increased from the Great Consensus Cleanup value of 600s to 7200s on the thesis that this allows miners to always create blocks with the current time. Sadly, doing so does allow for some nonzero inflation, even if not a huge amount.
While it could be that some hardware ignores the timestamp provided to it over Stratum and forces the block header timestamp to the current time, I'm not aware of any such hardware, and it would also likely suffer from random invalid blocks due to relying on NTP anyway, making its existence highly unlikely.
This leaves the only concern being pools, but most of those rely on work generated by Bitcoin Core (in one way or another, though when spy mining possibly not), and it seems likely that they will also not suffer any lost work. While its possible that a pool does generate invalid work due to spy mining or otherwise custom logic, it seems unlikely that a substantial portion of hashrate would do so, making the difference somewhat academic (any pool that screws this up will only do so once and the network would come out just fine).
Further, while we may end up deciding these assumptions were invalid and we should instead use 7200s, it seems prudent to try with the value we "want" on testnet4, giving us the ability to learn if the compatibility concerns are an issue before we go to mainnet.
ACKs for top commit:
fjahr:
tACK 16e95bda86
achow101:
ACK 16e95bda86
murchandamus:
crACK 16e95bda86
Tree-SHA512: ae46d03b728b6e23cb6ace64c9813bc01c01e38dd7f159cf0fab53b331ef84b3b811edab225453ccdfedb53b242f55b0efd69829782657490fe393d24dacbeb2
6ed424f2db wallet: fix, detect blank legacy wallets in IsLegacy (furszy)
Pull request description:
Blank legacy wallets do not have active SPKM. They can only be
detected by checking the descriptors' flag or the db format.
This enables the migration of blank legacy wallets in the GUI.
To test this:
1) Create a blank legacy wallet.
2) Try to migrate it using the GUI's toolbar "Migrate Wallet" button.
-> In master: The button will be disabled because `CWallet::IsLegacy()` returns false for blank legacy wallet.
-> In this PR: the button will be enabled, allowing the migration of legacy wallets.
ACKs for top commit:
achow101:
ACK 6ed424f2db
tdb3:
ACK 6ed424f2db
glozow:
ACK 6ed424f2db
Tree-SHA512: c06c4c4c2e546ccb033287b9aa3aee4ca36b47aeb2fac6fbed5de774b65caef9c818fc8dfdaac6ce78839b2d5d642a5632a5b44c5e889ea169ced80ed50501a7
The seeders now produce onion and i2p seeds, so there is no need to keep these
in the manual list.
Although should also be produced, there are not enough
good ones detected by the seeder, so we keep the manual seeds for them.
faa1b9b0e6 test: add functional test for XORed block/undo files (`-blocksxor`) (Sebastian Falbesoner)
6b3676be3e test: refactor: move `read_xor_key`/`util_xor` helpers to util module (Sebastian Falbesoner)
Pull request description:
This PR adds a dedicated functional test for XORed block data/undo file support (bitcoind option `-blocksxor`, see PR #28052). In order to verify that the XOR pattern has been applied, the {blk,rev}*.dat files are rewritten un-XORed manually by the test while the node is shut down; the node is then started again with `-blocksxor=0`, and both the data and undo files are verified via the `verifychain` RPC (with checklevel=2). Note that starting bitcoind with `-blocksxor=0` fails if a xor key is present already, which is also tested explicitly.
Fixes#30599.
ACKs for top commit:
glozow:
ACK faa1b9b0e6
maflcko:
ACK faa1b9b0e6
ismaelsadeeq:
Tested ACK faa1b9b0e6
Tree-SHA512: e1df106f6b4e3ba67eca108e36d762f1b991673b881934b84cd36946496a09ce9c329c1363c36aa29409137ae4881e2d177e651359686511632ddf2870f7ca8e
The re-init is expensive, so skip it if there is no need.
Also, add an even faster fuzz target utxo_snapshot_invalid, which does
not need any re-init at all.
f550a8e035 Rename ReleaseWallet to FlushAndDeleteWallet (furszy)
64e736d79e wallet: WaitForDeleteWallet, do not expect thread safety (Ryan Ofsky)
8872b4a6ca wallet: rename UnloadWallet to WaitForDeleteWallet (furszy)
5d15485aaf wallet: unload, notify GUI as soon as possible (furszy)
Pull request description:
Coming from #29073.
Applied ryanofsky suggested changes on https://github.com/bitcoin/bitcoin/issues/29073#issuecomment-2274237242 with few modifications coming from https://github.com/bitcoin/bitcoin/pull/18338#issuecomment-605060348.
The only point I did not tackle from https://github.com/bitcoin/bitcoin/pull/18338#issuecomment-605060348 is:
> * Move log print and flush out of ReleaseWallet into CWallet destructor
Because it would mean every `CWallet` object would flush data to disk during destruction. Which is not necessary for wallet tool utilities and unit tests.
ACKs for top commit:
achow101:
ACK f550a8e035
ryanofsky:
Code review ACK f550a8e035. Just a simple rename since last review
ismaelsadeeq:
Re-ACK f550a8e035
Tree-SHA512: e2eb69bf36883c514f601f4838ae6a41113996b9559abf8dc2b46e16bbcdad401195ac0f2b9d1fb55a10e78bb8ea9953788a168c80474e3f101350d208cb3bd2
The crawlers are not guaranteed to output nodes in a random order, so
shuffle the ips list after parsing to break any biasing that may be
caused by the output order.
Update the user agent regex to match all 3 digits of the version number,
not just the first 2 digits.
Also updates it to include 24.2, 25.2, 26.1, 27.0, 27.1, 27.99, 28.0 and
28.99.
1610643c8b chainparams: add mainnet assumeutxo param at height 840_000 (Sjors Provoost)
Pull request description:
This adds snapshot parameters for mainnet block 840,000.
You can generate the snapshot yourself using `./contrib/devtools/utxo_snapshot.sh` or download my torrent:
* torrent: `magnet:?xt=urn:btih:596c26cc709e213fdfec997183ff67067241440c&dn=utxo-840000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`
It would be a good idea to test:
1. That you can produce the same snapshot file, sha256 sum:
```
dc4bb43d58d6a25e91eae93eb052d72e3318bd98ec62a5d0c11817cefbba177b utxo-840000.dat
```
2. That the snapshot works
ACKs for top commit:
fjahr:
re-ACK 1610643c8b
achow101:
ACK 1610643c8b
theStack:
Tested ACK 1610643c8b
mzumsande:
tested ACK 1610643c8b
willcl-ark:
tACK 1610643c8b
Tree-SHA512: 581d8e86379bb044324f04f8559dd0a8946b6e2b145d5f25b38727b30b8cf13d6ac3c8777ff06554d3cf1a072809f7b5fbd693239868578f25dceafe5ba5f57c
9b29755520 Deduplicate list of chain strings in RPC help texts (Martin Saposnic)
Pull request description:
As mentioned in issue https://github.com/bitcoin/bitcoin/issues/30645:
Many command line parameter and RPC help texts currently contain the list of chain/network names hardcoded ("main, test, testnet4, signet, regtest"), which is error-prone as it can easily happen to miss an instance if the list ever changes again.
This PR deduplicates the list of possible chain/network strings in RPC/parameter help texts, and it creates a macro `LIST_CHAIN_NAMES` in src/chainparamsbase.h. In the future, there is only 1 place where that list of possible values lives, so maintainability is improved and errors are avoided.
All three places where this change impacts:
```
./bitcoin-cli --help
./bitcoin-cli help getblockchaininfo
./bitcoin-cli help getmininginfo
```
They all return the correct string `"main, test, testnet4, signet, regtest"`
See https://github.com/bitcoin/bitcoin/pull/30642#discussion_r1714711575
ACKs for top commit:
maflcko:
lgtm ACK 9b29755520
achow101:
ACK 9b29755520
MarnixCroes:
ACK 9b29755520
theStack:
ACK 9b29755520
danielabrozzoni:
ACK 9b29755520
Tree-SHA512: 1e961bcbe40b0f17a87a2437eb4ba1bb89468fd1b5a39599d72a00ef75cb4009e7d2f05d0a621bb904fecf681c55b8a219fcfe4d44d5d27f27cdda20882b1323
770b0348c0 qt: Update translation source file for v28.0 string freeze (Hennadii Stepanov)
Pull request description:
This PR updates the `src/qt/locale/bitcoin_en.xlf` translation source file according to the [Release schedule for 28.0](https://github.com/bitcoin/bitcoin/issues/29891).
Note for reviewers: it is expected to get a zero diff after running `make -C src translate` locally.
ACKs for top commit:
stickies-v:
re-ACK 770b0348c0
pablomartin4btc:
re-ACK 770b0348c0
Tree-SHA512: 11dd26c470411aefc2e4f897c605162027a00e2a0ab1dcec9a1784c053349a3feaeedda7b649476ff528231801629e0ef342a48430ef54a4ec75ac1548c56d4f
8f2522d242 gui: Use menu for wallet migration (Ava Chow)
d56a450bf5 gui: Use wallet name for wallet migration rather than WalletModel (Ava Chow)
c3918583dd gui: don't remove wallet manually before migration (furszy)
bfba63880f gui: Consolidate wallet display name to GUIUtil function (Ava Chow)
28fc562f26 wallet, interfaces: Include database format in listWalletDir (Ava Chow)
Pull request description:
Currently the Migrate Wallet menu item can only be used to migrate the currently loaded wallet. This is not suitable for the future when legacy wallets can no longer be loaded at all, but should still be able to be migrated. This PR changes that menu item into a menu list like Open Wallet and lets users migrate any legacy wallet in their wallet directory regardless of the wallets loaded.
One issue I ran into was dealing with encrypted wallets. Ideally, we would detect whether a wallet is encrypted, and prompt the user for their passphrase at that time. However, that's actually difficult to do in the GUI since migration will unload the wallet if it was already loaded, and reload it without connecting it to any signals or interfaces. Only then can it detect whether a wallet is encrypted, but then there is no `WalletModel` or even an `interfaces::Wallet` that the GUI could use to unlock it via a callback.
To deal with this, I've opted to just add a button to the migration dialog box that has the user enter their passphrase first, along with instructional text to use that button if their wallet was encrypted. If the user enters the wrong passphrase or clicked the other button that does not prompt for the passphrase, migration will fail with a message indicating that the passphrase was incorrect.
ACKs for top commit:
hebasto:
ACK 8f2522d242.
furszy:
ACK 8f2522d
Tree-SHA512: a0e3b70dbfcacb89617956510ebcea94cad8617a987c68fe39fa16ac1721190b7cf7afc156c39b9032920cfb67b5d4ca28791681f5021d92d16acc691387afa1
Once legacy wallets can no longer be loaded, we need to be able to
migrate them without loading. Thus we should use a menu that lists the
wallets in the wallet directory instead of an action which migrates the
currently loaded wallet.
In 6bfa26048d the testnet4 timewarp
attack fix block time variation was increased from the Great
Consensus Cleanup value of 600s to 7200s on the thesis that this
allows miners to always create blocks with the current time. Sadly,
doing so does allow for some nonzero inflation, even if not a huge
amount.
While it could be that some hardware ignores the timestamp provided
to it over Stratum and forces the block header timestamp to the
current time, I'm not aware of any such hardware, and it would also
likely suffer from random invalid blocks due to relying on NTP
anyway, making its existence highly unlikely.
This leaves the only concern being pools, but most of those rely on
work generated by Bitcoin Core (in one way or another, though when
spy mining possibly not), and it seems likely that they will also
not suffer any lost work. While its possible that a pool does
generate invalid work due to spy mining or otherwise custom logic,
it seems unlikely that a substantial portion of hashrate would do
so, making the difference somewhat academic (any pool that screws
this up will only do so once and the network would come out just
fine).
Further, while we may end up deciding these assumptions were
invalid and we should instead use 7200s, it seems prudent to try
with the value we "want" on testnet4, giving us the ability to
learn if the compatibility concerns are an issue before we go to
mainnet.
To prepare for migrating wallets that are not loaded, when migration
occurs in the GUI, it should not rely on a WalletModel existing.
Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
fa6fe43207 net: Clarify that m_addr_local is only set once (MarcoFalke)
Pull request description:
The function is supposed to be only called once when the version msg arrives (a single time). Calling it twice would be an internal logic bug. However, the `LogError` in this function has many issues:
* If the error happens in tests, as is the case for the buggy fuzz test, it will go unnoticed
* It is dead code, unless a bug is introduced to execute it
Fix all issues by using `Assume(!m_addr_local.IsValid())` instead. Idea taken from https://github.com/bitcoin/bitcoin/pull/30364#discussion_r1680530382
ACKs for top commit:
achow101:
ACK fa6fe43207
mzumsande:
utACK fa6fe43207
glozow:
ACK fa6fe43207
Tree-SHA512: 8c1e8c524768f4f36cc50110ae54ee423e057a963ff78f736f3bf92df1ce5af28e3e0149153780897944e1d5c22ddbca9dac9865d9f4d44afffa152bc8559405
It is expensive to construct, and only one test uses it.
Fix both issues by disallowing the construction and moving it to the
single test that uses it.
Previous rpcauth behavior was to sometimes
ignore empty -rpcauth= settings, and other times
treat them as errors.
Empty rpcauth is now consistently treated
as an error and prevents bitcoind from starting.
Updates associated test cases.
Also updates to non-deprecated logging macro.
Co-Authored-By: Luke Dashjr <luke-jr+git@utopios.org>
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
Blank legacy wallets do not have active SPKM. They can
only be detected by checking the descriptors' flag or
the db format.
This enables the migration of blank legacy wallets in
the GUI.
The following bitcoind parameters / RPC calls missed the "testnet4"
network string:
- `-chain=` parameter
- `getblockchaininfo` RPC, "chain" result
- `getmininginfo` RPC, "chain" result
5215c925d1 Compare ASMaps with respect to specific addresses (virtu)
Pull request description:
Right now, we have no way to quantify the "degradation" of an ASMap over time in the context of Bitcoin's P2P network in a meaningful way. However, such data would be useful for:
1. Making sure the minimum shelf life of ASMaps is compatible with the release cycle (we wouldn't want to start shipping ASMaps with releases before making sure ASMaps typically do not become obsolete before the time of the next release)
2. Node operators eager to keep their ASMaps up-to-date between releases.
While `asmap-tool.py` has a `diff` command to perform a prefix-based comparison of two ASMaps, it is hard to reason about whether an old ASMap still is "good enough" or should be replaced with a newer one based on a prefix-based diff such as the following:
```shell
$ ./asmap-tool.py diff 1704463200_asmap.dat 1710770400_asmap.dat
[...]
# 2c0f:fc98::/32 was AS37282
# 2c0f:fcb8::/32 was AS37323
2c0f:ff18::/32 AS37044 # was unassigned
2c0f:ff98::/32 AS37113 # was unassigned
2c0f:ffa0::/32 AS37273 # was unassigned
# 76082350 (2^26.18) IPv4 addresses changed; 834271985742505274886878979424260 (2^109.36) IPv6 addresses changed
```
One option for a more Bitcoin-centric ASMap comparison comprises comparing ASNs for the addresses of Bitcoin nodes and reporting on the number/share of addresses of nodes with disagreeing ASNs. By applying this approach to a node's set of known peers, a node operator can estimate how many of the node's peers are mapped to out-of-date AS when using the currently deployed and an up-to-date ASMap as input.
This PR adds this functionality to `asmap-tool.py` by introducing a `diff_addrs` subcommand. In addition to two ASMaps, the subcommand reads addresses from a (`getnodeaddresses`-compatible) file, and computes statistics for those addresses:
```bash
$ ./asmap-tool.py diff_addrs 1704463200_asmap.dat 1710770400_asmap.dat <(bitcoin-cli getnodeaddresses 0)
275 address(es) reassigned from unassigned to AS51167
84 address(es) reassigned from AS198949 to AS15557
66 address(es) reassigned from AS45758 to AS45629
33 address(es) reassigned from AS174 to AS212238
[...]
1 address(es) reassigned from unassigned to AS399619
Summary: 919 (1.67%) of 54,901 addresses were reassigned.
```
When the `-s / --show-addresses` flag is used, addresses subject to reassignment are included in the output.
ACKs for top commit:
fjahr:
tACK 5215c925d1
achow101:
ACK 5215c925d1
brunoerg:
reACK 5215c925d1
Tree-SHA512: ebcf47754bce92794fad9f4c3bfc1c5e9daf077db5975f444c5135092eb6a26ecaa1eca6748a03ae0c87d9e45532426966fe8f3c17249b17f9dcad490d6dd3bf
86b38529d5 qa: a fuzz target for the block index database (Antoine Poinsot)
Pull request description:
This introduces a small fuzz target for `CBlockTreeDB` which asserts a few invariants by using an in-memory LevelDb.
ACKs for top commit:
achow101:
ACK 86b38529d5
TheCharlatan:
Re-ACK 86b38529d5
maflcko:
review ACK 86b38529d5🥒
brunoerg:
utACK 86b38529d5
Tree-SHA512: ab75b4ae1c7e0a4b15f8a6ceffdf509fbc79833e6ea073ecef68558d53b83663d1b30362aaa2d77c22b8890a572f5b1d4b1c5abbca483c8c8f9b1fb5b276a59a
8fee5355ee guix: fix suggested fake date for openssl -1.1.1l (Sjors Provoost)
Pull request description:
Using `2020-10-01` as the fake timestamp will cause many test failures with `/gnu/store/bfirgq65ndhf63nn4q6vlkbha9zd931q-openssl-1.1.1l.drv`. I didn't investigate why, but I guess because it's _before_ the test certificates were created. They expired in June 2022. I tried a month before that, which worked.
Also fixes layout of instructions.
ACKs for top commit:
achow101:
ACK 8fee5355ee
maflcko:
review ACK 8fee5355ee
Tree-SHA512: df5dd3aa961e25bd57d0b8b73daeb3ec76856b06e35277f24b6b19be81774512228f75e2b779afa8ea92fcc39beb869f43e0c57fba19ad16a82812e7c0bea38b
77ff0ec1f1 contrib: support reading XORed blocks in linearize-data.py script (Sebastian Falbesoner)
Pull request description:
This PR is a small follow-up for #28052, adding support for the block linearization script to handle XORed blocksdir *.dat files. Note that if no xor.dat file exists, the XOR pattern is set to all-zeros, in order to still support blockdirs that have been created with versions earlier than 28.x.
Partly fixes issue #30599.
ACKs for top commit:
achow101:
ACK 77ff0ec1f1
tdb3:
ACK 77ff0ec1f1
hodlinator:
ACK 77ff0ec1f1
Tree-SHA512: 011eb02e2411de373cbbf4b26db4640fc693a20be8c2430529fba6e36a3a3abfdfdc3b005d330f9ec2846bfad9bfbf34231c574ba99289ef37dd51a68e6e7f3d
401cc4ec70 fuzz: improve scriptpubkeyman target (brunoerg)
Pull request description:
Fixes#30541
This PR aims to improve `scriptpubkeyman` target to avoid timeouts. The input provided in #30541 takes too much time to run because it basically calls only `MarkUnusedAddresses` (300 times * number of spks). The following changes were made to improve it:
- Reduce keypool size.
- When calling `MarkUnusedAddresses`, do it with one of the spks per iteration.
- Remove the specific `AddDescriptorKey` call since it is already covered with `AddWalletDescriptor`.
- Limit number of iterations to a reasonable value.
ACKs for top commit:
maflcko:
lgtm ACK 401cc4ec70
achow101:
ACK 401cc4ec70
Tree-SHA512: 941812bc6d991dd03675a2974ce1b839494ca7f6e6d8a22c689d4bf4fed2dac5491246998f19cb15dbff516fdd8eeda27e7628c3206d45f57dc292bc05624a5c
15aa7d0236 gui, qt: brintToFront workaround for Wayland (pablomartin4btc)
Pull request description:
There are known issues around handling windows focus in `Wayland` ([this one specific](https://bugs.kde.org/show_bug.cgi?id=462574) in KDE but also in [gnome](https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/730)).
The idea is that the workaround will be executed if `bitcoin-qt` is running using `Wayland` platform (e.g.: `QT_QPA_PLATFORM=wayland ./src/qt/bitcoin-qt -regtest`), since the workaround behaviour looks like re-opening the window again (which I tried to fix by moving the window to the original position and/ or re-setting the original geometry without success) while in `X11` (not sure in Mac) the current `GUIUtil::brintToFront` actually sets the focus to the desired window, keeping its original position as expected, and I didn't want to change that (`X11` behaviour).
The solution was [initially discussed](https://github.com/bitcoin-core/gui/pull/817#issuecomment-2256158902) with hebasto in #817.
ACKs for top commit:
hebasto:
ACK 15aa7d0236.
Tree-SHA512: 141d6cc4a618026e551627b9f4cc284285980db02a54a7b19c7de91e8c5adccf0c1d67380625146b5413e58c59f39c9e944ed5ba68cb8644f67647518918b6f7
6b2dcba076 wallet: List sqlite wallets with empty string name (Ava Chow)
3ddbdd1815 wallet: Ignore .bak files when listing wallet files (Ava Chow)
Pull request description:
When the default wallet is migrated, we do not rename the wallet so we end up having a descriptor wallet with the empty string as its name and the wallet.dat file in the root of the walletdir. This is supposed to be an unsupported configuration and there is no other way to achieve this (other than file copying), but the wallet loading code does not disallow loading such wallets. However `listwalletdir` does not currently list the default wallet if it is sqlite. This is confusing to users, so change `listwalletdir` to include these wallets.
Additionally, the migration of the default wallet, and of any plain wallet files in the walletdir, produces a backup file in the walletdir itself. Since these backups are a BDB file, `listwalletdir` will detect them as being another wallet that we could open, but this is erroneous and could lead to confusion and potentially funds loss if both the backup and the migrated wallet are in use simultaneously. To reduce the likelihood of this issue, don't list these wallets in `listwalletdir`.
***
Possibly we could have more stringent checks on loading to resolve these issues, but I'm concerned that that will just confuse users and gratuitously break things that already worked.
Since the original intent was to disallow default wallets for sqlite/descriptors, a possible alternative would be to prevent people from loading such wallets and change migration to rename those wallets. However, given that this behavior with migrating default wallets has existed since default wallet migration was fixed, I think that making such a change would be confusing and break things for no good reason. Although perhaps we should still do the renaming.
For the backups, we could also change loading to refuse to load any wallet named with `.bak` (or `.legacy.bak`) as such wallets can still be loaded by giving the path to them directly, which some users may do to "restore" the backup. However restricting what can be loaded based on filename seems a little heavyhanded. It wouldn't be funds loss though since the correct way to restore the backup is with `restorewallet`.
ACKs for top commit:
fjahr:
Code review ACK 6b2dcba076
furszy:
Code ACK 6b2dcba076
glozow:
ACK 6b2dcba076
Tree-SHA512: 0b033f6ed55830f8a054afea3fb2cf1fa82a94040053ebfaf123bda36c99f45d3f01a2aec4ed02fed9c61bb3d320b047ed892d7f6644b5a356a7bc5974b10cff
055bc05792 policy/feerate.h: avoid constraint self-dependency (Matt Whitlock)
138f867156 add missing #include <cstdint> for GCC 15 (Matt Whitlock)
Pull request description:
#30612 with changes made.
GCC 15 introduces three build failures:
* Two are related to missing includes. You can't use `uint16_t` et al. without including `<cstdint>`.
* The third is harder to understand but easy to fix. GCC changed something about the way templates are instantiated when checking type constraints, and now there is a dependency loop while checking `std::optional<CFeeRate>`. This manifests as the following compile-time mess:
```
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/format:48,
from /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/chrono_io.h:39,
from /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/chrono:3362,
from ./util/time.h:9,
from ./primitives/block.h:12,
from ./blockencodings.h:8,
from blockencodings.cpp:5:
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits: In substitution of 'template<class _Up> requires !(is_same_v<std::optional<_Tp>, typename std::remove_cvref<_It2>::type>) && (is_constructible_v<_Tp, const _Up&>) && (__construct_from_contained_value<_Up, typename std::remove_cv< <template-parameter-1-1> >::type>) constexpr std::optional<CFeeRate>::optional(const std::optional<_Tp>&) [with _Up = CFeeRate]':
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits:1140:25: required by substitution of 'template<class _Tp, class ... _Args> using std::__is_constructible_impl = std::__bool_constant<__is_constructible(_Tp, _Args ...)> [with _Tp = CFeeRate; _Args = {std::optional<CFeeRate>&}]'
1140 | = __bool_constant<__is_constructible(_Tp, _Args...)>;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits:1145:12: required from 'struct std::is_constructible<CFeeRate, std::optional<CFeeRate>&>'
1145 | struct is_constructible
| ^~~~~~~~~~~~~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits:178:35: required by substitution of 'template<class ... _Bn> std::__detail::__first_t<std::integral_constant<bool, false>, typename std::enable_if<(!(bool)(_Bn::value)), void>::type ...> std::__detail::__or_fn(int) [with _Bn = {std::is_constructible<CFeeRate, std::optional<CFeeRate>&>, std::is_convertible<std::optional<CFeeRate>&, CFeeRate>, std::is_constructible<CFeeRate, std::optional<CFeeRate> >, std::is_convertible<std::optional<CFeeRate>, CFeeRate>, std::is_constructible<CFeeRate, const std::optional<CFeeRate>&>, std::is_convertible<const std::optional<CFeeRate>&, CFeeRate>, std::is_constructible<CFeeRate, const std::optional<CFeeRate> >, std::is_convertible<const std::optional<CFeeRate>, CFeeRate>}]'
178 | __enable_if_t<!bool(_Bn::value)>...>;
| ^~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits:196:41: required from 'struct std::__or_<std::is_constructible<CFeeRate, std::optional<CFeeRate>&>, std::is_convertible<std::optional<CFeeRate>&, CFeeRate>, std::is_constructible<CFeeRate, std::optional<CFeeRate> >, std::is_convertible<std::optional<CFeeRate>, CFeeRate>, std::is_constructible<CFeeRate, const std::optional<CFeeRate>&>, std::is_convertible<const std::optional<CFeeRate>&, CFeeRate>, std::is_constructible<CFeeRate, const std::optional<CFeeRate> >, std::is_convertible<const std::optional<CFeeRate>, CFeeRate> >'
196 | : decltype(__detail::__or_fn<_Bn...>(0))
| ~~~~~~~~~~~~~~~~~~~~~~~~~^~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/optional:824:45: required from 'constexpr const bool std::optional<CFeeRate>::__construct_from_contained_value<CFeeRate, CFeeRate>'
824 | = !__converts_from_optional<_Tp, _From>::value;
| ^~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/optional:884:7: required by substitution of 'template<class _Up> requires !(is_same_v<std::optional<_Tp>, typename std::remove_cvref<_It2>::type>) && (is_constructible_v<_Tp, const _Up&>) && (__construct_from_contained_value<_Up, typename std::remove_cv< <template-parameter-1-1> >::type>) constexpr std::optional<CFeeRate>::optional(const std::optional<_Tp>&) [with _Up = CFeeRate]'
884 | && __construct_from_contained_value<_Up>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./validation.h:164:41: required from here
164 | return MempoolAcceptResult(state);
| ^
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/optional:886:2: required by the constraints of 'template<class _Tp> template<class _Up> requires !(is_same_v<std::optional<_Tp>, typename std::remove_cvref<_It2>::type>) && (is_constructible_v<_Tp, const _Up&>) && (__construct_from_contained_value<_Up, typename std::remove_cv< <template-parameter-1-1> >::type>) constexpr std::optional<_Tp>::optional(const std::optional<_From>&)'
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/optional:884:14: error: satisfaction of atomic constraint '__construct_from_contained_value<_Up, typename std::remove_cv< <template-parameter-1-1> >::type> [with _Tp = _Tp; _Up = _Up]' depends on itself
884 | && __construct_from_contained_value<_Up>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
It is easiest to solve this by changing the `static_assert` in the explicit `CFeeRate` constructor to a SFINAE by using a type constraint on the function template parameter.
We already [downstreamed](https://github.com/gentoo/gentoo/pull/38015) these fixes in Gentoo.
ACKs for top commit:
stickies-v:
ACK 055bc05792
Tree-SHA512: ce9cb27bcd9b0f4bbc80951e45cf7127112dcb7f9937bcb0167b362026d35beecb1255354746de0aac82e03c41eaccbe26acbfe0ddff2ee1e5a8634673f4f4ba
4c2cc63d84 Revert "depends: Fetch miniupnpc sources from an alternative website" (sunerok)
Pull request description:
This reverts commit 21b8a14d37.
started fresh fanquake
ACKs for top commit:
fanquake:
ACK 4c2cc63d84 - please don't @ mention anyone in your pull request descriptions, otherwise they are likely to just get spammed with notifications.
Tree-SHA512: ab3e6cc65e4531b6c129a6bc663ebc9859e0de1d4b745b29834344cc8aa83ea182b0a5d1fae14e609a85bbbe24c34f42502fb25b53a15b38dd63396913ea4682
16d8261181 doc: Drop no longer needed workaround for WSL (Hennadii Stepanov)
Pull request description:
This PR effectively reverts commit 4f890ba6bc from https://github.com/bitcoin/bitcoin/pull/11437, which fixed some build issues on WSL seven years ago.
Testing the current master branch @ 31a3ff5515 on Windows 11 + WSL using Ubuntu 24.04 or Debian images, I noticed that the workaround is no longer required. Moreover, it doesn't affect the build process at all, which means the hashes of the built packages in depends remain the same and the `configure` log in the main build system remains the same as well.
ACKs for top commit:
sipsorcery:
utACK 16d8261181.
Tree-SHA512: 703a2ac4647125c91aad47131e2723fd05af30b0cfae5677a26d3e89a77c2779e0197584208f3b378ed64dd7305512d9064fd073ec06517f86e9905af4ec8838
fa5755b0a8 doc: rpc: Use "output script" consistently (2/2) (MarcoFalke)
Pull request description:
Small follow-up to https://github.com/bitcoin/bitcoin/pull/30408 to fixup the RPCs that were forgotten.
ACKs for top commit:
theStack:
lgtm ACK fa5755b0a8
Tree-SHA512: f1fc0aabb59017da216d6fe0f08a2274336d04db332ad6ce3d9608cd6f03667be1c76423f24a489ac8e7d536011a129dca752ab64b4621b7bc1d4d53f68602e4
49d569cb1f p2p: For assumeutxo, download snapshot chain before background chain (Martin Zumsande)
7a885518d5 p2p: Restrict downloading of blocks for snapshot chain (Martin Zumsande)
Pull request description:
After loading a snapshot, `pindexLastCommonBlock` is usually already set to some block for existing peers. That means we'd continue syncing the background chain from those peers instead of prioritising the snapshot chain, which defeats the purpose of doing assumeutxo in the first place. Only existing peers are affected by this bug.
ACKs for top commit:
fjahr:
re-ACK 49d569cb1f
achow101:
ACK 49d569cb1f
Sjors:
tACK 49d569cb1f
Tree-SHA512: 0eaebe1c29a8510d5ced57e14c09b128ccb34b491692815291df68bf12e2a15b52b1e7bf8d9f34808904e7f7bc20f70b0ad0f7e14df93bbdf456bd12cc02a5d2
00618e8745 assumeutxo: Drop block height from metadata (Fabian Jahr)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/30514 which has more context and shows how the issue can be reproduced. Since the value in question is removed, there is no test to add to reproduce anything.
This is an alternative approach to #30516 with much of the [code being suggested there](https://github.com/bitcoin/bitcoin/pull/30516#discussion_r1689146902).
ACKs for top commit:
maflcko:
re-ACK 00618e8745🎌
achow101:
ACK 00618e8745
theStack:
Code-review ACK 00618e8745
ismaelsadeeq:
Re-ACK 00618e8745
mzumsande:
ACK 00618e8745
Tree-SHA512: db9575247bae838ad7742a27a216faaf55bb11e022f9afdd05752bb09bbf9614717d0ad64304ff5722a16bf41d8dea888af544e4ae26dcaa528c1add0269a4a8
Although it is not explicitly possible to create a default wallet with
descriptors, it is possible to migrate a default wallet and have it end
up being a default wallet with descriptors. These wallets should be
listed by ListDatabases so that it appears in wallet directory listings
to avoid user confusion.
Migration creates backup files in the wallet directory with .bak as the
extension. This pollutes the output of listwalletdir with backup files
that most users should not need to care about.
92c1d7d1f8 validation: Use MAX_TIMEWARP constant as testnet4 timewarp defense delta (Fabian Jahr)
4b2fad502e doc: Add release notes for 29775 (Fabian Jahr)
f7cc97313b doc: Align deprecation warnings (Fabian Jahr)
1163b08378 chainparams: Add initial minimum chain work for Testnet4 (Fabian Jahr)
Pull request description:
This completes follow-ups left open in #29775.
- Adds release notes
- Addresses the [misalignment](https://github.com/bitcoin/bitcoin/pull/29775#discussion_r1706982102) in deprecation warnings and hints at the intention to remove support for Testnet3.
- Adds initial minimum chainwork for Testnet4.
- Use the `MAX_TIMEWARP` constant as the timewarp defense delta, equal to `MAX_FUTURE_BLOCK_TIME`.
ACKs for top commit:
Sjors:
ACK 92c1d7d1f8
achow101:
ACK 92c1d7d1f8
tdb3:
re ACK 92c1d7d1f8
Tree-SHA512: 7ebdac7809f96231f75ca62706af59cd1ed27f713a4c7be5e2ad69fae95832b146b3ea23c712fb03b412da1deda7e8a5dae55bb2bbd2dcfd9f926e85c2a72666
2925bd537c refactor: use c++20 std::views::reverse instead of reverse_iterator.h (stickies-v)
Pull request description:
C++20 introduces [`std::ranges::views::reverse`](https://en.cppreference.com/w/cpp/ranges/reverse_view), which allows us to drop our own `reverse_iterator.h` implementation and also makes it easier to chain views (even though I think we currently don't use this).
ACKs for top commit:
achow101:
ACK 2925bd537c
maflcko:
ACK 2925bd537c🎷
Tree-SHA512: 567666ec44af5d1beb7a271836bcc89c4c577abc77f522fcc18bc6d4de516ae9b0df766d0bfa6dd217569e6878331c2aee1d9815620860375e3510dad7fed476
Enhanced efficiency and readability of CCoinsViewCache::FetchCoin by replacing separate find() and emplace() calls with a single try_emplace(), reducing map lookups and potential insertions.
Multipath descriptors will be imported as multiple separate descriptors.
When there are 2 multipath items, the first descriptor will be for receiving
addresses and the second for change. This mirrors importmulti.
Multipath descriptors will be imported as multiple separate descriptors.
When there are exactly 2 multipath items, the first descriptor will be
for receiving addreses, and the second for change
addresses. When importing a multipath descriptor, 'internal' cannot be
specified.
Instead of applying internal-ness to all keys being imported at the same
time, apply it on a per key basis. So each key that is imported will
carry with it whether it is for the change keypool.
When given a multipath descriptor, derive all of the descriptors.
The derived addresses will be returned in an object
consisting of multiple arrays. For compatibility, when given a single path
descriptor, the addresses are provided in a single array as before.
Multipath specifiers are derivation path indexes of the form `<i;j;k;...>`
used for specifying multiple derivation paths for a descriptor.
Only one multipath specifier is allowed per PubkeyProvider.
This is syntactic sugar which is parsed into multiple distinct descriptors.
One descriptor will have all of the `i` paths, the second all of the `j` paths,
the third all of the `k` paths, and so on.
ParseKeypath will always return a vector of keypaths with the same size
as the multipath specifier. The callers of this function are updated to deal
with this case and return multiple PubkeyProviders. Their callers have
also been updated to handle vectors of PubkeyProviders.
Co-Authored-By: furszy <matiasfurszyfer@protonmail.com>
To prepare for returning multipath descriptors which will be a shorthand
for specifying multiple descriptors, change ParseScript's signature to return a
vector.
589db872e1 validation: don't erase coins cache on prune flushes (Andrew Toth)
0e8918755f Add linked-list test to CCoinsViewCache::SanityCheck (Pieter Wuille)
05cf4e1875 coins: move Sync logic to CoinsViewCacheCursor (Andrew Toth)
7825b8b9ae coins: pass linked list of flagged entries to BatchWrite (Andrew Toth)
a14edada8a test: add cache entry linked list tests (Andrew Toth)
24ce37cb86 coins: track flagged cache entries in linked list (Andrew Toth)
58b7ed156d coins: call ClearFlags in CCoinsCacheEntry destructor (Andrew Toth)
8bd3959fea refactor: require self and sentinel parameters for AddFlags (Andrew Toth)
75f36d241d refactor: add CoinsCachePair alias (Andrew Toth)
f08faeade2 refactor: move flags to private uint8_t and rename to m_flags (Andrew Toth)
4e4fb4cbab refactor: disallow setting flags in CCoinsCacheEntry constructors (Andrew Toth)
8737c0cefa refactor: encapsulate flags setting with AddFlags and ClearFlags (Andrew Toth)
9715d3bf1e refactor: encapsulate flags get access for all other checks (Andrew Toth)
df34a94e57 refactor: encapsulate flags access for dirty and fresh checks (Andrew Toth)
Pull request description:
Since https://github.com/bitcoin/bitcoin/pull/17487 we no longer need to clear the coins cache when syncing to disk. A warm coins cache significantly speeds up block connection, and only needs to be fully flushed when nearing the `dbcache` limit.
For frequent pruning flushes there's no need to empty the cache and kill connect block speed. However, simply using `Sync` in place of `Flush` actually slows down a pruned full IBD with a high `dbcache` value. This is because as the cache grows, sync takes longer since every coin in the cache is scanned to check if it's dirty. For frequent prune flushes and a large cache this constant scanning starts to really slow IBD down, and just emptying the cache on every prune becomes faster.
To fix this, we can add two pointers to each cache entry and construct a doubly linked list of dirty entries. We can then only iterate through all dirty entries on each `Sync`, and simply clear the pointers after.
With this approach a full IBD with `dbcache=16384` and `prune=550` was 32% faster than master. For default `dbcache=450` speedup was ~9%. All benchmarks were run with `stopatheight=800000`.
| | prune | dbcache | time | max RSS | speedup |
|-----------:|----------:|------------:|--------:|-------------:|--------------:|
| master | 550 | 16384 | 8:52:57 | 2,417,464k | - |
| branch | 550 | 16384 | 6:01:00 | 16,216,736k | 32% |
| branch | 550 | 450 | 8:05:08 | 2,818,072k | 8.8% |
| master | 10000 | 5000 | 8:19:59 | 2,962,752k | - |
| branch | 10000 | 5000| 5:56:39 | 6,179,764k | 28.8% |
| master | 0 | 16384 | 4:51:53 | 14,726,408k | - |
| branch | 0 | 16384 | 4:43:11 | 16,526,348k | 2.7% |
| master | 0 | 450 | 7:08:07 | 3,005,892k | - |
| branch | 0 | 450 | 6:57:24 | 3,013,556k |2.6%|
While the 2 pointers add memory to each cache entry, it did not slow down IBD. For non-pruned IBD results were similar for this branch and master. When I performed the initial IBD, the full UTXO set could be held in memory when using the max `dbcache` value. For non-pruned IBD with max `dbcache` to tip ended up using 12% more memory, but it was also 2.7% faster somehow. For smaller `dbcache` values the `dbcache` limit is respected so does not consume more memory, and the potentially more frequent flushes were not significant enough to cause any slowdown.
For reviewers, the commits in order do the following:
First 4 commits encapsulate all accesses to `flags` on cache entries, and then the 5th makes `flags` private.
Commits `refactor: add CoinsCachePair alias` to `coins: call ClearFlags in CCoinsCacheEntry destructor` create the linked list head nodes and cache entry self references and pass them into `AddFlags`.
Commit `coins: track flagged cache entries in linked list` actually adds the entries into a linked list when they are flagged DIRTY or FRESH and removes them from the linked list when they are destroyed or the flags are cleared manually. However, the linked list is not yet used anywhere.
Commit `test: add cache entry linked list tests` adds unit tests for the linked list.
Commit `coins: pass linked list of flagged entries to BatchWrite` uses the linked list to iterate through DIRTY entries instead of using the entire coins cache.
Commit `validation: don't erase coins cache on prune flushes` uses `Sync` instead of `Flush` for pruning flushes, so the cache is no longer cleared.
Inspired by [this comment](https://github.com/bitcoin/bitcoin/pull/15265#issuecomment-457720636).
Fixes https://github.com/bitcoin/bitcoin/issues/11315.
ACKs for top commit:
paplorinc:
ACK 589db872e1
sipa:
reACK 589db872e1
achow101:
ACK 589db872e1
mzumsande:
re-ACK 589db872e1
Tree-SHA512: 23b2bc01c83edacb5b39aa60bb0b766de9a74ce17f0c59bf13b97b4328a7b758ad9aff6581c3ca88e2973f7658380651530d497444f48d6e22ea0bfc51cc921d
6bfa26048d testnet: Add timewarp attack prevention for Testnet4 (Fabian Jahr)
0100907ca1 testnet: Add Testnet4 difficulty adjustment rules fix (Fabian Jahr)
74a04f9e7a testnet: Introduce Testnet4 (Fabian Jahr)
Pull request description:
To supplement the [ongoing conceptual discussion about a testnet reset](https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/9yCPo3uUBwAJ) I have drafted a move to v4 including a fix to the difficulty adjustment mechanism, which was part of the motivation that started the discussion.
Conceptual considerations:
- The conceptual discussion about doing a testnet4 or softforking the fix into testnet3 is outside of the scope of this PR and I would ask reviewers to contribute their opinions on this on the ML instead. However, I am happy to adapt this PR to a softfork change on testnet3 if there is consensus for that instead.
- The difficulty adjustment fix suggested here touches the `CalculateNextWorkRequired` function and uses the same logic used in `GetNextWorkRequired` to find the last previous block that was not mined with difficulty 1 under the exceptionf. An alternative fix briefly mentioned on the mailing list by Jameson Lopp would be to "restrict the special testnet minimum difficulty rule so that it can't be triggered on the block right before a difficulty retarget". That would also fix the issue but I find my suggestion here a bit more elegant.
ACKs for top commit:
jsarenik:
tACK 6bfa26048d
achow101:
ACK 6bfa26048d
murchandamus:
tACK 6bfa26048d
Tree-SHA512: 0b8b69a621406a944da5be551b863d065358ba94d85dd3b80d83c412660e230ee93b27316081fbee9b4851cc4ff8585db64c7dfa26cb5148ac835663f2712c3d
1f93e3c360 add deprecation warning for mempoolfullrbf (glozow)
4400c979a3 [doc] update documentation for new mempoolfullrbf default (glozow)
Pull request description:
Followup to #30493. Update bips.md and policy/*.md to reflect new default rules around signaling requirements in RBF.
Also, log a warning when `-mempoolfullrbf=0` that this config option is deprecated and will be removed in a future release.
ACKs for top commit:
petertodd:
ACK 1f93e3c360
instagibbs:
ACK 1f93e3c360
tdb3:
ACK 1f93e3c360
Tree-SHA512: f60a9524f15cfaa4c10c40b6f62b787d3f9865aac48ca883def30efac4f8a118f1359532f1b209ea34e201f0b1c92398abc8bc1e439e6b60910cc7f75c51e9ae
Refactor satoshi_round function to accept different rounding modes.
Updated call site to use the revised `satoshi_round` function.
Co-authored-by: Kate Salazar <52637275+katesalazar@users.noreply.github.com>
ec973dd197 refactor: remove un-tested early returns (josibake)
72a5822d43 tests: add tests for KeyPair (josibake)
cebb08b121 refactor: move SignSchnorr to KeyPair (josibake)
c39fd39ba8 crypto: add KeyPair wrapper class (josibake)
5d507a0091 tests: add key tweak smoke test (josibake)
f14900b6e4 bench: add benchmark for signing with a taptweak (josibake)
Pull request description:
Broken out from #28201
---
The wallet returns an untweaked internal key for taproot outputs. If the output commits to a tree of scripts, this key needs to be tweaked with the merkle root. Even if the output does not commit to a tree of scripts, BIP341/342 recommend commiting to a hash of the public key.
Previously, this logic for applying the taptweak was implemented in the `CKey::SignSchnorr` method.
This PR moves introduces a KeyPair class which wraps a `secp256k1_keypair` type and refactors SignSchnorr to use this new KeyPair. The KeyPair class is created with an optional merkle_root argument and the logic from BIP341 is applied depending on the state of the merkle_root argument.
The motivation for this refactor is to be able to use the tap tweak logic outside of signing, e.g. in silent payments when retrieving the private key (see #28201).
Outside of silent payments, since we almost always convert a `CKey` to a `secp256k1_keypair` when doing anything with taproot keys, it seems generally useful to have a way to model this type in our code base.
ACKs for top commit:
paplorinc:
ACK ec973dd197 - will happily reack if you decide to apply @ismaelsadeeq's suggestions
ismaelsadeeq:
Code review ACK ec973dd197
itornaza:
trACK ec973dd197
theStack:
Code-review ACK ec973dd197
Tree-SHA512: 34947e3eac39bd959807fa21b6045191fc80113bd650f6f08606e4bcd89aa17d6afd48dd034f6741ac4ff304b104fa8c1c1898e297467edcf262d5f97425da7b
6714276d72 miniscript: Use `ToIntegral` instead of `ParseInt64` (brunoerg)
Pull request description:
Currently, miniscript code uses `ParseInt64` function for `after`, `older`, `multi` and `thresh` fragments. It means that a leading `+` or whitespace, among other things, are accepted into the fragments. However, these cases are not useful and cause Bitcoin Core to behave differently compared to other miniscript implementations (see https://github.com/brunoerg/bitcoinfuzz/issues/34). This PR fixes it.
ACKs for top commit:
achow101:
ACK 6714276d72
tdb3:
cr ACK 6714276d72
danielabrozzoni:
tACK 6714276d72
darosior:
utACK 6714276d72
Tree-SHA512: d9eeb93f380f346d636513eeaf26865285e7b0907b8ed258fe1e02153a9eb69d484c82180eb1c78b0ed77ad5f0e5b244be6672c2f890b1d9fddc9e844bee6dde
After loading a snapshot, pindexLastCommonBlock is usually already set
to some block for existing peers. That means we'd continue syncing the
background chain from those peers instead of prioritising the snapshot
chain, which defeats the purpose of doing assumeutxo in the first place.
Only existing peers are affected by this bug.
If the best chain of the peer doesn't include the snapshot
block, it is futile to download blocks from this chain,
because we couldn't reorg to it. We'd also crash
trying to reorg because this scenario is not handled.
fa18fc7050 log: Remove NOLINT(bitcoin-unterminated-logprintf) (MarcoFalke)
Pull request description:
`NOLINT(bitcoin-unterminated-logprintf)` is used to document a missing trailing `\n` char in the format string. This has many issues:
* It is just documentation, assuming that a trailing `\n` ends up in the formatted string. It is not enforced at compile-time, so it is brittle.
* If the newline was truly missing and `NOLINT(bitcoin-unterminated-logprintf)` were used to document a "continued" line, the log stream would be racy/corrupt, because any other thread may inject a log message in the meantime.
* If the newline was accidentally missing, nothing is there to correct the mistake.
* The intention of all code is to always end a log line with a new line. However, historic code exists to deal with the case where the new line was missing (`m_started_new_line`). This is problematic, because the presumed dead code has to be maintained (https://github.com/bitcoin/bitcoin/pull/30386#discussion_r1682963306).
Fix almost all issues by removing the `NOLINT(bitcoin-unterminated-logprintf)`, ensuring that a new line is always present.
A follow-up will remove the dead logging code.
ACKs for top commit:
TheCharlatan:
ACK fa18fc7050
ryanofsky:
Code review ACK fa18fc7050
Tree-SHA512: bf8a83723cca84e21187658edc19612da79c34f7ef2e1f6e9353e7ba70e4ecc0a878a2ae32290045fb90cba9a44451e35341a36ef2ec1169d13592393aa4a8ca
bbcba09cd5 build: remove check for __declspec(dllexport) (fanquake)
37c9abdc43 build: remove check for __attribute__((visibility.. (fanquake)
Pull request description:
These are unused (since libbitcoinconsensus / #29648), and the current CMake port doesn't quite match behaviour, such that there's no real point in doing the check. So rather than port anything, just remove it. If these are needed again in future (i.e for kernel or similar), they can be revisted, and it might be the case that build-system level checks will not be wanted.
ACKs for top commit:
hebasto:
ACK bbcba09cd5. I've verified that neither `HAVE_DEFAULT_VISIBILITY_ATTRIBUTE` nor `HAVE_DLLEXPORT_ATTRIBUTE` are used or evaluated in the current codebase.
TheCharlatan:
ACK bbcba09cd5
willcl-ark:
ACK bbcba09cd5
Tree-SHA512: 332f018c50a159d2cbfd2f9ce018538fa11cf06a94e27ed42146945b86645af5881095df39cadd2f775a8ae348ebfc949d54f7eb4b62264bf48119c9f9952c20
e9de0a76b9 doc: release note for 30212 (willcl-ark)
87b1880525 rpc: clarify ALREADY_IN_CHAIN rpc errors (willcl-ark)
Pull request description:
Closes: #19363
Renaming this error improves clarity around the returned error both internally and externally when a transactions' outputs are already found in the utxo set (`TransactionError::ALREADY_IN_CHAIN -> TransactionError::ALREADY_IN_UTXO_SET`)
ACKs for top commit:
tdb3:
ACK e9de0a76b9
ismaelsadeeq:
ACK e9de0a76b9
ryanofsky:
Code review ACK e9de0a76b9.
Tree-SHA512: 7d2617200909790340951fe56a241448f9ce511900777cb2a712e8b9c0778a27d1f912b460f82335844224f1abb4322bc898ca076440959edade55c082a09237
59c0ece0a7 fuzz: replace hardcoded numbers for bech32 limits (josibake)
Pull request description:
Follow-up to #30047 to replace a hardcoded value that was missed in the original PR
ACKs for top commit:
paplorinc:
ACK 59c0ece0a7
dergoegge:
utACK 59c0ece0a7
marcofleon:
ACK 59c0ece0a7. Ran the test a bit to be sure, lgtm.
brunoerg:
utACK 59c0ece0a7
Tree-SHA512: 89799928feb6752a533259117340b087ff7299f9bf204b165dd87708e15b99a338521f2ac9f9e1fd91dc48b93be839059768d9e68b172e36328232174d1dfa3f
a0a9a11642 depends: fix ZMQ CMake getcachesize check (fanquake)
Pull request description:
Fixes#30587.
ACKs for top commit:
maflcko:
ACK a0a9a11642
hebasto:
ACK a0a9a11642. On Ubuntu (s390x), I was able to reproduce https://github.com/bitcoin/bitcoin/issues/30587. With this PR building `zeromq` succeeds.
TheCharlatan:
ACK a0a9a11642
Tree-SHA512: 70ca50ebe8e36d5a10a2354a1fbed49f5f802ae5115e09686bccca7e5d1da35168e84a6cab40dd2c83f8918889cdfdcbd5d1cbe25273b844c8ddd21865ea6c51
Erase spent cache entries and clear flags of unspent
entries inside the BatchWrite loop, instead of an
additional loop after BatchWrite.
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
No visible behavior change. This commit tracks the flagged
entries internally but the list is not iterated by anything.
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
Co-Authored-By: l0rinc <pap.lorinc@gmail.com>
fa895c7283 mingw: Document mode wbx workaround (MarcoFalke)
fa359255fe Add -blocksxor boolean option (MarcoFalke)
fa7f7ac040 Return XOR AutoFile from BlockManager::Open*File() (MarcoFalke)
Pull request description:
Currently the *.dat files in the blocksdir store the data received from remote peers as-is. This may be problematic when a program other than Bitcoin Core tries to interpret them by accident. For example, an anti-virus program or other program may scan them and move them into quarantine, or delete them, or corrupt them. This may cause Bitcoin Core to fail a reorg, or fail to reply to block requests (via P2P, RPC, REST, ...).
Fix this, similar to https://github.com/bitcoin/bitcoin/pull/6650, by rolling a random XOR pattern over the dat files when writing or reading them.
Obviously this can only protect against programs that accidentally and unintentionally are trying to mess with the dat files. Any program that intentionally wants to mess with the dat files can still trivially do so.
The XOR pattern is only applied when the blocksdir is freshly created, and there is an option to disable it (on creation), so that people can disable it, if needed.
ACKs for top commit:
achow101:
ACK fa895c7283
TheCharlatan:
Re-ACK fa895c7283
hodlinator:
ACK fa895c7283
Tree-SHA512: c92a6a717da83bc33a9b8671a779eeefde2c63b192362ba1d71e6535ee31d08e2802b74acc908345197de9daac6930e4771595ee25b09acd5a67f7ea34854720
f3cfbd65f5 net: log connections failures via SOCKS5 with less severity (Vasil Dimov)
Pull request description:
It is expected to have some Bitcoin nodes unreachable some of the time. A failure to connect to an IPv4 or IPv6 node is already properly logged under category=net/severity=debug. Do the same when a connection fails when using a SOCKS5 proxy. This could be either to an .onion address or to an IPv4 or IPv6 address (via a Tor exit node).
Related: https://github.com/bitcoin/bitcoin/issues/29759
ACKs for top commit:
achow101:
ACK f3cfbd65f5
mzumsande:
Code Review ACK f3cfbd65f5
tdb3:
Code Review ACK f3cfbd65f5
Tree-SHA512: c6e83568783cb5233edac7840a00f708d27be9af87480fc73093ad99fe4bd8670d3f2c97fd6b6e2c54b8d9337746eacb9a5db6eefecc1486951996bfbb0a37f7
172c1ad026 test: expand LimitOrphan and EraseForPeer coverage (Greg Sanders)
28dbe218fe refactor: move orphanage constants to header file (Greg Sanders)
Pull request description:
Inspired by refactorings in #30000 as the coverage appeared a bit sparse.
Added some minimal border value testing, timeouts, and tightened existing assertions.
ACKs for top commit:
achow101:
ACK 172c1ad026
rkrux:
reACK [172c1ad](172c1ad026)
glozow:
reACK 172c1ad026
Tree-SHA512: e8fa9b1de6a8617612bbe9b132c9c0c9b5a651ec94fd8c91042a34a8c91c5f9fa7ec4175b47e2b97d1320d452c23775be671a9970613533e68e81937539a7d70
fa530ec543 rpc: Return precise loadtxoutset error messages (MarcoFalke)
faa5c86dbf refactor: Use untranslated error message in ActivateSnapshot (MarcoFalke)
Pull request description:
The error messages should never happen in normal operation. However, if
they do, they are helpful to return to the user to debug the issue. For
example, to notice a truncated file.
This fixes https://github.com/bitcoin/bitcoin/issues/28621
Also includes a minor refactor commit.
ACKs for top commit:
fjahr:
Code review ACK fa530ec543
ryanofsky:
Code review ACK fa530ec543, just adjusting error messages a little since last review. (Thanks!)
Tree-SHA512: 224968c9b13d082ca2ed1f6a8fcc5f51ff16d6c96bd38c3679699505b54337b99cccaf7a8474391f6b11f9ccb101977b4e626898c1217eae95802e290cf105f1
2d9d752e4f scripted-diff: Replace uint256S("str") -> uint256{"str"} (Hodlinator)
c06f2368e2 refactor: Hand-replace some uint256S -> uint256 (Hodlinator)
b74d8d58fa refactor: Add consteval uint256(hex_str) (Hodlinator)
Pull request description:
Motivation:
* Validates and converts the hex string at compile time instead of at runtime into the resulting bytes.
* Makes it possible to derive other compile time constants from `uint256`.
* Potentially eliminates runtime dependencies (`SetHexDeprecated()` is called in less places).
* Has stricter requirements than the deprecated `uint256S()` (requiring 64 chars exactly, disallows garbage at the end) and replaces it in a bunch of places.
* Makes the binary smaller (tested Guix-built x86_64-linux-gnu bitcoind binary).
* Minor: should shave off a few cycles of start-up time.
Extracted from #30377 which diverged into exploring `consteval` `ParseHex()` solutions.
ACKs for top commit:
maflcko:
rebase re-cr-ACK 2d9d752e4f🎐
stickies-v:
re-ACK 2d9d752e4f
paplorinc:
ACK 2d9d752e4f
Tree-SHA512: 39bd9320db0ed81950b5d71495eaa1d06508cc008466f2308874d70ac9ff32bc69798d2e3ef6a784868c1633fb519f60cc2111a9d0718c2663b28e78b67f7cde
When using `sendrawtransaction` the ALREADY_IN_CHAIN error help string
may be confusing.
Rename TransactionError::ALREADY_IN_CHAIN to
TransactionError::ALREADY_IN_UTXO_SET and update the rpc help string.
Remove backwards compatibility alias as no longer required.
fa3ea3b83c test: Fix intermittent issue in p2p_v2_misbehaving.py (MarcoFalke)
55555574d1 net: Log accepted connection after m_nodes.push_back (MarcoFalke)
Pull request description:
Fix the two issues reported in https://github.com/bitcoin/bitcoin/pull/30468/files#r1688444784:
* Delay a debug log line for consistency.
* Fix an intermittent test issue.
They are completely separate fixes, but both `net` related.
ACKs for top commit:
0xB10C:
Code Review ACK fa3ea3b83c
stratospher:
tested ACK fa3ea3b.
Tree-SHA512: cd6b6e164b317058a305a5c3e38c56c9a814a7469039e1143f1d7addfbc91b0a28506873356b373d97448b46cb6fbe94a1309df82e34c855540b241a09489e8b
bfd3c29e4f fuzz: fix timeout in crypter target (brunoerg)
Pull request description:
Fixes#30503
- Move SetKeyFromPassphrase to out of LIMITED_WHILE
- Remove `SetKey` calls since it is already called internally by other functions.
- Reduce number of iterations (100 is enough, no need for 10,000).
ACKs for top commit:
maflcko:
review ACK bfd3c29e4f📆
dergoegge:
utACK bfd3c29e4f
Tree-SHA512: 275ab7d07a20bfd07279a23613678993c10c166f40cdc900213b9f4d5afb107462d5f88518a0f4ce2a52f3b7950ff2c01cf74292042f16996909fcb96f827d3e
chainparams.cpp - workaround for MSVC bug triggering C7595 - Calling consteval constructors in initializer lists fails, but works on GCC (13.2.0) & Clang (17.0.6).
Complements uint256::FromHex() nicely in that it naturally does all error checking at compile time and so doesn't need to return an std::optional.
Will be used in the following 2 commits to replace many calls to uint256S(). uint256S() calls taking C-string literals are littered throughout the codebase and executed at runtime to perform parsing unless a given optimizer was surprisingly efficient. While this may not be a hot spot, it's better hygiene in C++20 to store the parsed data blob directly in the binary, without any parsing at runtime.
6d33e13bd4 doc: tor.md: use -bind=127.0.0.1:8334=onion for the Tor bind (David Gumberg)
a7f5d188cc doc: add release notes for #22729 (Vasil Dimov)
Pull request description:
Add release notes for #22729.
ACKs for top commit:
davidgumberg:
reACK 6d33e13bd4
willcl-ark:
ACK 6d33e13bd4
Tree-SHA512: 9d7e66ee1d0bb1d75b8273707d30f20915d5040a768c2c5cd47c84997df2645c8bec35db6c09dc77ab917836622411b924373816cbc83c4be38e2e9156a139d8
bf0efb4fc7 scripted-diff: Modernize naming of nChainTx and nTxCount (Fabian Jahr)
72e5d1be1f test: Add basic check for nChainTx type (Fabian Jahr)
dc2938e979 chainparams: Change nChainTx to uint64_t (Fabian Jahr)
Pull request description:
This picks up the work from #29331 and closes#29258.
This simply changes the type and addresses the comments from #29331 by changing the type in all relevant places and removing unnecessary casts. This also adds an extremely simple unit test.
Additionally this modernizes the name of `nChainTx` which helps reviewers check all use of the symbol and can make silent merge conflicts.
ACKs for top commit:
maflcko:
only rebase in scripted-diff, re-ACK bf0efb4fc7🔈
glozow:
reACK bf0efb4fc7 via range-diff
Tree-SHA512: ee4020926d0800236fe655d0c7b127215ab36b553b04d5f91494f4b7fac6e1cfe7ee298b07c0983db5a3f4786932acaa54f5fd2ccd45f2fcdcfa13427358dc3b
5e87f30f7c test: check that keyless P2A 'signing' via `signrawtransactionwithkey` succeeds (Sebastian Falbesoner)
Pull request description:
This small PR adds a sanity check to verify that transactions with P2A inputs can be 'signed' successfully, using the non-wallet RPC `signrawtransactionwithkey`. Note that in the this flow, `SignStep` (which was also extended for the new `ANCHOR` output type in #30352) is never called, as signing is only tried if the locking script verification isn't successful already. See the review discussion https://github.com/bitcoin/bitcoin/pull/30352#discussion_r1690530356 ff.
ACKs for top commit:
instagibbs:
ACK 5e87f30f7c
tdb3:
ACK 5e87f30f7c
glozow:
code review ACK 5e87f30f7c
Tree-SHA512: dfea75b4bf8fa0b9c265ddd63dab36374c2430c31220f0c8eb1b53dd847c183f9e1c493a0173e2da317553a1d4cb1b35aa9ffde1268c430cc610368d23b9c942
bbcee5a0d6 clusterlin: improve rechunking in LinearizationChunking (optimization) (Pieter Wuille)
04d7a04ea4 clusterlin: add MergeLinearizations function + fuzz test + benchmark (Pieter Wuille)
4f8958d756 clusterlin: add PostLinearize + benchmarks + fuzz tests (Pieter Wuille)
0e2812d293 clusterlin: add algorithms for connectedness/connected components (Pieter Wuille)
0e52728a2d clusterlin: rename Intersect -> IntersectPrefixes (Pieter Wuille)
Pull request description:
Part of cluster mempool: #30289
Depends on #30126, and was split off from it. #28676 depends on this.
This adds the algorithms for merging & postprocessing linearizations.
The `PostLinearize(depgraph, linearization)` function performs an in-place improvement of `linearization`, using two iterations of the [Linearization post-processing](https://delvingbitcoin.org/t/linearization-post-processing-o-n-2-fancy-chunking/201/8) algorithm. The first running from back to front, the second from front to back.
The `MergeLinearizations(depgraph, linearization1, linearization2)` function computes a new linearization for the provided cluster, given two existing linearizations for that cluster, which is at least as good as both inputs. The algorithm is described at a high level in [merging incomparable linearizations](https://delvingbitcoin.org/t/merging-incomparable-linearizations/209).
For background and references, see [Introduction to cluster linearization](https://delvingbitcoin.org/t/introduction-to-cluster-linearization/1032).
ACKs for top commit:
sdaftuar:
ACK bbcee5a0d6
glozow:
code review ACK bbcee5a0d6
instagibbs:
ACK bbcee5a0d6
Tree-SHA512: d2b5a3f132d1ef22ddf9c56421ab8b397efe45b3c4c705548dda56f5b39fe4b8f57a0d2a4c65b338462d80bb5b9b84a9a39efa1b4f390420a8005ce31817774e
73e3fa10b4 doc + test: Correct uint256 hex string endianness (Hodlinator)
Pull request description:
This PR is a follow-up to #30436.
Only changes test-code and modifies/adds comments.
Byte order of hex string representation was wrongfully documented as little-endian, but are in fact closer to "big-endian" (endianness is a memory-order concept rather than a numeric concept). `[arith_]uint256` both store their data in arrays with little-endian byte order (`arith_uint256` has host byte order within each `uint32_t` element).
**uint256_tests.cpp** - Avoid using variable from the left side of the condition in the right side. Credits to @maflcko: https://github.com/bitcoin/bitcoin/pull/30436#discussion_r1688273553
**setup_common.cpp** - Skip needless ArithToUint256-conversion. Credits to @stickies-v: https://github.com/bitcoin/bitcoin/pull/30436#discussion_r1688621638
---
<details>
<summary>
## Logical reasoning for endianness
</summary>
1. Comparing an `arith_uint256` (`base_uint<256>`) to a `uint64_t` compares the beginning of the array, and verifies the remaining elements are zero.
```C++
template <unsigned int BITS>
bool base_uint<BITS>::EqualTo(uint64_t b) const
{
for (int i = WIDTH - 1; i >= 2; i--) {
if (pn[i])
return false;
}
if (pn[1] != (b >> 32))
return false;
if (pn[0] != (b & 0xfffffffful))
return false;
return true;
}
```
...that is consistent with little endian ordering of the array.
2. They have the same endianness (but `arith_*` has host-ordering of each `uint32_t` element):
```C++
arith_uint256 UintToArith256(const uint256 &a)
{
arith_uint256 b;
for(int x=0; x<b.WIDTH; ++x)
b.pn[x] = ReadLE32(a.begin() + x*4);
return b;
}
```
### String conversions
The reversal of order which happens when converting hex-strings <=> uint256 means strings are actually closer to big-endian, see the end of `base_blob<BITS>::SetHexDeprecated`:
```C++
unsigned char* p1 = m_data.data();
unsigned char* pend = p1 + WIDTH;
while (digits > 0 && p1 < pend) {
*p1 = ::HexDigit(trimmed[--digits]);
if (digits > 0) {
*p1 |= ((unsigned char)::HexDigit(trimmed[--digits]) << 4);
p1++;
}
}
```
Same reversal here:
```C++
template <unsigned int BITS>
std::string base_blob<BITS>::GetHex() const
{
uint8_t m_data_rev[WIDTH];
for (int i = 0; i < WIDTH; ++i) {
m_data_rev[i] = m_data[WIDTH - 1 - i];
}
return HexStr(m_data_rev);
}
```
It now makes sense to me that `SetHexDeprecated`, upon receiving a shorter hex string that requires zero-padding, would pad as if the missing hex chars where towards the end of the little-endian byte array, as they are the most significant bytes. "Big-endian" string representation is also consistent with the case where `SetHexDeprecated` receives too many hex digits and discards the leftmost ones, as a form of integer narrowing takes place.
### How I got it wrong in #30436
Previously I used the less than (`<`) comparison to prove endianness, but for `uint256` it uses `memcmp` and thereby gives priority to the *lower* bytes at the beginning of the array.
```C++
constexpr int Compare(const base_blob& other) const { return std::memcmp(m_data.data(), other.m_data.data(), WIDTH); }
```
`arith_uint256` is different in that it begins by comparing the bytes from the end, as it is using little endian representation, where the bytes toward the end are more significant.
```C++
template <unsigned int BITS>
int base_uint<BITS>::CompareTo(const base_uint<BITS>& b) const
{
for (int i = WIDTH - 1; i >= 0; i--) {
if (pn[i] < b.pn[i])
return -1;
if (pn[i] > b.pn[i])
return 1;
}
return 0;
}
```
(The commit documents that `base_blob::Compare()` is doing lexicographic ordering unlike the `arith_*`-variant which is doing numeric ordering).
</details>
ACKs for top commit:
paplorinc:
ACK 73e3fa10b4
ryanofsky:
Code review ACK 73e3fa10b4
Tree-SHA512: 121630c37ab01aa7f7097f10322ab37da3cbc0696a6bbdbf2bbd6db180dc5938c7ed91003aaa2df7cf4a4106f973f5118ba541b5e077cf3588aa641bbd528f4e
a7432dd6ed logging: clarify -debug and -debugexclude descriptions (Anthony Towns)
74dd33cb0a rpc: make logging method reject "0" category and correct the help text (Vasil Dimov)
8c6f3bf163 logging, refactor: minor encapsulation improvement and use BCLog::NONE instead of 0 (Vasil Dimov)
160706aa38 logging, refactor: make category special cases explicit (Ryan Ofsky)
Pull request description:
* Move special cases from `LOG_CATEGORIES_BY_STR` to `GetLogCategory()` (suggested [here](https://github.com/bitcoin/bitcoin/pull/29419#discussion_r1547990373)).
* Remove `"none"` and `"0"` from RPC `logging` help because that help text was wrong. `"none"` resulted in an error and `"0"` was ignored itself (contrary to what the help text suggested).
* Remove unused `LOG_CATEGORIES_BY_STR[""]` (suggested [here](https://github.com/bitcoin/bitcoin/pull/29419#discussion_r1548018694)).
This is a followup to https://github.com/bitcoin/bitcoin/pull/29419, addressing leftover suggestions + more.
ACKs for top commit:
LarryRuane:
ACK a7432dd6ed
ryanofsky:
Code review ACK a7432dd6ed. Only changes since last review are removing dead if statement and adding AJ's suggested -debug and -debugexclude help improvements, which look accurate and much more clear.
Tree-SHA512: 41b997b06fccdb4c1d31f57d4752c83caa744cb3280276a337ef4a9b7012a04eb945071db6b8fad24c6a6cf8761f2f800fe6d8f3d8836f5b39c25e4f11c85bf0
Replace early returns in KeyPair::KeyPair() with asserts.
The if statements imply there is an error we are handling, but keypair_xonly_pub
and xonly_pubkey_serialize can only fail if the keypair object is malformed, i.e.,
it was created with a bad secret key. Since we check that the keypair was created
successfully before attempting to extract the public key, using asserts more
accurately documents what we expect here and removes untested branches from the code.
Move `SignSchnorr` to `KeyPair`. This makes `CKey::SignSchnorr` now
compute a `KeyPair` object and then call `KeyPair::SignSchorr`. The
notable changes are:
* Move the merkle_root tweaking out of the sign function and into
the KeyPair constructor
* Remove the temporary secp256k1_keypair object and have the
functions access m_keypair->data() directly
Current logging RPC method documentation claims to accept "0" and "none"
categories, but the "none" argument is actually rejected and the "0"
argument is ignored. Update the implementation to refuse both
categories, and remove the help text claiming to support them.
* Make the standalone function `LogCategoryToStr()` private inside
`logging.cpp` (aka `static`) - it is only used in that file.
* Make the method `Logger::GetLogPrefix()` `private` - it is only
used within the class.
* Use `BCLog::NONE` to initialize `m_categories` instead of `0`.
We later check whether it is `BCLog::NONE` (in
`Logger::DefaultShrinkDebugFile()`).
Make special cases explicit in GetLogCategory() and LogCategoryToStr()
functions. Simplify the LOG_CATEGORIES_BY_STR and LOG_CATEGORIES_BY_FLAG
mappings and LogCategoriesList() function.
This makes the maps `LOG_CATEGORIES_BY_STR` and `LOG_CATEGORIES_BY_FLAG`
consistent (one is exactly the opposite of the other).
Follow-up to #30436.
uint256 string representation was wrongfully documented as little-endian due to them being reversed by GetHex() etc, and base_blob::Compare() giving most significance to the beginning of the internal array. They are closer to "big-endian", but this commit tries to be even more precise than that.
uint256_tests.cpp - Avoid using variable from the left side of the condition in the right side.
setup_common.cpp - Skip needless ArithToUint256-conversion.
Add a `KeyPair` class which wraps the `secp256k1_keypair`. This keeps
the secret data in secure memory and enables passing the
`KeyPair` object directly to libsecp256k1 functions expecting a
`secp256k1_keypair`.
Motivation: when passing `CKeys` for taproot outputs to libsecp256k1 functions,
the first step is to create a `secp256k1_keypair` data type and use that
instead. This is so the libsecp256k1 function can determine if the key
needs to be negated, e.g., when signing.
This is a bit clunky in that it creates an extra step when using a `CKey`
for a taproot output and also involves copying the secret data into a
temporary object, which the caller must then take care to cleanse. In
addition, the logic for applying the merkle_root tweak currently
only exists in the `SignSchnorr` function.
In a later commit, we will add the merkle_root tweaking logic to this
function, which will make the merkle_root logic reusable outside of
signing by using the `KeyPair` class directly.
Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
Sanity check that using CKey/CPubKey directly vs using secp256k1_keypair objects
returns the same results for BIP341 key tweaking.
Co-authored-by: l0rinc <pap.lorinc@gmail.com>
Add benchmarks for signing with null and non-null merkle_root arguments.
Null and non-null merkle_root arguments will apply the taptweaks
H_TapTweak(P) and H_TapTweak(P | merkle_root), respectively, to the
private key during signing.
This benchmark is added to verify there are no significant performance
changes after moving the taptweak signing logic in a later commit.
Co-authored-by: l0rinc <pap.lorinc@gmail.com>
75648cea5a test: add P2A ProduceSignature coverage (Greg Sanders)
7998ce6b20 Add release note for P2A output feature (Greg Sanders)
71c9b02a04 test: add P2A coverage for decodescript (Greg Sanders)
1349e9ec15 test: Add anchor mempool acceptance test (Greg Sanders)
9d89209937 policy: stop 3rd party wtxid malleability of anchor spend (Greg Sanders)
b60aaf8b23 policy: make anchor spend standard (Greg Sanders)
455fca86cf policy: Add OP_1 <0x4e73> as a standard output type (Greg Sanders)
Pull request description:
This is a sub-feature taken out of the original proposal for ephemeral anchors #30239
This PR makes *spending* of `OP_1 <0x4e73>` (i.e. `bc1pfeessrawgf`) standard. Creation of this output type is already standard.
Any future witness output types are considered relay-standard to create, but not to spend. This preserves upgrade hooks, such as a completely new output type for a softfork such as BIP341. It also gives us a bit of room to use a new output type for policy uses.
This particular sized witness program has no other known use-cases (https://bitcoin.stackexchange.com/a/110664/17078), s it affords insufficient cryptographic security for a secure commitment to data, such as a script or a public key. This makes this type of output "keyless", or unauthenticated.
As a witness program, the `scriptSig` of the input MUST be blank, by BIP141. This helps ensure txid-stability of the spending transaction, which may be required for smart contracting wallets. If we do not use segwit, a miner can simply insert an `OP_NOP` in the `scriptSig` without effecting the result of program execution.
An additional relay restriction is to disallow non-empty witness data, which an adversary may use to penalize the "honest" transactor when RBF'ing the transaction due to the incremental fee requirement of RBF rules.
The intended use-case for this output type is to "anchor" the transaction with a spending child to bring exogenous CPFP fees into the transaction package, encouraging the inclusion of the package in a block. The minimal size of creation and spending of this output makes it an attractive contrast to outputs like `p2sh(OP_TRUE)` and `p2wsh(OP_TRUE)` which
are significantly larger in vbyte terms.
Combined with TRUC transactions which limits the size of child transactions significantly, this is an attractive option for presigned transactions that need to be fee-bumped after the fact.
ACKs for top commit:
sdaftuar:
utACK 75648cea5a
theStack:
re-ACK 75648cea5a
ismaelsadeeq:
re-ACK 75648cea5a via [diff](e7ce6dc070..75648cea5a)
glozow:
ACK 75648cea5a
tdb3:
ACK 75648cea5a
Tree-SHA512: d529de23d20857e6cdb40fa611d0446b49989eaafed06c28280e8fd1897f1ed8d89a4eabbec1bbf8df3d319910066c3dbbba5a70a87ff0b2967d5205db32ad1e
189c987386 Showing local addresses on the Node Window (Jadi)
a5d7aff867 net: Providing an interface for mapLocalHost (Jadi)
Pull request description:
This change adds a new row to the Node Window (debugwindow.ui)
under the Network section which shows the LocalAddresses.
fixes#564
<!--
*** Please remove the following help text before submitting: ***
Pull requests without a rationale and clear improvement may be closed
immediately.
GUI-related pull requests should be opened against
https://github.com/bitcoin-core/gui
first. See CONTRIBUTING.md
-->
<!--
Please provide clear motivation for your patch and explain how it improves
Bitcoin Core user experience or Bitcoin Core developer experience
significantly:
* Any test improvements or new tests that improve coverage are always welcome.
* All other changes should have accompanying unit tests (see `src/test/`) or
functional tests (see `test/`). Contributors should note which tests cover
modified code. If no tests exist for a region of modified code, new tests
should accompany the change.
* Bug fixes are most welcome when they come with steps to reproduce or an
explanation of the potential issue as well as reasoning for the way the bug
was fixed.
* Features are welcome, but might be rejected due to design or scope issues.
If a feature is based on a lot of dependencies, contributors should first
consider building the system outside of Bitcoin Core, if possible.
* Refactoring changes are only accepted if they are required for a feature or
bug fix or otherwise improve developer experience significantly. For example,
most "code style" refactoring changes require a thorough explanation why they
are useful, what downsides they have and why they *significantly* improve
developer experience or avoid serious programming bugs. Note that code style
is often a subjective matter. Unless they are explicitly mentioned to be
preferred in the [developer notes](/doc/developer-notes.md), stylistic code
changes are usually rejected.
-->
<!--
Bitcoin Core has a thorough review process and even the most trivial change
needs to pass a lot of eyes and requires non-zero or even substantial time
effort to review. There is a huge lack of active reviewers on the project, so
patches often sit for a long time.
-->
ACKs for top commit:
pablomartin4btc:
re-ACK 189c987386
furszy:
utACK 189c987
Tree-SHA512: 93f201bc6d21d81b27b87be050a447b841f01e3efb69b9eca2cc7af103023d7cd69eb5e16e2875855573ef51a5bf74a6ee6028636c1b6798cb4bb11567cb4996
fa46a1b74b test: Avoid CScript() as default function argument (MarcoFalke)
fadf621825 test: Make leaf_script mandatory when scriptpath is set in TaprootSignatureMsg (MarcoFalke)
Pull request description:
Unlike other function calls in default arguments, CScript should not cause any issues in the tests, because they are const.
However, this change allows to enable the "function-call-in-default-argument (B008)" lint rule, which will help to catch severe test bugs, such as https://github.com/bitcoin/bitcoin/issues/30543#issuecomment-2259260024 .
The lint rule will be enabled in a follow-up, when all violations are fixed.
ACKs for top commit:
instagibbs:
utACK fa46a1b74b
theStack:
lgtm ACK fa46a1b74b
ismaelsadeeq:
Tested ACK fa46a1b74b
Tree-SHA512: bc68b15121d50ead0fc70ad772360a7829908aedeaff8426efcb8a67f33117f67d26b4f5da94fa735dd8de9c9ff65fc10a29323f1b12f238b75486fa7cc32a89
Move `SetKeyFromPassphrase` to out of LIMITED_WHILE,
remove `SetKey` calls since it is already called
internally by other functions and reduce the number
of iterations.
faed533743 test: Disable known broken USDT test for now (MarcoFalke)
Pull request description:
(cherry picked from commit faf8be7c32)
Sadly, it still happens: https://github.com/bitcoin/bitcoin/issues/27380#issuecomment-2265205214
ACKs for top commit:
fanquake:
ACK faed533743
Tree-SHA512: 7108c468efd31a1f062646b7b21d69ddaaa9808cdc44db75c78d7a840830f85d016d4a95571c239402f0b6639b714224720182bcda8f53b147a0be06cfbd2b25
642c885b61 Merge bitcoin-core/secp256k1#1575: release: prepare for 0.5.1
cdf08c1a2b Merge bitcoin-core/secp256k1#1576: doc: mention `needs-changelog` github label in release process
40d87b8e45 release: prepare for 0.5.1
5770226176 changelog: clarify CMake option
759bd4bbc8 doc: mention `needs-changelog` github label in release process
fded437c4c Merge bitcoin-core/secp256k1#1574: Fix compilation when extrakeys module isn't enabled
763d938cf0 ci: only enable extrakeys module when schnorrsig is enabled
af551ab9db tests: do not use functions from extrakeys module
0055b86780 Merge bitcoin-core/secp256k1#1551: Add ellswift usage example
ea2d5f0f17 Merge bitcoin-core/secp256k1#1563: doc: Add convention for defaults
ca06e58b2c Merge bitcoin-core/secp256k1#1564: build, ci: Adjust the default size of the precomputed table for signing
e2af491263 ci: Switch to the new default value of the precomputed table for signing
d94a9273f8 build: Adjust the default size of the precomputed table for signing
fcc5d7381b Merge bitcoin-core/secp256k1#1565: cmake: Bump CMake minimum required version up to 3.16
9420eece24 cmake: Bump CMake minimum required version up to 3.16
16685649d2 doc: Add convention for defaults
a5269373fa Merge bitcoin-core/secp256k1#1555: Fixed O3 replacement
b8fe33332b cmake: Fixed O3 replacement
31f84595c4 Add ellswift usage example
fe4fbaa7f3 examples: fix case typos in secret clearing paragraphs (s/, Or/, or/)
git-subtree-dir: src/secp256k1
git-subtree-split: 642c885b6102725e25623738529895a95addc4f4
7231c7630e qt: Replace deprecated LogPrintf with LogInfo in GUIUtil::LogQtInfo() (Hennadii Stepanov)
b3d3ae0680 qt, build: Drop `QT_STATICPLUGIN` macro (Hennadii Stepanov)
Pull request description:
Broken out of https://github.com/bitcoin/bitcoin/pull/30454.
Our `QT_STATICPLUGIN` macro is effectively equivalent to the Qt's `QT_STATIC` macro.
It is easy to see in the `_BITCOIN_QT_IS_STATIC` macro implementation: ebd82fa9fa/build-aux/m4/bitcoin_qt.m4 (L269-L292)
No need to handle both macros.
ACKs for top commit:
maflcko:
re-ACK 7231c7630e
TheCharlatan:
ACK 7231c7630e
Tree-SHA512: abbf21859b7ac2aaf47c5b0e075403e4cc9bc540b1565d23f51650b8932dde314586aca67fd4ed5daadebc89268baf8c18f65348fa2b836078ac24543c14cfd6
2e86f2b201 rpc: fix maybe-uninitialized compile warning in getchaintxstats (Michael Dietz)
Pull request description:
This resolves the compiler warning about potential uninitialized use of window_tx_count introduced in fa2dada.
The warning:
```
CXX rpc/libbitcoin_node_a-blockchain.o
rpc/blockchain.cpp: In function ‘getchaintxstats()::<lambda(const RPCHelpMan&, const JSONRPCRequest&)>’:
rpc/blockchain.cpp:1742:38: warning: ‘*(std::_Optional_payload_base<unsigned int>::_Storage<unsigned int, true>*)((char*)&window_tx_count + offsetof(const std::optional<unsigned int>,std::optional<unsigned int>::<unnamed>.std::_Optional_base<unsigned int, true, true>::<unnamed>)).std::_Optional_payload_base<unsigned int>::_Storage<unsigned int, true>::_M_value’ may be used uninitialized in this function [-Wmaybe-uninitialized]
1742 | ret.pushKV("txrate", double(*window_tx_count) / nTimeDiff);
|
```
ACKs for top commit:
maflcko:
lgtm ACK 2e86f2b201
theStack:
ACK 2e86f2b201
tdb3:
ACK 2e86f2b201
Tree-SHA512: c087e8f1cd68dd8df734a8400d30a95abe57ebd56cd53aef4230e425b33a23aa55b3af42abfd162e3be8c937a4c27e56abb70a4fedb10e2df64d52d577e0f262
Contributes to #564 by providing an interface for mapLocalHost
through net -> node interface -> clientModel. Later this value can be
read by GUI to show the local addresses.
No behavior change because any entries that are added in EmplaceCoinInternalDANGER
have DIRTY assigned to them after, and if they
are not inserted then they will not be
modified as before.
This prepares moving the cache entry
flags field to private access.
Co-Authored-By: Martin Leitner-Ankerl <martin.ankerl@gmail.com>
When the transactions being marked done exactly match the first chunk of
what remains of the linearization, we can just remember to skip that
chunk instead of computing a full rechunking.
Further, chop off prefixes of the input linearization that are already done,
so they don't need to be reconsidered for further rechunkings.
be419674da qt: Update translation source file (Hennadii Stepanov)
e49d858aab qt: Bump Transifex slug for 28.x (Hennadii Stepanov)
31b33019b7 qt: Pull recent translations from Transifex (Hennadii Stepanov)
Pull request description:
This PR follows our [Release Process](4c62f4b535/doc/release-process.md).
Note: (possible) vandalism/damage has been prevented by reverting the deletion of `bitcoin_af`, `bitcoin_es_MX`, and `bitcoin_ru` translations.
Required to open Transifex translations for v28.0 as it's scheduled in https://github.com/bitcoin/bitcoin/issues/29891.
The previous similar PR: https://github.com/bitcoin/bitcoin/pull/29397.
ACKs for top commit:
stickies-v:
ACK be419674da
Tree-SHA512: 76f7947af9c156c2aaf24c7f926f82e4d8e2664beb5ebde5c7cda8dd7a8dbf672b4a886302c8d189e0cb2145c0ed755f45f9cdb545e29d38bb1ec90ca18fa539
f553e6d86f refactor: remove TxidFromString (stickies-v)
285ab50ace test: replace WtxidFromString with Wtxid::FromHex (stickies-v)
9a0b2a69c4 fuzz: increase FromHex() coverage (stickies-v)
526a87ba6b test: add uint256::FromHex unittest coverage (stickies-v)
Pull request description:
Since fab6ddbee6, `TxidFromString()` has been deprecated because it is less robust than the `transaction_identifier::FromHex()` introduced in [the same PR](https://github.com/bitcoin/bitcoin/pull/30482). Specifically, it tries to recover from length-mismatches, recover from untrimmed whitespace, 0x-prefix and garbage at the end, instead of simply requiring exactly 64 hex-only characters.
In this PR, `TxidFromString` is removed completely to clean up the code and prevent further unsafe usage. Unit and fuzz test coverage on `uint256::FromHex()` and functions that wrap it is increased.
Note: `TxidFromSring` allowed users to prefix strings with "0x", this is no longer allowed for `transaction_identifier::FromHex()`, so a helper function for input validation may prove helpful in the future _(this overlaps with the `uint256::uint256S()` vs `uint256::FromHex()` future cleanup)_. It is not relevant to this PR, though, besides the fact that this unused (except for in tests) functionality is removed.
The only users of `TxidFromString` are:
- `test`, where it is straightforward to drop in the new `FromHex()` methods without much further concern
- `qt` coincontrol. There is no need for input validation here, but txids are not guaranteed to be 64 characters. This is already handled by the existing code, so again, using `FromHex()` here seems quite straightforward.
Addresses @maflcko's suggestion: https://github.com/bitcoin/bitcoin/pull/30482#discussion_r1691826934
Also removes `WtxidFromString()`, which is a test-only helper function.
### Testing GUI changes
To test the GUI coincontrol affected lines, `regtest` is probably the easiest way to quickly get some test coins, you can use e.g.
```
alias cli="./src/bitcoin-cli -regtest"
cli createwallet "coincontrol"
# generate 10 spendable outputs on 1 address
cli generatetoaddress 10 $(cli -rpcwallet=coincontrol getnewaddress)
# generate 10 spendable outputs on another address
cli generatetoaddress 10 $(cli -rpcwallet=coincontrol getnewaddress)
# make previous outputs spendable
cli generatetoaddress 100 $(cli -rpcwallet=coincontrol getnewaddress)
```
ACKs for top commit:
maflcko:
re-ACK f553e6d86f🔻
hodlinator:
ACK f553e6d86f
paplorinc:
ACK f553e6d86f
TheCharlatan:
Nice, ACK f553e6d86f
Tree-SHA512: c1c7e6ea4cbf05cf660ba178ffc4f35f0328f7aa6ad81872e2462fb91a6a22e4681ff64b3d0202a5a9abcb650c939561585cd309164a69ab6081c0765ee271ef
89b1d5c818 depends: Fix `zeromq` build on OpenBSD (Hennadii Stepanov)
Pull request description:
On the master branch @ 66e82dc90c, the `zeromq` package fails to build on OpenBSD 7.5:
```
[ 19%] Building CXX object CMakeFiles/objects.dir/src/io_thread.cpp.o
/home/hebasto/bitcoin/depends/work/build/amd64-unknown-openbsd7.5/zeromq/4.3.5-df5b1b9f936/src/io_thread.cpp:14:22: error: static_cast from 'std::nullptr_t' to 'poller_t::handle_t' (aka 'int') is not allowed
_mailbox_handle (static_cast<poller_t::handle_t> (NULL))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
```
This [regression](https://github.com/bitcoin/bitcoin/pull/29723#issuecomment-2261513105) was overlooked by me in https://github.com/bitcoin/bitcoin/pull/29723.
This PR fixes the issue by backporting an upstream commit from https://github.com/zeromq/libzmq/pull/4659.
ACKs for top commit:
theStack:
tACK 89b1d5c818
Tree-SHA512: 48d22ea99dfd44c5adf858c74e64082390da27b8ccad8c0d5a91d4dabfa3d12267cef98e4bb8c088e4cd0ec477c242cb1d47aace5c88cd86f796715bba957ed8
afd237bb5d [fuzz] Harness for version handshake (dergoegge)
a90ab4aec9 scripted-diff: Rename lazily initialized bloom filters (dergoegge)
82de1bc478 [net processing] Lazily initialize m_recent_confirmed_transactions (dergoegge)
fa0c87f19c [net processing] Lazily initialize m_recent_rejects_reconsiderable (dergoegge)
662e8db2d3 [net processing] Lazily initialize m_recent_rejects (dergoegge)
Pull request description:
This adds a fuzzing harness dedicated to the version handshake. To avoid determinism issues, the harness creates necessary components each iteration (addrman, peerman, etc). A harness like this would have easily caught https://bitcoincore.org/en/2024/07/03/disclose-timestamp-overflow/.
As a performance optimization, this PR includes a change to `PeerManager` to lazily initialize various filters (to avoid large unnecessary memory allocations each iteration).
ACKs for top commit:
brunoerg:
ACK afd237bb5d
marcofleon:
Tested ACK afd237bb5d. I compared the coverage of `net_processing` from this harness to the `process_message` and `process_messages` harnesses to see the differences. This target hits more specific parts of the version handshake. The stability looks good as well, at about 94%.
glozow:
utACK afd237bb5d lazy blooms look ok
mzumsande:
Code Review ACK afd237bb5d
Tree-SHA512: 62bba20aec0cd220e62368354891f9790b81ad75e8adf7b22a76a6d4663bd26aedc4cae8083658a75ea9043d60aad3f0e58ad36bd7bbbf93ff1d16e317bf15cc
3cd24aadb2 doc: remove obsolete mention and link to closed PR (Marnix)
Pull request description:
Remove the mention and link as the PR (https://github.com/bitcoin/bitcoin/pull/22341) is closed and the description is wrong/outdated anyway.
ACKs for top commit:
BrandonOdiwuor:
ACK 3cd24aadb2
tdb3:
ACK 3cd24aadb2
Tree-SHA512: 5cd97029337f0cdfe81b6be9401adc4fe51ae2868f8fcadcb03828531a38380a587c32840850a924b6428f62df7d20a1e16ef7414d4078e7bb2c4e359b1fae40
fa2269dd65 test: Try previously intermittently broken USDT test again (MarcoFalke)
Pull request description:
Seems fine to try it again, given that the infra changed in the meantime.
Should be trivial to disable again, on the first failure.
Ref: https://github.com/bitcoin/bitcoin/issues/27380#issuecomment-1637971779
ACKs for top commit:
fanquake:
ACK fa2269dd65 - seems fine to re-try this now.
Tree-SHA512: e203625b49019517def8e92cc465dd947d5aa85a080d370aabce7a45442efc3900b5c3783f6cc15720f81e5aaa3ff193a26be3d33048f20a73cd7b1fce320118
This does not cause any issues, because CScript in the tests are const.
However, this change allows to enable the
"function-call-in-default-argument (B008)" lint rule.
f70eb0eeef doc: Remove reference to resolved issue (Daniela Brozzoni)
b27ef8ec7f doc: Update issue reference for libbitcoinkernel (Daniela Brozzoni)
Pull request description:
- The discussion of libbitcoinkernel has moved from 24303 to 27587
- Issue 15732 has been resolved, removing it from the document
ACKs for top commit:
maflcko:
ACK f70eb0eeef
Tree-SHA512: 11b597d9710504010945aae66f7e488403895aa8e1e091f3a8f6737dc128a4fde185daff8d4709cbbb69f454d3a649c4217e82a6bfc8ee2b25c8a1c047b57f1b
93fb0e7897 kernel: Only setup kernel context globals once (TheCharlatan)
Pull request description:
The globals setup by the function calls when creating a new kernel context only need to be setup once. Calling them multiple times may be wasteful and has no apparent benefit.
Besides kernel users potentially creating multiple contexts, this change may also be useful for tests creating multiple setups.
ACKs for top commit:
stickies-v:
re-ACK 93fb0e7897
maflcko:
ACK 93fb0e7897👝
tdb3:
re ACK 93fb0e7897
Tree-SHA512: c8418c23b34883b9b6af2b93c48760a931c246c9190fae372fb808f573408d332f53ca43b9c783eef561c4a6681e2fb63f215c939b40a87d597c0518dabea22a
a6efc7e16e test: fix intermittent failures in feature_proxy.py (Martin Zumsande)
Pull request description:
Fixes#29871
If addnode connections are made with v2transport and the peer immediately disconnects us, reconnections with v1 are scheduled. This could interfere with later checks depending on timing. Avoid this by using `v2transport=False` in the addnode rpc - this test isn't about the message layer anyway, so running it with v2 would add no value.
ACKs for top commit:
maflcko:
ACK a6efc7e16e
tdb3:
cr re ACK a6efc7e16e
Tree-SHA512: 39353a392e75e4c6257d971ceecb65fb76ec6d3b121a087869831c24b767a18f57e2ae2968da445c7fa731cb03053c90df37dd2cd6e86f786ad4121bc68ca235
ec5e294e4b test: fix constructor of msg_tx (Martin Zumsande)
Pull request description:
In python, if the default value is a mutable object (here: a class) it is shared over all instances, so that one instance being changed would affect others to be changed as well.
This was the source of #30543, and possibly various other intermittent bugs in the functional tests, see
https://github.com/bitcoin/bitcoin/issues/29621#issuecomment-1999298224.
Fixes#30543Fixes#29621Fixes#25128
ACKs for top commit:
sipa:
utACK ec5e294e4b. I believe some linters even warn about doing this.
maflcko:
ACK ec5e294e4b
vasild:
ACK ec5e294e4b❤️
theStack:
ACK ec5e294e4b
Tree-SHA512: a6204fb1a326de3f9aa965f345fd658f6a4dcf78731db25cc905ff6eb8d4eeb65d14cc316305eebd89387aec8748c57c3a4f4ca62408f8e5ee53f535b88b1411
903def1ffd doc: mention optional dependencies (qrencode, zmq) in OpenBSD build docs (Sebastian Falbesoner)
Pull request description:
The wording is taken from the FreeBSD build docs.
Tested on OpenBSD 7.5. See the following links for the package names:
- https://openbsd.app/?search=libqrencode
- https://openbsd.app/?search=zeromq
Thanks to hebasto for noticing that this was missing.
ACKs for top commit:
maflcko:
review ACK 903def1ffd
hebasto:
ACK 903def1ffd, I can successfully build with the `libqrencode` and `zeromq` packages on my OpenBSD 7.5 installation.
Tree-SHA512: 955e4892948a7703627d304a41a774f7cca0e4c672bdfa0edf531587d6970444aa49195b0f6f531ce375c8e7c2af6bbfa1a12e0612ae7a65f3e454fb17958672
This removes the default value, because there should not be a use-case
to fall back to a an empty leaf_script by default. (If there was, it
could trivially be added back)
In python, if the default value is a mutable object (here: a class)
its shared over all instances, so that one instance being changed
would affect others to be changed as well.
This was likely the source of various intermittent bugs in the
functional tests.
These outputs are called anchors, and allow
key-less anchor spends which are vsize-minimized
versus keyed anchors which require larger outputs
when creating and inputs when spending.
41a1a8615d gui: Hide peers details (@RandyMcMillan)
Pull request description:
Add a close (X) button to the Peers Detail panel.
Reuse the same icon used in the Console Tab.
The close button deselects the peer highlighted
in the PeerTableView and hides the detail panel.
fixes#485
Co-authored-by: @w0xlt <w0xlt@users.noreply.github.com>
ACKs for top commit:
pablomartin4btc:
re ACK 41a1a8615d
hebasto:
ACK 41a1a8615d, tested on Ubuntu 23.10.
Tree-SHA512: fc692891eec61bd1e6878f2433b478de3c69bf0b3ce3471f2faafda6f63d371e2cc125ae8290fd2ac3e4d8659031b79d85665318cfc5a9481e967ef99d245f9c
The globals setup by the function calls when creating a new kernel
context only need to be setup once. Calling them multiple times may be
wasteful and has no apparent benefit.
Besides kernel users potentially creating multiple contexts, this change
may also be useful for tests creating multiple setups.
Co-authored-by: stickies-v <stickies-v@protonmail.com>
If addnode connections are made with v2transport and the peer immediately disconnects us, reconnections
with v1 are scheduled. This could interfere with later checks depending on timing. Avoid this by using
`v2transport=False` in the addnode rpc - this test isn't about the message layer anyway, so running it
with v2 would add no value.
bda537f7c4 depends: remove ENV unsetting for darwin (fanquake)
1807760f09 guix: improve ENV unsetting for macOS (fanquake)
0b2aeee21d depends: patch explicit -lm usage out of Qt tools (fanquake)
Pull request description:
Now that we use the native compiler, and have fixed Qt, and these vars
are (almost) unset in Guix, we can remove the unsetting from our compiler
command here.
I couldn't manage to make a darwin-clang-cross only exclusion of `-lm` work properly
for Qt, so opted for just removing the explicit link entirely. I do not think this should have
any other unwanted side-effects.
Fixes#21552.
ACKs for top commit:
TheCharlatan:
ACK bda537f7c4
Tree-SHA512: 97a2d85de7d4b1d65717ecb521399ecba5f53863b8aef21af62ede5ceee59ee1a9392663da3a3852cad1b6d8b420dd4b0b5f0eea38d30a81785d8b2718620b5f
93ee17c1d6 ci: enable berkley db on test each commit job (Max Edwards)
Pull request description:
As the "test each commit" job installs `libdb++-dev` it looks like it was intended that it would compile with Berkeley DB support.
This PR enables it.
CI run log with the change: https://github.com/m3dwards/bitcoin/actions/runs/10142921800/job/28043223197?pr=1
ACKs for top commit:
maflcko:
ACK 93ee17c1d6
danielabrozzoni:
ACK 93ee17c1d6
hebasto:
ACK 93ee17c1d6.
Tree-SHA512: b6c2a7cea104a84221814fd121fd4fef2d7c0c3717f8c3fe35ec3c42e850e72085e9e6407b61ca1f9e86571346fa33a2cdd924210b26155956835011775320b2
e6df3485ed guix: move bison from global scope, to Linux (fanquake)
Pull request description:
This is only needed for the Qt build (libxkbcommon), on Linux, so does not need to be built/present for the macOS or Windows builds.
ACKs for top commit:
hebasto:
ACK e6df3485ed.
TheCharlatan:
ACK e6df3485ed
Tree-SHA512: b66111e398b4fce88f912adfd808d537e2d85e1f0078befd264bb700b201ca1bbe322810e80a212e0023657e9e3693a106761c43743d66aabd16e2afe7f599e6
17845e7f21 rpc: add utxo's blockhash and number of confirmations to scantxoutset output (Luis Schwab)
Pull request description:
This PR resolves#30478 by adding two fields to the `scantxoutset` RPC:
- blockhash: the blockhash that an UTXO was created
- confirmations: the number of confirmations an UTXO has relative to the chaintip.
The rationale for the first field is that a blockhash is a much more reliable identifier than the height:
> When using the scantxoutset RPC, the current behaviour is to show the block height of the UTXO. This is not optimal, as block height is ambiguous, especially in the case of a block reorganization happening at the same instant of the query. In this case, an UTXO that does not exist would be assumed to exist, unless the chain's tip hash is recorded before the scan, and make sure it still exists after, as per https://github.com/bitcoindevkit/bdk/issues/895#issuecomment-1475766797 comment by evanlinjin.
The second one was suggested by maflcko, and I agree it's useful for human users:
> While touching this, another thing to add could be the number of confirmations? I understand that this wouldn't help machine consumers of the interface, but human callers may find it useful?
This will yield an RPC output like so:
```diff
bitcoin-cli scantxoutset start "[\"addr(bc1q5q9344vdyjkcgv79ve3tldz4jmx4lf7knmnx6r)\"]"
{
"success": true,
"txouts": 185259116,
"height": 853622,
"bestblock": "00000000000000000002e97d9be8f0ddf31829cf873061b938c10b0f80f708b2",
"unspents": [
{
"txid": "fae435084345fe26e464994aebc6544875bca0b897bf4ce52a65901ae28ace92",
"vout": 0,
"scriptPubKey": "0014a00b1ad58d24ad8433c56662bfb45596cd5fa7d6",
"desc": "addr(bc1q5q9344vdyjkcgv79ve3tldz4jmx4lf7knmnx6r)#smk4xmt7",
"amount": 0.00091190,
"coinbase": false,
"height": 852741,
+ "blockhash": "00000000000000000002eefe7e7db44d5619c3dace4c65f3fdcd2913d4945c13",
+ "confirmations": 882
}
],
"total_amount": 0.00091190
}
```
ACKs for top commit:
sipa:
utACK 17845e7f21
Eunovo:
ACK 17845e7f21
tdb3:
ACK 17845e7f21
Tree-SHA512: 02366d0004e5d547522115ef0efe6794a35978db53dda12c675cfae38197bf43f0bf89ca99a3d79e3d2cff95186015fe1ab764abb8ab82bda440ae9302ad973b
The error messages should never happen in normal operation. However, if
they do, they are helpful to return to the user to debug the issue. For
example, to notice a truncated file.
b4dd7ab43e logging: use std::string_view (Anthony Towns)
558df5c733 logging: Apply formatting to early log messages (Anthony Towns)
6cf9b34440 logging: Limit early logging buffer (Anthony Towns)
0b1960f1b2 logging: Add DisableLogging() (Anthony Towns)
6bbc2dd6c5 logging: Add thread safety annotations (Anthony Towns)
Pull request description:
In order to cope gracefully with `Log*()` calls that are invoked prior to logging being fully configured (indicated by calling `StartLogging()` we buffer early log messages in `m_msgs_before_open`. This has a couple of minor issues:
* if there are many such log messages the buffer can become arbitrarily large; this can be a problem for users of libkernel that might not wish to worry about logging at all, and as a result never invoke `StartLogging()`
* early log messages are formatted before the formatting options are configured, leading to inconsistent output
Fix those issues by buffering the log info prior to formatting it, and setting a limit on the size of the buffer (dropping the oldest lines, and reporting the number of lines skipped).
Also adds some thread safety annotations, and the ability to invoke `LogInstance().DisableLogging()` if you want to disable logging entirely, for a minor efficiency improvement.
ACKs for top commit:
maflcko:
re-ACK b4dd7ab43e 🕴
ryanofsky:
Code review ACK b4dd7ab43e
TheCharlatan:
Nice, ACK b4dd7ab43e
Tree-SHA512: 966660181276939225a9f776de6ee0665e44577d2ee9cc76b06c8937297217482e6e426bdc5772d1ce533a0ba093a8556b6a50857d4c876ad8923e432a200440
fae0db0360 fuzz: Deglobalize signature cache in sigcache test (TheCharlatan)
Pull request description:
The body of the fuzz test should ideally be a pure function. If data is persisted in the cache over many iterations, and there is a crash, reproducing it from the input might be difficult. Solve this by getting rid of the global state. This is a follow-up from #30425.
ACKs for top commit:
dergoegge:
utACK fae0db0360
ryanofsky:
Code review ACK fae0db0360
Tree-SHA512: 93dcbb9f2497f13856970469042d6870f04de10fe206827a8db1aae7fc8f3ac7fd900bee7945b5fe4c9e33883268dabb15be7e7bc91cf353ffc0d118cd60e97d
647fa37cdb bench: add cluster linearization improvement benchmark (Pieter Wuille)
28549791b3 clusterlin: permit passing in existing linearization to Linearize (Pieter Wuille)
97d98718b0 clusterlin: add LinearizationChunking class (Pieter Wuille)
d5918dc3c6 clusterlin: randomize the SearchCandidateFinder search order (Pieter Wuille)
991ff9a9a4 clusterlin: use bounded BFS exploration (optimization) (Pieter Wuille)
d9b235e7d2 bench: Candidate finding and linearization benchmarks (Pieter Wuille)
46aad9b099 clusterlin: add Linearize function (Pieter Wuille)
ee0ddfe4f6 clusterlin: add chunking algorithm (Pieter Wuille)
2a41f151af clusterlin: add SearchCandidateFinder class (Pieter Wuille)
4828079db3 clusterlin: add AncestorCandidateFinder class (Pieter Wuille)
58f7e01db4 tests: framework for testing DepGraph class (Pieter Wuille)
a6e07e769a clusterlin: introduce cluster_linearize.h with Cluster and DepGraph types (Pieter Wuille)
Pull request description:
Part of cluster mempool: #30289
This introduces low-level cluster linearization code, including tests and some benchmarks. It is currently not hooked up to anything.
Ultimately, what this PR adds is a function `Linearize` which operates on instances of `DepGraph` (instances of which represent pre-processed transaction clusters) to produce and/or improve linearizations for that cluster.
To provide assurance, the code heavily relies on fuzz tests. A novel approach is used here, where the fuzz input is parsed using the serialization.h framework rather than `FuzzedDataProvider`, with a custom serializer/deserializer for `DepGraph` objects. By including serialization, it's possible to ascertain that the format can represent every relevant cluster, as well as potentially permitting the construction of ad-hoc fuzz inputs from clusters (not included in this PR, but used during development).
---
The `Linearize(depgraph, iteration_limit, rng_seed, old_linearization)` function is an implementation of the (single) LIMO algorithm, with the $S$ in every iteration found as the best out of (a) the best remaining ancestor set and (b) randomized computationally-bounded search. It incrementally builds up a linearization by finding good topologically-valid subsets to move to the front, in such a way that the resulting linearization has a diagram that is at least as good as the `old_linearization` passed in (if any).
* Despite using both best ancestor set and search, this is not Double LIMO, as no intersections between these are involved; just the best of the two.
* The `iteration_limit` and `rng_seed` only control the (b) randomized search. Even with 0 iterations, the result will be as good as the old linearization, and the included sets at every point will have a feerate at least as high as the best remaining ancestor set at that point.
The search algorithm used in the (b) step is very basic, and largely matches Section 2.1 of [How to Linearize your Cluster.](https://delvingbitcoin.org/t/how-to-linearize-your-cluster/303#h-21-searching-6). See #30286 for optimizations to make it more efficient.
For background and references, see [Introduction to cluster linearization](https://delvingbitcoin.org/t/introduction-to-cluster-linearization/1032).
ACKs for top commit:
instagibbs:
reACK 647fa37cdb
glozow:
reACK 647fa37cdb, both code and mermaid diagram look correct to me
sdaftuar:
ACK 647fa37cdb
Tree-SHA512: 52c8aa3d1d91190bf1265a947d2712e9d12f745313ffceef6ae7e3ff517d01d8b3b9b4ce6066298d59751c4ba90555a3c0171229868ba50100f588a2aa6a486d
e4b0dabb21 test: add functional test for tagged MiniWallet instances (Sebastian Falbesoner)
3162c917e9 test: fix MiniWallet internal key derivation for tagged instances (Sebastian Falbesoner)
c9f7364ab2 test: fix MiniWallet script-path spend (missing parity bit in leaf version) (Sebastian Falbesoner)
7774c314fb test: refactor: return TaprootInfo from P2TR address creation routine (Sebastian Falbesoner)
Pull request description:
This PR fixes a dormant bug in MiniWallet that exists since support for P2TR was initially added in #23371 (see commit 041abfebe4).
In the course of spending the output, the leaf version byte of the control block in the witness stack doesn't set the parity bit, i.e. we were so far just lucky that the used combinations of relevant data (internal pubkey, leaf script / version) didn't result in a tweaked pubkey with odd y-parity. If that was the case, we'd get the following validation error:
`mandatory-script-verify-flag-failed (Witness program hash mismatch) (-26)`
Since MiniWallets can now optionally be tagged (#29939), resulting in different internal pubkeys, the issue is more prevalent now. Fix it by passing the parity bit, as specified in BIP341.
Can be tested with the following patch (fails on master, succeeds on PR):
```diff
diff --git a/test/functional/test_framework/mempool_util.py b/test/functional/test_framework/mempool_util.py
index 148cc935ed..7ebe858681 100644
--- a/test/functional/test_framework/mempool_util.py
+++ b/test/functional/test_framework/mempool_util.py
@@ -42,7 +42,7 @@ def fill_mempool(test_framework, node):
# Generate UTXOs to flood the mempool
# 1 to create a tx initially that will be evicted from the mempool later
# 75 transactions each with a fee rate higher than the previous one
- ephemeral_miniwallet = MiniWallet(node, tag_name="fill_mempool_ephemeral_wallet")
+ ephemeral_miniwallet = MiniWallet(node, tag_name="fill_mempool_ephemeral_wallet3")
test_framework.generate(ephemeral_miniwallet, 1 + num_of_batches * tx_batch_size)
# Mine enough blocks so that the UTXOs are allowed to be spent
```
In addition to that, another bug is fixed where the internal key derivation failed, as not every pseudorandom hash results in a valid x-only pubkey. Fix this by treating the hash result as private key and calculate the x-only public key out of that, to be used then as internal key.
Fixes#30528.
ACKs for top commit:
glozow:
ACK e4b0dabb21
rkrux:
reACK [e4b0dab](e4b0dabb21)
hodlinator:
ACK e4b0dabb21
Tree-SHA512: a16f33f76bcb1012857cc3129438a9f6badf28aa2b1d25696da0d385ba5866b46de0f1f93ba777ed9263fe6952f98d7d9c44ea0c0170a2bcc86cbef90bf6ac58
7aa8994c6f refactor: Add FlatFileSeq member variables in BlockManager (TheCharlatan)
Pull request description:
Instead of constructing a new class every time a file operation is done, construct them once for each of the undo and block file when a new BlockManager is created.
In future, this might make it easier to introduce an abstract block store.
Historically, this was not easily possible prior to #27125.
ACKs for top commit:
danielabrozzoni:
ACK 7aa8994c6f
tdb3:
ACK 7aa8994c6f
stickies-v:
ACK 7aa8994c6f
brunoerg:
utACK 7aa8994c6f
Tree-SHA512: 7c181968c270956c90fa0f3687562239912a973b6a35ddbf49fc58733247ea9d986303cbf6f8fc16e8c2d9bf4505e866aed37f030a8c9be72e95bf3752902aa6
c399c80a09 cleanse: Use SecureZeroMemory for mingw-w64 (release) builds (fanquake)
Pull request description:
This PR switches our Windows release builds to use the [`SecureZeroMemory()`](https://learn.microsoft.com/en-us/previous-versions/windows/desktop/legacy/aa366877(v=vs.85)) provided by mingw-w64.
ACKs for top commit:
sipa:
utACK c399c80a09
TheCharlatan:
ACK c399c80a09
Tree-SHA512: dbb20b16c85061d2f9408a3cf69cecc16765f8f61b25a1707146767b664c7ad0caf36975380814ef8e7c49a30199daebac6d5d7a3585354d1adac8e9770199c6
It encapsulates a given linearization in chunked form, permitting arbitrary
subsets of transactions to be removed from the linearization. Its purpose
is adding the Intersect function, which is a crucial operation that will
be used in a further commit to make Linearize improve existing linearizations.
Switch to BFS exploration of the search tree in SearchCandidateFinder
instead of DFS exploration. This appears to behave better for real
world clusters.
As BFS has the downside of needing far larger search queues, switch
back to DFS temporarily when the queue grows too large.
Add benchmarks for known bad graphs for the purpose of search (as
an upper bound on work per search iterations) and ancestor sorting
(as an upper bound on linearization work with no search iterations).
This adds a first version of the overall linearization interface, which given
a DepGraph constructs a good linearization, by incrementally including good
candidate sets (found using AncestorCandidateFinder and SearchCandidateFinder).
This introduces a bespoke fuzzing-focused serialization format for DepGraphs,
and then tests that this format can represent any graph, roundtrips, and then
uses that to test the correctness of DepGraph itself.
This forms the basis for future fuzz tests that need to work with interesting
graphs.
This primarily adds the DepGraph class, which encapsulates precomputed
ancestor/descendant information for a given transaction cluster, with a
number of utility features (inspectors for set feerates, computing
reduced parents/children, adding transactions, adding dependencies), which
will become needed in future commits.
Now that we use the native compiler, and have fixed Qt, and these vars
are unset it Guix, we can remove the unsetting from our compiler command
here.
Fixes#21552.
d1592d2eee guix: use gcc-12 to compile winpthreads (fanquake)
b23690e821 guix: use GCC 12.4.0 over 12.3.0 (fanquake)
8b41ede55e guix: consolidate back to GCC 12 toolchain for all HOSTS (fanquake)
Pull request description:
This PR contains 3 changes:
* Bump GCC in Guix from [12.3.0 to 12.4.0](https://gcc.gnu.org/gcc-12/). A patch was sent upstream, https://lists.gnu.org/archive/html/guix-patches/2024-06/msg01025.html, but has not landed.
* Consolidate all build environments back to using a GCC 12 toolchain. After #21778, the macOS environment is no-longer pinned to 11 (12 would otherwise cause issues building cctools). So, instead of requiring all builders to compile an additional GCC toolchain, use 12.
* Use GCC 12 to compile winpthreads. Currently, GCC 11 is used; which became apparent in https://github.com/bitcoin/bitcoin/pull/30452#issuecomment-2244715566.
ACKs for top commit:
TheCharlatan:
ACK d1592d2eee
hebasto:
ACK d1592d2eee.
Tree-SHA512: e3aa1fa3e69500c93180e07cb4684661247ec6bc45245f746538d81406ff1d8777131590307496dda3287a112b6633e4991168586ca4c2036fa3a57b1efa9c87
f46b220256 fuzz: Use BasicTestingSetup for coins_view target (TheCharlatan)
9e2a723d5d test: Add arguments for creating a slimmer setup (TheCharlatan)
Pull request description:
This adds arguments to some of the testing setup constructors for creating an environment without networking and a validation interface. This is useful for improving the performance of the utxo snapshot fuzz test, which constructs a new TestingSetup on each iteration.
Using this slimmed down `TestingSetup` in future might also make the tests a bit faster when run in aggregate.
ACKs for top commit:
maflcko:
review ACK f46b220256
dergoegge:
utACK f46b220256
Tree-SHA512: 9dc62512b127b781fc9e2d8ef2b5a9b06ebb927a8294b6d872001c553984a7eb1f348e0257b32435b34b5505b5d0323f73bdd572a673da272d3e1e8538ab49d6
fac0c3d4bf doc: Add release notes for two pull requests (MarcoFalke)
fa7b57e5f5 refactor: Replace ParseHashStr with FromHex (MarcoFalke)
fa90777245 rest: Reject truncated hex txid early in getutxos parsing (MarcoFalke)
fab6ddbee6 refactor: Expose FromHex in transaction_identifier (MarcoFalke)
fad2991ba0 refactor: Implement strict uint256::FromHex() (MarcoFalke)
fa103db2bb scripted-diff: Rename SetHex to SetHexDeprecated (MarcoFalke)
fafe4b8051 test: refactor: Replace SetHex with uint256 constructor directly (MarcoFalke)
Pull request description:
In `rest_getutxos` truncated txids such as `aa` or `ff` are accepted. This is brittle at best.
Fix it by rejecting any truncated (or overlarge) input.
----
Review note: This also starts a major refactor to rework hex parsing in Bitcoin Core, meaning that a few refactor commits are included as well. They are explained individually in the commit message and the work will be continued in the future.
ACKs for top commit:
stickies-v:
re-ACK fac0c3d4bf - only doc and test updates to address review comments, thanks!
hodlinator:
ACK fac0c3d4bf
Tree-SHA512: 473feb3fcf6118443435d1dd321006135b0b54689bfbbcb1697bb5811a449bef51f475c715de6911ff3c4ea3bdb75f601861ff93347bc4414d6b9e5298105dd7
faa3598772 ci: Add missing qttools5-dev install to Asan task (MarcoFalke)
Pull request description:
This is required, according to the docs:
```
$ git grep --line-number 'qtbase5-dev qttools5-dev qttools5-dev-tools' doc
doc/build-unix.md:84: sudo apt-get install qtbase5-dev qttools5-dev qttools5-dev-tools
```
Also, needed for cmake.
ACKs for top commit:
hebasto:
ACK faa3598772.
Tree-SHA512: c986908f757d70d958267c1e902b5d7d94589360db61ddf7b9b398cd635b2172e83510c0c77fd6032810166342a286c0f95225b6c6639acd869e1e51c3348ea7
25bf86a225 [test]: ensure `estimatesmartfee` default mode is `economical` (ismaelsadeeq)
41a2545046 [fees]: change `estimatesmartfee` default mode to `economical` (ismaelsadeeq)
Pull request description:
Fixes#30009
This PR changes the `estimatesmartfee` default mode to `economical`.
This was also suggested on IRC https://bitcoin-irc.chaincode.com/bitcoin-core-dev/2024-04-26#1021609
- `conservative` mode: This is the `estimatesmartfee` RPC mode which considers a longer history of blocks. It potentially returns a higher fee rate and is more likely to be sufficient for the desired target, but it is not as responsive to short-term drops in the prevailing fee market.
- `economical` mode: This is the `estimatesmartfee` RPC mode where estimates are potentially lower and more responsive to short-term drops in the prevailing fee market.
Since users are likely to use the default mode, this change will reduce overestimation for many users. The conservative mode remains available for those who wish to opt-in.
For an in-depth analysis of how significantly the `conservative` mode overestimates, see
https://delvingbitcoin.org/t/bitcoind-policy-estimator-modes-analysis/964.
ACKs for top commit:
instagibbs:
reACK 25bf86a225
glozow:
ACK 25bf86a225
willcl-ark:
ACK 25bf86a225
Tree-SHA512: 78ebda667eb9c8f87dcc2f0e6c14968bd1de30358dc77a13611b186fb8427ad97d9f537bad6e32e0a1aa477ccd8c64fee4d41e19308ef3cb184ff1664e6ba8a6
e3edaccd9d ci: add _LIBCPP_REMOVE_TRANSITIVE_INCLUDES to TSAN job (fanquake)
6e786165ca refactor: fix missing includes (fanquake)
Pull request description:
Add `_LIBCPP_REMOVE_TRANSITIVE_INCLUDES` to one of the libc++ CI jobs, to catch missing includes, that are otherwise hidden by transitive includes inside libc++. A more appropriate place for this might be the tidy job, but that does not use libc++.
See https://libcxx.llvm.org/DesignDocs/HeaderRemovalPolicy.html for more information.
ACKs for top commit:
maflcko:
re-ACK e3edaccd9d
Tree-SHA512: 3fb2e9bbbf4bb1570633d52939875ee674d934b645a4037a309643f84ab69edf0fb5b6cfcbd02fa7d92052a64fa63f31979a58fede23593c4df7c33a8cb2953a
This is a safe replacement of the previous SetHex, which now returns an
optional to indicate success or failure.
The code is similar to the ParseHashStr helper, which will be removed in
a later commit.
The current behavior of seednode fetching is pretty eager: we do it as the first
step under `ThreadOpenNetworkConnections` even if some peers may be queryable
from our addrman. This poses two potential issues:
- First, if permanently set (e.g. running with seednode in a config file) we'd
be signaling such seed every time we restart our node
- Second, we will be giving the seed node way too much influence over our addrman,
populating the latter even with data from the former even when unnecessary
This changes the behavior to only add seednodes to `m_addr_fetch` if our addrman
is empty, or little by little after we've spent some time trying addresses from
our addrman. Also, seednodes are added to `m_addr_fetch` in random order, to avoid
signaling the same node in case more than one seed is added and we happen to try
them over multiple restarts
These cause compile failures with _LIBCPP_REMOVE_TRANSITIVE_INCLUDES.
i.e:
```bash
In file included from init.cpp:8:
./init.h:46:54: error: no template named 'atomic' in namespace 'std'
46 | bool AppInitBasicSetup(const ArgsManager& args, std::atomic<int>& exit_status);
| ~~~~~^
1 error generated.
```
See: https://libcxx.llvm.org/DesignDocs/HeaderRemovalPolicy.html.
Now that m_txrequest and m_recent_confirmed_transactions are guarded by
the same mutex, there is no benefit to processing them separately.
Instead, just loop through pblock->vtx once.
1bc9f64bee contrib: assume binary existence in sec/sym checks (fanquake)
51d8f435c9 contrib: simplify ELF test-security-check (fanquake)
1810e20677 contrib: simplify PE test-security-check (fanquake)
6c9746ff92 contrib: simplify MACHO test-security-check (fanquake)
Pull request description:
The current `test-security-check` script is hard to understand, and change (i.e https://github.com/bitcoin/bitcoin/pull/29987/files#diff-52aa0cda44721f089e53b128cb1232a876006ef257b211655456b17dfb2ec712); tests are also not done in isolation (when-possible). Fix that, and add missing checks. Simplifies future toolchain/security/hardening changes.
ACKs for top commit:
hebasto:
ACK 1bc9f64bee (assuming my Guix hashes match; I'll provide them shortly).
TheCharlatan:
ACK 1bc9f64bee
Tree-SHA512: 1885d0ce63a94ffa61345327f919da20b63de6dd4148d6db3ee8bad4485253a36e8ab0dbee48cecc02ea35d139edfed75453af45fc364bcbef6fe16b6823bc7a
c85accecaf [refactor] delete EraseTxNoLock, just use EraseTx (glozow)
6ff84069a5 remove obsoleted TxOrphanage::m_mutex (glozow)
61745c7451 lock m_recent_confirmed_transactions using m_tx_download_mutex (glozow)
723ea0f9a5 remove obsoleted hashRecentRejectsChainTip (glozow)
18a4355250 update recent_rejects filters on ActiveTipChange (glozow)
36f170d879 add ValidationInterface::ActiveTipChange (glozow)
3eb1307df0 guard TxRequest and rejection caches with new mutex (glozow)
Pull request description:
See #27463 for full project tracking.
This contains the first few commits of #30110, which require some thinking about thread safety in review.
- Introduce a new `m_tx_download_mutex` which guards the transaction download data structures including `m_txrequest`, the rolling bloom filters, and `m_orphanage`. Later this should become the mutex guarding `TxDownloadManager`.
- `m_txrequest` doesn't need to be guarded using `cs_main` anymore
- `m_recent_confirmed_transactions` doesn't need its own lock anymore
- `m_orphanage` doesn't need its own lock anymore
- Adds a new `ValidationInterface` event, `ActiveTipChanged`, which is a synchronous callback whenever the tip of the active chainstate changes.
- Flush `m_recent_rejects` and `m_recent_rejects_reconsiderable` on `ActiveTipChanged` just once instead of checking the tip every time `AlreadyHaveTx` is called. This should speed up calls to that function (no longer comparing a block hash each time) and removes the need to lock `cs_main` every time it is called.
Motivation:
- These data structures need synchronization. While we are holding `m_tx_download_mutex`, these should hold:
- a tx hash in `m_txrequest` is not also in `m_orphanage`
- a tx hash in `m_txrequest` is not also in `m_recent_rejects` or `m_recent_confirmed_transactions`
- In the future, orphan resolution tracking should also be synchronized. If a tx has an entry in the orphan resolution tracker, it is also in `m_orphanage`, and not in `m_txrequest`, etc.
- Currently, `cs_main` is used to e.g. sync accesses to `m_txrequest`. We should not broaden the scope of things it locks.
- Currently, we need to know the current chainstate every time we call `AlreadyHaveTx` so we can decide whether we should update it. Every call compares the current tip hash with `hashRecentRejectsChainTip`. It is more efficient to have a validation interface callback that updates the rejection filters whenever the chain tip changes.
ACKs for top commit:
instagibbs:
reACK c85accecaf
dergoegge:
Code review ACK c85accecaf
theStack:
Light code-review ACK c85accecaf
hebasto:
ACK c85accecaf, I have reviewed the code and it looks OK.
Tree-SHA512: c3bd524b5de1cafc9a10770dadb484cc479d6d4c687d80dd0f176d339fd95f73b85cb44cb3b6b464d38a52e20feda00aa2a1da5a73339e31831687e4bd0aa0c5
a517029646 depends: switch to building expat with CMake (fanquake)
Pull request description:
Switch to building Expat with CMake, instead of Autotools.
ACKs for top commit:
hebasto:
re-ACK a517029646.
Tree-SHA512: ca040545dd83fb81a8b209aa24cae6e22eaeff04f44bdabc4454adf6ea63d34f4ae27bd5980c65db2d2542e23eb2712102719023c262ab63a933c90b5999c11e
Instead of constructing a new class every time a file operation is done,
construct them once for each of the undo and block file when a new
BlockManager is created.
In future, this might make it easier to introduce an abstract block
store.
SetHex is fragile, because it accepts any non-hex input or any length of
input, without error feedback. This can lead to issues when the input is
truncated or otherwise corrupted.
Document the problem by renaming the method.
In the future, the fragile method should be removed from the public
interface.
-BEGIN VERIFY SCRIPT-
sed -i 's/SetHex/SetHexDeprecated/g' $( git grep -l SetHex ./src )
-END VERIFY SCRIPT-
09ce3501fa fix: Make TxidFromString() respect string_view length (Hodlinator)
01e314ce0a refactor: Change base_blob::SetHex() to take std::string_view (Hodlinator)
2f5577dc2e test: uint256 - Garbage suffixes and zero padding (Hodlinator)
f11f816800 refactor: Make uint256_tests no longer use deprecated BOOST_CHECK() (Hodlinator)
f0eeee2dc1 test: Add test for TxidFromString() behavior (Ryan Ofsky)
Pull request description:
### Problem
Prior to this, `TxidFromString()` was passing `string_view::data()` into `uint256S()` which meant it would only receive the a naked `char*` pointer and potentially scan past the `string_view::length()` until it found a null terminator (or some other non-hex character).
Appears to have been a fully dormant bug as callers were either passing a string literal or `std::string` directly to `TxidFromFromString()`, meaning a null terminator always existed at `pointer[length()]`. Bug existed since original merge of `TxidFromString()`.
### Solution
Make `uint256S()` (and `base_blob::SetHex()`) take and operate on `std::string_view` instead of `const char*` and have `TxidFromString()` pass that in.
(PR was prompted by comment in https://github.com/bitcoin/bitcoin/pull/30377#issuecomment-2208857200 (referring to https://github.com/bitcoin/bitcoin/pull/28922#discussion_r1404437378)).
ACKs for top commit:
maflcko:
re-ACK 09ce3501fa🕓
paplorinc:
ACK 09ce3501fa
ryanofsky:
Code review ACK 09ce3501fa. I think the current code changes are about as small as you could make to fix the bug without introducing a string copy, and the surrounding test improvements are all very nice and welcome.
Tree-SHA512: c2c10551785fb6688d1e2492ba42a8eee4c19abbe8461bb0774d56a70c23cd6b0718d2641632890bee880c06202dee148126447dd2264eaed4f5fee7e1bcb581
29eafd5733 rpc: doc: use "output script" terminology consistently in "asm"/"hex" results (Sebastian Falbesoner)
Pull request description:
The wording "public key script" was likely chosen as a human-readable form of the technical term `scriptPubKey`, but it doesn't seem to be really widespread. Replace it by the more (probably most?) common term "output script" instead. Note that the argument for the `decodescript` RPC is not necessarily an output script (it could e.g. be also a redeem script), so in this case we just stay generic and use "script".
See also the draft BIP "Terminology for Transaction Components" (https://github.com/murchandamus/bips/blob/2022-04-tx-terminology/bip-tx-terminology.mediawiki) from murchandamus which suggests to use "output script" as well.
Affects the help text of the following RPCs:
- decodepsbt
- decoderawtransaction
- decodescript
- getblock (if verbosity=3)
- getrawtransaction (if verbosity=2,3)
- gettxout
ACKs for top commit:
maflcko:
ACK 29eafd5733
achow101:
ACK 29eafd5733
BrandonOdiwuor:
ACK 29eafd5733
tdb3:
ACK 29eafd5733
Tree-SHA512: 62eb92d42bc44e36dc3090df7b248a123868a74af253d2046de02086e688bf6ff98307b927ba2fee3d599f85e073aeb8eca90ed15105ca63b648b6796cfa340b
Without the fix, the test could fail intermittently. For example:
node0 2024-07-22T16:31:54.104994Z [httpworker.0] [rpc/request.cpp:232] [parse] [rpc] ThreadRPCServer method=setmocktime user=__cookie__
test 2024-07-22T16:31:54.291000Z TestFramework (INFO): Sending first 4 bytes of ellswift which match network magic
test 2024-07-22T16:31:54.292000Z TestFramework (INFO): If a response is received, assertion failure would happen in our custom data_received() function
test 2024-07-22T16:31:54.292000Z TestFramework.p2p (DEBUG): Connecting to Bitcoin Node: 127.0.0.1:12644
test 2024-07-22T16:31:54.293000Z TestFramework.p2p (DEBUG): Connected & Listening: 127.0.0.1:12644
test 2024-07-22T16:31:54.588000Z TestFramework.p2p (DEBUG): sending 4050 bytes of garbage data
test 2024-07-22T16:31:54.588000Z TestFramework (INFO): Sending remaining ellswift and garbage which are different from V1_PREFIX. Since a response is
test 2024-07-22T16:31:54.588000Z TestFramework (INFO): expected now, our custom data_received() function wouldn't result in assertion failure
node0 2024-07-22T16:31:55.523868Z (mocktime: 2024-07-22T16:31:54Z) [net] [net.cpp:3764] [CNode] [net] Added connection peer=0
node0 2024-07-22T16:31:55.625145Z (mocktime: 2024-07-22T16:31:54Z) [net] [net.cpp:1814] [CreateNodeFromAcceptedSocket] [net] connection from 127.0.0.1:45154 accepted
node0 2024-07-22T16:31:55.625769Z (mocktime: 2024-07-22T16:31:54Z) [http] [httpserver.cpp:305] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:33320
node0 2024-07-22T16:31:55.626543Z (mocktime: 2024-07-22T16:31:54Z) [httpworker.1] [rpc/request.cpp:232] [parse] [rpc] ThreadRPCServer method=getpeerinfo user=__cookie__
test 2024-07-22T16:31:55.818000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 132, in main
self.run_test()
File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/p2p_v2_misbehaving.py", line 133, in run_test
self.test_earlykeyresponse()
File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/p2p_v2_misbehaving.py", line 151, in test_earlykeyresponse
self.wait_until(lambda: node0.getpeerinfo()[-1]["bytesrecv"] > 4)
File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 791, in wait_until
return wait_until_helper_internal(test_function, timeout=timeout, timeout_factor=self.options.timeout_factor)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/util.py", line 289, in wait_until_helper_internal
if predicate():
^^^^^^^^^^^
File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/p2p_v2_misbehaving.py", line 151, in <lambda>
self.wait_until(lambda: node0.getpeerinfo()[-1]["bytesrecv"] > 4)
~~~~~~~~~~~~~~~~~~~^^^^
IndexError: list index out of range
Otherwise, the debug log could read confusingly, when the getpeerinfo()
RPC (calling GetNodeStats) happens after the "accepted connection" log
line, but returns an empty list.
For example, the following timeline in the debug log could correspond to
a getpeerinfo reply that is empty:
[net] [net.cpp:3764] [CNode] Added connection peer=0
[net] [net.cpp:1814] [CreateNodeFromAcceptedSocket] connection from 127.0.0.1:45154 accepted
[http] [httpserver.cpp:305] [http_request_cb] Received a POST request for / from 127.0.0.1:33320
[httpworker.1] [rpc/request.cpp:232] [parse] ThreadRPCServer method=getpeerinfo user=__cookie__
Fix it by moving the log line.
d63ef73800 test: Add loadtxoutset test with tip on snapshot block (Fabian Jahr)
c2f86d4bcb test: Remove already resolved assumeutxo todo comments (Fabian Jahr)
Pull request description:
The first commit removes three Todos that have been addressed previously (see commit message for details).
The second message resolves another todo by adding the missing test case. This is a special case of "the tip has more work than the snapshot" where the tip is the same block as the snapshot base block.
Related to #28648.
ACKs for top commit:
jrakibi:
ACK [d63ef73](d63ef73800)
achow101:
ACK d63ef73800
maflcko:
ACK d63ef73800
alfonsoromanz:
Re ACK d63ef73800
Tree-SHA512: 8d5a25fc0b26531db3a9740132694138f2103b7b42eeb1d4a64095bfc901c1372e23601c0855c7def84c8a4e185d10611e4e830c4e479f1b663ae6ed53abb130
Using GCC 11 for the macOS build hasn't been required since #21778, and
at this point, given a toolchain is still needed (#30206), it makes more
sense to (re-)use 12, rather than make all builders compile another
GCC toolchain.
Prior to this, passing string_view::data() into uint256S() meant the latter would only receive the a naked char* pointer and potentially scan past the string_view::length() until it found a null terminator (or some other non-hex character).
Appears to have been a fully dormant bug as callers were either passing a string literal or std::string directly to TxidFromFromString(), meaning null terminator always existed at pointer[length()]. Bug existed since original merge of TxidFromString(), discussed in https://github.com/bitcoin/bitcoin/pull/28922#discussion_r1404437378.
6a5e9e40e1 doc: use proper doxygen formatting for CTxMemPool::cs (Vasil Dimov)
Pull request description:
Having `@par title` followed by an empty line renders improperly in Doxygen - it results in a paragraph with a title but without a body.
https://www.doxygen.nl/manual/commands.html#cmdpar
This also results in a compiler warning (or error) with Clang 19:
```
./txmempool.h:368:34: error: empty paragraph passed to '@par' command [-Werror,-Wdocumentation]
368 | * @par Consistency guarantees
| ~~~~~~~~~~~~~~~~~~~~~~~~~~^
1 error generated.
```
ACKs for top commit:
maflcko:
review ACK 6a5e9e40e1
tdb3:
ACK 6a5e9e40e1
Tree-SHA512: 2c4c9e5fd4bd44754800a9bcfff74df101afc060b84451c45aa098e4ceb05a47f28a36f8473b31222552fad6339b752a148e6b1c7d41c2003f515b3eb4060902
Having `@par title` followed by an empty line renders improperly in
Doxygen - it results in a paragraph with a title but without a body.
https://www.doxygen.nl/manual/commands.html#cmdpar
This also results in a compiler warning (or error) with Clang 19:
```
./txmempool.h:368:34: error: empty paragraph passed to '@par' command [-Werror,-Wdocumentation]
368 | * @par Consistency guarantees
| ~~~~~~~~~~~~~~~~~~~~~~~~~~^
1 error generated.
```
fa33a63bd9 fuzz: Speed up PickValue in txorphan (MarcoFalke)
Pull request description:
`PickValue` will advance a begin iterator on the `outpoints` set, which is expensive, because it only has a `++` operator. As it is called in a loop of `num_in` (~`outpoints.size()`), the runtime is `O(outpoints.size() ^ 2)`.
Fix it by making the runtime linear.
ACKs for top commit:
glozow:
ACK fa33a63bd9, thanks for taking the suggestion
dergoegge:
utACK fa33a63bd9
Tree-SHA512: 33f440d97c6834d907d43a8d29e4fb2c995f0d244460bd079af100f13d3607a53e44a0db52f4eb5c487d98df0ff4f2f6d987bf94b922ae9f4506f1295ad6214c
8c935e625e depends: Fix CMake-generated `libevent*.pc` files (Hennadii Stepanov)
Pull request description:
Broken out of #30454. This is a backport of the merged upstream PR: https://github.com/libevent/libevent/pull/1622.
Note that after #29835 we might end up dropping pkg-config and using the installed CMake files directly, but that depends on whether or not enough distros actually ship those files.
Either way, having fixed up .pc files won't hurt.
ACKs for top commit:
hebasto:
ACK 8c935e625e.
fanquake:
ACK 8c935e625e
Tree-SHA512: 259c2ad78fb9e90370a7205dc71c40acda1a872f6509435133bc1c4c2c3de57366e80679aa083e13ed85e7966883dc470c0147ee171a2ed0171a18cd5ffc99b3
fa7bee13bf lint: Use git clone --depth=1 (MarcoFalke)
fadb7c2a91 lint: Add missing docker.io prefix to ci/lint_imagefile (MarcoFalke)
Pull request description:
Currently, the `ci/lint_imagefile` may pick the wrong (non-native) architecture due to the missing prefix.
For example, assuming the user has previously pulled an s390x image:
```
$ podman run --rm 'docker.io/s390x/debian:bookworm' dpkg --print-architecture
exec /usr/bin/dpkg: exec format error
```
Now, `debian:bookworm` will refer to the same image:
```
$ podman run --rm 'debian:bookworm' dpkg --print-architecture
exec /usr/bin/dpkg: exec format error
```
However, `docker.io/debian:bookworm` works fine:
```
$ podman run --rm 'docker.io/debian:bookworm' dpkg --print-architecture
arm64
```
(Also includes a nit-fix from https://github.com/bitcoin/bitcoin/pull/30499#discussion_r1686470495)
ACKs for top commit:
paplorinc:
utACK fa7bee13bf
hebasto:
ACK fa7bee13bf.
Tree-SHA512: 4b6d562c14c67bef984ad25f6a3a1ef7f1059dc2859c603c45083b36bcacafa3248fc74176e2e4626fdc39507e9353f458ddbc4077f805c03e970df46af02224
0388ad0d65 depends: switch zmq to CMake (Cory Fields)
fefb3bbe5b depends: add zeromq no librt patch (fanquake)
a522ef1542 depends: add zeromq cmake minimum patch (fanquake)
cbbc229adf depends: add zeromq windows usage patch (fanquake)
2de68d6d38 depends: add zeromq builtin sha1 patch (fanquake)
0c8605253a depends: add zeromq mktemp macos patch (fanquake)
Pull request description:
This picks up a change, which is a switch to building zeromq with CMake. It includes a number of patches, some which have already been upstreamed (see each patch for details).
ACKs for top commit:
hebasto:
ACK 0388ad0d65.
Tree-SHA512: 5567e432b4e4e0446c41d502bd61810a80b329dea2399b5d9d9f6e79acc450d1c6ba861c8238ba895de98338cfc5dc44ad2bf86ee8c222ecb3fbf47d6eb60da4
fa8d73e86e lint: Use consistent out-of-tree build for python and test_runner (MarcoFalke)
fa0f859885 doc: Clarify intent of ./ci/lint_run_all.sh (MarcoFalke)
fa9ad59f87 lint: Use $CI_RETRY_EXE when building ./ci/lint_imagefile (MarcoFalke)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/30496
Seems odd to sometimes do an out-of-tree build (via `./ci/lint_imagefile`, see `test/lint/README.md`) and sometimes not (via Cirrus CI, see `./ci/lint_run_all.sh`).
Fix it by doing an out-of-tree build consistently in the same location.
Also, fix `$CI_RETRY_EXE`, while touching this.
ACKs for top commit:
josibake:
utACK fa8d73e86e
willcl-ark:
utACK fa8d73e86e
paplorinc:
utACK fa8d73e86e
Tree-SHA512: 4181ca14299a798850f5e05f180f3305a3378081ca8dabf6ab2da6115997cc17f6ef0f10db9b2b31618e59231083e5c4a971432d27b4d77903e655be21155abb
Previous code was confusing and brittle. For example, the full import
"source ./ci/test/00_setup_env.sh" and $PATH overwrite was not needed.
Fix it by simply copying the exe to /ci_retry and use that in
$CI_RETRY_EXE.
This is also a fix, because previously ci/lint_imagefile did use an
empty $CI_RETRY_EXE.
a8e3af1a82 qa: Do not assume running `feature_asmap.py` from source directory (Hennadii Stepanov)
9bf7ca6cad qa: Consider `cache` and `config.ini` relative to invocation directory (Hennadii Stepanov)
a0473442d1 scripted-diff: Add `__file__` argument to `BitcoinTestFramework.init()` (Hennadii Stepanov)
Pull request description:
This PR includes changes split from https://github.com/bitcoin/bitcoin/pull/30454. They improve the functional test framework, allowing users to [run individual functional tests](https://github.com/hebasto/bitcoin/issues/146) from the build directory in the new CMake-based build system.
This functionality is not available for out-of-source builds using the current Autotools-based build system, which always requires write permissions for the source directory. Nevertheless, this PR can be tested as suggested in https://github.com/bitcoin/bitcoin/pull/30463#issuecomment-2232618421:
1. Make an out-of-source build:
```
$ ./autogen.sh
$ mkdir ../build && cd ../build
$ ../bitcoin/configure
$ make
```
2. Create a symlink in the build directory to a functional test:
```
$ ln --symbolic ../../../bitcoin/test/functional/wallet_disable.py ./test/functional/
```
3. Run this symlink:
```
$ ./test/functional/wallet_disable.py
```
The last command fails on the master branch:
```
Traceback (most recent call last):
File "/home/hebasto/git/build/./test/functional/wallet_disable.py", line 31, in <module>
DisableWalletTest().main()
^^^^^^^^^^^^^^^^^^^
File "/home/hebasto/git/bitcoin/test/functional/test_framework/test_framework.py", line 106, in __init__
self.parse_args()
File "/home/hebasto/git/bitcoin/test/functional/test_framework/test_framework.py", line 210, in parse_args
config.read_file(open(self.options.configfile))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/hebasto/git/bitcoin/test/config.ini'
```
and succeeds with this PR.
ACKs for top commit:
maflcko:
tested ACK a8e3af1a82🎨
glozow:
ACK a8e3af1a82, tested with the steps in op
stickies-v:
ACK a8e3af1a82
Tree-SHA512: 899e4efc09edec13ea3f5b47825d03173fb21d3569c360deda7fa6a56b99b4d24e09ad4f0883bad1ee926b1c706e47ba07c6a6160c63c07c82b3cf4ae5816e91
7703884ab1 Fix MSVC warning C4273 "inconsistent dll linkage" (Hennadii Stepanov)
Pull request description:
Broken out of https://github.com/bitcoin/bitcoin/pull/30454.
When using CMake, the user can select the MSVC runtime library to be:
1) Statically-linked (with the corresponding `x64-windows-static` vcpkg triplet) or
2) Dynamically-linked (with the corresponding `x64-windows` vcpkg triplet)
In the latter case, the compiler emits the [C4273](https://learn.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-1-c4273) warning.
As the "Necessary on some platforms" comment does not apply to MSVC, skip the declaration for MSVC.
The MSVC build system in the master branch supports the statically-linked runtime only: ed739d14b5/build_msvc/common.init.vcxproj.in (L65)
ACKs for top commit:
sipa:
utACK 7703884ab1
sipsorcery:
utACK 7703884ab1.
theuni:
utACK 7703884ab1
Tree-SHA512: a42e1a0d48973217462e703c418f3e9ef9cb5236267c1bf32912aacaf68976cdd2b9229168523f7c2a99ee3f2fb1bf8add4f342796bdb1e4063ca026b761db51
d318c4ef56 depends: bump libmultiprocess for CMake fixes (Cory Fields)
Pull request description:
Broken out of #30454 . Bumped [even further](4883197abc (r1684802528)) after https://github.com/chaincodelabs/libmultiprocess/pull/98 was merged upstream.
hebasto Presumably this approach works now with the CMake branch?
ACKs for top commit:
ryanofsky:
Code review ACK d318c4ef56.
Tree-SHA512: 4b5491f73c0063d09e8339829cd831b1f4c441dd7b55a22037c9337c80cce19bb00a3e5cf925efa77d6d4e89ab45482f40f5799bc14948f8cabcbad3c3549430
When using CMake, the user can select the MSVC runtime library to be:
1) Statically-linked (with the corresponding `x64-windows-static` vcpkg
triplet) or
2) Dynamically-linked (with the corresponding `x64-windows` vcpkg
triplet)
In the latter case, the compiler emits the C4273 warning.
As the "Necessary on some platforms" comment does not apply to MSVC,
skip the declaration for MSVC.
The body of the fuzz test should ideally be a pure function. If data is
persisted in the cache over many iterations, and there is a crash,
reproducing it from the input might be difficult.
The CMake WIN32_WINNT autodetection is broken, and must be set
manually. We may want to set is explicitly in any case, but the
brokenness should also be fixed upstream.
Also patch out depends paths, that would cause non-determinism.
Co-authored-by: fanquake <fanquake@gmail.com>
Adds more testing options for creating an environment without networking
and a validation interface. This is useful for improving the performance
of the utxo snapshot fuzz test, which constructs a new TestingSetup on
each iteration.
ff4f3deb7b depends: use CMake to build FreeType (fanquake)
Pull request description:
Switches Freetype to be built with CMake.
ACKs for top commit:
theuni:
ACK ff4f3deb7b
hebasto:
ACK ff4f3deb7b, I've verified the actual compile options, they look sane.
Tree-SHA512: e9e4348975998539fde88a84d110d53dbac50ae9cc3fa692d15e09313d6fdb6acb3bb23533786a645fc836091075b4487d6de42ef78ba3a44de46d06360aef4f
The formatting of log messages isn't defined until StartLogging() is
called; so can't be correctly applied to early log messages from prior
to that call. Instead of saving the output log message, save the inputs
to the logging invocation (including time, mocktime and thread name),
and format those inputs into a log message when StartLogging() is called.
Also pulls out the guarding assert and calls it explicitly before the test function is called. This is already done before the existing call of the test function so it was not needed there.
- "Valid snapshot file, but referencing a snapshot block that turns out
to be invalid, or has an invalid parent" has been addressed in #30267
- "An ancestor of snapshot block" - If chain tip refers to blocks in this context then any successful load is addressing this because if we have synced past the snapshot base block we fail because we don't need assumeutxo anymore. And if this is about headers then this is the `test_headers_not_synced()` case.
- "A descendant of the snapshot block" - If this refers to blocks the
`test_snapshot_with_less_work()` addressed this and if it is just headers in this case again it would be represented in all of the successful loads in the test.
Co-authored-by: Alfonso Roman Zubeldia <alfonsoromanz24@gmail.com>
23333b7ed2 net: Allow DNS lookups on nodes with IPV6 lo only (Max Edwards)
Pull request description:
This is similar to (but does not fix) https://github.com/bitcoin/bitcoin/issues/13155 which I believe is the same issue but in libevent.
The issue is on a host that has IPV6 enabled but only a loopback IP address `-proxy=[::1]` will fail as `[::1]` is not considered valid by `getaddrinfo` with `AI_ADDRCONFIG` flag. I think the loopback interface should be considered valid and we have a functional test that will try to test this: `feature_proxy.py`.
To replicate the issue, run `feature_proxy.py` inside a docker container that has IPV6 loopback ::1 address without specifically giving that container an external IPV6 address. This should be the default with recent versions of docker. IPV6 on loopback interface was enabled in docker engine 26 and later ([https://docs.docker.com/engine/release-notes/26.0/#bug-fixes-and-enhancements-2](https://docs.docker.com/engine/release-notes/26.0/#bug-fixes-and-enhancements-2)).
`AI_ADDRCONFIG` was introduced to prevent slow DNS lookups on systems that were IPV4 only.
References:
Man section on `AI_ADDRCONFIG`:
```
If hints.ai_flags includes the AI_ADDRCONFIG flag, then IPv4 addresses are returned in the list pointed to by res only if the local system has at least one IPv4 address configured, and IPv6 addresses
are returned only if the local system has at least one IPv6 address configured. The loopback address is not considered for this case as valid as a configured address. This flag is useful on, for ex‐
ample, IPv4-only systems, to ensure that getaddrinfo() does not return IPv6 socket addresses that would always fail in connect(2) or bind(2).
```
[AI_ADDRCONFIG considered harmful Wiki entry by Fedora](https://fedoraproject.org/wiki/QA/Networking/NameResolution/ADDRCONFIG)
[Mozilla discussing slow DNS without AI_ADDRCONFIG and also localhost issues with it](https://bugzilla.mozilla.org/show_bug.cgi?id=467497)
ACKs for top commit:
achow101:
ACK 23333b7ed2
tdb3:
ACK 23333b7ed2
pinheadmz:
ACK 23333b7ed2
Tree-SHA512: 5ecd8c72d1e1c28e3ebff07346381d74eaddef98dca830f6d3dbf098380562fa68847d053c0d84cc8ed19a45148ceb5fb244e4820cf63dccb10ab3db53175020
55b6d7be68 validation: Don't load a snapshot if it's not in the best header chain. (Martin Zumsande)
Pull request description:
This was suggested by me in the discussion of #30288, which has more context.
If the snapshot is not an ancestor of the most-work header (`m_best_header`), syncing from that alternative chain leading to `m_best_header` should be prioritised. Therefore it's not useful loading the snapshot in this situation.
If the other chain turns out to be invalid or the chain with the snapshot retrieves additional headers so that it's the most-work one again (see functional test), `m_best_header` will change and loading the snapshot will be possible again.
Because of the work required to generate a conflicting headers chain, a situation with two conflicting chains should only be possible under extreme circumstances, such as major forks.
ACKs for top commit:
fjahr:
re-ACK 55b6d7be68
achow101:
ACK 55b6d7be68
alfonsoromanz:
Re ACK 55b6d7be68
Tree-SHA512: 4fbea5ab1038ae353fc949a186041cf9b397e7ce4ac59ff36f881c9437b4f22ada922490ead5b2661389eb1ca0f3d1e7e7e6a4261057678643e71594a691ac36
fac932bf93 refactor: Use util::Split to avoid a harmless unsigned-integer-overflow (MarcoFalke)
fab54db9f1 rest: Reject negative outpoint index in getutxos parsing (MarcoFalke)
Pull request description:
In `rest_getutxos` outpoint indexes such as `+N` or `-N` are accepted. This should be harmless, because any index out of range should be treated as a non-existent utxo. However, a negative index can't exist ever, so it seems better to reject all signs, whether `+` or `-`.
ACKs for top commit:
achow101:
ACK fac932bf93
hodlinator:
ut-ACK fac932bf93
tdb3:
re ACK fac932bf93
danielabrozzoni:
ACK fac932bf93
brunoerg:
reACK fac932bf93
Tree-SHA512: 8f1a75248cb61e1c4beceded6ed170db83b07f30fbcf93a26acfffc00ec4546572366eff87907a7e1423d7d3a2a9e57a0a7a9bacb787c86463f842d7161c16bc
faed5d3870 test: Non-Shy version sender (MarcoFalke)
Pull request description:
After `add_outbound_p2p_connection`, the test framework normally sends a version message only in reply to a received version. This is fine, but the protocol does not require this and tolerates a version to be sent earlier.
However, this is untested, and the missing test coverage leads to bugs being missed. For example https://github.com/bitcoin/bitcoin/pull/30394#pullrequestreview-2166824948
Fix it by adding a test.
ACKs for top commit:
brunoerg:
ACK faed5d3870
tdb3:
ACK faed5d3870
theStack:
tACK faed5d3870
glozow:
ACK faed5d3870
Tree-SHA512: dbf527a39c932e994a1e8248ba78058000811a4bf69275278f1fd1e545716ac4d2d3be5dcf362976bbafa2a49f91d13e3601daf71d29e9c556179b01af62c03c
c504b6997b refactor: add coinbase constraints to BlockCreateOptions (Sjors Provoost)
6b4c817d4b refactor: pass BlockCreateOptions to createNewBlock (Sjors Provoost)
323cfed595 refactor: use CHECK_NONFATAL to avoid single-use symbol (Sjors Provoost)
Pull request description:
When generating a block template through e.g. getblocktemplate RPC, we reserve 4000 weight units and 400 sigops. Pools use this space for their coinbase outputs.
At least one pool patched their Bitcoin Core node to adjust these hardcoded values. They eventually [produced an invalid block](https://bitcoin.stackexchange.com/questions/117837/how-many-sigops-are-in-the-invalid-block-783426) which exceeded the sigops limit.
The existince of such patches suggests it may be useful to make this value configurable. This PR would make such a change easier. However, the main motivation is that in the Stratum v2 spec requires the pool to communicate the maximum bytes they intend
to add to the coinbase outputs.
Specifically the `CoinbaseOutputDataSize` message which is part of the [Template Distribution Protocol](https://github.com/stratum-mining/sv2-spec/blob/main/07-Template-Distribution-Protocol.md#71-coinbaseoutputdatasize-client---server) has a field `coinbase_output_max_additional_size`.
A proposed change to the spec adds the max additional sigops as well: https://github.com/stratum-mining/sv2-spec/pull/86. Whether that change makes it into the spec is not important though, as adding both to `BlockAssembler::Options` makes sense.
The first commit is a test refactor followup for #30335, related to the code that's changed here, but not required.
The second commit introduces BlockCreateOptions, with just `use_mempool`.
The thirds commit adds `coinbase_max_additional_weight` and `coinbase_output_max_additional_sigops` to `BlockCreateOptions`. They use the originally hardcoded values, and no existing caller overrides these defaults. This changes in #29432.
ACKs for top commit:
itornaza:
tested ACK c504b6997b
ryanofsky:
Code review ACK c504b6997b
ismaelsadeeq:
Code review ACK c504b6997b
Tree-SHA512: de2fa085f47048c91d95524e03f909f6f27f175c1fefa3d6106445e7eb5cf5b710eda6ea5b641cf3b4704a4e4e0181a0c829003b9fd35465f2a46167e5d64487
If the binaries don't exist, the Guix build has failed for some other
reason.
There's no need to check for unknown architectures, or executable
formats, as the only ones that could be built are those that we've
configured toolchains for in Guix.
We've also been doing this inconsistently across the two scripts.
44f08786f4 test: Fix MSVC warning C4101 "unreferenced local variable" (Hennadii Stepanov)
5d25a82b9a univalue, refactor: Convert indentation tabs to spaces (Hennadii Stepanov)
Pull request description:
This PR is split from https://github.com/bitcoin/bitcoin/pull/30454 and addresses MSVC warning [C4101](https://learn.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-3-c4101) "unreferenced local variable". The current MSVC build system in the master branch skips building univalue tests, so it is not affected.
No behaviour changes.
ACKs for top commit:
kevkevinpal:
utACK [44f0878](44f08786f4)
maflcko:
ACK 44f08786f4
theuni:
trivial ACK 44f08786f4.
Tree-SHA512: 661d3b40ddb4f7915de7a65ccb27a24da88ae499ce03c036099007260b0597e83738f1a3a420985b51f798ee309ade32988c6d78f4ffed401099b175a0b2025b
734076c6de [wallet, rpc]: add `max_tx_weight` to tx funding options (ismaelsadeeq)
b6fc5043c1 [wallet]: update the data type of `change_output_size`, `change_spend_size` and `tx_noinputs_size` to `int` (ismaelsadeeq)
baab0d2d43 [doc]: update reason for deducting change output weight (ismaelsadeeq)
7f61d31a5c [refactor]: update coin selection algorithms input parameter `max_weight` name (ismaelsadeeq)
Pull request description:
This PR taken over from #29264
The PR added an option `max_tx_weight` to transaction funding RPC's that ensures the resulting transaction weight does not exceed the specified `max_tx_weight` limit.
If `max_tx_weight` is not given `MAX_STANDARD_TX_WEIGHT` is used as the max threshold.
This PR addressed outstanding review comments in #29264
For more context and rationale behind this PR see https://delvingbitcoin.org/t/lightning-transactions-with-v3-and-ephemeral-anchors/418/11?u=instagibbs
ACKs for top commit:
achow101:
ACK 734076c6de
furszy:
utACK 734076c6de
rkrux:
reACK [734076c](734076c6de)
Tree-SHA512: 013501aa443d239ee2ac01bccfc5296490c27b4edebe5cfca6b96c842375e895e5cfeb5424e82e359be581460f8be92095855763a62779a18ccd5bdfdd7ddce7
When generating a block template through e.g. getblocktemplate RPC,
we reserve 4000 weight units and 400 sigops. Pools use this space
for their coinbase outputs.
At least one pool patched their Bitcoin Core node to adjust
these hardcoded values. They eventually produced an invalid
block which exceeded the sigops limit.
https://bitcoin.stackexchange.com/questions/117837/how-many-sigops-are-in-the-invalid-block-783426
The existince of such patches suggests it may be useful to
make this value configurable. This commit would make such a
change easier.
The main motivation however is that the Stratum v2 spec
requires the pool to communicate the maximum bytes they intend
to add to the coinbase outputs. A proposed change to the spec
would also require them to communicate the maximum number of sigops.
This commit also documents what happens when
-blockmaxweight is lower than the coinbase
reserved value.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
d440f13db0 crypto: Guard code with `ENABLE_SSE41` macro (Hennadii Stepanov)
6ec1ca7c85 build: Fix test for SSE4.1 intrinsics (Hennadii Stepanov)
Pull request description:
1. Fix the test for SSE4.1 intrinsics during build system configuration, which currently can be false positive, for example, when `CXXFLAGS="-mno-sse4.1"` provided.
This PR fixes the test by adding the `_mm_blend_epi16` SSE4.1 function used in our codebase.
2. Guard `sha_x86_shani.cpp` code with `ENABLE_SSE41` macro as it uses the `_mm_blend_epi16` function from
the SSE4.1 instruction set.
It is possible that SHA-NI is enabled even when SSE4.1 is disabled, which causes compile errors in the master branch.
Closes https://github.com/bitcoin/bitcoin/issues/28864.
ACKs for top commit:
sipa:
utACK d440f13db0
willcl-ark:
tACK d440f13db0
theuni:
utACK d440f13db0
Tree-SHA512: a6e1e8c94e1b94874ff51846815ef445e6135cbdb01b08eb695b3548115f2340dd835ebe53673ae46a553fe6be4815e68d8642c34235dd7af5106c4b7c9ea6f3
c322bddd08 test: bump mocktime after node has received and sent bytes (stratospher)
Pull request description:
Fixes an intermittent failure for `p2p_v2_misbehaving.py` reported in https://github.com/bitcoin/bitcoin/pull/30420#discussion_r1680462164.
A [different error message](262260ce1e/src/net.cpp (L1970)) `"socket no message in first %i seconds"` will be displayed if `m_last_send=0` or if `m_last_recv is 0`. Fix this by:
1. mocktime bump is done after all the bytes are received. (`m_last_recv is not 0 now`)
2. wait until bytes are sent by `TestNode`/`bitcoind` (`m_last_send is not 0 now`)
See https://cirrus-ci.com/task/5359619151757312?logs=ci#L3935 for an example failure (I wasn't able to reproduce the intermittent failure locally but I think the fix is logical)
ACKs for top commit:
maflcko:
reACK c322bddd08
Tree-SHA512: 1c05524c2819041eb2001c2baf2c912d4f812a39347f784f212634e8c53131357a73116a46b4b7542bc7fc8c1370c4d36fc9898a2cbdb40bcee61105123c4a35
fa927055dd refactor: Make m_last_notified_header private (MarcoFalke)
Pull request description:
Seems brittle to expose mutable fields public.
Fix it by making it private.
Fixes https://github.com/bitcoin/bitcoin/pull/30425#discussion_r1677633601
ACKs for top commit:
dergoegge:
utACK fa927055dd
Tree-SHA512: d9841c42571144ced0edeaa4bb1d96a177a011dca37c8342c66513477c37278602a1b88beb93068b94fc4443b1552c8fc9f98bcf0bda7d0fc101e61e90c33944
fa6390df20 doc: getaddressinfo[isscript] is optional (MarcoFalke)
Pull request description:
`isscript` is unknown for unknown witness versions, so it should be marked optional in the docs
Fixes https://github.com/bitcoin/bitcoin/issues/30456
ACKs for top commit:
stickies-v:
ACK fa6390df20
tdb3:
ACK fa6390df20
Tree-SHA512: f728f18e0871923225e0bf29594f8095997456cf55409f42087b5f70f95bef10f984323b48d2b484b6705f23b04e9e8a3fe42446830638fdd70453c18fd7f189
a different error message "socket no message in first %i seconds"
will be displayed if m_last_send=0 or if m_last_recv is 0. make
the test robust by ensuring that they will not be 0 before
bumping mocktime.
The previous commit added a test which would fail the
unsigned-integer-overflow sanitizer. The warning is harmless and can be
triggered on any commit, since the code was introduced.
For reference, the warning would happen when the separator `-` was not
present.
For example:
GET /rest/getutxos/6a297bfa5cb8dd976ab0207a767d6cbfaa5e876f30081127ec8674c8c52b16c0_+1.json
would result in:
rest.cpp:792:77: runtime error: unsigned integer overflow: 18446744073709551615 + 1 cannot be represented in type 'size_type' (aka 'unsigned long')
#0 0x55ad42c16931 in rest_getutxos(std::any const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) src/rest.cpp:792:77
#1 0x55ad4319e3c0 in std::function<bool (HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&)>::operator()(HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) const /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9
#2 0x55ad4319e3c0 in HTTPWorkItem::operator()() src/httpserver.cpp:59:9
#3 0x55ad431a3eea in WorkQueue<HTTPClosure>::Run() src/httpserver.cpp:114:13
#4 0x55ad4318f961 in HTTPWorkQueueRun(WorkQueue<HTTPClosure>*, int) src/httpserver.cpp:403:12
#5 0x7f078ebcbbb3 (/lib/x86_64-linux-gnu/libstdc++.so.6+0xeabb3) (BuildId: 40b9b0d17fdeebfb57331304da2b7f85e1396ef2)
#6 0x55ad4277e01c in asan_thread_start(void*) asan_interceptors.cpp.o
#7 0x7f078e840a93 (/lib/x86_64-linux-gnu/libc.so.6+0x9ca93) (BuildId: 08134323d00289185684a4cd177d202f39c2a5f3)
#8 0x7f078e8cdc3b (/lib/x86_64-linux-gnu/libc.so.6+0x129c3b) (BuildId: 08134323d00289185684a4cd177d202f39c2a5f3)
SUMMARY: UndefinedBehaviorSanitizer: unsigned-integer-overflow rest.cpp:792:77
7e36dca657 test: add test for modififed walletprocesspsbt calls (willcl-ark)
39cea21ec5 wallet: fix FillPSBT errantly showing as complete (willcl-ark)
Pull request description:
Fixes: #30077
Fix cases of calls to `FillPSBT` returning `complete=true` when it's not
the case.
This can happen when some inputs have been signed but the transaction is
subsequently modified, e.g. in the context of PayJoins.
Also fixes a related bug where a finalized hex string is attempted to be
added during `walletprocesspsbt` but a CHECK_NONFATAL causes an abort.
ACKs for top commit:
achow101:
ACK 7e36dca657
ismaelsadeeq:
Tested ACK 7e36dca657
pinheadmz:
re-ACK 7e36dca657
Tree-SHA512: e35d19789899c543866d86d513506494d672e4bed9aa36a995dbec4e72f0a8ec5536b57c4a940a18002ae4a8efd0b007c77ba64e57cd52af98e4ac0e7bf650d6
In CMake-based build system (1) `config.ini` is created in the build
directory, and (2) `cache` must also be created in the same directory.
This change enables running individual functional tests from the build
directory.
bca346a970 net: require P2P binds to succeed (Vasil Dimov)
af552534ab net: report an error if unable to bind on the Tor listening addr:port (Vasil Dimov)
9a7e5f4d68 net: don't extra bind for Tor if binds are restricted (Vasil Dimov)
Pull request description:
Make it possible to disable the Tor binding on `127.0.0.1:8334` and stop startup if any P2P bind fails instead of "if all P2P binds fail".
Fixes https://github.com/bitcoin/bitcoin/issues/22726
Fixes https://github.com/bitcoin/bitcoin/issues/22727
ACKs for top commit:
achow101:
ACK bca346a970
cbergqvist:
ACK bca346a970
pinheadmz:
ACK bca346a970
Tree-SHA512: fabef89a957191eea4f3e3b6109d2b8389f27ecc74440a920b0c10f31fff00a85bcfd1eb3c91826c7169c618f4de8a8d0a260e2caf40fd854f07ea9a980d8603
fe92c15f0c script/sign: avoid duplicated signature verification after signing (Sebastian Falbesoner)
080089567c bench: add benchmark for `SignTransaction` (Sebastian Falbesoner)
Pull request description:
This PR is a small performance improvement on the `SignTransaction` function, which is used mostly by the wallet (obviously) and the `signrawtransactionwithkey` RPC. The lower-level function `ProduceSignature` already calls `VerifyScript` internally as last step in order to check whether the signature data is complete:
daa56f7f66/src/script/sign.cpp (L568-L570)
If and only if that is the case, the `complete` field of the `SignatureData` is set to `true` accordingly and there is no need then to verify the script after again, as we already know that it would succeed.
This leads to a rough ~20% speed-up for `SignTransaction` for single-input ECDSA or Taproot transactions, according to the newly introduced `SignTransaction{ECDSA,Taproot}` benchmarks:
```
$ ./src/bench/bench_bitcoin --filter=SignTransaction.*
```
without commit 18185f4f578b8795fdaa75926630a691e9c8d0d4:
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 185,597.79 | 5,388.00 | 1.6% | 0.22 | `SignTransactionECDSA`
| 141,323.95 | 7,075.94 | 2.1% | 0.17 | `SignTransactionSchnorr`
with commit 18185f4f578b8795fdaa75926630a691e9c8d0d4:
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 149,757.86 | 6,677.45 | 1.4% | 0.18 | `SignTransactionECDSA`
| 108,284.40 | 9,234.94 | 2.0% | 0.13 | `SignTransactionSchnorr`
Note that there are already signing benchmarks in the secp256k1 library, but `SignTransaction` does much more than just the cryptographical parts, i.e.:
* calculate the unsigned tx's `PrecomputedTransactionData` if necessary
* apply Solver on the prevout scriptPubKey, fetch the relevant keys from the signing provider
* perform the actual signing operation (for ECDSA signatures, that could be more than once due to low-R grinding)
* verify if the signatures are correct by calling `VerifyScript` (more than once currently, which is fixed by this PR)
so it probably makes sense to also have benchmarks from that higher-level application perspective.
ACKs for top commit:
achow101:
ACK fe92c15f0c
furszy:
utACK fe92c15f0c
glozow:
light review ACK fe92c15f0c
Tree-SHA512: b7225ff9e8a640ca5222dea5b2a463a0f9b9de704e4330b5b9a7bce2d63a1f4620575c474a8186f4708d7d9534eab55d000393d99db79c0cfc046b35f0a4a778
fa6270737e rpc: Use CHECK_NONFATAL over Assert (MarcoFalke)
Pull request description:
Any RPC method should not abort the whole node when an internal logic error happens.
Fix it by just aborting this single RPC method call when an error happens.
Also, fix the linter to find the fixed cases.
ACKs for top commit:
achow101:
ACK fa6270737e
stickies-v:
ACK fa6270737e
tdb3:
ACK fa6270737e
hodlinator:
ACK fa6270737e
Tree-SHA512: dad2f31b01a66578949009499e4385fb4d72f0f897419f2a6e0ea02e799b9a31e6ecb5a67fa5d27fcbc7939fe8acd62dc04e877b35831493b7f2c604dec7dc64
5fd4836019 init: change shutdown order of load block thread and scheduler (Martin Zumsande)
Pull request description:
This avoids situations during a reindex, in which the shutdown doesn't finish since `LimitValidationInterfaceQueue()` is called by the load block thread when the scheduler is already stopped, in which case it would block indefinitely. This can lead to intermittent failures in `feature_reindex.py` (#30424), which I could locally reproduce with
```diff
diff --git a/src/validation.cpp b/src/validation.cpp
index 74f0e4975c..be1706fdaf 100644
--- a/src/validation.cpp
+++ b/src/validation.cpp
@@ -3446,6 +3446,7 @@ static void LimitValidationInterfaceQueue(ValidationSignals& signals) LOCKS_EXCL
AssertLockNotHeld(cs_main);
if (signals.CallbacksPending() > 10) {
+ std::this_thread::sleep_for(std::chrono::milliseconds(50));
signals.SyncWithValidationInterfaceQueue();
}
}
```
It has also been reported by users running `reindex-chainstate` (#23234).
I thought for a bit about potential downsides of changing this order, but couldn't find any.
Fixes#30424Fixes#23234
ACKs for top commit:
maflcko:
review ACK 5fd4836019
hebasto:
re-ACK 5fd4836019.
tdb3:
ACK 5fd4836019
BrandonOdiwuor:
Code Review ACK 5fd4836019
Tree-SHA512: 3b8894e99551c5d4392b55eaa718eee05841a7287aeef2978699e1d633d5234399fa2f5a3e71eac1508d97845906bd33e0e63e5351855139e7be04c421359b36
81d4dc8e87 build: use -no_exported_symbols on macOS (fanquake)
Pull request description:
This reduces the size of the binary by ~1% when building with `--enable-reduce-exports`.
> -no_exported_symbols
> Useful for main executable that don't have plugins and thus need no symbol exports.
Can be tested with `dyld_info -exports src/bitcoind`. The only exported symbol should be `__mh_execute_header`.
ACKs for top commit:
theuni:
utACK 81d4dc8e87
hebasto:
ACK 81d4dc8e87.
Tree-SHA512: ae46065a05d190753ba807943c0734a06cfe6d2cf9eaf3c3aa93250bf8639da8bc53b81c6b0390e6d572a74c6bb31a695f8c5924810bfa358a3c9b08caff03f7
51fa26239a refactor: Mark some static global vars as const (TheCharlatan)
39f9b80fba refactor: De-globalize last notified header index (TheCharlatan)
3443943f86 refactor: De-globalize validation benchmark timekeeping (TheCharlatan)
Pull request description:
In future, users of the kernel library might run multiple chainstates in parallel, or create and destroy multiple chainstates over the lifetime of a process. Having static, mutable variables could lead to state inconsistencies in these scenarios.
---
This pull request is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).
ACKs for top commit:
dergoegge:
Code review ACK 51fa26239a
maflcko:
ACK 51fa26239a🍚
tdb3:
code review ACK 51fa26239a
Tree-SHA512: da91aa7ffa343325cabb8764ef03c8358845662cf0ba8a6cc1dd38e40e5462d88734f2b459c2de8e7a041551eda9143d92487842609f7f30636f61a0cd3c57ee
8607773750 Add fuzz test for FSChaCha20Poly1305 (stratospher)
c807f33228 Add fuzz test for AEADChacha20Poly1305 (stratospher)
Pull request description:
This PR adds fuzz tests for `AEADChaCha20Poly1305` and `FSChaCha20Poly1305` introduced in #28008.
Run using:
```
$ FUZZ=crypto_aeadchacha20poly1305 src/test/fuzz/fuzz
$ FUZZ=crypto_fschacha20poly1305 src/test/fuzz/fuzz
```
ACKs for top commit:
dergoegge:
tACK 8607773750
marcofleon:
Tested ACK 8607773750. Ran both targets for ~200 CPU hours. Coverage of intended targets looks good to me. The simulation of damaged keys and checks that follow seem useful as well.
Tree-SHA512: b6b85661d896e653caeed330f941fde665fc2bbd97ecd340808a3f365c469fe9134aa77316569a771dc36d1158cac1a5f76700bcfc45fff12aef07562e48feb9
Resetting m_recent_rejects once per block is more efficient than
comparing hashRecentRejectsChainTip with the chain tip every time we
call AlreadyHaveTx. We keep hashRecentRejectsChainTip for now to assert
that updates happen correctly; it is removed in the next commit.
This is a synchronous callback notifying clients of all tip changes.
It allows clients to respond to a new block immediately after it is
connected. The synchronicity is important for things like
m_recent_rejects, in which a transaction's validity can change (rejected
vs accepted) when this event is processed. For example, the transaction
might have a timelock condition that has just been met. This is distinct
from something like m_recent_confirmed_transactions, in which the
validation outcome is the same (valid vs already-have), so it does not
need to be reset immediately.
We need to synchronize between various tx download structures.
TxRequest does not inherently need cs_main for synchronization, and it's
not appropriate to lock all of the tx download logic under cs_main.
9010b1343b contrib: c++ify test stubs after switching to c++ compilers (Cory Fields)
261f770333 contrib: rename cc to cxx in binary checking scripts (Cory Fields)
a38c960005 contrib: use c++ rather than c for binary tests (Cory Fields)
Pull request description:
From hebasto's CMake repo. See discussion here: https://github.com/hebasto/bitcoin/pull/252#discussion_r1664657488
Use CXX/CXXFLAGS rather than CC/CFLAGS to test our actual compiler for binary checks rather than the one we only forward to secp256k1.
ACKs for top commit:
hebasto:
ACK 9010b1343b.
fanquake:
ACK 9010b1343b
Tree-SHA512: 7b8788d7d3760103062eff10056c995e1ad14c0c487d9414683ad54d816c255d0ca751f4d0e2d2ad7f9e8a7101d8c7f1e9333fa5b137558ed68fa593c4b4ce6d
16bd283b3a Reapply "test: p2p: check that connecting to ourself leads to disconnect" (Sebastian Falbesoner)
0dbcd4c148 net: prevent sending messages in `NetEventsInterface::InitializeNode` (Sebastian Falbesoner)
66673f1c13 net: fix race condition in self-connect detection (Sebastian Falbesoner)
Pull request description:
This PR fixes a recently discovered race condition in the self-connect detection (see #30362 and #30368).
Initiating an outbound network connection currently involves the following steps after the socket connection is established (see [`CConnman::OpenNetworkConnection`](bd5d1688b4/src/net.cpp (L2923-L2930)) method):
1. set up node state
2. queue VERSION message (both steps 1 and 2 happen in [`InitializeNode`](bd5d1688b4/src/net_processing.cpp (L1662-L1683)))
3. add new node to vector `m_nodes`
If we connect to ourself, it can happen that the sent VERSION message (step 2) is received and processed locally *before* the node object is added to the connection manager's `m_nodes` vector (step 3). In this case, the self-connect remains undiscovered, as the detection doesn't find the outbound peer in `m_nodes` yet (see `CConnman::CheckIncomingNonce`).
Fix this by swapping the order of 2. and 3., by taking the `PushNodeVersion` call out of `InitializeNode` and doing that in the `SendMessages` method instead, which is only called for `CNode` instances in `m_nodes`.
The temporarily reverted test introduced in #30362 is readded. Fixes#30368.
Thanks go to vasild, mzumsande and dergoegge for suggestions on how to fix this (see https://github.com/bitcoin/bitcoin/issues/30368#issuecomment-2200625017 ff. and https://github.com/bitcoin/bitcoin/pull/30394#discussion_r1668290789).
ACKs for top commit:
naiyoma:
tested ACK [16bd283b3a), built and tested locally, test passes successfully.
mzumsande:
ACK 16bd283b3a
tdb3:
ACK 16bd283b3a
glozow:
ACK 16bd283b3a
dergoegge:
ACK 16bd283b3a
Tree-SHA512: 5b8aced6cda8deb38d4cd3fe4980b8af505d37ffa0925afaa734c5d81efe9d490dc48a42e1d0d45dd2961c0e1172a3d5b6582ae9a2d642f2592a17fbdc184445
Rather than pass options individually to createNewBlock and then
combining them into BlockAssembler::Options, this commit introduces
BlockCreateOptions and passes that instead.
Currently there's only one option (use_mempool) but the next
commit adds more.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
c6d43367a1 test: Fix intermittent failure in p2p_v2_misbehaving.py (stratospher)
Pull request description:
Fixes#30419.
Make sure that ellswift computation is complete in the `NetworkThread` in `test/functional/p2p_v2_misbehaving.py` before sending the ellswift in the `MainThread`.
One way to reproduce this failure on master would be:
```diff
diff --git a/test/functional/test_framework/v2_p2p.py b/test/functional/test_framework/v2_p2p.py
index 87600c36de..ea0615ef3b 100644
--- a/test/functional/test_framework/v2_p2p.py
+++ b/test/functional/test_framework/v2_p2p.py
@@ -111,6 +111,7 @@ class EncryptedP2PState:
def generate_keypair_and_garbage(self, garbage_len=None):
"""Generates ellswift keypair and 4095 bytes garbage at max"""
+ import time; time.sleep(3)
self.privkey_ours, self.ellswift_ours = ellswift_create()
if garbage_len is None:
garbage_len = random.randrange(MAX_GARBAGE_LEN + 1)
```
ACKs for top commit:
maflcko:
ACK c6d43367a1
mzumsande:
Code Review ACK c6d43367a1
tdb3:
cr and t ACK c6d43367a1
Tree-SHA512: dfc3a6afa09773b7e84d35aff0aa14e0b8a4475860e0b31ab5c1a8d54911c814f07138f624fea651fba90cc5c526c0d05c3fe33d2ce0ad833b2be3a3caa9f522
fa14e1d9d5 log: Fix __func__ in LogError in blockstorage module (MarcoFalke)
fad59a2f0f log: LogError with FlatFilePos in UndoReadFromDisk (MarcoFalke)
aaaa3323f3 refactor: Mark IsBlockPruned const (MarcoFalke)
Pull request description:
These errors should never happen in normal operation. If they do,
knowing the `FlatFilePos` may be useful to determine if data corruption
happened. Also, handle the error `pos.IsNull()` as part of `OpenUndoFile`,
because it may as well have happened due to data corruption.
This mirrors the `LogError` behavior from `ReadBlockFromDisk`.
Also, two other fixup commits in this module.
ACKs for top commit:
kevkevinpal:
ACK [fa14e1d](fa14e1d9d5)
tdb3:
cr and light test ACK fa14e1d9d5
ryanofsky:
Code review ACK fa14e1d9d5. This should make logging clearer and more consistent
Tree-SHA512: abb492a919b4796698d1de0a7874c8eae355422b992aa80dcd6b59c2de1ee0d2949f62b3cf649cd62892976fee640358f7522867ed9d48a595d6f8f4e619df50
fa690c8e53 test: [refactor] Pass TestOpts (MarcoFalke)
Pull request description:
Currently optional test context setup settings are passed by adding a new optional argument to the constructors. For example `extra_args`. This is problematic, because:
* Adding more optional settings in the future requires touching all affected constructors, increasing their verbosity.
* Setting only a later option requires setting the earlier ones.
* Clang-tidy named args passed to `std::make_unique` are not checked.
Fix all issues by adding a new struct `TestOpts`, which holds all options. Notes:
* The chain type is not an option in the struct for now, because the default values vary.
* The struct holds all possible test options globally. Not all fields may be used by all constructors. Albeit harmless, it is up to the test author to not set a field that is unused.
ACKs for top commit:
kevkevinpal:
utACK [fa690c8](fa690c8e53)
dergoegge:
utACK fa690c8e53
TheCharlatan:
Nice, ACK fa690c8e53
Tree-SHA512: 8db8efa5dff854a73757d3f454f8f902e41bb4358f5f9bae29dbb3e251e20ee93489605de51d0822ba31d97835cd15526a29c075278dd6a8bbde26134feb4f49
The comment in the code regarding the use of an "&"
on a menu item is misleading. If a wallet name has an "&" in it,
it is not supposed to be interpreted as a hot-key, but it should be
shown as it is without replacing it to an underscore.
The ellswift bytes are computed in the NetworkThread and sent in
the MainThread. Add a `wait_until()` to make sure that ellswift
computation is completed in NetworkThread before sending it in
the MainThread. Also wait until bytes sent are actually received
and use mocktime for more robust disconnection checking.
bc34bc2888 fuzz: limit the number of nested wrappers in descriptors (Antoine Poinsot)
8d7340105f fuzz: limit the number of sub-fragments per fragment for descriptors (Antoine Poinsot)
Pull request description:
Some of the logic in the miniscript module is quadratic. It only becomes an issue for very large uninteresting descriptors (like a `thresh` with 130k sub-fragments or a fragment with more than 60k nested `j:` wrappers).
This PR fixes the two types of fuzz timeouts reported by Marco in https://github.com/bitcoin/bitcoin/issues/28812 by trying to pinpoint the problematic descriptors through a simple analysis of the string, without limiting the size of the string itself. This is the same approach as was adopted for limiting the depth of derivation paths.
ACKs for top commit:
dergoegge:
utACK bc34bc2888
stickies-v:
Light ACK bc34bc2888
marcofleon:
Code review ACK bc34bc2888. The added comments are useful, thanks for those. Tested on the three inputs in https://github.com/bitcoin/bitcoin/issues/28812 that caused the timeouts.
Tree-SHA512: 8811c7b225684c5ecc1eb1256cf39dfa60d4518161e70210086c8a01b38927481ebe747af86aa5f4803187672d43fadabcfdfbf4e3b049738d629a25143f0e77
b71bfd9eef GUI/OptionsDialog: Prefer to stretch actual options area rather than waste space (Luke Dashjr)
Pull request description:
ACKs for top commit:
hebasto:
ACK b71bfd9eef
Tree-SHA512: b706a07292fe81379e303f9069fca6efd5ceb15ee5bb77c6aeddbf63f736494ce877b76767ff17d7becf98d07209e51c74bdb99365596b7b9f4904a30438d72d
4383dc90ba fuzz: fix key size in crypter target (brunoerg)
Pull request description:
Fixes#30251
This PR:
1. Limits `cipher_text_ed` and `random_string` (`SecureString`) size.
2. Replace `ConsumeRandomLengthByteVector` for keys to `ConsumeFixedLengthByteVector` with `WALLET_CRYPTO_KEY_SIZE`.
3. Replace `ConsumeRandomLengthByteVector` for `chSalt` to `ConsumeFixedLengthByteVector` with `WALLET_CRYPTO_SALT_SIZE`.
ACKs for top commit:
marcofleon:
Tested ACK 4383dc90ba. I ran this:
dergoegge:
utACK 4383dc90ba
Tree-SHA512: 6f09cca0b4627f49152b685ac03659c01004f2131c6aada7654606ea01f6619b1611b1d17624d2cddce277c1afdddda5f656d99f6ca8f72a22f5c0541762c964
992b1bbd5d qt: keep focus on "Hide" while ModalOverlay is visible (Jadi)
Pull request description:
During the initial sync, the Tab moves the focus to the widgets of the main window, even when the ModalOverlay is visible. This creates some weird rectangular *selections on the screen*.
This PR fixes this by keeping the focus on the "Hide" button while the ModalOverlay is visible.
Fixes#783
ACKs for top commit:
pablomartin4btc:
Concept & approach ACK 992b1bbd5d
hebasto:
re-ACK 992b1bbd5d
Tree-SHA512: f702a3fd51db4bc10780bccf76394e35a6b5fb45db72c9c23cd10d777106b08c61077d2d989003838921e76d2cb44f809399f31df76448e4305a6c2a71b5c6a3
09370529fb fuzz: mini_miner_selection fixups. (glozow)
de273d5300 MiniMiner: use FeeFrac in AncestorFeerateComparator (glozow)
Pull request description:
Closes#30284. Closes#30367, see https://github.com/bitcoin/bitcoin/issues/30367#issuecomment-2217459257
Previously, we were only comparing feerates up to 1/1000 precision, since CFeeRate comparison just looks at their respective nSatoshisPerK. This could lead to MiniMiner selecting packages in the wrong order (i.e. by txid) if their feerates were less than 0.001sat/vB different. Fix this by creating + comparing `FeeFrac`s instead.
Also, `FeeFrac::Mul` doesn't have the overflow problem.
Also added a few minor fuzzer fixups that caught my eye while I was debugging this.
ACKs for top commit:
ismaelsadeeq:
Tested ACK 09370529fb
murchandamus:
ACK 09370529fb with nits
dergoegge:
tACK 09370529fb
Tree-SHA512: e5b6d6c3f7289f30cd8280d0a47cd852d0180b83d1b27ff9514f50c97103b0f069484e48cba2ca3a57419beadc1996c1b9dd8d0a0f34bc4f4223d8adaf414ce5
The script building logic performs a quadratic number of copies in the
number of nested wrappers in the miniscript. Limit the number of nested
wrappers to avoid fuzz timeouts.
Thanks to Marco Falke for reporting the fuzz timeouts and providing a
minimal input to reproduce.
This target may call into logic quadratic over the number of
sub-fragments. Limit the number of sub-fragments to keep the runtime
reasonable.
Thanks to Marco Falke for reporting the fuzz timeouts with a minimized
input.
This avoids situations during a reindex in which shutdown
doesn't finish since SyncWithValidationInterfaceQueue is
called by the load block thread when the scheduler is already stopped.
fa360b047f util: Use SteadyClock in RandAddSeedPerfmon (MarcoFalke)
Pull request description:
`GetTime` is mockable in tests and system-changeable in production. This should be fine and not lead to issues, but using `SteadyClock` is more correct in this context to do an expensive task only so often.
ACKs for top commit:
sipa:
utACK fa360b047f
TheCharlatan:
ACK fa360b047f
Tree-SHA512: 1958b9e9e356c9801ac981014b4b528cfc8ce6612853d8b45f6519b16f0b1839ff765abb8b3368b86f00958ddc6a686f6b90278c57a7ad4858bdf3ea33775cca
f170fe04ca depends: update doc in Qt pwd patch (fanquake)
Pull request description:
Now that upstream has gotten around to fixing this. We don't need any more of the patch, and it likely wont apply to our version of Qt in any case. See: 3388de698b.
ACKs for top commit:
theuni:
ACK f170fe04ca
Tree-SHA512: f6db8ccad591b1bf144ce71f873f42a115d394c432a95b6b855e3e32751e6331145e0d9676657599b25fd369af8c72c1bd34e192a7a1062c15f152421422a9ed
00b8e26bd6 test: fix inconsistency in fundrawtransaction weight limits test (furszy)
Pull request description:
Fix https://github.com/bitcoin/bitcoin/pull/30309#discussion_r1657628378 inconsistency.
Currently, the test is passing due to a mistake in the test inputs
selection process. We are selecting the parent transaction change
output as one of the inputs of the transaction to fund, which
helps to surpass the target amount when it shouldn't due to the
fee reduction.
The failure arises when the test behaves as intended by its coder;
that is, when it does not select the change output. In this case,
the pre-selected inputs aren't enough to cover the target amount.
Fix this by excluding the parent transaction's change output from
the inputs selection and including an extra input to cover the tx
fee.
The CI failure can be replicated with the following patch in master:
```diff
diff --git a/test/functional/wallet_fundrawtransaction.py b/test/functional/wallet_fundrawtransaction.py
--- a/test/functional/wallet_fundrawtransaction.py(revision 9b480f7a25)
+++ b/test/functional/wallet_fundrawtransaction.py(date 1720652934739)
@@ -1322,7 +1322,7 @@
outputs = []
for _ in range(1472):
outputs.append({wallet.getnewaddress(address_type="legacy"): 0.1})
- txid = self.nodes[0].send(outputs=outputs)["txid"]
+ txid = self.nodes[0].send(outputs=outputs, change_position=0)["txid"]
self.generate(self.nodes[0], 1)
# 272 WU per input (273 when high-s); picking 1471 inputs will exceed the max standard tx weight.
@@ -1330,7 +1330,7 @@
# 1) Try to fund transaction only using the preset inputs
input_weights = []
- for i in range(1471):
+ for i in range(1, 1472): # skip first output as it is the parent tx change output
input_weights.append({"txid": txid, "vout": i, "weight": 273})
assert_raises_rpc_error(-4, "Transaction too large", wallet.fundrawtransaction, hexstring=rawtx, input_weights=input_weights)
```
ACKs for top commit:
achow101:
ACK 00b8e26bd6
ismaelsadeeq:
Code review and Tested ACK 00b8e26bd6
Tree-SHA512: 5ef792961b7fad4999fc30aa03366432103ddf672ca5cbb366c9eab4c2e46d5ae1ab0c073dfc4fbb2b4e63203653bc0e54463c731c5f8655140207ba5f8e542e
26a7f70b5d ci: enable self-assignment clang-tidy check (Cory Fields)
32b1d13792 refactor: add self-assign checks to classes which violate the clang-tidy check (Cory Fields)
Pull request description:
See comment here: https://github.com/bitcoin/bitcoin/pull/30161#issuecomment-2148229582
Our code failed these checks in three places, which have been fixed up here. Though these appear to have been harmless, adding the check avoids the copy in the self-assignment case so there should be no downside.
~Additionally, minisketch failed the check as well. See https://github.com/sipa/minisketch/pull/87~
Edit: Done
After fixing up the violations, turn on the aggressive clang-tidy check.
Note for reviewers: `git diff -w` makes this trivial to review.
ACKs for top commit:
hebasto:
ACK 26a7f70b5d, I have reviewed the code and it looks OK.
TheCharlatan:
ACK 26a7f70b5d
Tree-SHA512: 74d8236a1b5a698f2f61c4740c4fc77788b7f882c4b395acc4e6bfef1ec8a4554ea8821a26b14d70cfa6c8e2e9ea305deeea3fbf323967fa19343c007a53c5ba
3333bae9b2 tidy: modernize-use-equals-default (MarcoFalke)
Pull request description:
Prior to C++20, `modernize-use-equals-default` could have been problematic because it could turn a non-aggregate into an aggregate. The risk would be that aggregate initialization would be enabled where the author did not intend to enable it.
With C++20, aggregate for those is forbidden either way. (https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p1008r1.pdf)
So enabled it for code clarity, consistency, and possibly unlocking compiler optimizations. See https://clang.llvm.org/extra/clang-tidy/checks/modernize/use-equals-default.html
ACKs for top commit:
stickies-v:
ACK 3333bae9b2
Tree-SHA512: ab42ff01be7ca7e7d8b4c6a485e68426f59627d83dd827cf292304829562348dc17a52ee009f5f6f3c1c2081d7166ffac4baef23197ebeba8de7767c6ddfe255
34c9cee380 clang-tidy: add check for non-trivial thread_local vars (Cory Fields)
Pull request description:
Forbid thread_local vars with non-trivial destructors.
This is a follow-up from: https://github.com/bitcoin/bitcoin/pull/30095#discussion_r1608423170
ACKs for top commit:
maflcko:
ACK 34c9cee380
TheCharlatan:
Re-ACK 34c9cee380
Tree-SHA512: 3a798607fb189a5bbc714ed6e86dea462fe29d366b790e96d10a7b4ffcf1f194da9b8f4cd0b82154408709b8e3c58d3f613d6311903bd65a76d8b556ab230d21
fa601ab9f7 util: Catch translation string errors at compile time (MarcoFalke)
Pull request description:
The translation helper function `_()` has many problems. For example, the following compiles:
```cpp
auto ptr{"wrong"};
_(ptr);
_(nullptr);
_(0);
_(NULL);
```
However, it is wrong, because none of the arguments passed to the function can be picked up by the translation tooling for transifex.
Fix all issues by enforcing only real string literals can be passed to the function.
ACKs for top commit:
ryanofsky:
Code review ACK fa601ab9f7
hebasto:
ACK fa601ab9f7.
Tree-SHA512: 33aed02d7e8fc9bfb8f90746f5c8072a8c0910fa900ec3516af2e732780b0fee8b07b6596c0fc210b018c0869111d6c34bf8d083de0e88ecdb4dee88e809186d
e233ec036d refactor: Use designated initializer (Hodlinator)
Pull request description:
Block was recently touched (e2d1f84858) and the codebase recently switched to C++20 which allows this to improve robustness.
Follow-up suggested in https://github.com/bitcoin/bitcoin/pull/29625#discussion_r1664818014
ACKs for top commit:
maflcko:
ACK e233ec036d
Tree-SHA512: ce3a18f513421e923710a43c8f97db1badb7ff5c6bdbfd62d9543312d2225731db5c14bef16feb47c43b84fad4dc24485086634b680feba422d2b7b363e13fa6
If the snapshot is not an ancestor of the most-work header (m_best_header),
syncing from that alternative chain should be prioritised.
Therefore don't accept loading a snapshot in this situation.
If that other chain turns out to be invalid, m_best_header
would be reset and loading the snapshot should be possible again.
Because of the work required to generate a conflicting headers chain,
this should only be possible under extreme circumstances, such as major forks.
e8c3b7172c remove truc_policy.cpp from libbitcoin_common_a_SOURCES (glozow)
Pull request description:
Hebasto pointed out that it doesn't need to be there since it's in `libbitcoin_node_a_SOURCES`
ACKs for top commit:
maflcko:
ACK e8c3b7172c
hebasto:
ACK e8c3b7172c, this change follows the design [docs](https://github.com/bitcoin/bitcoin/blob/master/doc/design/libraries.md).
ismaelsadeeq:
ACK e8c3b7172c
Tree-SHA512: ebe6b0dda2d097d88c37d2b071ac99da3e9c519ec473d4b8f870a50f1b24d00e2e5deef317fb0f6a91c96103e7f37468cb8f13395818eab55a42af48df4e0fc6
8ce3739edb test: verify wallet is still active post-migration failure (furszy)
771bc60f13 wallet: Use LegacyDataSPKM when loading (Ava Chow)
61d872f1b3 wallet: Move MigrateToDescriptor and DeleteRecords to LegacyDataSPKM (Ava Chow)
b231f4d556 wallet: Move LegacyScriptPubKeyMan::IsMine to LegacyDataSPKM (Ava Chow)
7461d0c006 wallet: Move LegacySPKM data storage and handling to LegacyDataSPKM (Ava Chow)
517e204bac Change MigrateLegacyToDescriptor to reopen wallet as BERKELEY_RO (Ava Chow)
Pull request description:
#26606 introduced `BerkeleyRODatabase` which is an independent parser for BDB files. This PR uses this in legacy wallet migration so that migration will continue to work once the legacy wallet and BDB are removed. `LegacyDataSPKM` is introduced to have the minimum data and functions necessary for a legacy wallet to be loaded for migration.
ACKs for top commit:
cbergqvist:
ACK 8ce3739edb
theStack:
Code-review ACK 8ce3739edb
furszy:
Code review ACK 8ce3739edb
Tree-SHA512: dccea12d6c597de15e3e42f97ab483cfd069e103611200279a177e021e8e9c4e74387c4f45d2e58b3a1e7e2bdb32a1d2d2060b1f8086c03eeaa0c68579d9d54e
These errors should never happen. However, when they do happen, it is
useful to log the correct error location (function name).
For example, this fixes an incorrect "ConnectBlock()" in
"WriteUndoDataForBlock".
These errors should never happen in normal operation. If they do,
knowing the FlatFilePos may be useful to determine if data corruption
happened. Also, handle the error pos.IsNull() as part of OpenUndoFile,
because it may as well have happened due to data corruption.
This mirrors the LogError behavior from ReadBlockFromDisk.
Currently, the test is passing due to a mistake in the test inputs
selection process. We are selecting the parent transaction change
output as one of the inputs of the transaction to fund, which
helps to surpass the target amount when it shouldn't due to the
fee reduction.
The failure arises when the test behaves as intended by its coder;
that is, when it does not select the change output. In this case,
the pre-selected inputs aren't enough to cover the target amount.
Fix this by excluding the parent transaction's change output from
the inputs selection and including an extra input to cover the tx
fee.
8789dc8f31 doc: Add note to getblockfrompeer on missing undo data (Fabian Jahr)
4a1975008b rpc: Make pruneheight also reflect undo data presence (Fabian Jahr)
96b4facc91 refactor, blockstorage: Generalize GetFirstStoredBlock (Fabian Jahr)
Pull request description:
The function `GetFirstStoredBlock()` helps us find the first block for which we have data. So far this function only looked for a block with `BLOCK_HAVE_DATA`. However, this doesn't mean that we also have the undo data of that block, and undo data might be required for what a user would like to do with those blocks. One example of how this might happen is if some blocks were fetched using the `getblockfrompeer` RPC. Blocks fetched from a peer will have data but no undo data.
The first commit here allows `GetFirstStoredBlock()` to check for undo data as well by passing a parameter. This alone is useful for #29553 and I would use it there.
In the second commit I am applying the undo check to the RPCs that report `pruneheight` to the user. I find this much more intuitive because I think the user expects to be able to do all operations on blocks up until the `pruneheight` but that is not the case if undo data is missing. I personally ran into this once before and now again when testing for assumeutxo when I had used `getblockfrompeer`. The following commit adds test coverage for this change of behavior.
The last commit adds a note in the docs of `getblockfrompeer` that undo data will not be available.
ACKs for top commit:
achow101:
ACK 8789dc8f31
furszy:
Code review ACK 8789dc8f31.
stickies-v:
ACK 8789dc8f31
Tree-SHA512: 90ae8bdd07a496ade579aa25240609c61c9ed173ad38d30533f6c631fe674e5a41727478ade69ca4b71a571ad94c9da4b33ebba6b5d8821109313c2de3bdfb3d
5b7f70ba26 test: loadtxoutset in divergent chain with less work (Alfonso Roman Zubeldia)
d35efe1efc p2p: Start downloading historical blocks from common ancestor (Martin Zumsande)
Pull request description:
This PR adds a test to cover the scenario of loading an assumeutxo snapshot when the current chain tip is not an ancestor of the snapshot block but has less work.
During the review process, a bug was discovered where blocks between the last common ancestor and the background tip were not being requested if the background tip was not an ancestor of the snapshot block. mzumsande suggested a fix (65343ec49a6b73c4197dfc38e1c2f433b0a3838a) to start downloading historical blocks from the last common ancestor to address this issue. This fix has been incorporated into the PR with a slight modification.
Related to https://github.com/bitcoin/bitcoin/issues/28648
ACKs for top commit:
fjahr:
tACK 5b7f70ba26
achow101:
ACK 5b7f70ba26
mzumsande:
Code Review ACK 5b7f70ba26
Tree-SHA512: f8957349686a6a1292165ea9e0fd8c912d21466072632a10f8ef9d852a5f430bc6b2a531e6884a4dbf2e3adb28b3d512b25919e78f5804a67320ef54c3b1aaf6
576828e732 ci: test-each-commit merge base optional (Sjors Provoost)
e9bfbb5414 ci: forks can opt-out of CI branch push (Cirrus only) (Sjors Provoost)
Pull request description:
Maintainer note: `SKIP_BRANCH_PUSH=true` must be set in Cirrus for `bitcoin-core/gui` before merging this. See `https://cirrus-ci.com/github/bitcoin-core/gui` -> Settings.
---
I find myself making pull requests against my fork (mostly on top of https://github.com/bitcoin/bitcoin/pull/28983, or asking others to do so. Currently only the Github actions are run on forks, because we use self-hosted runners for the Cirrus tasks.
While setting up my own self-hosted runners for my fork, I ran into a number of issues. Some of those were addressed by https://github.com/bitcoin/bitcoin/pull/29441, but remaining issues are:
1. When PRs are opened in the fork, cirrus CI jobs are run twice because PRs and branches reside in the same repository, rather than a main repository and a fork repository, as is the case with bitcoin/bitcoin PRs. Fix this by adding a `SKIP_BRANCH_PUSH` configuration option that allows skipping CI runs not directly associated with a PR. The fix is a generalization of [#20328](https://github.com/bitcoin/bitcoin/pull/20328), which fixed a similar problem for the bitcoin-core/gui mirror repository, and it allows removing a hardcoded reference to that repository.
Github actions jobs will still run twice despite this change, see [#29274 (comment)](https://github.com/bitcoin/bitcoin/pull/29274#issuecomment-2188840483). Initially this PR tried to prevent that with b9fdd0dc75, but this had some potentially negative side effects, see [#29274 (comment)](https://github.com/bitcoin/bitcoin/pull/29274#discussion_r1457587805), so that commit was dropped for now.
2. When PRs are opened in the fork, the "test-each-commit" github action can fail due to not being able to find a recent merge commit. This problem doesn't happen in the bitcoin/bitcoin repository because branches in this repository used as the base for pull requests always point at merge commits.
This PR replaces https://github.com/bitcoin/bitcoin/pull/29259 using the self hosted workers via Cirrus instead of Github.
You can see this PR in action on this pull request to my fork: https://github.com/Sjors/bitcoin/pull/30
To test it yourself:
1. spin up at least two [self hosted runners](https://github.com/cirruslabs/cirrus-cli/blob/master/PERSISTENT-WORKERS.md). Either use a seperate VM for each, or give them their own user.
3. Install Podman and other CI dependencies (see .cirrus.yml)
4. Give Cirrus access to your fork at https://cirrus-ci.com/settings/github/YOU
5. Get a token from Cirrus and use it to start your worker(s)
6. Optionally set SKIP_BRANCH_PUSH=true ~and NO_ARM=true~ env variables (see .cirrus.yml)
make a pull request to your own fork, with this PR as the base branch
Security wise: when dealing with code from strangers on the internet, review it first before running the CI. There's a Cirrus check-box that requires approval for people without write access to trigger CI.
ACKs for top commit:
maflcko:
ACK 576828e732
ryanofsky:
Code review ACK 576828e732.
Tree-SHA512: fb6be2f228aa62f45a65ce5c613c979b6f387df396f9601ce4622b27aa317a66f198e7d7a194592b0bb397b32a2f50f8be47065834d74af4ea09407c5c8d306d
d93b794709 tests: improve wallet multisig descriptor test and docs (Michael Dietz)
Pull request description:
It is best to store all key origin information
(master key fingerprint and all derivation steps)
in the multisig descriptor. Being explicit with
this information should be beneficial if this approach is used with other wallets/signers (whether hardware or software). There is no harm including all of this with xpubs (if anything it simplifies the test code) and makes this example/docs more complete and safer incase it is referenced by others.
ACKs for top commit:
S3RK:
Code Review ACK d93b794709
achow101:
ACK d93b794709
Tree-SHA512: 0e5c4d13f060489405e6cf50c8a09911f5a0cee71023649235afd80a5e3aae38d52c6e12ad4660205b9357b09f45596941391bdcf6fceccbe07c4e5a1592a482
6ecda04fef random: drop ad-hoc Shuffle in favor of std::shuffle (Pieter Wuille)
da28a26aae bench random: benchmark more functions, and add InsecureRandomContext (Pieter Wuille)
0a9bbc64c1 random bench refactor: move to new bench/random.cpp (Pieter Wuille)
Pull request description:
This adds benchmarks for various operations on `FastRandomContext` and `InsecureRandomContext`, and then removes the ad-hoc `Shuffle` functions, now that it appears that standard library `std::shuffle` has comparable performance. The other reason for keeping `Shuffle`, namely the fact that libstdc++ used self-move (which debug mode panics on) has been fixed as well (see https://github.com/bitcoin/bitcoin/pull/29625#discussion_r1658344049).
ACKs for top commit:
achow101:
ACK 6ecda04fef
hodlinator:
ACK 6ecda04fef
dergoegge:
Code review ACK 6ecda04fef
Tree-SHA512: 2560b7312410581ff2b9bd0716e0f1558d910b5eadb9544785c972384985ac0f11f72d6b2797cfe2e7eb71fa57c30cffd98cc009cb4ee87a18b1524694211417
c9dacd958d test: Check that non empty version packet is ignored and no disconnection happens (stratospher)
997cc00b95 test: Check that disconnection happens when AAD isn't filled (stratospher)
b5e6238fdb test: Check that disconnection happens when garbage sent/received are different (stratospher)
ad1482d5a2 test: Check that disconnection happens when wrong garbage terminator is sent (stratospher)
e351576862 test: Check that disconnection happens when >4095 garbage bytes is sent (stratospher)
e075fd131d test: Introduce test types and modify v2 handshake function accordingly (stratospher)
7d07daa623 log: Add V2 handshake timeout (stratospher)
d4a1da8543 test: Make global TRANSPORT_VERSION variable an instance variable (stratospher)
c642b08c4e test: Log when the garbage is actually sent to transport layer (stratospher)
86cca2cba2 test: Support disconnect waiting for add_p2p_connection (stratospher)
bf9669af9c test: Rename early key response test and move random_bitflip to util (stratospher)
Pull request description:
Add tests for the following v2 handshake scenarios:
1. Disconnection happens when > `MAX_GARBAGE_LEN` bytes garbage is sent
2. Disconnection happens when incorrect garbage terminator is sent
3. Disconnection happens when garbage bytes are tampered with
4. Disconnection happens when AAD of first encrypted packet after the garbage terminator is not filled
5. bitcoind ignores non-empty version packet and no disconnection happens
All these tests require a modified v2 P2P class (different from `EncryptedP2PState` used in `v2_p2p.py`) to implement our custom handshake behaviour based on different scenarios and have been kept in a single test file (`test/functional/p2p_v2_misbehaving.py`). Shifted the test in `test/functional/p2p_v2_earlykeyresponse.py` which is of the same pattern to this file too.
ACKs for top commit:
achow101:
ACK c9dacd958d
mzumsande:
ACK c9dacd958d
theStack:
Code-review ACK c9dacd958d
Tree-SHA512: 90df81f0c7f4ecf0a47762d290a618ded92cde9f83d3ef3cc70e1b005ecb16125ec39a9d80ce95f99e695d29abd63443240cb5490aa57c5bc8fa2e52149a0672
de71d4dece fuzz: improve utxo_snapshot target (Martin Zumsande)
Pull request description:
Add the possibility of giving more guidance to the creation of the metadata and/or coins, so that the fuzzer gets the chance
to reach more error conditions in ActivateSnapshot and sometimes successfully creates a valid snapshot.
This also changes the asserts for the success case that were outdated (after #29370) and only didn't result in a crash because the fuzzer wasn't able to reach this code before.
ACKs for top commit:
maflcko:
re-ACK de71d4dece🎆
fjahr:
utACK de71d4dece
TheCharlatan:
ACK de71d4dece
Tree-SHA512: 346974d594164544d8cd3df7d8362c905fd93116215e9f5df308dfdac55bab04d727bfd7fd001cf11318682d11ee329b4b4a43308124c04d64b67840ab8a58a0
Now that the queueing of the VERSION messages has been moved out of
`InitializeNode`, there is no need to pass a mutable `CNode` reference any
more. With a const reference, trying to send messages in this method would
lead to a compile-time error, e.g.:
----------------------------------------------------------------------------------------------------------------------------------
...
net_processing.cpp: In member function ‘virtual void {anonymous}::PeerManagerImpl::InitializeNode(const CNode&, ServiceFlags)’:
net_processing.cpp:1683:21: error: binding reference of type ‘CNode&’ to ‘const CNode’ discards qualifiers
1683 | PushNodeVersion(node, *peer);
...
----------------------------------------------------------------------------------------------------------------------------------
Initiating an outbound network connection currently involves the
following steps after the socket connection is established (see
`CConnman::OpenNetworkConnection` method):
1. set up node state
2. queue VERSION message
3. add new node to vector `m_nodes`
If we connect to ourself, it can happen that the sent VERSION message
(step 2) is received and processed locally *before* the node object
is added to the connection manager's `m_nodes` vector (step 3). In this
case, the self-connect remains undiscovered, as the detection doesn't
find the outbound peer in `m_nodes` yet (see `CConnman::CheckIncomingNonce`).
Fix this by swapping the order of 2. and 3., by taking the `PushNodeVersion`
call out of `InitializeNode` and doing that in the `SendMessages` method
instead, which is only called for `CNode` instances in `m_nodes`.
Thanks go to vasild, mzumsande, dergoegge and sipa for suggestions on
how to fix this.
Comparing using FeeFracs is more precise, allows us to simply the
code since FeeFrac comparison internally does cross-multiplication,
and avoids potential overflow in the multiplication.
Previously, we were only comparing feerates up to 0.001sat/vB precision,
since CFeeRate comparison just looks at their respective nSatoshisPerK.
This could lead to MiniMiner selecting packages in the wrong order (i.e.
by txid) if their feerates were less than 0.001sat/vB different.
In future, users of the kernel library might run multiple chainstates in
parallel, or create and destroy multiple chainstates over the lifetime
of a process. Having static, mutable variables could lead to state
inconsistencies in these scenarios.
In future, users of the kernel library might run multiple chainstates in
parallel, or create and destroy multiple chainstates over the lifetime
of a process. Having static, mutable variables could lead to state
inconsistencies in these scenarios.
9e13ccc50e psbt: Check non witness utxo outpoint early (Ava Chow)
Pull request description:
A common issue that our fuzzers keep finding is that outpoints don't exist in the non witness utxos. Instead of trying to track this down and checking in various individual places, do the check early during deserialization. This also unifies the error message returned for this class of problems.
ACKs for top commit:
maflcko:
lgtm ACK 9e13ccc50e
S3RK:
tACK 9e13ccc50e
dergoegge:
utACK 9e13ccc50e
Tree-SHA512: 81b8055b146c6358052226578ddfec0ae5bd877968c7f4f62dc3d6a684545ea568f37c7f1bd619918441af9e453ba8b26531a2280d218da37fa15480f1b45d0e
606a7ab862 kernel: De-globalize signature cache (TheCharlatan)
66d74bfc45 Expose CSignatureCache class in header (TheCharlatan)
021d38822c kernel: De-globalize script execution cache hasher (TheCharlatan)
13a3661aba kernel: De-globalize script execution cache (TheCharlatan)
ab14d1d6a4 validation: Don't error if maxsigcachesize exceeds uint32::max (TheCharlatan)
Pull request description:
The validation caches are currently setup independently from where the rest of the validation code is initialized. This makes their ownership semantics unclear. There is also no clear enforcement on when and in what order they need to be initialized. The caches are always initialized in the `BasicTestingSetup` although a number of tests don't actually need them.
Solve this by moving the caches from global scope into the `ChainstateManager` class. This simplifies the usage of the kernel library by no longer requiring manual setup of the caches prior to using the `ChainstateManager`. Tests that need to access the caches can instantiate them independently.
---
This pull request is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).
ACKs for top commit:
stickies-v:
re-ACK 606a7ab862
glozow:
reACK 606a7ab
ryanofsky:
Code review ACK 606a7ab862. Just small formatting, include, and static_assert changes since last review.
Tree-SHA512: e7f3ee41406e3b233832bb67dc3a63c4203b5367e5daeed383df9cb590f227fcc62eae31311029c077d5e81b273a37a88a364db3dee2efe91bb3b9c9ddc8a42e
The wording "public key script" was likely chosen as a human-readable form of
the technical term `scriptPubKey`, but it doesn't seem to be really widespread.
Replace it by the more common term "output script" instead. Note that the
argument for the `decodescript` RPC is not necessarily an output script (it
could e.g. be also a redeem script), so in this case we just stay generic and
use "script".
See also the draft BIP "Terminology for Transaction Components"
(https://github.com/murchandamus/bips/blob/2022-04-tx-terminology/bip-tx-terminology.mediawiki)
which suggests to use "output script" as well.
Affects the help text of the following RPCs:
- decodepsbt
- decoderawtransaction
- decodescript
- getblock (if verbosity=3)
- getrawtransaction (if verbosity=2,3)
- gettxout
fa8f53273c refactor: Remove no longer needed clang-15 workaround for std::span (MarcoFalke)
9999dbc1bd fuzz: Clarify Apple-Clang-16 workaround (MarcoFalke)
fa7462c67a build: Bump clang minimum supported version to 16 (MarcoFalke)
Pull request description:
Most supported operating systems ship with clang-16 (or later), so bump the minimum to that and allow new code to drop workarounds for previous clang bugs.
For reference:
* https://packages.debian.org/bookworm/clang-16
* https://packages.ubuntu.com/noble/clang (clang-18)
* CentOS-like 8/9 Stream: All Clang versions from 16 to 17
* FreeBSD 12/13: All Clang versions from 16 to 18
* OpenSuse Tumbleweed ships with https://software.opensuse.org/package/clang (`clang18`); No idea about OpenSuse Leap
On operating systems where the clang version is not shipped by default, the user would have to use GCC, or install clang in a different way. For example:
* https://packages.debian.org/bookworm/g++ (g++-12)
* https://packages.ubuntu.com/jammy/g++ (g++-11)
* https://apt.llvm.org/, or nix, or guix, or compile clang from source, ...
**Ubuntu 22.04 LTS does not ship with clang-16**, so one of the above workarounds is needed there.
macOS 13 is unaffected, and the previous minimum requirement of Xcode15.0 remains, see also b1ba1b178f/.github/workflows/ci.yml (L93). For macOS 11 (Big Sur) and 12 (Monterey) you need to install a more recent version of llvm, this remains unchanged as well, see b1ba1b178f/doc/build-osx.md (L54).
ACKs for top commit:
hebasto:
ACK fa8f53273c, I have reviewed the code and it looks OK.
TheCharlatan:
Re-ACK fa8f53273c
stickies-v:
ACK fa8f53273c
Tree-SHA512: 18b79f88301a63bb5e367d2f52fffccd5fb84409061800158e51051667f6581a4cd71d4859d4cfa6d23e47e92963ab637e5ad87e3170ed23b5bebfbe99e759e2
6af51e8198 Use WITH_LOCK in Warnings::Set (Ava Chow)
Pull request description:
The scope of the lock should be limited to just guarding m_warnings as anything listening on `NotifyAlertChanged` may execute code that requires the lock as well.
Fixes#30400
ACKs for top commit:
maflcko:
lgtm ACK 6af51e8198
TheCharlatan:
ACK 6af51e8198
glozow:
ACK 6af51e8198
willcl-ark:
ACK 6af51e8198
stickies-v:
ACK 6af51e8198
Tree-SHA512: 9884046c70dcad996276931b6d154f0330200332403828f34f7f7b285fc0e770ba7b25056131ab24dcb8a4b18f58d31633aa17fbb09b0eaea8a29e28fca10ec4
46819f5df6 wallet: use LogTrace for walletdb log messages at trace level (Anthony Towns)
Pull request description:
Wallet sqlite logging is enabled by `-debug=walletdb -loglevel=walletdb:trace` however the actual log messages are sent at `BCLog::Level::Info`. Switch to the trace level to make this consistent. This adds `[walletdb:trace]` to the log output, eg:
```
[httpworker.3] [wallet/sqlite.cpp:55] [TraceSqlCallback] [/tmp/bitcoin_func_test_4fsnatpg/node0/regtest/wallets/boring/wallet.dat] SQLite Statement: BEGIN EXCLUSIVE TRANSACTION
```
becomes
```
[httpworker.0] [wallet/sqlite.cpp:55] [TraceSqlCallback] [walletdb:trace] [/tmp/bitcoin_func_test_9lcwth4z/node0/regtest/wallets/boring/wallet.dat] SQLite Statement: BEGIN EXCLUSIVE TRANSACTION
```
ACKs for top commit:
maflcko:
ACK 46819f5df6
ryanofsky:
Code review ACK 46819f5df6. Nice catch!
furszy:
ACK 46819f5df6
luke-jr:
utACK 46819f5df6
Tree-SHA512: 6fc1bc63c2ee686d4ca8f4f558f06c0cd9e7813b5fce1588351f55ef8bedfc23c97ea443e54a6a447008fa79ea022b6d631cb010929932f1db23fa8e255e6482
The scope of the lock should be limited to just guarding m_warnings as
anything listening on `NotifyAlertChanged` may execute code that
requires the lock as well.
Move its ownership to the ChainstateManager class.
Next to simplifying usage of the kernel library by no longer requiring
manual setup of the cache prior to using validation code, it also slims
down the amount of memory allocated by BasicTestingSetup.
Use this opportunity to make SignatureCache RAII styled
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Add the possibility of giving more guidance to the creation of the
metadata and/or coins, so that the fuzzer gets the chance
to reach more error conditions in ActivateSnapshot and sometimes
successfully creates a valid snapshot.
This also changes the asserts for the success case that were outdated,
and only didn't result in a crash because the fuzzer wasn't able
to reach this code before.
Move its ownership to the ChainstateManager class.
Next to simplifying usage of the kernel library by no longer requiring
manual setup of the cache prior to using validation code, it also slims
down the amount of memory allocated by BasicTestingSetup.
f59e9057e2 depends: switch libevent to CMake (Cory Fields)
Pull request description:
Switches libevent in depends to be built with CMake.
ACKs for top commit:
TheCharlatan:
ACK f59e9057e2
willcl-ark:
ACK f59e9057e2
Tree-SHA512: 875bf9bc57653c78775a1f8192a2c964fea8f4490d733ff796d9efb00e786f0ca9a7c1a3fd610cda032273c4f2ae06394585b03567d5f241ab073c83a47cf927
33c48c106c validation: Check if mempool exists before asserting in ActivateSnapshot (TheCharlatan)
Pull request description:
The mempool is an optional component of the chainstate manager, so don't assume its presence and instead check if it is there first.
ACKs for top commit:
maflcko:
re-ACK 33c48c106c
fjahr:
ACK 33c48c106c
Tree-SHA512: 7a3568d5b7af45efa7bf54bae7bac1f00dc99bc9d47a744d73594f283c952be9500168f680d72f4aee09761da4e878ddca83ba675cdea8ee9e44eeff00ac09da
ce8094246e random: replace construct/assign with explicit Reseed() (Pieter Wuille)
2ae392d561 random: use LogError for init failure (Pieter Wuille)
97e16f5704 tests: make fuzz tests (mostly) deterministic with fixed seed (Pieter Wuille)
2c91330dd6 random: cleanup order, comments, static (Pieter Wuille)
8e31cf9c9b net, net_processing: use existing RNG objects more (Pieter Wuille)
d5fcbe966b random: improve precision of MakeExponentiallyDistributed (Pieter Wuille)
cfb0dfe2cf random: convert GetExponentialRand into rand_exp_duration (Pieter Wuille)
4eaa239dc3 random: convert GetRand{Micros,Millis} into randrange (Pieter Wuille)
82de1b80d9 net: use GetRandMicros for cache expiration (Pieter Wuille)
ddc184d999 random: get rid of GetRand by inlining (Pieter Wuille)
e2d1f84858 random: make GetRand() support entire range (incl. max) (Pieter Wuille)
810cdf6b4e tests: overhaul deterministic test randomness (Pieter Wuille)
6cfdc5b104 random: convert XoRoShiRo128PlusPlus into full RNG (Pieter Wuille)
8cc2f45065 random: move XoRoShiRo128PlusPlus into random module (Pieter Wuille)
8f5ac0d0b6 xoroshiro128plusplus: drop comment about nonexisting copy() (Pieter Wuille)
8924f5120f random: modernize XoRoShiRo128PlusPlus a bit (Pieter Wuille)
ddb7d26cfd random: add RandomMixin::randbits with compile-known bits (Pieter Wuille)
21ce9d8658 random: Improve RandomMixin::randbits (Pieter Wuille)
9b14d3d2da random: refactor: move rand* utilities to RandomMixin (Pieter Wuille)
40dd86fc3b random: use BasicByte concept in randbytes (Pieter Wuille)
27cefc7fd6 random: add a few noexcepts to FastRandomContext (Pieter Wuille)
b3b382dde2 random: move rand256() and randbytes() to .h file (Pieter Wuille)
493a2e024e random: write rand256() in function of fillrand() (Pieter Wuille)
Pull request description:
This PR contains a number of vaguely-related improvements to the random module.
The specific changes and more detailed rationale is in the commit messages, but the highlights are:
* `XoRoShiRo128PlusPlus` (previously a test-only RNG) moves to random.h and becomes `InsecureRandomContext`, which is even faster than `FastRandomContext` but non-cryptographic. It also gets all helper randomness functions (`randrange`, `fillrand`, ...), making it a lot more succinct to use.
* During tests, **all** randomness is made deterministic (except for `GetStrongRandBytes`) but non-repeating (like `GetRand()` used to be when `g_mock_deterministic_tests` was used), either fixed, or from a random seed (overridden by env var).
* Several infrequently used top-level functions (`GetRandMillis`, `GetRandMicros`, `GetExponentialRand`) are converted into member functions of `FastRandomContext` (and `InsecureRandomContext`).
* `GetRand<T>()` (without argument) can now return the maximum value of the type (previously e.g. `GetRand<uint32_t>()` would never return 0xffffffff).
ACKs for top commit:
achow101:
ACK ce8094246e
maflcko:
re-ACK ce8094246e🐈
hodlinator:
ACK ce8094246e
dergoegge:
utACK ce8094246e
Tree-SHA512: 79bc0cbafaf27e95012c1ce2947a8ca6f9a3c78af5f1f16e69354b6fc9b987a28858adf4cd356dc5baf21163e9af8dcc24e70f8d7173be870e8a3ddcdd47c02c
dea7afd5e4 lint: remove unneeded trailing line fix (willcl-ark)
4d942547a8 lint: ignore files ignored by git in mlc (willcl-ark)
Pull request description:
Updating to MLC v0.18.0 includes a new feature which will ignore all files ignored by git: `mlc --gitignore`.
This helps avoid false-positives flagged by this linter in non-project files, such as a developer might expect to have in their working directory (e.g. guix-builds, python venvs, etc.)
Top commit has no ACKs.
Tree-SHA512: 1752448e0c85abd3c73570a17cc69294de2248d7773c6499833ae33806f6c03f3f345261aa7b855a557b45982fbdcb8190e758d087c43b4fb0254fbb39173432
Updating to MLC v0.18.0 includes a new feature which will ignore all
files ignored by git: `--gitignore`.
This helps avoid false-positives flagged by this linter in non-project
files, such as a developer might expect to have in their directory (e.g.
guix-builds, python venvs, etc.)
This test checks that we can successfully process PSBTs and opt out of
finalization.
Previously trying to call `walletprocesspsbt` would attempt to
auto-finalize (as a convenience), and would not permit opt-out of
finalization, instead aborting via `CHECK_NONFATAL`.
2342b46c45 test: Add coverage for getchaintxstats in assumeutxo context (Fabian Jahr)
faf2a6750b rpc: Reorder getchaintxstats output (MarcoFalke)
fa2dada0c9 rpc: Avoid getchaintxstats invalid results (MarcoFalke)
Pull request description:
The `getchaintxstats` RPC reply during AU background download may return non-zero, but invalid, values for `window_tx_count` and `txrate`.
For example, `txcount` may be zero for a to-be-downloaded block, but may be non-zero for an ancestor block which is already downloaded. Thus, the values returned may be negative (and cause intermediate integer sanitizer violations).
Also, `txcount` may be accurate for the snapshot base block, or a descendant of it. However it may be zero for an ancestor block that still needs to be downloaded. Thus, the values returned may be positive, but wrong.
Fix all issues by skipping the returned value if either `txcount` is unset (equal to zero).
Also, skip `txcount` in the returned value, if it is unset (equal to zero).
Fixes https://github.com/bitcoin/bitcoin/issues/29328
ACKs for top commit:
fjahr:
re-ACK 2342b46c45
achow101:
ACK 2342b46c45
mzumsande:
ACK 2342b46c45
Tree-SHA512: 931cecc40ee5dc0f96be728db7eb297155f8343076cd29c8b8c050c99fd1d568b80f54c9459a34ca7a9489c2474c729796d00eeb1934d6a9f7b4d6a53e3ec430
926b8e39dc [doc] add release note for TRUC (glozow)
19a9b90617 use version=3 instead of v3 in debug strings (glozow)
881fac8e60 scripted-diff: change names from V3 to TRUC (glozow)
a573dd2617 [doc] replace mentions of v3 with TRUC (glozow)
089b5757df rename mempool_accept_v3.py to mempool_truc.py (glozow)
f543852a89 rename policy/v3_policy.* to policy/truc_policy.* (glozow)
Pull request description:
Adds a release note for TRUC policy which will be live in v28.0.
For clarity, replaces mentions of "v3" with "TRUC" in most places. Suggested in
- https://github.com/bitcoin/bitcoin/pull/29496#discussion_r1629749583
- https://github.com/bitcoin/bitcoin/pull/29496#discussion_r1624500904
I changed error strings from "v3-violation" to "TRUC-violation" but left v3 in the debug strings because I think it might be clearer for somebody who is debugging. Similarly, I left some variables unchanged because I think they're more descriptive this way, e.g. `tx_v3_from_v2_and_v3`. I'm happy to debate places that should or shouldn't be documented differently in this PR, whatever is clearest to everyone.
ACKs for top commit:
instagibbs:
reACK 926b8e39dc
achow101:
ACK 926b8e39dc
ismaelsadeeq:
Code review ACK 926b8e39dc
Tree-SHA512: 16c88add0a29dc6d1236c4d45f34a17b850f6727b231953cbd52eb9f7268d1d802563eadfc8b7928c94ed3d7a615275dd103e57e81439ebf3ba2b12efa1e42af
2f9bde69f4 test: Remove unnecessary restart in assumeutxo test (Fabian Jahr)
19ce3d407e assumeutxo: Check snapshot base block is not marked invalid (Fabian Jahr)
80315c0118 refactor: Move early loadtxoutset checks into ActiveSnapshot (Fabian Jahr)
Pull request description:
This was discovered in a discussion in #29996
If the base block of the snapshot is marked invalid or part of an invalid chain, we currently still load the snapshot and get stuck in a weird state where we have the snapshot chainstate but it will never connect to our valid chain.
While this scenario is highly unlikely to occur on mainnet, it still seems good to prevent this inconsistent state.
The behavior change described above is in the second commit.
The first commit refactors the early checks in the `loadtxoutset` RPC by moving them into `ActivateSnapshot()` in order to have the chance to cover them by unit tests in the future and have a more consistent interface. Previously checks were spread out between `rpc/blockchain.cpp` and `validation.cpp`. In order to be able to return the error message to users of the RPC, the return type of `ActivateSnapshot()` is changed from `bool` to `util::Result`.
The third commit removes an unnecessary restart introduced in #29428.
ACKs for top commit:
mzumsande:
re-ACK 2f9bde6
alfonsoromanz:
Re-ACK 2f9bde69f4. The RPC code looks much cleaner after the refactor. Also, it seems very useful to get the error message in the RPC response rather than having to rely on the logs in some scenarios if you are an RPC user.
achow101:
ACK 2f9bde69f4
Tree-SHA512: 5328dd88c3c7be3f1be97c9eef52ac3666c27188c30a798b3e949f3ffcb83be075127c107e4046f7f39f961a79911ea3d61b61f3c11e451b3e4c541c264eeed4
323ce30308 Moved the repeated -printpriority fetching out of AddToBlock (Lőrinc)
Pull request description:
`AddToBlock` was called repeatedly from `addPackageTxs` where the constant value of `printpriority` is recalculated every time.
<img src="https://github.com/bitcoin/bitcoin/assets/1841944/6fd89647-7b6c-4f44-bd04-98d16cd2a938">
This showed up during profiling of AssembleBlock, fetching it once in the constructor results in a small speed increase for many iterations.
> ./src/bench/bench_bitcoin --filter='AssembleBlock' --min-time=10000
before:
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 156,460.15 | 6,391.40 | 0.1% | 11.03 | `AssembleBlock`
after:
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 149,289.55 | 6,698.39 | 0.3% | 10.97 | `AssembleBlock`
---
The slight speedup shows up in CI as well:
<img src="https://github.com/bitcoin/bitcoin/assets/1841944/3be779c9-2dce-4a96-ae5f-cab5435bd72f">
ACKs for top commit:
maflcko:
ACK 323ce30308
achow101:
ACK 323ce30308
tdb3:
re ACK 323ce30308
furszy:
utACK 323ce30308
Tree-SHA512: c2a0aab429646453ad0470956529f1cac8c38778c4c53f82c92c6cbaaaeb69f3d3603c0014ff097844b151e9da7caa2371a4676244caea96527cb540e66825a3
8ec24bdad8 test: Added coverage to Block not found error using gettxoutsetinfo (kevkevinpal)
Pull request description:
#### Description
There were no tests that checked for the `Block not found` error called in `ParseHashOrHeight` when using `gettxoutsetinfo`, this change adds coverage to it.
You can see there are no tests that do the following by doing the below
`grep -nri "Block not found.*gettxoutsetinfo" ./test/functional/`
which leads to no results
ACKs for top commit:
achow101:
ACK 8ec24bdad8
tdb3:
ACK 8ec24bdad8
kristapsk:
ACK 8ec24bdad8
brunoerg:
crACK 8ec24bdad8
alfonsoromanz:
Re ACK 8ec24bdad8
Tree-SHA512: 2c61c681e7304c679cc3d7dd13af1b795780e85716c25c7423d68104e253d01271e048e21bc21be35dbc7ec1a4fde94e439542f3cfd669fe5a16478c5fa982ab
e38eadb2c2 test: change comments to `self.log.info` for `test_addnode_getaddednodeinfo` (brunoerg)
c838e3b610 test: add coverage for `node` field of `getaddednodeinfo` RPC (brunoerg)
Pull request description:
We currently do not test a successful call to `getaddednodeinfo` filtering by `node`, we only test it with an unknown address and checks whether it fails. This PR adds coverage to it.
ACKs for top commit:
kevkevinpal:
ACK [e38eadb](e38eadb2c2)
achow101:
ACK e38eadb2c2
tdb3:
re ACK e38eadb2c2
BrandonOdiwuor:
Code Review ACK e38eadb2c2
rkrux:
tACK [e38eadb](e38eadb2c2)
Tree-SHA512: e9f768b7aa86e58b0b0ced089ead57040ff9a5204493da1ab99c8bc897b6dcdce7c856855f74c52010fceef19af1e12a39eee9f8f2e7294b42476b6f980fe754
In the Tor case, this prevents us from telling the Tor daemon to send
our incoming connections from the Tor network to an address where we
do not listen (we tried to listen but failed probably because another
application is already listening).
In the other cases (IPv4/IPv6 binds) this also prevents unpleasant
surprises caused by continuing operations even on bind failure. For
example, another application may be listening on portX, bitcoind tries
to bind on portX and portY, only succeeds with portY and continues
operation leaving the user thinking that his bitcoind is listening on
portX whereas another application is listening (the error message in
the log could easily be missed).
Avoid having the functional testing framework start multiple `bitcoind`s
that try to listen on the same `127.0.0.1:18445` (Tor listen for
regtest) if `bind_to_localhost_only` is set to `False`.
Also fix a typo in `test-shell.md` related to `bind_to_localhost_only`.
Fixes https://github.com/bitcoin/bitcoin/issues/22727
If only `-bind=addr:port` is given (without `-bind=...=onion`) then we
would bind to `addr:port` _and_ to `127.0.0.1:8334` in addition which
may be unexpected, assuming the semantic of `-bind=addr:port` is
"bind _only_ to `addr:port`".
Change the above to not do the additional bind: if only
`-bind=addr:port` is given (without `-bind=...=onion`) then bind to
`addr:port` (only). If we are creating a Tor hidden service then use
`addr:port` as target (same behavior as before
https://github.com/bitcoin/bitcoin/pull/19991).
This allows disabling binding on the onion port.
Fixes https://github.com/bitcoin/bitcoin/issues/22726
Keep mentions of v3 in debug strings to help people who might not know
that TRUC is applied when version=3.
Also keep variable names in tests, as it is less verbose to keep v3 and v2.
f1478c0545 mempool: move LoadMempool/DumpMempool to node (Cory Fields)
6d242ff1e9 kernel: remove mempool_persist.cpp (Cory Fields)
Pull request description:
DumpMempool/LoadMempool are not necessary for the kernel.
Noticed while working on instantiated logging.
I suppose these could have been left in on purpose, but I'm assuming it was probably just an oversight.
ACKs for top commit:
TheCharlatan:
Re-ACK f1478c0545
glozow:
ACK f1478c0545
stickies-v:
ACK f1478c0545
Tree-SHA512: 5825da0cf2e67470524eb6ebe397eb90755a368469a25f184df99ab935b3eb6d89eb802b41a6c3661e869bba3bbfa8ba9d95281bc75ebbf790ec5d9d1f79c66f
Fix cases of calls to `FillPSBT` returning `complete=true` when it's not
the case.
This can happen when some inputs have been signed but the transaction is
subsequently modified, e.g. in the context of PayJoins.
Also fixes a related bug where a finalized hex string is attempted to be
added during `walletprocesspsbt` but a CHECK_NONFATAL causes an abort.
Reported in #30077.
9ec2c53701 Revert "test: p2p: check that connecting to ourself leads to disconnect" (Sebastian Falbesoner)
Pull request description:
As suggested in https://github.com/bitcoin/bitcoin/issues/30368#issuecomment-2200786670, this PR reverts the recently added test #30362 that causes frequent CI failures. A TODO is added in the functional test file to re-add it later when the race condition is fixed.
ACKs for top commit:
mzumsande:
utACK 9ec2c53701
brunoerg:
utACK 9ec2c53701
tdb3:
ACK 9ec2c53701
Tree-SHA512: df211ab194dc47f2ff8192f3827382974db922ed9fa54bc44fac75de4edfb3af43c1340cd5434b15b0b573f7b0ddd4451a0bbbbd7deaf7f4244e4865b9d5977e
The migration process reloads the wallet after all failures.
This commit tests the behavior by trying to obtain a new address
after a decryption failure during migration.
In SetupLegacyScriptPubKeyMan, a base LegacyDataSPKM will be created if
the database has the format "bdb_ro" (i.e. the wallet was opened only
for migration purposes).
All of the loading functions are now called with a LegacyDataSPKM object
instead of LegacyScriptPubKeyMan.
In order to load the necessary data for migrating a legacy wallet
without the full LegacyScriptPubKeyMan, move the data storage and
loading components to LegacyDataSPKM. LegacyScriptPubKeyMan now
subclasses that.
There are only a few call sites of these throughout the codebase, so
move the functionality into FastRandomContext, and rewrite all call sites.
This requires the callers to explicit construct FastRandomContext objects,
which do add to the verbosity, but also make potentially apparent locations
where the code can be improved by reusing a FastRandomContext object (see
further commit).
fa6beb8cfc ci: Clear unused /msan/llvm-project (MarcoFalke)
Pull request description:
Could help to fix the `no space left on device` that are sometimes seen.
ACKs for top commit:
theuni:
utACK fa6beb8cfc
Tree-SHA512: 0bedf4b26eed842c7bfa2caeac4df578cdbb00a658e8d0037b8b7b90150d8a9d1b8140437d1cf40b50d82a9085ea50cf9a010764c4439b2a03a457d399191319
The existing code uses GetRand(nMax), with a default value for nMax, where nMax is the
range of values (not the maximum!) that the output is allowed to take. This will always
miss the last possible value (e.g. GetRand<uint32_t>() will never return 0xffffffff).
Fix this, by moving the functionality largely in RandomMixin, and also adding a
separate RandomMixin::rand function, which returns a value in the entire (non-negative)
range of an integer.
The existing code provides two randomness mechanisms for test purposes:
- g_insecure_rand_ctx (with its wrappers InsecureRand*), which during tests is
initialized using either zeros (SeedRand::ZEROS), or using environment-provided
randomness (SeedRand::SEED).
- g_mock_deterministic_tests, which controls some (but not all) of the normal
randomness output if set, but then makes it extremely predictable (identical
output repeatedly).
Replace this with a single mechanism, which retains the SeedRand modes to control
all randomness. There is a new internal deterministic PRNG inside the random
module, which is used in GetRandBytes() when in test mode, and which is also used
to initialize g_insecure_rand_ctx. This means that during tests, all random numbers
are made deterministic. There is one exception, GetStrongRandBytes(), which even
in test mode still uses the normal PRNG state.
This probably opens the door to removing a lot of the ad-hoc "deterministic" mode
functions littered through the codebase (by simply running relevant tests in
SeedRand::ZEROS mode), but this isn't done yet.
Convert XoRoShiRo128PlusPlus into a full RandomMixin-based RNG class,
providing all utility functionality that FastRandomContext has. In doing so,
it is renamed to InsecureRandomContext, highlighting its non-cryptographic
nature.
To do this, a fillrand fallback is added to RandomMixin (where it is used by
InsecureRandomContext), but FastRandomContext still uses its own fillrand.
In many cases, it is known at compile time how many bits are requested from
randbits. Provide a variant of randbits that accepts this number as a template,
to make sure the compiler can make use of this knowledge. This is used immediately
in rand32() and randbool(), and a few further call sites.
The previous randbits code would, when requesting more randomness than available
in its random bits buffer, discard the remaining entropy and generate new.
Benchmarks show that it's usually better to first consume the existing randomness
and only then generate new ones. This adds some complexity to randbits, but it
doesn't weigh up against the reduced need to generate more randomness.
Rather than make all the useful types of randomness be exclusive to
FastRandomContext, move it to a separate RandomMixin class where it can be reused by
other RNGs.
A Curiously Recurring Template Pattern (CRTP) is used for this, to provide the ability
for individual RNG classes to override one or more randomness functions, without
needing the runtime-cost of virtual classes.
Specifically, RNGs are expected to only provide fillrand and rand64, while all others
are derived from those:
- randbits
- randrange
- randbytes
- rand32
- rand256
- randbool
- rand_uniform_delay
- rand_uniform_duration
- min(), max(), operator()(), to comply with C++ URBG concept.
55eea003af test: Make blockencodings_tests deterministic (AngusP)
4c99301220 test: Add ReceiveWithExtraTransactions Compact Block receive test. (AngusP)
4621e7cc8f test: refactor: Rename extra_txn to const empty_extra_txn as it is empty in all test cases (AngusP)
Pull request description:
This test uses the `extra_txn` (`vExtraTxnForCompact`) vector of optional orphan/conflicted/etc. transactions to provide transactions to a PartiallyDownloadedBlock that are not otherwise present in the mempool, and check that they are used.
This also covers a former nullptr deref bug that was fixed in #29752 (bf031a517c) where the `extra_txn` vec/circular-buffer was null-initialized and not yet filled when dereferenced in `PartiallyDownloadedBlock::InitData`.
ACKs for top commit:
marcofleon:
Code review ACK 55eea003af. I ran the `blockencodings` unit test and no issues with the new test case.
dergoegge:
Code review ACK 55eea003af
glozow:
ACK 55eea003af
Tree-SHA512: d7909c212bb069e1f6184b26390a5000dcc5f2b18e49b86cceccb9f1ec4f874dd43bc9bc92abd4207c71dd78112ba58400042c230c42e93afe55ba51b943262c
e009bf681c Don't use iterator addresses in IteratorComparator (dergoegge)
Pull request description:
See #29018.
Stability for `txorphan` is now >90%. `mini_miner` needs further investigation, stability still low (although slightly improved by this PR) at ~62%.
ACKs for top commit:
marcofleon:
Tested ACK e009bf681c. Using afl++, stability for `txorphan` went from 82% to ~94% and for `mini_miner` it went from 84% to 97%. I ran them both using the corpora in qa-assets.
glozow:
utACK e009bf681c
Tree-SHA512: 6d0a20fd7ceedca8e702d8adde5fca500d8b0187147aee8d43b4e9eb5176dcacf60180f42a7158f037d18dbb27e479b6c069a0f3c912226505cbff5aa073a415
4d81b4de33 fuzz: FuzzedSock::Recv() don't lose bytes from MSG_PEEK read (Vasil Dimov)
b51d75ea97 fuzz: simplify FuzzedSock::m_peek_data (Vasil Dimov)
Pull request description:
Problem:
If `FuzzedSock::Recv(N, MSG_PEEK)` is called then `N` bytes would be
retrieved from the fuzz provider, saved in `m_peek_data` and returned
to the caller (ok).
If after this `FuzzedSock::Recv(M, 0)` is called where `M < N`
then the first `M` bytes from `m_peek_data` would be returned
to the caller (ok), but the remaining `N - M` bytes in `m_peek_data`
would be discarded/lost (not ok). They must be returned by a subsequent
`Recv()`.
To resolve this, only remove the head `N` bytes from `m_peek_data`.
---
This is a followup to https://github.com/bitcoin/bitcoin/pull/30211, more specifically:
https://github.com/bitcoin/bitcoin/pull/30211#discussion_r1633199919https://github.com/bitcoin/bitcoin/pull/30211#discussion_r1633216366
ACKs for top commit:
marcofleon:
ACK 4d81b4de33. Tested this with the I2P fuzz target and there's no loss in coverage. I think overall this is an improvement in the robustness of `Recv` in `FuzzedSock`.
dergoegge:
Code review ACK 4d81b4de33
brunoerg:
utACK 4d81b4de33
Tree-SHA512: 73b5cb396784652447874998850e45899e8cba49dcd2cc96b2d1f63be78e48201ab88a76cf1c3cb880abac57af07f2c65d673a1021ee1a577d0496c3a4b0c5dd
fa1bc7c88b scripted-diff: Log parameter interaction not thrice (MarcoFalke)
fafb7875e1 doc: Fix outdated dev comment about logging (MarcoFalke)
Pull request description:
Seems a bit overkill to log the words "parameter interaction" thrice, when at least once is enough. So do that.
Before:
```
2024-06-28T15:30:57Z [init.cpp:745] [InitParameterInteraction] InitParameterInteraction: parameter interaction: -connect or -maxconnections=0 set -> setting -dnsseed=0
```
After:
```
2024-06-28T15:47:27Z [init.cpp:745] [InitParameterInteraction] parameter interaction: -connect or -maxconnections=0 set -> setting -dnsseed=0
ACKs for top commit:
paplorinc:
ACK fa1bc7c88b
fjahr:
utACK fa1bc7c88b
TheCharlatan:
Nice, ACK fa1bc7c88b
hodlinator:
utACK fa1bc7c88b
Tree-SHA512: 83cd92e20dffa38737d4fd31764481284383e12671d9e4b33cfa496743c95c10921a113b1da2caafeb44fca3759a28a8e230df5e30c29fb55d5854ff1531382c
5d2fb14baf test: p2p: check that connecting to ourself leads to disconnect (Sebastian Falbesoner)
Pull request description:
This small PR adds test coverage for the scenario of connecting to ourself, leading to an immediate disconnect:
2f6dca4d1c/src/net_processing.cpp (L3729-L3735)
This logic has been first introduced by Satoshi in October 2009, together with a couple of other changes and a version bump to "v0.1.6 BETA" (see commit cc0b4c3b62).
ACKs for top commit:
kevkevinpal:
tACK [5d2fb14](5d2fb14baf)
maflcko:
ACK 5d2fb14baf
fjahr:
tACK 5d2fb14baf
tdb3:
ACK 5d2fb14baf
Tree-SHA512: 30fb8c82cef94701affeca386ecd59daa32231635fa770fe225feb69fdab2ffedbfa157edd563f65099ec209f2dafffc1154f7f9292c2ea68bbd114750904875
AddToBlock was called repeatedly from `addPackageTxs` where the constant value of `printpriority` is recalculated every time.
Since its behavior was changed in 400b151, I've named the variable accordingly.
This showed up during profiling of AssembleBlock, fetching it once in the constructor results in a measurable speed increase for many iterations.
> ./src/bench/bench_bitcoin --filter='AssembleBlock' --min-time=1000
before:
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 155,558.97 | 6,428.43 | 0.1% | 1.10 | `AssembleBlock`
after:
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 148,083.68 | 6,752.94 | 0.1% | 1.10 | `AssembleBlock`
Co-authored-by: furszy <mfurszy@protonmail.com>
Otherwise, if the background tip is not an ancestor of the snapshot, blocks in between that ancestor up to the height of the background tip will never be requested.
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
Co-authored-by: Alfonso Roman Zubeldia <19962151+alfonsoromanz@users.noreply.github.com>
The "connect to ourself" detection logic has been first introduced
by Satoshi in October 2009, together with a couple of other changes
and a version bump to "v0.1.6 BETA" (see commit
cc0b4c3b62).
a74b0f93ef Have testBlockValidity hold cs_main instead of caller (Sjors Provoost)
f6dc6db44d refactor: use CHECK_NONFATAL to avoid single-use symbol (Sjors Provoost)
5fb2b70489 Drop unneeded lock from createNewBlock (Sjors Provoost)
75ce7637ad refactor: testBlockValidity make out argument last (Sjors Provoost)
83a9bef0e2 Add missing include for mining interface (Sjors Provoost)
Pull request description:
Followups from #30200
Fixes:
- `std::unique_ptr` needs `#include <memory>` (noticed while working on #30332, which has fewer includes than its parent PR that I originally tested with)
- Drop lock from createNewBlock that was spuriously added
- Have testBlockValidity hold cs_main instead of caller (also fixes a race condition in test-only code)
Refactor:
- Use CHECK_NONFATAL to avoid single-use symbol (refactor)
- move output argument `state` to the end of `testBlockValidity`, see https://github.com/bitcoin/bitcoin/pull/30200#discussion_r1647987176
ACKs for top commit:
AngusP:
Code Review ACK a74b0f93ef
itornaza:
Tested ACK a74b0f93ef
ryanofsky:
Code review ACK a74b0f93ef. Just new error string is added since last review, and a commit message was updated
Tree-SHA512: 805e133bb59303fcee107d6f02b3e2761396c290efb731a85e6a29ae56b4b1b9cd28ada9629e979704dcfd98cf35034e7e6b618e29923049eb1eca2f65630e41
73f0a6cbd0 doc: detail -rpccookieperms option (willcl-ark)
d2afa2690c test: add rpccookieperms test (willcl-ark)
f467aede78 init: add option for rpccookie permissions (willcl-ark)
7df03f1a92 util: add perm string helper functions (willcl-ark)
Pull request description:
This PR picks up #26088 by aureleoules which adds a bitcoind launch option `-rpccookieperms` to set the file permissions of the cookie generated by bitcoin core.
Example usage to make the generated cookie group-readable: `./src/bitcoind -rpccookieperms=group`.
Accepted values for `-rpccookieperms` are `[owner|group|all]`. We let `fs::perms` handle platform-specific permissions changes.
ACKs for top commit:
achow101:
ACK 73f0a6cbd0
ryanofsky:
Code review ACK 73f0a6cbd0. Main change since last review is no longer throwing a skip exception in the rpc test on windows, so other checks can run after it, and overall test result is passing, not skipped. Also were clarifying renames and documentation improvements.
tdb3:
cr ACK 73f0a6cbd0
Tree-SHA512: e800d59a44aca10e1c58ca69bf3fdde9f6ccf5eab4b7b962645af6d6bc0cfa3a357701e409c8c60d8d7744fcd33a91e77ada11790aa88cd7811ef60fab86ab11
a9c7300135 move-only: refactor CreateTransactionInternal (josibake)
adc6ab25bb wallet: use CRecipient instead of CTxOut (josibake)
Pull request description:
Broken out from #28201
---
In order to estimate fees properly, we need to know what the final serialized transaction size will be. This PR refactors `CreateTransactionInternal` to:
* Get the serialized size directly from the `CRecipient`: this sets us up in a future PR to calculate the serialized size of silent payment `CTxDestinations` (see 797e21c8c1)
* Use the new `GetSerializeSizeForRecipient` to move the serialize size calculation to *before* coin selection and the output creation to *after* coin selection: this also sets us up for silent payments sending in a future PR in that silent payments outputs cannot be created until after the inputs to the transaction have been selected
Aside from the silent payments use case, I think this structure logically makes more sense. As a reminder, move-only commits are best reviewed with something like `git diff -w --color-moved=dimmed-zebra`
ACKs for top commit:
S3RK:
reACK a9c7300135
achow101:
ACK a9c7300135
rkrux:
tACK [a9c7300](a9c7300135)
Tree-SHA512: 412e1764b98f7428c8530c3a68f55e32063d6b66ab2ff613e1c7e12d49b049807cb60055cfe7f7e8ffe7ac7f0f9931427cbfd3efe7d4f97a5a0f6d1bf1aaac58
Introduce diff_addrs subcommand as means for a Bitcoin-centric
comparison of two ASMaps.
In addition to two ASMaps, the subcommand reads addresses from
a (getnodeaddresses-compatible) file, and provides information on
addresses that have mismatching ASN according to the two ASMaps.
This allows a transaction's weight to be bound under a certain
weight if possible and desired. This can be beneficial for future
RBF attempts, or whenever a more restricted spend topology is
desired.
Co-authored-by: Greg Sanders <gsanders87@gmail.com>
PermsToSymbolicString will convert from fs::perms to string type
'rwxrwxrwx'.
InterpretPermString will convert from a user-supplied "perm string" such
as 'owner', 'group' or 'all, into appropriate fs::perms.
- This change ensures consistency in transaction size and weight calculation
within the wallet and prevents conversion overflow when calculating
`max_selection_weight`.
`CoinGrinder` will also produce change output, listing all the
Coin selection algorithms that produces change output is not maintainable,
just infer that remaining algorithms all might produce change.
- This commit renames the coin selection algorithms input parameter `max_weight`
to `max_selection_weight` for clarity.
The parameter represent the maximum weight of the UTXOs the coin selection algorithm
should select, not the transaction maximum weight.
- The commit updates the parameter docstring to provide correct description.
- Also updates coin selection unit and fuzzing test variables to match the new name.
The goal of interfaces is to eventually run in their own process,
so we can't use EXCLUSIVE_LOCKS_REQUIRED in their declaration.
However TestBlockValidaty will crash (in its call to ConnectBlock)
if the tip changes from under the proposed block.
Have the testBlockValidity implementation hold the lock instead,
and non-fatally check for this condition.
7d3662fbe3 i2p: fix log when an interruption happens during `Accept` (brunoerg)
3d3a83fab2 i2p: log errors properly according to their severity (brunoerg)
Pull request description:
This PR improves and fixes i2p logs (joint work with vasild).
- It replaces `LogPrint` to `LogPrintLevel` so we can log according to the severity.
- Fix log when interruption happens during `Accept`. Before this PR, when an interruption happens, it just logs "Error accepting:", no reason is logged as it does for other situations. This PR changes it to log "Accept interrupted".
- Log errors according to the severity. Stuff like creating SAM session, destroying SAM session, etc... are logged as 'debug'.
ACKs for top commit:
achow101:
ACK 7d3662fbe3
marcofleon:
ACK 7d3662fbe3.
vasild:
ACK 7d3662fbe3
Tree-SHA512: 1c3d92108dbc22833f37a78e18b4efd723433d10f28166d17c74eab884cd97e908b4e0a0908fd16288df895eb2eb480f781de37b2ec6a6d414abfb71e0c86fe2
72b226882f wallet: notify when preset + automatic inputs exceed max weight (furszy)
Pull request description:
Small change. Found it while finishing my review on #29523. This does not interfere with it.
Basically, we are erroring out early when the automatic coin selection process exceeds the maximum weight, but we are not doing so when the user-preselected inputs combined with the wallet-selected inputs exceed the maximum weight.
This change avoids signing all inputs before erroring out and introduces test coverage for `fundrawtransaction`.
ACKs for top commit:
achow101:
ACK 72b226882f
tdb3:
re ACK for 72b226882f
rkrux:
tACK [72b2268](72b226882f)
ismaelsadeeq:
utACK 72b226882f
Tree-SHA512: d77be19231023383a9c79a5d66b642dcbc6ebfc31a363e0b9f063c44898720a7859ec211cdbc0914ac7a3bfdf15e52fb8fc20d97f171431f70492c0f159dbc36
c0b5ea5901 build: Drop redundant `sys/sysctl.h` header check (Hennadii Stepanov)
Pull request description:
The `AC_CHECK_HEADERS` macro defines `HAVE_SYS_SYSCTL_H` if the `sys/sysctl.h` header is found. However, in the source code, this header is guarded by `HAVE_SYSCTL` and `HAVE_SYSCTL_ARND` macros, which have their own checks. Since `HAVE_SYS_SYSCTL_H` is not used, we can skip the `AC_CHECK_HEADERS(... sys/sysctl.h ...)` check.
ACKs for top commit:
laanwj:
ACK c0b5ea5901
fanquake:
ACK c0b5ea5901 - we could got the other way, and add nested #defs, but that doesn't seem worthwhile.
Tree-SHA512: 73bc4bbfc5c457cd2c38e40f8e57d2a70c06ef661d76d4148d683d262be45b9405b8cda1958ac611c312ca7d9e2f9624cf2cac1b61f1008af0856875c62f0eac
b5fc6d46a3 guix: use glibc 2.31 (fanquake)
Pull request description:
Set minimum required glibc to 2.31.
The glibc 2.31 branch is still maintained: https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/release/2.31/master.
Remove the stack-protector check from test-security-check, as the test
no-longer fails, and given the control we have of the end, the actual
security-check test seems sufficient (this might also be applied to some
of the other checks).
Drops runtime support for Ubuntu Bionic 18.04 and RHEL-8 from the release binaries.
ACKs for top commit:
TheCharlatan:
ACK b5fc6d46a3
Tree-SHA512: ba7e727240fa0ebebfb8b749024c71cbfdec37c33b39627866d78f9318ccdc687fd5103a63ee0e98cf809d9954dde56b1b305691c33d1de275ed0519f716c921
2721d64989 chainparams: Add achow101 DNS seeder (Ava Chow)
Pull request description:
I wrote a [DNS seeder](https://github.com/achow101/dnsseedrs) and have been running it for the past 2 months now. I believe it is ready/good enough to be used as an additional DNS seeder for all of our supported public networks.
ACKs for top commit:
laanwj:
ACK 2721d64989
1440000bytes:
~~reACK 2721d64989~~
mzumsande:
ACK 2721d64989
willcl-ark:
reACK 2721d64989
Tree-SHA512: 857a6cf7dd33962f0008a89db4d6b57d3c6aa622704cdcca6ab710babeead3a2970d9a6fa190949c7bbf7cb7d006e814d6314be3d8c8180eed29013c7c1ac7e1
3ab2520190 contrib: Fixup verify-binaries OS platform parsing (Ben Westgate)
Pull request description:
Closes#30145.
This PR solves two major issues in the `parse_version_string` function of verify-binaries:
1. `-aarch64` binaries cannot be specifically downloaded. The -platform string gets interpreted as a release candidate that doesn't exist due to containing sub-string "rc".
2. Specifying a platform with a "-" in the name causes the parser to ignore both "-platform" AND "-rcN" and download the potentially wrong (non-rc) version for every platform. This also prevented specifying just one platform binary the user wished to download.
It also updates the accompanying `test.py` to cover problem two and adds two examples that were formerly broken to `README.md` to show what is now possible. Including the most useful case of downloading only 1 specific platform's binary.
This improves the Bitcoin verify-binaries tools user experience by not:
1. Failing to download for inexplicable reasons,
2. Downloading more files than what the user told it to, or in the worst case
3. Downloading only the wrong files.
* A test was added to cover the command `verify-binaries/verify.py pub 22.0-x86_64-linux-gnu.tar.gz` which checks that _bitcoin-22.0-x86_64-linux-gnu.tar.gz_ downloads successfully AND ONLY _bitcoin-22.0-x86_64-linux-gnu.tar.gz_ downloads.
* The steps to reproduce each bug are in the referenced issue #30145. Explanation of the potential issue as well as reasoning for the way the bug was fixed are in my commit descriptions.
* This delivers the promised feature of "only download the binaries for a certain platform", by allowing strings with '-' to be accepted, allowing for single file downloads for any specific platform which was not always possible before.
* Removes 6 lines of code from the offending `parse_version_string` function, while fixing the bugs/errors, and extending the functionality to be practical for users with slow connections.
* Makes the error message more helpful when no file matches the provided platform string, now prints "Did you mean: `closest-match`" to help correct typos.
Thanks for reading my PR. I look forward to getting this helpful tool in its best shape yet.
Log of this branch passing the new test.py:
```
python3 test.py
✓ 'Nonexistent version should fail' passed
✓ 'Malformed version should fail' passed
✓ '--min-good-sigs 20 should fail' passed
- testing verification (22.0-x86_64-linux-gnu.tar.gz)
✓ '22.0-x86_64-linux-gnu.tar.gz should succeed' passed
- testing verification (22.0)
✓ '22.0 should succeed' passed
```
Log of master failing the new test.py:
```
python3 test.py
✓ 'Nonexistent version should fail' passed
✓ 'Malformed version should fail' passed
✓ '--min-good-sigs 20 should fail' passed
- testing verification (22.0-x86_64-linux-gnu.tar.gz)
✓ '22.0-x86_64-linux-gnu.tar.gz should succeed' passed
Traceback (most recent call last):
File "/home/ben/Documents/GitHub/bitcoin/contrib/verify-binaries/test.py", line 74, in <module>
main()
File "/home/ben/Documents/GitHub/bitcoin/contrib/verify-binaries/test.py", line 27, in main
assert len(v) == 1
^^^^^^^^^^^
AssertionError
```
ACKs for top commit:
stickies-v:
re-ACK 3ab2520190
willcl-ark:
re-ACK 3ab2520190
Tree-SHA512: 6093228bb876cd0ac84d1cd2630074e17a3f09c4b23325b9419d859a5721a802f928844575233b135df52b882287dd18d6fadf4419d88ec0a2cdf82db315329e
The ci "test-each-commit" job fetches the PR branch being tested with a depth of (# of commits in PR + 2), and then tries to run tests on commits after the most recent merge commit.
When a PR is opened against a bitcoin core branch, a merge commit is always guaranteed to exist within the fetch depth, because bitcoin core branches always point at merge commits.
However, in fork repositories, pull requests can be opened based on other branches that don't contain recent merge commits, and this will currently cause git rev-list to fail with fatal: bad revision '^^@'.
Work around this problem by not requiring a recent merge commit, and just testing on all fetched commits if a merge commit can't be found.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Parse platform strings with "-" or '.' correctly such as "linux-gnu" or
"x86_64-linux-gnu.tar.gz" to download the matching files or file. String
partition() is used to tolerate more dashes. Update `VERSION_EXAMPLE`
with a new string parsed correctly now. Fix "-aarch64" interpreted as a
release candidate due to sub-string "rc", causing all downloads to fail.
Now "rc" must immediately follow first "-" to indicate an [-rc] string.
Local variables `version_rc`, `version_os` renamed to `rc`, `platform`.
If "-rcN" is specified, `platform` is reassigned to remove the '-rcN'.
Changes are useful to only download one bitcoin core binary on slow
connections. Making `verify.py pub` more intuitive, robust, and
versatile. Closes#30145
When user types a platform string not found in any filename lets help
and say the platform closest to what they typed in a `f"No files
matched the platform specified. Did you mean: {closest_match}"` log.
Improves UX when unaware how we name our files.
Uses the difflib Python built-in which was already imported elsewhere.
Update test.py to test single file verification
verify-binaries/verify.py can accept an entire filename filter for its
"-platform" parameter now so let us test that it succeeds and downloads
and verifies only one file. `verify.py pub 22.0-x86_64-linux-gnu.tar.gz`
should get and verify only the requested binary. It is placed before the
existing <version> wide verification as it is a faster test and possibly
easier to break.
Update doc with examples now possible after bugfix
Add example to show release candidates now work with "-platform" strings
containing "-" and string provided can be from the middle of filename:
`./contrib/verify-binaries/verify.py --json pub 23.0-rc5-linux-gnu`
Change example 5 to not match example 3.
New examples to show platform can now be provided specifically enough to
download only a single binary down to its file extension:
`./contrib/verify-binaries/verify.py pub 25.2-x86_64-linux`
`./contrib/verify-binaries/verify.py pub 24.1-rc1-darwin`
`./contrib/verify-binaries/verify.py pub 27.0-win64-setup.exe`
This is the most common use if not verifying all files so users see it
as the first example for "only download the binaries for a certain
architecture and/or platform". Downloading one file is intuitively what
most will think this meant and this change delivers on that expectation.
Co-authored-by: stickies-v
Now that upstream has gotten around to fixing this. We don't need any
more of the patch, and it likely wont apply to our version of Qt in any
case. See:
3388de698b.
a9716c53f0 rpc: call IsInitialBlockDownload via miner interface (Sjors Provoost)
dda0b0834f rpc: minize getTipHash() calls in gbt (Sjors Provoost)
7b4d3249ce rpc: call processNewBlock via miner interface (Sjors Provoost)
9e228351e7 rpc: getTransactionsUpdated via miner interface (Sjors Provoost)
64ebb0f971 Always pass options to BlockAssembler constructor (Sjors Provoost)
4bf2e361da rpc: call CreateNewBlock via miner interface (Sjors Provoost)
404b01c436 rpc: getblocktemplate getTipHash() via Miner interface (Sjors Provoost)
d8a3496b5a rpc: call TestBlockValidity via miner interface (Sjors Provoost)
8ecb681678 Introduce Mining interface (Sjors Provoost)
Pull request description:
Introduce a `Mining` interface for the `getblocktemplate`, `generateblock` and other mining RPCs to use now, and for Stratum v2 to use later.
Suggested here: https://github.com/bitcoin/bitcoin/pull/29346#issuecomment-2108528652
The selection of methods added to the interface is mostly based on what the Template Provider in #29432 uses. It could be expanded further so that `rpc/mining.cpp` no longer needs `EnsureMemPool` and `EnsureChainman`.
This PR should be a pure refactor.
ACKs for top commit:
tdb3:
re ACK a9716c53f0
itornaza:
Code review and std-tests ACK a9716c53f0
ryanofsky:
Code review ACK a9716c53f0 with one minor suggestion in case you update. Only changes since last review were other small changes to the interface.
Tree-SHA512: cf97f87d6e9ed89da3835a0730da3b24a7b14c8605ea221149103a5915e79598cf082a95f2bc88e33f1c450e3d4aad88aed1163a29195acca88bcace055af724
e3dc64f499 build: add -Wundef (fanquake)
82b43955f7 refactor: use #ifdef HAVE_SOCKADDR_UN (fanquake)
40cd7585a0 randomenv: use ifdef over if (fanquake)
7839503b30 zmq: use #ifdef ENABLE_ZMQ (fanquake)
79e197b175 build: Suppress warnings from boost and capnproto in multiprocess code (Ryan Ofsky)
Pull request description:
Turn on `-Wundef`.
[> Warn if an undefined identifier is evaluated in an #if directive. Such identifiers are replaced with zero.](https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wundef).
Note that this is still beneficial with CMake, and may even be nice to have enabled prior, to catch any change in behaviour.
If we end up with this enabled, it should probably be enough to fix#16419.
ACKs for top commit:
hebasto:
ACK e3dc64f499, I have reviewed the code and it looks OK.
Tree-SHA512: 73436ead07f3a09ba0d30f7105df50d9b2ec8452f11e866bc1c7ebc10c005772ee77fedaa125f444175663c04dfc472f98c2699c63711da356089b66a8cc3e0a
The `AC_CHECK_HEADERS` macro defines `HAVE_SYS_SYSCTL_H` if the
`sys/sysctl.h` header is found. However, in the source code, this header
is guarded by `HAVE_SYSCTL` and `HAVE_SYSCTL_ARND` macros, which have
their own checks. Since `HAVE_SYS_SYSCTL_H` is not used, we can skip the
`AC_CHECK_HEADERS(... sys/sysctl.h ...)` check.
da205dda14 ci: increase available ccache size to 300MB (Max Edwards)
4ecbbd9b7f ci: add option for running tests without volume (Max Edwards)
Pull request description:
Fixes: https://github.com/bitcoin/bitcoin/pull/30193#discussion_r1645950272
Cache wasn't being saved when run on GHA because the default behaviour of the CI script was to store cache items in a docker volume. This works on Cirrus CI as the volumes are shared but it does not work on Github Actions in which each run is ephemeral.
Kept the default behaviour the same so hopefully this continues to work for the Cirrus CI jobs.
ACKs for top commit:
maflcko:
utACK da205dda14
hebasto:
ACK da205dda14.
Tree-SHA512: 3b35482c0628adb60574a1462181ecfcb06cb237ed48beb6fe9aa51110be82f863dc9147e7f8d82960450aa6ecc3a24a70e3c8283fd24cdad075dbfb8fc93095
This test type is represented using SEND_NO_AAD. If AAD of the first encrypted packet
sent after the garbage terminator (optional decoy packet/version packet) hasn't been
filled, disconnection happens.
This test type is represented using WRONG_GARBAGE.
Here, garbage bytes sent to TestNode are assumed to be tampered with and
do not correspond to the garbage bytes which P2PInterface calculated and
uses.
This test type is represented using WRONG_GARBAGE_TERMINATOR.
since the wrong garbage terminator is sent to TestNode, TestNode
will interpret all of the gabage bytes, wrong garbage terminator,
decoy messages and version packet it receives as garbage bytes.
If the length of all these is more than 4095 + 16, it will result
in a missing garbage terminator error. otherwise, it will result
in a V2 handshake timeout error.
Send only MAX_GARBAGE_LEN//2 bytes of garbage data to TestNode
so that the total length received by the TestNode is at max
= (MAX_GARBAGE_LEN//2) + 16 + 10*120 + 20 = 3283 bytes
(which is less than 4095 + 16 bytes) and we get a consistent
V2 handshake timeout error message.
If we do not limit the garbage length sent, we will intermittently
get both missing garbage terminator error and V2 handshake
timeout error based on the garbage length and decoy packets length
which are chosen at random.
GetFirstStoredBlock is generalized to check for any data status with a
status mask that needs to be passed as a parameter. To reflect this the
function is also renamed to GetFirstBlock.
Co-authored-by: stickies-v <stickies-v@protonmail.com>
c67f215ea5 ci: clarify Cirrus self-hosted workers setup (Sjors Provoost)
Pull request description:
Taken from #29274 (except for two paragraphs that require the other commits in that PR).
ACKs for top commit:
maflcko:
ACK c67f215ea5
tdb3:
ACK c67f215ea5
Tree-SHA512: 321cc327bfbf0b8e55eb84cb259cf55a66d480c99abe6824248f8b5fdb9a31a079f7ce2c5a6c27afa809aa343d1efb0744a19dd379c17162b21fdf24b6b8836b
5729dbbb74 refactor: remove extraneous lock annotations from function definitions (Cory Fields)
Pull request description:
These annotations belong in the declarations rather than the definitions. While harmless now, future versions of clang may warn about these.
Discovered these using the upstream WIP: https://github.com/llvm/llvm-project/pull/67520
ACKs for top commit:
instagibbs:
ACK 5729dbbb74
maflcko:
ACK 5729dbbb74🦋
Tree-SHA512: c82c6b269dd353b140cbb36b5519ab2637e54034f159d6ad3eb78c6f4019aa053a5973c626395f0ed3366b9f4117ecc4fe7926b83e9714b1e21c97d5e4bed8d7
randomenv.cpp:48:5: warning: 'HAVE_VM_VM_PARAM_H' is not defined, evaluates to 0 [-Wundef]
randomenv.cpp:51:5: warning: 'HAVE_SYS_RESOURCES_H' is not defined, evaluates to 0 [-Wundef]
randomenv.cpp:424:5: error: 'HAVE_SYSCTL' is not defined, evaluates to 0 [-Werror,-Wundef]
Without this change there are errors from boost like:
/ci_container_base/depends/i686-pc-linux-gnu/include/boost/signals2/expired_slot.hpp:23:28: error: 'what' overrides a member function but is not marked 'override' [-Werror,-Wsuggest-override]
/ci_container_base/depends/i686-pc-linux-gnu/include/boost/signals2/detail/signal_template.hpp:750:32: error: 'lock_pimpl' overrides a member function but is not marked 'override' [-Werror,-Wsuggest-override]
/ci_container_base/depends/i686-pc-linux-gnu/include/boost/signals2/connection.hpp:150:22: error: 'connected' overrides a member function but is not marked 'override' [-Werror,-Wsuggest-override]
There do not seem to be errors from capnproto currently, but add a suppression
for it, too, to be consistent with other libraries.
1245d1388b netbase: extend CreateSock() to support creating arbitrary sockets (Vasil Dimov)
Pull request description:
Allow the callers of `CreateSock()` to pass all 3 arguments to the `socket(2)` syscall. This makes it possible to create sockets of any domain/type/protocol. In addition to extending arguments, some extra safety checks were put in place.
The need for this came up during the discussion in https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1618837102
ACKs for top commit:
achow101:
ACK 1245d1388b
tdb3:
re ACK 1245d1388b
theStack:
re-ACK 1245d1388b
Tree-SHA512: cc86b56121293ac98959aed0ed77812d20702ed7029b5a043586f46e74295779c5354bb0d5f9e80be6c29e535df980d34c1dbf609064fb7ea3e5ca0f0ed54d6b
6eecba475e net_processing: make MaybePunishNodeFor{Block,Tx} return void (Pieter Wuille)
ae60d485da net_processing: remove Misbehavior score and increments (Pieter Wuille)
6457c31197 net_processing: make all Misbehaving increments = 100 (Pieter Wuille)
5120ab1478 net_processing: drop 8 headers threshold for incoming BIP130 (Pieter Wuille)
944c54290d net_processing: drop Misbehavior for unconnecting headers (Pieter Wuille)
9f66ac7cf1 net_processing: do not treat non-connecting headers as response (Pieter Wuille)
Pull request description:
So far, discouragement of peers triggers when their misbehavior score exceeds 100 points. Most types of misbehavior increment the score by 100, triggering immediate discouragement, but some types do not. This PR makes all increments equal to either 100 (meaning any misbehavior will immediately cause disconnection and discouragement) or 0 (making the behavior effectively unconditionally allowed), and then removes the logic for score accumulation.
This simplifies the code a bit, but also makes protocol expectations clearer: if a peer misbehaves, they get disconnected. There is no good reason why certain types of protocol violations should be permitted 4 times (howmuch=20) or 9 times (howmuch=10), while many others are never allowed. Furthermore, the distinction between these looks arbitrary.
The specific types of misbehavior that are changed to 100 are:
* Sending us a `block` which does not connect to our header tree (which necessarily must have been unsollicited). [used to be score 10]
* Sending us a `headers` with a non-continuous headers sequence. [used to be score 20]
* Sending us more than 1000 addresses in a single `addr` or `addrv2` message [used to be score 20]
* Sending us more than 50000 invs in a single `inv` message [used to be score 20]
* Sending us more than 2000 headers in a single `headers` message [used to be score 20]
The specific types of misbehavior that are changed to 0 are:
* Sending us 10 (*) separate BIP130 headers announcements that do not connect to our block tree [used to be score 20]
* Sending us more than 8 headers in a single `headers` message (which thus does not get treated as a BIP130 announcement) that does not connect to our block tree. [used to be score 10]
I believe that none of these behaviors are unavoidable, except for the one marked (*) which can in theory happen still due to interaction between BIP130 and variations in system clocks (the max 2 hour in the future rule). This one has been removed entirely. In order to remove the impact of the bug it was designed to deal with, without relying on misbehavior, a separate improvement is included that makes `getheaders`-tracking more accurate.
In another unrelated improvement, this also gets rid of the 8 header limit heuristic to determine whether an incoming non-connecting `headers` is a potential BIP130 announcement, as this rule is no longer needed to prevent spurious Misbehavior. Instead, any non-connecting `headers` is now treated as a potential announcement.
ACKs for top commit:
sr-gi:
ACK [6eecba4](6eecba475e)
achow101:
ACK 6eecba475e
mzumsande:
Code Review ACK 6eecba475e
glozow:
light code review / concept ACK 6eecba475e
Tree-SHA512: e11e8a652c4ec048d8961086110a3594feefbb821e13f45c14ef81016377be0db44b5311751ef635d6e026def1960aff33f644e78ece11cfb54f2b7daa96f946
DANGER_CI_ON_HOST_CACHE_FOLDERS if set will mount caches in directories on the host rather than in docker volumes. Supports saving and restoring caches on Github Actions.
fa7bc9bbca fuzz: Fix wallet_bdb_parser 32-bit unhandled fseek error (MarcoFalke)
Pull request description:
`std::fseek` on 64-bit past the end of the file may work fine (the following read would fail). However, on 32-bit it may fail early.
Fix it, by ignoring the error, treating it similar to a read error.
This was found by OSS-Fuzz.
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=69414
ACKs for top commit:
TheCharlatan:
ACK fa7bc9bbca
brunoerg:
utACK fa7bc9bbca
Tree-SHA512: 7a752a005837bae6846ce315a7b3b1a5fe1f440c7797c750f2c0bbb20b1ef1537cd390c425747c0c85d012655e2f908bd300ea084f82e5ada19badbf826e1ec9
fa9cb101cf refactor: Add explicit cast to expected_last_page to silence fuzz ISan (MarcoFalke)
Pull request description:
Fixes#30247
I don't think this implicit cast can lead to any bugs, so make it explicit to silence the fuzz integer sanitizer.
Can be tested with:
```
FUZZ=wallet_bdb_parser UBSAN_OPTIONS="suppressions=$(pwd)/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1" ./src/test/fuzz/fuzz /tmp/1376869be72eebcc87fe737020add634b1a29533
```
After downloading the raw fuzz input from 1376869be7
ACKs for top commit:
dergoegge:
utACK fa9cb101cf
Tree-SHA512: 226dcc58be8d70b4eec1657f232c9c6648b5dac5eb2706e7390e65ce0a031fbaf8afce97d71a535c8294467dca4757c96f294d8cc03d5e6a1c0a036b0e070325
refactor: CBlockHeaderAndShortTxIDs constructor now always takes an explicit nonce.
test: Make blockencodings_tests deterministic using fixed seed providing deterministic
CBlockHeaderAndShortTxID nonces and dummy transaction IDs.
Fixes very rare flaky test failures, where the ShortIDs of test transactions collide, leading to
`READ_STATUS_FAILED` from PartiallyDownloadedBlock::InitData and/or `IsTxAvailable` giving `false`
when the transaction should actually be available.
* Use a new `FastRandomContext` with a fixed seed in each test, to ensure 'random' uint256s
used as fake prevouts are deterministic, so in-turn test txids and short IDs are deterministic
and don't collide causing very rare but flaky test failures.
* Add new test-only/internal initializer for `CBlockHeaderAndShortTxIDs` that takes a specified
nonce to further ensure determinism and avoid rare but undesireable short ID collisions.
In a test context this nonce is set to a fixed known-good value. Normally it is random, as
previously.
Flaky test failures can be reproduced with:
```patch
diff --git a/src/blockencodings.cpp b/src/blockencodings.cpp
index 695e8d806a..64d635a97a 100644
--- a/src/blockencodings.cpp
+++ b/src/blockencodings.cpp
@@ -44,7 +44,8 @@ void CBlockHeaderAndShortTxIDs::FillShortTxIDSelector() const {
uint64_t CBlockHeaderAndShortTxIDs::GetShortID(const Wtxid& wtxid) const {
static_assert(SHORTTXIDS_LENGTH == 6, "shorttxids calculation assumes 6-byte shorttxids");
- return SipHashUint256(shorttxidk0, shorttxidk1, wtxid) & 0xffffffffffffL;
+ // return SipHashUint256(shorttxidk0, shorttxidk1, wtxid) & 0xffffffffffffL;
+ return SipHashUint256(shorttxidk0, shorttxidk1, wtxid) & 0x0f;
}
```
to increase the likelihood of a short ID collision; and running
```shell
set -e;
n=0;
while (( n++ < 5000 )); do
src/test/test_bitcoin --run_test=blockencodings_tests;
done
```
4ccb3d6d0d fuzz: have package_rbf always make small txns (Greg Sanders)
Pull request description:
hopefully resolves https://github.com/bitcoin/bitcoin/issues/30241
The fuzz target is generating a large amount of
transactions, but the core of the logic is
ConsumeTxMemPoolEntry making the mempool
entries for adding to the mempool. Since
ConsumeTxMemPoolEntry generates its own transaction "vsize",
we can improve efficiency of the target
by explicitly creating very small transactions,
reducing the hashing and memory burden.
ACKs for top commit:
maflcko:
lgtm ACK 4ccb3d6d0d
hodlinator:
ACK 4ccb3d6d0d
glozow:
ACK 4ccb3d6d0d
Tree-SHA512: 5d2e7e98460c6144dfe7deac554865e2e8e0e5f934dbdf5857dc4b4f471a64dc933297dc0dcf516f748a4348be6bd184808b7ece17ce073fdcc77f81b74c64de
The addresses of the iterator values are non-deterministic (i.e. they
depend on where the values were allocated). This causes stability issues
when fuzzing (e.g. in the `txorphan` and `mini_miner` harnesses), due
the orders (derived from IteratorComparator) not being deterministic.
Improve stability by comparing the first element in the iterator value
pair instead of using the the value addresses.
8acdf66540 upnp: add compatibility for miniupnpc 2.2.8 (Cory Fields)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/30266
Miniupnpc 2.2.8 [changed the function signature of `UPNP_GetValidIGD`](c0a50ce33e (diff-5a0d7cff00628c2c64a617edb347c0f283e3a75e7df910e7e8438fc6db23f610R122)) without taking much care with the abi :(
~This is the minimal change to cope with that. Also included in this PR is a temporary bump to 2.2.8 to verify that it builds correctly. I'm happy to revert that and discuss the bump separately, as miniupnpc bumps require some scrutiny.~
I believe that this is problematic if we build against one version and encounter a different one at runtime. This is not a problem for depends because we build statically. But for users who are self-building against shared system libs, care must be taken to run against the same version used for linking.
Some quick digging shows that at least Ubuntu/Arch make the distinction between soversions:
`libminiupnpc.so.17` -> `libminiupnpc.so.18`. So in practice, I suppose this shouldn't be much of a problem.
Boooo for the upstream loose abi policy.
ACKs for top commit:
edilmedeiros:
reACK 8acdf66540
fanquake:
ACK 8acdf66540
Tree-SHA512: d2236ec8aef57a5c879065fbbe20080a14e4bf7b44c0bf506707eb946f72aa5837aba2fb2426d6853d21a9b77db5d72561d29d7ea645714d90309e11fe11d354
Set tip at the start of the function and only update it for a long poll.
Additionally have getTipHash return an optional, so the
caller can explicitly check that a tip exists.
This makes the options argument for BlockAssembler constructor mandatory,
dropping implicit use of ArgsManager. The caller i.e. the Mining
interface implementation now handles this.
In a future Stratum v2 change the Options object needs to be
mofified after arguments have been processed. Specifically
the pool communicates how many extra bytes it needs for
its own outputs (payouts, extra commitments, etc). This will need
to be substracted from what the user set as -blockmaxweight.
Such a change can be implemented in createNewBlock, after
ApplyArgsManOptions.
The fuzz target is generating a large amount of
transactions, but the core of the logic is
ConsumeTxMemPoolEntry making the mempool
entries for adding to the mempool. Since
ConsumeTxMemPoolEntry generates its own transaction
"vsize", we can improve efficiency of the target
by explicitly creating very small transactions,
reducing the hashing and memory burden.
See: c0a50ce33e
The return value of 2 now indicates:
"A valid connected IGD has been found but its IP address is reserved (non routable)"
We continue to ignore any return value other than 1.
7c298fe0df doc: rewrite some of the macdeploy docs (fanquake)
d042230f7a depends: swap mmacosx-version-min for mmacos-version-min (fanquake)
Pull request description:
Whilst `-mmacosx-version-min` and `-mmacos-version-min` remain aliases for each other, the later is preferred,
and I assume the former will be removed at some point in the future; see: https://github.com/llvm/llvm-project/pull/95374.
Somewhat of a followup to #21778. Rewrite some of the mac deploy docs.
ACKs for top commit:
theuni:
utACK 7c298fe0df
TheCharlatan:
ACK 7c298fe0df
hebasto:
ACK 7c298fe0df.
Tree-SHA512: 6493f087fde93e0eec319af0e105d163b3f047d8a03f7d4b0d6cd7c64b58d0a978b7d67c6b8dba5c6ccf8b10e188aab5dc98eec400b0546dc9ee801a689b4332
b03a45b13e Revert "contrib: macdeploy: monkey-patch gen-sdk to be deterministic" (fanquake)
Pull request description:
This reverts commit ba30a5407e.
We no-longer support Python 3.8, so remove the monkey patching.
ACKs for top commit:
hebasto:
ACK b03a45b13e, I have reviewed the code and it looks OK.
Tree-SHA512: 5bf68c2b332f18a620a8a6f77812ed93afa988016847bec1d3b7355670301dc957442ac47191a0cb7c3fe607d902914fb00c96345c8170f2a64429638c00b3c4
94ed4fbf8e Add release note for size 2 package rbf (Greg Sanders)
afd52d8e63 doc: update package RBF comment (Greg Sanders)
6e3c4394cf mempool: Improve logging of replaced transactions (Greg Sanders)
d3466e4cc5 CheckPackageMempoolAcceptResult: Check package rbf invariants (Greg Sanders)
316d7b63c9 Fuzz: pass mempool to CheckPackageMempoolAcceptResult (Greg Sanders)
4d15bcf448 [test] package rbf (glozow)
dc21f61c72 [policy] package rbf (Suhas Daftuar)
5da3967815 PackageV3Checks: Relax assumptions (Greg Sanders)
Pull request description:
Allows any 2 transaction package with no in-mempool ancestors to do package RBF when directly conflicting with other mempool clusters of size two or less.
Proposed validation steps:
1) If the transaction package is of size 1, legacy rbf rules apply.
2) Otherwise the transaction package consists of a (parent, child) pair with no other in-mempool ancestors (or descendants, obviously), so it is also going to create a cluster of size 2. If larger, fail.
3) The package rbf may not evict more than 100 transactions from the mempool(bip125 rule 5)
4) The package is a single chunk
5) Every directly conflicted mempool transaction is connected to at most 1 other in-mempool transaction (ie the cluster size of the conflict is at most 2).
6) Diagram check: We ensure that the replacement is strictly superior, improving the mempool
7) The total fee of the package, minus the total fee of what is being evicted, is at least the minrelayfee * size of the package (equivalent to bip125 rule 3 and 4)
Post-cluster mempool this will likely be expanded to general package rbf, but this is what we can safely support today.
ACKs for top commit:
achow101:
ACK 94ed4fbf8e
glozow:
reACK 94ed4fbf8e via range-diff
ismaelsadeeq:
re-ACK 94ed4fbf8e
theStack:
Code-review ACK 94ed4fbf8e
murchandamus:
utACK 94ed4fbf8e
Tree-SHA512: 9bd383e695964f362f147482bbf73b1e77c4d792bda2e91d7f30d74b3540a09146a5528baf86854a113005581e8c75f04737302517b7d5124296bd7a151e3992
260f8da71a refactor: remove warnings globals (stickies-v)
9c4b0b7ce4 node: update uiInterface whenever warnings updated (stickies-v)
b071ad9770 introduce and use the generalized `node::Warnings` interface (stickies-v)
20e616f864 move-only: move warnings from common to node (stickies-v)
bed29c481a refactor: remove unnecessary AppendWarning helper function (stickies-v)
Pull request description:
This PR:
- moves warnings from common to the node library and into the node namespace (as suggested in https://github.com/bitcoin/bitcoin/pull/29845#discussion_r1570069541)
- generalizes the warnings interface to `Warnings::Set()` and `Warnings::Unset()` methods, instead of having a separate function and globals for each warning. As a result, this simplifies the `kernel::Notifications` interface.
- removes warnings.cpp from the kernel library
- removes warning globals
- adds testing for the warning logic
Behaviour change introduced:
- the `-alertnotify` command is executed for all `KernelNotifications::warningSet` calls, which now also covers the `LARGE_WORK_INVALID_CHAIN` warning
- the GUI is updated automatically whenever a warning is (un)set, covering some code paths where it previously wouldn't be, e.g. when `node::AbortNode()` is called, or for the `LARGE_WORK_INVALID_CHAIN` warning
Some discussion points:
- ~is `const std::string& id` the best way to refer to warnings? Enums are an obvious alternative, but since we need to define warnings across libraries, strings seem like a straightforward solution.~ _edit: updated approach to use `node::Warning` and `kernel::Warning` enums._
ACKs for top commit:
achow101:
ACK 260f8da71a
ryanofsky:
Code review ACK 260f8da71a. Only change since last review was rebasing
TheCharlatan:
Re-ACK 260f8da71a
Tree-SHA512: a3fcedaee0d3ad64e9c111aeb30665162f98e0e72acd6a70b76ff2ddf4f0a34da4f97ce353c322a1668ca6ee4d8a81cc6e6d170c5bbeb7a43cffdaf66646b588
9eea51d905 ci: move Asan / LSan / USDT job to Github Actions (Max Edwards)
4b527fa93b ci: add IPV6 network to ci container (Max Edwards)
Pull request description:
PR for moving the ASAN + LSAN + USDT + friends job to github actions from Cirrus.
The motivation for this PR is that this task needs a full VM (or bare metal) to function, because of the tracepoints. It can not run in a container on an arbitrary Linux, because the outside machine must exactly match the specification of the distro used in the CI task config. This requires more maintenance for the persistent worker, and I think moving to GHA will reduce the maintenance burden, or at least make it possible for anyone to work on.
Also, it makes it easier to run the task on forks (bitcoin-inquisition, bitcoin-knots, devel forks, ...) without having to set-up a real machine.
ACKs for top commit:
maflcko:
review ACK 9eea51d905
achow101:
ACK 9eea51d905
hebasto:
ACK 9eea51d905.
Tree-SHA512: 1111c1c9e3a11e725dff1344643fff3c91fb9b4d7c1cc9a7d507a8f146f5223316a00272030b41ae37ecb59d044f2e90e1cd907450049b25f094f0b60643d4c7
881724d443 test: Added test coverage to listsinceblock rpc (kevkevinpal)
Pull request description:
This change is meant to add test coverage to this rpc error https://github.com/bitcoin/bitcoin/blob/master/src/wallet/rpc/transactions.cpp#L666C53-L666C79
This is done by renaming the first block in the blocks folder
---
Doing a quick grep for the error code in our functional tests leads to zero results
`grep -nri "Can't read block from disk" ./test/functional/`
ACKs for top commit:
achow101:
ACK 881724d443
tdb3:
re ACK for 881724d443
rkrux:
tACK [881724](881724d443)
Tree-SHA512: c5dff20cf014d0181f49d6b161f1364e1c6b79e8661047f77f07e21e59f4d1f2fd6f745538c8fc5bd6d4244650a840dd64d184634366f7c21fa67141a60af44a
5cf0a1f230 test: add `createmultisig` P2MS encoding test for all n (1..20) (Sebastian Falbesoner)
0570d2c204 test: add unit test for `keys_to_multisig_script` (Sebastian Falbesoner)
0c41fc3fa5 test: fix `keys_to_multisig_script` (P2MS) helper for n/k > 16 (Sebastian Falbesoner)
Pull request description:
While reviewing #28307, I noticed that the test framework's `key_to_multisig_script` helper (introduced in #23305) is broken for pubkey count (n) and threshold (k) values larger than 16. This is due to the implementation currently enforcing a direct single-byte data push (using `CScriptOp.encode_op_n`), which obviously fails for values 17+. Fix that by passing the numbers directly to the CScript list, where it's automatically converted to minimally-encoded pushes (see class method `CScript.__coerce_instance`, branch `isinstance(other, int)`).
The second commit adds a unit test to ensure that the encoding is correct.
ACKs for top commit:
achow101:
ACK 5cf0a1f230
tdb3:
ACK 5cf0a1f230
rkrux:
reACK [5cf0a1f](5cf0a1f230)
Tree-SHA512: 4168a165c3f483ec8e37a27dba1628a7ea0063545a2b7e74d9e20d753fddd7e33d37e1a190434fa6dca39adf9eef5d0211f7a0c1c7b44979f0a3bb350e267562
ad06e68399 test: write functional test results to csv (tdb3)
Pull request description:
Adds argument `--resultsfile` to test_runner.py.
Enables functional test results to be written to a (csv) file for processing by other applications (or for historical archiving).
Test name, status, and duration are written to the file provided with the argument.
Since `test_runner.py` is being touched, also fixes a misspelling (linter warning). Can split into its own commit if desired.
#### Notes
- Total runtime of functional tests has seemed to have increased on my development machines over the past few months (more tests added, individual test runtime increase, etc.). Was interested in recording test runtime data over time to detect trends. Initially searched `doc/benchmarking.md`, existing PRs, and Issues, but didn't immediately see this type of capability or alternate solutions (please chime in if you know of one!). Thought it would be beneficial to add this capability to `test_runner` to facilitate this type of data analysis (and potentially other use cases)
- Saw https://github.com/bitcoin/bitcoin/blob/master/test/functional/README.md#benchmarking-with-perf, and this PR's higher level data seems complimentary.
- Was on the fence as to whether to expand `print_results()` (i.e. take advantage of the same loop over `test_results`) or implement in a separate `write_results()` function. Decided on the latter for now, but interested in reviewers' thoughts.
#### Example 1: all tests pass
```
$ test/functional/test_runner.py --resultsfile functional_test_results.csv --cachedir=/mnt/tmp/cache --tmpdir=/mnt/tmp feature_blocksdir wallet_startup feature_config_args mempool_accept
Temporary test directory at /mnt/tmp/test_runner_₿_🏃_20240614_201625
Test results will be written to functional_test_results.csv
...
$ cat functional_test_results.csv
test,status,duration(seconds)
feature_blocksdir.py,Passed,1
feature_config_args.py,Passed,29
mempool_accept.py,Passed,9
wallet_startup.py,Passed,2
ALL,Passed,29
```
#### Example 2: one test failure
```
$ cat functional_test_results.csv
test,status,duration(seconds)
feature_blocksdir.py,Passed,1
feature_config_args.py,Passed,28
wallet_startup.py,Passed,2
mempool_accept.py,Failed,1
ALL,Failed,28
```
ACKs for top commit:
maflcko:
re-ACK ad06e68399
kevkevinpal:
tACK [ad06e68](ad06e68399)
achow101:
ACK ad06e68399
rkrux:
tACK [ad06e68](ad06e68399)
brunoerg:
ACK ad06e68399
marcofleon:
Good idea, tested ACK ad06e68399
Tree-SHA512: 561194406cc744905518aa5ac6850c07c4aaecdaf5d4d8b250671b6e90093d4fc458f050e8a85374e66359cc0e0eaceba5eb24092c55f0d8f349d744a32ef76c
Move the output serialization size and dust calculation into the loop where the
outputs are iterated over to calculate the total sum.
Move the code for adding the the txoutputs to the transaction to after
coin selection.
While this code structure generally follows a more logical flow,
the primary motivation for moving the code for adding outputs to the
transaction sets us up nicely for silent payments (in a future PR):
we need to know the input set before generating the final output scriptPubKeys.
Now that a CRecipient holds a CTxDestination, we can get the serialized
size and determine if the output is dust using the CRecipient directly.
This does not change any current behavior, but provides a nice generalization
that can be used to apply special logic to a CTxDestination serialization
and dust calculations in the future.
Specifically, in a later PR when support for `V0SilentPayment` destinations is
added, we need to use `WitnessV1Taproot` as the scriptPubKey for serialized
size calcuations whenever the `CRecipient` destination is a `V0SilentPayment`
destination.
Prior to this commit, TestEncryptedP2PState would always
send initial_v2_handshake bytes in 2 parts (as required
by early key response test).
For generalising this test and having different v2 handshake
behaviour based on the test type, special behaviours like
sending initial_v2_handshake bytes in 2 parts are executed
only if test_type is set to EARLY_KEY_RESPONSE.
Adds argument --resultsfile to test_runner.py.
Writes comma-separated functional test name, status,
and duration to the file provided with the argument.
Also fixes minor typo in test_runner.py
f58beabe75 test: bumpfee with user specified fee_rate ignores walletIncrementalRelayFee (ismaelsadeeq)
436e88f433 bumpfee: ignore WALLET_INCREMENTAL_RELAY_FEE when user specifies fee rate (ismaelsadeeq)
Pull request description:
Fixes#26973
When using the `bumpfee` RPC and manually specifying `fee_rate`, there should be no requirement that the new fee must be at least the sum of the original fee and `incrementalFee` (maximum of `relayIncrementalFee` and `WALLET_INCREMENTAL_RELAY_FEE`).
This restriction should only apply when user did not specify `fee_rate`.
> because the GUI doesn't let the user specify the new fee rate yet (https://github.com/bitcoin-core/gui/issues/647), it would be very annoying to have to bump 20 times to increment by 20 sat/vbyte.
The restriction should instead be the new fee must be at least the sum of the original fee and `incrementalFee` (`relayIncrementalFee`)
ACKs for top commit:
achow101:
ACK f58beabe75
murchandamus:
ACK f58beabe75
Tree-SHA512: 193259f87173b7d5a8e68e0e29f2ca7e75c550e3cf0dee3d6d822b5b1e07c2e6dec0bfc8fb435855736ebced97a10dbdbfef72e8c5abde06fdefcba122f2e7f1
fae3a1f006 log: use error level for critical log messages (MarcoFalke)
Pull request description:
This picks up the first commit from https://github.com/bitcoin/bitcoin/pull/29231, but extends it to also cover cases that were missed in it.
As per https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#logging, LogError should be used for severe problems that require the node to shut down.
ACKs for top commit:
stickies-v:
re-ACK fae3a1f006, I'm ~0 on the latest force push as `user_error` was already logged at the right level through `GetNotifications().fatalError(user_error);` so I'd be in favour of deduplicating/cleaning up this logging logic but can be done in follow-up.
kevkevinpal:
ACK [fae3a1f](fae3a1f006)
achow101:
ACK fae3a1f006
Tree-SHA512: 3f99fd25d5a204d570a42d8fb2b450439aad7685692f9594cc813d97253c4df172a6ff3cf818959bfcf25dfcf8ee9a9c9ccc6028fcfcecdb47591e18c77ef246
Problem:
If `FuzzedSock::Recv(N, MSG_PEEK)` is called then `N` bytes would be
retrieved from the fuzz provider, saved in `m_peek_data` and returned
to the caller (ok).
If after this `FuzzedSock::Recv(M, 0)` is called where `M < N`
then the first `M` bytes from `m_peek_data` would be returned
to the caller (ok), but the remaining `N - M` bytes in `m_peek_data`
would be discarded/lost (not ok). They must be returned by a subsequent
`Recv()`.
To resolve this, only remove the head `N` bytes from `m_peek_data`.
Allow the callers of `CreateSock()` to pass all 3 arguments to the
`socket(2)` syscall. This makes it possible to create sockets of
any domain/type/protocol.
a37778d4d3 Squashed 'src/leveldb/' changes from e2f10b4e47..688561cba8 (fanquake)
Pull request description:
Includes https://github.com/bitcoin-core/leveldb-subtree/pull/41 which is used in #30234.
ACKs for top commit:
theuni:
utACK 95812d912b
Tree-SHA512: 3d943695a3d33816cf5558b183f5629aa92a500a1544eecedf84952e93c8592a8cf0d554b88281fc0bad3c9e920ebcff1ed8edc12f8e73f36ed5335482beb829
Whilst these remain aliases for each other, the later is preferred,
and I assume the former will be removed at some point in the future;
see: https://github.com/llvm/llvm-project/pull/95374.
e2779ce98b test: cover more errors for `signrawtransactionwithkey` RPC (brunoerg)
Pull request description:
This PR adds test coverage for the following errors for the `signrawtransactionwithkey` RPC:
- Invalid private key
- TX decode failed
For reference: https://maflcko.github.io/b-c-cov/total.coverage/src/rpc/rawtransaction.cpp.gcov.html
ACKs for top commit:
maflcko:
ACK e2779ce98b
kevkevinpal:
ACK [e2779ce](e2779ce98b)
tdb3:
ACK e2779ce98b
BrandonOdiwuor:
Code Review ACK e2779ce98b
Tree-SHA512: 41c7e990684b60645cf4ccec8aad5ebbe61da221871eb3c1685b2bb1eebda58b29358502cb1525b7c7a2b612e2bebf449ed0bae14ab663b4641c528a9c013b5b
07f64177a4 Reduce memory copying operations in bech32 encode (Lőrinc)
d5ece3c4b5 Reserve hrp memory in Decode and LocateErrors (Lőrinc)
Pull request description:
Started optimizing the base conversions in [TryParseHex](https://github.com/bitcoin/bitcoin/pull/29458), [Base58](https://github.com/bitcoin/bitcoin/pull/29473) and [IsSpace](https://github.com/bitcoin/bitcoin/pull/29602) - this is the next step.
Part of this change was already merged in https://github.com/bitcoin/bitcoin/pull/30047, which made decoding `~26%` faster.
Here I've reduced the memory reallocations and copying operations in bech32 encode, making it `~15%` faster.
> make && ./src/bench/bench_bitcoin --filter='Bech32Encode' --min-time=1000
Before:
```
| ns/byte | byte/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 19.97 | 50,074,562.72 | 0.1% | 1.06 | `Bech32Encode`
```
After:
```
| ns/byte | byte/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 17.33 | 57,687,668.20 | 0.1% | 1.10 | `Bech32Encode`
```
ACKs for top commit:
josibake:
ACK 07f64177a4
sipa:
utACK 07f64177a4
achow101:
ACK 07f64177a4
Tree-SHA512: 511885217d044ad7ef2bdf9203b8e0b94eec8b279bc193bb7e63e29ab868df6d21e9e4c7a24390358e1f9c131447ee42039df72edcf1e2b11e1856eb2b3e10dd
Support package RBF where the conflicting package would result
in a mempool cluster of size two, and each of its direct
conflicts are also part of an up-to-size-2 mempool cluster.
This restricted topology allows for exact calculation
of miner scores for each side of the equation, reducing
the surface area for new pins, or incentive-incompatible
replacements.
This allows wallets to create simple CPFP packages
that can fee bump other simple CPFP packages. This,
leveraged with other restrictions such as V3 transactions,
can create pin-resistant applications.
Future package RBF relaxations can be considered when appropriate.
Co-authored-by: glozow <gloriajzhao@gmail.com>
Co-authored-by: Greg Sanders <gsanders87@gmail.com>
This commit introduces slight behaviour change. Previously, the
GUI status bar would be updated for most warnings, namely
UNKNOWN_NEW_RULES_ACTIVATED, CLOCK_OUT_OF_SYNC and
PRE_RELEASE_TEST_BUILD, but not for LARGE_WORK_INVALID_CHAIN
(and not for FATAL_INTERNAL_ERROR, but that is not really
meaningful).
Fix this by always updating the status bar when the warnings are
changed.
Instead of having separate warning functions (and globals) for each
different warning that can be raised, encapsulate this logic into
a single class and allow to (un)set any number of warnings.
Introduces behaviour change:
- the `-alertnotify` command is executed for all
`KernelNotifications::warningSet` calls, which now also covers the
`LARGE_WORK_INVALID_CHAIN` warning.
- previously, warnings were returned based on a predetermined order,
e.g. with the "pre-release test build" warning always first. This
is no longer the case, and Warnings::GetMessages() will return
messages sorted by the id of the warning.
Removes warnings.cpp from kernel.
0fcbfdb7ad Support running individual lint checks (David Gumberg)
Pull request description:
This PR was split out from #29965:
Adds support for running individual tests in the rust lint suite by passing `--lint=LINT_TO_RUN` to the lint runner. This PR also adds a corresponding help message.
When running with `cargo run`, arguments after a double dash (`--`) are passed to the binary instead of the cargo command. For example, in order to run the linter check that tabs are not used as whitespace:
```console
cd test/lint/test_runner && cargo run -- --lint=tabs_whitespace
```
ACKs for top commit:
maflcko:
ACK 0fcbfdb7ad
achow101:
ACK 0fcbfdb7ad
marcofleon:
Tested ACK 0fcbfdb7ad. Ran `cargo run` with various of the individual tests and with bad input. Also ran it with no arguments. Everything works as expected and help message looks good.
Tree-SHA512: 48fe4aa9fbb2acef5f8e3c17382ae22e0e350ae6ad9aeeb1a3c0a9192de98809f98728e32b8db24a36906ace999e35626ebd6cb2ca05f74146d21e9b6fb14615
c7376babd1 doc: Clarify distinction between util and common libraries in libraries.md (Ryan Ofsky)
4f74c59334 util: Move util/string.h functions to util namespace (Ryan Ofsky)
4d05d3f3b4 util: add TransactionError includes and namespace declarations (Ryan Ofsky)
680eafdc74 util: move fees.h and error.h to common/messages.h (Ryan Ofsky)
02e62c6c9a common: Add PSBTError enum (Ryan Ofsky)
0d44c44ae3 util: move error.h TransactionError enum to node/types.h (Ryan Ofsky)
9bcce2608d util: move spanparsing.h to script/parsing.h (Ryan Ofsky)
6dd2ad4792 util: move spanparsing.h Split functions to string.h (Ryan Ofsky)
23cc8ddff4 util: move HexStr and HexDigit from util to crypto (TheCharlatan)
6861f954f8 util: move util/message to common/signmessage (Ryan Ofsky)
cc5f29fbea build: move memory_cleanse from util to crypto (Ryan Ofsky)
5b9309420c build: move chainparamsbase from util to common (Ryan Ofsky)
ffa27af24d test: Add check-deps.sh script to check for unexpected library dependencies (Ryan Ofsky)
Pull request description:
Remove `fees.h`, `errors.h`, and `spanparsing.h` from the util library. Specifically:
- Move `Split` functions from `util/spanparsing.h` to `util/string.h`, using `util` namespace for clarity.
- Move remaining spanparsing functions to `script/parsing.h` since they are used for descriptor and miniscript parsing.
- Combine `util/fees.h` and `util/errors.h` into `common/messages.h` so there is a place for simple functions that generate user messages to live, and these functions are not part of the util library.
Motivation for this change is that the util library is a dependency of the kernel, and we should remove functionality from util that shouldn't be called by kernel code or kernel applications. These changes should also improve code organization and make functions easier to discover. Some of these same moves are (or were) part of #28690, but did not help with code organization, or made it worse, so it is better to move them and clean them up in the same PR so code only has to change one time.
ACKs for top commit:
achow101:
ACK c7376babd1
TheCharlatan:
Re-ACK c7376babd1
hebasto:
re-ACK c7376babd1.
Tree-SHA512: 5bcef16c1255463b1b69270548711e7ff78ca0dd34e300b95e3ca1ce52ceb34f83d9ddb2839e83800ba36b200de30396e504bbb04fa02c6d0c24a16d06ae523d
Before, interruption was printed as an error. Also,
it did not log the reason when an interruption happened,
e.g. "Error accepting:".
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
faa41e29d5 fuzz: Use std::span in FuzzBufferType (MarcoFalke)
Pull request description:
The use of `Span` is problematic, because it lacks methods such as `rbegin`, leading to compile failures when used:
```
error: no member named 'rbegin' in 'Span<const unsigned char>'
```
One could fix `Span`, but it seems better to use `std::span`, given that `Span` will be removed anyway in the long term.
ACKs for top commit:
dergoegge:
utACK faa41e29d5
Tree-SHA512: 54bcaf51c83a1b48739cd7f1e8445c6eba0eb04231bce5c35591a47dddb3890ffcaf562cf932930443c80ab0e66950c4619560e6692240de0c52aeef3214facd
193c748e44 fuzz: add I2P harness (marcofleon)
Pull request description:
Addresses https://github.com/bitcoin/bitcoin/issues/28803. This updated harness sets mock time at the beginning of each iteration and deletes the private key file at the end of each iteration. Mock time is used to make the fuzz test more stable, as `GetTime` is called at points in `i2p`. Deleting the private key file ensures that each iteration is independent from the last. Now, a new key is generated in `i2p` every time, so the fuzzer can eventually make progress through the target code.
Re-working this harness also led me and dergoegge to resolve a couple of issues in `FuzzedSock`, which allows for full coverage of the `i2p` code. Those changes can be seen in https://github.com/bitcoin/bitcoin/pull/30211.
The SAM protocol for interacting with I2P requires some specifc inputs so it's best to use a dictionary when running this harness.
<details>
<summary>I2P dict</summary>
```
"HELLO VERSION"
"HELLO REPLY RESULT=OK VERSION="
"HELLO REPLY RESULT=NOVERSION"
"HELLO REPLY RESULT=I2P_ERROR"
"SESSION CREATE"
"SESSION STATUS RESULT=OK DESTINATION="
"SESSION STATUS RESULT=DUPLICATED_ID"
"SESSION STATUS RESULT=DUPLICATED_DEST"
"SESSION STATUS RESULT=INVALID_ID"
"SESSION STATUS RESULT=INVALID_KEY"
"SESSION STATUS RESULT=I2P_ERROR MESSAGE="
"SESSION ADD"
"SESSION REMOVE"
"STREAM CONNECT"
"STREAM STATUS RESULT=OK"
"STREAM STATUS RESULT=INVALID_ID"
"STREAM STATUS RESULT=INVALID_KEY"
"STREAM STATUS RESULT=CANT_REACH_PEER"
"STREAM STATUS RESULT=I2P_ERROR MESSAGE="
"STREAM ACCEPT"
"STREAM FORWARD"
"DATAGRAM SEND"
"RAW SEND"
"DEST GENERATE"
"DEST REPLY PUB= PRIV="
"DEST REPLY RESULT=I2P_ERROR"
"NAMING LOOKUP"
"NAMING REPLY RESULT=OK NAME= VALUE="
"DATAGRAM RECEIVED DESTINATION= SIZE="
"RAW RECEIVED SIZE="
"NAMING REPLY RESULT=INVALID_KEY NAME="
"NAMING REPLY RESULT=KEY_NOT_FOUND NAME="
"MIN"
"MAX"
"STYLE"
"ID"
"SILENT"
"DESTINATION"
"NAME"
"SIGNATURE_TYPE"
"CRYPTO_TYPE"
"SIZE"
"HOST"
"PORT"
"FROM_PORT"
"TRANSIENT"
"STREAM"
"DATAGRAM"
"RAW"
"MASTER"
"true"
"false"
```
</details>
I'll add this dict to qa-assets later on.
ACKs for top commit:
dergoegge:
tACK 193c748e44
brunoerg:
ACK 193c748e44
vasild:
ACK 193c748e44
Tree-SHA512: 09ae4b3fa0738aa6f159f4d920493bdbce786b489bc8148e7a135a881e9dba93d727b40f5400c9510e218dd2cfdccc7ce2d3ac9450654fb29c78aac59af92ec3
eb37a9b8e7 Merge sipa/minisketch#87: Avoid copy in self-assign
fe6557642e Merge sipa/minisketch#88: build: Add `-Wundef`
8ea298bfa7 Avoid copy in self-assign
978a3d8869 build: Add `-Wundef`
3387044179 Merge sipa/minisketch#86: doc: fix typo in sketch_impl.h
15c2d13b60 doc: fix typo in sketch_impl.h
7be08b8a46 Merge sipa/minisketch#85: Fixes for integer precision loss
00fb4a4d83 Avoid or make integer precision conversion explicit
9d62a4d27c Avoid the need to cast/convert to size_t for vector operations
19e06cc7af Prevent overflows from large capacity/max_elements
git-subtree-dir: src/minisketch
git-subtree-split: eb37a9b8e79f9e49d73b96a49bf97a96d9eb676c
f51da34ec1 utils: add missing include (Cory Fields)
Pull request description:
Noticed when testing `VecDeque` with no other includes.
For libc++, need type_traits for `std::is_trivially_destructible_v`.
ACKs for top commit:
maflcko:
ACK f51da34ec1
glozow:
ACK f51da34ec1
sipa:
utACK f51da34ec1
Tree-SHA512: bf96910abe9aaddd8586e6cc8f68a9bbac4c26d976ebeebcfa86b86c0da5783c1cbdbc7fa09b62cdcfde19e6442eb65a66bf1e2e80408d68e9dd9689dc22b0fa
Set minimum required glibc to 2.31.
The glibc 2.31 branch is still maintained:
https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/release/2.31/master.
Remove the stack-protector check from test-security-check, as the test
no-longer fails, and given the control we have of the end, the actual
security-check test seems sufficient (this might also be applied to some
of the other checks).
Drops runtime support for Ubuntu Bionic 18.04 and RHEL-8 from the release binaries.
429ec1aaaa refactor: Rename CTransaction::nVersion to version (Ava Chow)
27e70f1f5b consensus: Store transaction nVersion as uint32_t (Ava Chow)
Pull request description:
Given that the use of a transaction's nVersion is always as an unsigned int, it doesn't make sense to store it as signed and then cast it to unsigned everywhere it is used and displayed.
Since a few alternative implementations have recently been revealed to have made an error with this signedness that would have resulted in consensus failure, I think it makes sense for us to just make this always unsigned to make it clear that the version is treated as unsigned. This would also help us avoid future potential issues with signedness of this value.
I believe that this is safe and does not actually change what transactions would or would not be considered both standard and consensus valid. Within consensus, the only use of the version in consensus is in BIP68 validation which was already casting it to uint32_t. Within policy, although it is used as a signed int for the transaction version number check, I do not think that this change would change standardness. Standard transactions are limited to the range [1, 2]. Negative numbers would have fallen under the < 1 condition, but by making it unsigned, they are still non-standard under the > 2 condition.
Unsigned and signed ints are serialized and unserialized the same way so there is no change in serialization.
ACKs for top commit:
maflcko:
ACK 429ec1aaaa 🐿
glozow:
ACK 429ec1aaaa
shaavan:
ACK 429ec1aaaa💯
Tree-SHA512: 0bcd92a245d7d16c3665d2d4e815a4ef28207ad4a1fb46c6f0203cdafeab1b82c4e95e4bdce7805d80a4f4a46074f6542abad708e970550d38a00d759e3dcef1
3d4ca62d88 doc: add release note for 29091 and 29165 (fanquake)
Pull request description:
GCC 11.x or Clang 15.x are now required to compile Bitcoin Core.
ACKs for top commit:
hebasto:
ACK 3d4ca62d88.
Tree-SHA512: 6469a920ff9512897eeeb85d9c5538fd24884aabb20754d0f699f9975d81a320547de9e41758c242c4271bd45b7a76c363efe2ef703e156497e2c8cb9f3c14da
47f705b33f tests: add fuzz tests for BitSet (Pieter Wuille)
59a6df6bd5 util: add BitSet (Pieter Wuille)
Pull request description:
Extracted from #30126.
This introduces the `BitSet` data structure, inspired by `std::bitset`, but with a few features that cannot be implemented on top without efficiency loss:
* Finding the first set bit (`First`)
* Finding the last set bit (`Last`)
* Iterating over all set bits (`begin` and `end`).
And a few other operators/member functions that help readability for #30126:
* `operator-` for set subtraction
* `Overlaps()` for testing whether intersection is non-empty
* `IsSupersetOf()` for testing (non-strict) supersetness
* `IsSubsetOf()` for testing (non-strict) subsetness
* `Fill()` to construct a set with all numbers from 0 to n-1, inclusive
* `Singleton()` to construct a set with one specific element.
Everything is tested through a simulation-based fuzz test that compares the behavior with normal `std::bitset` equivalent operations.
ACKs for top commit:
instagibbs:
ACK 47f705b33f
achow101:
ACK 47f705b33f
cbergqvist:
re-ACK 47f705b33f
theStack:
Code-review ACK 47f705b33f
Tree-SHA512: e451bf4b801f193239ee434b6b614f5a2ac7bb49c70af5aba24c2ac0c54acbef4672556800e4ac799ae835632bdba716209c5ca8c37433a6883dab4eb7cd67c1
5bc2077e8f validation: allow to specify frequency for -checkblockindex (Martin Zumsande)
d5a631b959 validation: improve performance of CheckBlockIndex (Martin Zumsande)
32c80413fd bench: add benchmark for checkblockindex (Martin Zumsande)
Pull request description:
`CheckBlockIndex() ` are consistency checks that are currently enabled by default on regtest.
The function is rather slow, which is annoying if you
* attempt to run it on other networks, especially if not fully synced
* want to generate a long chain on regtest and see block generation slow down because you forgot to disable `-checkblockindex` or don't know it existed.
One reason why it's slow is that in order to be able to traverse the block tree depth-first from genesis, it inserts pointers to all block indices into a `std::multimap` - for which inserts and lookups become slow once there are hundred thousands of entries.
However, typically the block index is mostly chain-like with just a few forks so a multimap isn't really needed for the most part. This PR suggests to store the block indices of the chain ending in the best header in a vector instead, and store only the rest of the indices in a multimap. This does not change the actual consistency checks that are being performed for each index, just the way the block index tree is stored and traversed.
This adds a bit of complication to make sure each block is visited (note that there are asserts that check it), making sure that the two containers are traversed correctly, but it speeds up the function considerably:
On master, a single invocation of `CheckBlockIndex` takes ~1.4s on mainnet for me (4.9s on testnet which has >2.4 million blocks).
With this branch, the runtime goes down to ~0.27s (0.85s on testnet).This is a speedup by a factor ~5.
ACKs for top commit:
achow101:
ACK 5bc2077e8f
furszy:
ACK 5bc2077e8f
ryanofsky:
Code review ACK 5bc2077e8f. Just added suggested assert and simplification since last review
Tree-SHA512: 6b9c3e3e5069d6152b45a09040f962380d114851ff0f9ff1771cf8cad7bb4fa0ba25cd787ceaa3dfa5241fb249748e2ee6987af0ccb24b786a5301b2836f8487
24bc46c83b cli: Add warning for duplicate port definition (tdb3)
e208fb5d3b cli: Sanitize ports in rpcconnect and rpcport (tdb3)
Pull request description:
Adds invalid port detection to bitcoin-cli for -rpcconnect and -rpcport.
In addition to detecting malformed/invalid ports (e.g. those outside of the 16-bit port range, not numbers, etc.), bitcoin-cli also now considers usage of port 0 to be invalid. bitcoin-cli previously considered port 0 to be valid and attempted to use it to reach bitcoind.
Functional tests were added for invalid port detection as well as port prioritization.
Additionally, a warning is provided when a port is specified in both -rpcconnect and -rpcport.
This PR is an alternate approach to PR #27820 (e.g. SplitHostPort is unmodified).
Considered an alternative to 127.0.0.1 being specified in functional tests, but at first glance, this might need an update to test_framework/util.py (e.g. rpc_url), which might be left to a future PR.
ACKs for top commit:
S3RK:
light code review ACK 24bc46c83b
achow101:
ACK 24bc46c83b
cbergqvist:
re ACK 24bc46c83b
Tree-SHA512: c83ab6a30a08dd1ac8b368a7dcc2b4f23170f0b61dd67ffcad7bcda05096d333bcb9821fba11018151f55b2929c0a333bfec15b8bb863d83f41fc1974c6efca5
09ef322acc [[refactor]] Check CTxMemPool options in constructor (TheCharlatan)
Pull request description:
The tests should run the same checks on the mempool options that the init code also applies. The downside to this patch is that the log line may now be printed more than once in the for loop.
This was originally noticed here https://github.com/bitcoin/bitcoin/pull/25290#discussion_r900272797.
ACKs for top commit:
stickies-v:
re-ACK 09ef322acc . Fixed unreachable assert and updated docstring, and also added an exception for "-maxmempool must be at least " in the `tx_pool` fuzz test, which makes sense when looking at how the mempool options are constructed in `SetMempoolConstraints`.
achow101:
ACK 09ef322acc
ryanofsky:
Code review ACK 09ef322acc. Just fuzz test error checking fix and updated comment since last review
Tree-SHA512: eb3361411c2db70be17f912e3b14d9cb9c60fb0697a1eded952c3b7e8675b7d783780d45c52e091931d1d80fe0f0280cee98dd57a3100def13af20259d9d1b9e
Not every pseudorandom hash result is a valid x-only public key,
so the pubkey tweaking in the course of creating the output public
key would fail about every second time.
Fix this by treating the hash result as private key and calculate
the x-only public key out of that, to be used then as internal key.
7cbfd7a7ce refactor: rename (macho) ld64 to lld (fanquake)
d851451705 ci: update deps for macOS cross build (fanquake)
9ebdd5e9e0 depends: update install docs for macOS cross compilation (fanquake)
fb74fd66cb depends: remove no-longer used llvm_* vars from macOS build (fanquake)
9043f12425 depends: no-longer pass -B to clang in macOS cross-compile (fanquake)
f9994b025e depends: remove native LLVM package (fanquake)
e9a44faf14 depends: remove FORCE_USE_SYSTEM_CLANG (fanquake)
9946618f61 guix: use clang-toolchain-18 for macOS build (fanquake)
Pull request description:
Remove `FORCE_USE_SYSTEM_CLANG` in favour of always using the system Clang and lld for macOS cross-compilation; rather than downloading precompiled blobs.
For example, anyone using Ubuntu 24.04 should be able to `apt install clang llvm lld .. etc`, and then cross-compile for macOS using:
```bash
# clang --version
Ubuntu clang version 18.1.3 (1)
make -C depends HOST=arm64-apple-darwin FORCE_USE_SYSTEM_CLANG=1
./autogen.sh
CONFIG_SITE=/path/to/depends/arm64-apple-darwin/share/config.site ./configure
make
# file src/qt/bitcoin-qt
src/qt/bitcoin-qt: Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK|PIE|HAS_TLV_DESCRIPTORS>
```
Note that the minimum supported version of Clang we will support for macOS cross-compilation will likely be more recent than our other minimum supported version of Clang, due to compiler/linker option usage.
ACKs for top commit:
Sjors:
tACK 7cbfd7a7ce
theuni:
ACK 7cbfd7a7ce
TheCharlatan:
Nice, ACK 7cbfd7a7ce
Tree-SHA512: 1499e29b3b238c5c85278c38e8fb6bb5e7883db3443f62b6bf397c5d761bedbc054962be645a9defce15266f0a969bb4b3ccd28b6e4dd874472857b928f185d1
ab98e6fd03 test: add coverage for errors for `combinerawtransaction` RPC (brunoerg)
Pull request description:
This PR adds test coverage for the following errors for the `combinerawtransaction` RPC:
* Tx decode failed
* Missing transactions
* Input not found or already spent
For reference: https://maflcko.github.io/b-c-cov/total.coverage/src/rpc/rawtransaction.cpp.gcov.html
ACKs for top commit:
maflcko:
lgtm ACK ab98e6fd03
tdb3:
ACK ab98e6fd03
Tree-SHA512: 8a133c25dad2e1b236e0278a88796f60f763e3fd6fbbc080f926bb23f9dcc55599aa242d6e0c4ec15a179d9ded10a1f17ee5b6063719107ea84e6099f10416b2
0000276b31 test: Remove redundant verack check (MarcoFalke)
Pull request description:
Currently the sync in `connect_nodes` mentions the `version` and `verack` message types, but only checks the `verack`. Neither check is required, as the `pong` check implies both. In case of failure, the debug log will have to be consulted anyway, so the redundant check doesn't add value.
Also clarify in the comments that the goal is to check the flag `fSuccessfullyConnected` indirectly.
ACKs for top commit:
furszy:
utACK 0000276b31
brunoerg:
ACK 0000276b31
tdb3:
ACK 0000276b31
Tree-SHA512: f9ddcb1436d2f70da462a8dd470ecfc90a534dd6507c23877ef7626e7c02326c077001a42ad0171a87fba5c5275d1970d8c5e5d82c56c8412de944856fdfd6db
This commit fixes a dormant bug in MiniWallet that exists since
support for P2TR was initially added in #23371 (see commit
041abfebe4).
In the course of spending the output, the leaf version byte of the
control block in the witness stack doesn't set the parity bit, i.e.
we were so far just lucky that the used combinations of relevant
data (internal pubkey, leaf script / version) didn't result in a
tweaked pubkey with odd y-parity. If that was the case, we'd get the
following validation error:
`mandatory-script-verify-flag-failed (Witness program hash mismatch) (-26)`
Since MiniWallets can now optionally be tagged (#29939), resulting
in different internal pubkeys, the issue is more prevalent now.
Fix it by passing the parity bit, as specified in BIP341.
Rather than only returning the internal key from the P2TR anyone-can-spend
address creation routine, provide the whole TaprootInfo object, which in turn
contains a dictionary of TaprootLeafInfo object for named leaves.
This data is used in MiniWallet for the default ADDRESS_OP_TRUE mode, in order
to deduplicate the witness script and leaf version of the control block.
39d135e79f test: MiniWallet: respect fee_rate for target_weight, use in mempool_limit.py (Sebastian Falbesoner)
b2f0a9f8b0 test: add framework functional test for MiniWallet's tx padding (Sebastian Falbesoner)
c17550bc3a test: MiniWallet: fix tx padding (`target_weight`) for large sizes, improve accuracy (Sebastian Falbesoner)
Pull request description:
MiniWallet allows to create padded transactions that are equal or slightly above a certain `target_weight` (first introduced in PR #25379, commit 1d6b438ef0), which can be useful especially for mempool-related tests, e.g. for policy limit checks or scenarios to trigger mempool eviction. Currently the `target_weight` parameter doesn't play together with `fee_rate` though, as the fee calculation is incorrectly based on the tx vsize before the padding output is added, so the fee-rate is consequently far off. This means users are forced to pass an absolute fee, which can be quite inconvenient and leads to lots of duplicated "calculate absolute fee from fee-rate and vsize" code with the pattern `fee = (feerate / 1000) * (weight // 4)` on the call-sites.
This PR first improves the tx padding itself to be more accurate, adds a functional test for it, and fixes the `fee_rate` treatment for the `{create,send}_self_transfer` methods. (Next step would be to enable this also for the `_self_transfer_multi` methods, but those currently don't even offer a `fee_rate` parameter). Finally, the ability to pass both `target_weight` and `fee_rate` is used in the `mempool_limit.py` functional test. There might be more use-cases in other tests, that could be done in a follow-up.
ACKs for top commit:
rkrux:
tACK [39d135e](39d135e79f)
ismaelsadeeq:
Code Review ACK 39d135e79f🚀
glozow:
light review ACK 39d135e79f
Tree-SHA512: 6bf8e853a921576d463291d619cdfd6a7e74cf92f61933a563800ac0b3c023a06569b581243166906f56b3c5e8858fec2d8a6910d55899e904221f847eb0953d
d1581c6048 test: doc: fix units in tx size standardness test (s/vbytes/weight units) (Sebastian Falbesoner)
Pull request description:
This small fixup PR is a late follow-up for #17947 (commit 4537ba5f21), where the wrong units has been used in the comments for the large tx composition.
ACKs for top commit:
tdb3:
ACK d1581c6048
ismaelsadeeq:
ACK d1581c6048
glozow:
ACK d1581c6048
Tree-SHA512: ea2de42174f9dca0608275ea377c852ebddc5a04a2b32248ce808aea33d7e00cdee3a225b24c0cf426c69646cccbbc31273c62f7bc1647bb3443a61de3b15670
Relax assumptions about in-mempool children of in-mempool
parents. With package RBF, we will allow a package of size
2 with conflicts on its parent and reconsider the parent
if its fee is insufficient on its own.
Consider:
TxA (in mempool) <- TxB (in mempool)
TxA (in mempool) <- TxB' (in package, conflicts with TxB) <-
TxC (in package)
If TxB' fails to RBF TxB due to insufficient feerate, the
package TxB' + TxC will be considered. PackageV3Checks
called on TxB' will see an in-mempool parent TxA, and
see the in-mempool child TxB. We cannot assume there is
no in-mempool sibling, rather detect it and fail normally.
Prior to package RBF, this would have failed on the first
conflict in package.
f68cba29b3 blockman: Replace m_reindexing with m_blockfiles_indexed (Ryan Ofsky)
1b1c6dcca0 test: Add functional test for continuing a reindex (TheCharlatan)
201c1a9282 indexes: Don't wipe indexes again when already reindexing (TheCharlatan)
804f09dfa1 kernel: Add less confusing reindex options (Ryan Ofsky)
e172553223 validation: Remove needs_init from LoadBlockIndex (TheCharlatan)
533eab7d67 bugfix: Streamline setting reindex option (TheCharlatan)
Pull request description:
When restarting `bitcoind` during an ongoing reindex without setting the `-reindex` flag again, the block and coins db is left intact, but any data from the optional indexes is discarded. While not a bug per se, wiping the data again is
wasteful, both in terms of having to write it again, as well as potentially leading to longer startup times. So keep the index data instead when continuing a prior reindex.
Also includes a bugfix and smaller code cleanups around the reindexing code. The bug was introduced in b47bd95920: "kernel: De-globalize fReindex".
ACKs for top commit:
stickies-v:
ACK f68cba29b3
fjahr:
Code review ACK f68cba29b3
furszy:
Code review ACK f68cba29b3
ryanofsky:
Code review ACK f68cba29b3. Only changes since last review were cherry-picking suggested commits that rename variables, improving comments, and making some tweaks to test code.
Tree-SHA512: b252228cc76e9f1eaac56d5bd9e4eac23408e0fc04aeffd97a85417f046229364673ee1ca7410b9b6e7b692b03f13ece17c42a10176da0d7e975a8915deb98ca
This adds a bitset module that implements a BitSet<N> class, a variant
of std::bitset with a few additional features that cannot be implemented
in a wrapper without performance loss (specifically, finding first and
last bit set, or iterating over all set bits).
As per doc/developer-notes#logging, LogError should be used for
severe problems that require the node to shut down.
Co-Authored-By: stickies-v <stickies-v@protonmail.com>
fa780e1c25 build: Remove --enable-gprof (MarcoFalke)
Pull request description:
It is unclear what benefit this option has, given that:
* `gprof` requires re-compilation (`perf` and other tools can instead be used on existing executables)
* `gprof` requires hardening to be disabled
* `gprof` doesn't work with `clang`
* `perf` is documented in the dev-notes, and test notes, and embedded into the functional test framework; `gprof` isn't
* Anyone really wanting to use it could pass the required flags to `./configure`
* I couldn't find any mention of the use of `gprof` in the discussions in this repo, apart from the initial pull request adding it (cfaac2a60f)
* Keeping it means that it needs to be maintained and ported to CMake
Fix all issues by removing it.
ACKs for top commit:
TheCharlatan:
ACK fa780e1c25
hebasto:
ACK fa780e1c25, I have reviewed the code and it looks OK.
willcl-ark:
crACK fa780e1c25
Tree-SHA512: 0a9ff363ac2bec8b743878a4e3147f18bc16823d00c5007568432c36320bd0199b13b6d0ce828a9a83c2cc434c058afaa64eb2eccfbd93ed85b81ce10c41760c
0d3ef83433 ci: Use relative paths in `win64-native` CI job consistently (Hennadii Stepanov)
501aceefcf ci: Remove no longer needed workaround for GHA Windows images (Hennadii Stepanov)
Pull request description:
This PR:
1. Removes no longer needed workaround for GHA Windows images.
GHA Windows images previously had multiple VC Build Tools installed, which required specifying the `VCPKG_PLATFORM_TOOLSET_VERSION` explicitly to avoid linker errors. This issue has been resolved as per
https://github.com/actions/runner-images/issues/9701.
2. Switches all references to temporary files to relative ones for consistency and readability.
ACKs for top commit:
sipsorcery:
ACK 0d3ef83433.
maflcko:
ACK 0d3ef83433
Tree-SHA512: e832b87fc6dee1e9d1eb452797f16b732e776c2ecdbe3dc9e64cc48ce9b5b89c569d5b96b999423ae1261ff4bf684b7003af84d8024ef5260682f531c4e8ff5e
15796d4b61 build: warn on self-assignment (Cory Fields)
53372f2176 refactor: disable self-assign warning for tests (Cory Fields)
Pull request description:
Belt-and suspenders after #30234. Self-assignment should be safe _and_ discouraged.
We used to opt out of this warning because something deep in our serialization/byteswapping code could self-assign, but that doesn't appear to be the case anymore.
ACKs for top commit:
maflcko:
ACK 15796d4b61
fanquake:
ACK 15796d4b61 - not a huge fan of inline pragma usage, but this seems fine, given it's to work around an already-fixed compiler bug, and we'll only be carrying it for a shortish time in any case.
Tree-SHA512: 1f95f7c730b974ad1da55ebd381040bac312f2f380fff9d569ebab91d7c1963592a84d1613d81d96238c6f5a66aa40deebba68a76f6b24b02150d0a77c769654
fab01b5220 refactor: performance-for-range-copy in psbt.h (MarcoFalke)
Pull request description:
A copy of the partial signatures is not required before serializing them.
For reference, this was found by switching the codebase to `std::span` (not sure why it wasn't found with `Span` though):
```
./psbt.h:240:23: error: loop variable is copied but only used as const reference; consider making it a const reference [performance-for-range-copy,-warnings-as-errors]
240 | for (auto sig_pair : partial_sigs) {
| ^
| const &
ACKs for top commit:
tdb3:
ACK fab01b5220
theStack:
utACK fab01b5220
Tree-SHA512: b55513d8191118499716684190ee568d171b50ae9171d246ca6e047f0cfd3ec14c9453d721e88af55e47bb41fa66cbafdbfb47bc5f9b8d82789e0a9b634b371b
1f6ab1215b minor: remove unnecessary semicolons from RPC content type examples (Matthew Zipkin)
b225295298 test: use json-rpc 2.0 in all functional tests by default (Matthew Zipkin)
391843b029 bitcoin-cli: use json-rpc 2.0 (Matthew Zipkin)
d39bdf3397 test: remove unused variable in interface_rpc.py (Matthew Zipkin)
0ead71df8c doc: update and link for JSON-RPC 2.0 (Matthew Zipkin)
Pull request description:
This is a follow-up to #27101.
- Addresses [post-merge comments ](https://github.com/bitcoin/bitcoin/pull/27101#discussion_r1606723428)
- bitcoin-cli uses JSON-RPC 2.0
- functional tests use JSON-RPC 2.0 by default (exceptions are in the regression tests added by #27101)
ACKs for top commit:
tdb3:
ACK 1f6ab1215b
cbergqvist:
ACK 1f6ab1215b
Tree-SHA512: 49bf14c70464081280216ece538a2f5ec810bac80a86a83ad3284f0f1b017edf755a1a74a45be279effe00218170cafde7c2de58aed07097a95c2c6b837a6b6c
In order to ensure that the change of nVersion to a uint32_t in the
previous commit has no effect, rename nVersion to version in this commit
so that reviewers can easily spot if a spot was missed or if there is a
check somewhere whose semantics have changed.
This is a just a mechanical change, renaming and inverting the meaning
of the indexing variable.
"m_blockfiles_indexed" is a more straightforward name for this variable
because this variable just indicates whether or not
<datadir>/blocks/blk?????.dat files have been indexed in the
<datadir>/blocks/index LevelDB database. The name "m_reindexing" was
more confusing, it could be true even if -reindex was not specified, and
false when it was specified. Also, the previous name unnecessarily
required thinking about the whole reindexing process just to understand
simple checks in validation code about whether blocks were indexed.
The motivation for this change is to follow up on previous commits,
moving away from having multiple variables called "reindex" internally,
and instead naming variables individually after what they do and
represent.
Before this change continuing a reindex without the -reindex flag set
would leave the block and coins db intact, but discard the data of the
optional indexes. While not a bug per se, wiping the data again is
wasteful, both in terms of having to write it again, and potentially
leading to longer startup times.
When initially running a reindex, both the block index and any further
activated indexes are wiped. On an index's Init(), both the best block
stored by the index and the chain's tip are null. An index's m_synced
member is therefore true. This means that it will process blocks through
validation events while the reindex is running.
Currently, if the reindex is continued without the user re-specifying
the reindex flag, the block index is preserved but further index data is
wiped. This leads to the stored best block being null, but the chain tip
existing. The m_synced member will be set to false. The index will not
process blocks through the validation interface, but instead use the
background sync once the reindex is completed.
If the index is preserved (this change) after a restart its best block
may potentially match the chain tip. The m_synced member will be set to
true and the index can process validation events during the rest of the
reindex.
Drop confusing kernel options:
BlockManagerOpts::reindex
ChainstateLoadOptions::reindex
ChainstateLoadOptions::reindex_chainstate
Replacing them with more straightforward options:
ChainstateLoadOptions::wipe_block_tree_db
ChainstateLoadOptions::wipe_chainstate_db
Having two options called "reindex" which did slightly different things
was needlessly confusing (one option wiped the block tree database, and
the other caused block files to be rescanned). Also the previous set of
options did not allow rebuilding the block database without also
rebuilding the chainstate database, when it should be possible to do
those independently.
30a01134cd [doc] update bips.md for 431 (glozow)
9dbe6a03f0 [test] wallet uses CURRENT_VERSION which is 2 (glozow)
539404fe0f [policy] make v3 transactions standard (glozow)
052ede75af [refactor] use TRUC_VERSION in place of 3 (glozow)
Pull request description:
Make `nVersion=3` (which is currently nonstandard on mainnet) standard.
Note that we will treat these transactions as Topologically Restricted Until Confirmation (TRUC). Spec is in BIP 431 and implementation is in #28948, #29306, and #29873
See #27463 for overall project tracking, and #29319 for information about relevance to cluster mempool.
ACKs for top commit:
sdaftuar:
utACK 30a01134c
achow101:
ACK 30a01134cd
instagibbs:
utACK 30a01134cd
murchandamus:
ACK 30a01134cd
ismaelsadeeq:
ACK 30a01134cd🛰️
Tree-SHA512: 2a4aec0442c860e792a061d83e36483c1f1b426f946efbdf664c8db97a596e498b535707e1d3a900218429486ea69fd4552e3d476526a6883cbd5556c6534b48
7b8eea067f tests: add fuzz tests for VecDeque (Pieter Wuille)
62fd24af6a util: add VecDeque (Pieter Wuille)
Pull request description:
Extracted from #30126.
This adds a `VecDeque` data type, inspired by `std::deque`, but backed by a single allocated memory region used as a ring buffer instead of a linked list of arrays. This gives better memory locality and less allocation overhead, plus better guarantees (some C++ standard library implementations, though not libstdc++ and libc++, use a separate allocation per element in a deque).
It is intended for the candidate set search queue in #30126, but may be useful as a replacement for `std::deque` in other places too. It's not a full drop-in replacement, as I did not add iteration support which is unnecessary for the intended use case, but nothing prevents adding that if needed.
Everything is tested through a simulation-based fuzz test that compares the behavior with normal `std::deque` equivalent operations, both for trivially-copyable/destructible types and others.
ACKs for top commit:
instagibbs:
reACK 7b8eea067f
cbergqvist:
re-ACK 7b8eea067f
hebasto:
re-ACK 7b8eea067f, I've verified changes since my recent [review](https://github.com/bitcoin/bitcoin/pull/30161#pullrequestreview-2103018546) with
glozow:
ACK 7b8eea067f
Tree-SHA512: 1b62f3ba1a43a1293d8c9de047e2399442e74c46de2df81406151fe27538716ce265f35fb6779ee56d77a39cddf8fb4b4e15bda8f04ebf3b149e2f05fa55cb21
It does not control any actual logic and the log message as well as the
comment are obsolete, since no database initialization takes place there
anymore. Log messages indicating when indexes and chainstate databases
are loaded exist in other places.
Reverts a bug introduced in b47bd95920
"kernel: De-globalize fReindex". The change leads to a GUI user being
prompted to re-index on a chainstate loading failure more than once as
well as the node actually not reindexing if the user chooses to. Fix
this by setting the reindexing option instead of the atomic, which can
be safely re-used to indicate that a reindex should be attempted.
The bug specifically is caused by the chainman, and thus the blockman
and its m_reindexing atomic being destroyed on every iteration of
the for loop.
The reindex option for ChainstateLoadOptions is currently also set in a
confusing way. By using the reindex atomic, it is not obvious in which
scenario it is true or false.
The atomic is controlled by both the user passing the -reindex option,
the user chosing to reindex if something went wrong during chainstate
loading when running the gui, and by reading the reindexing flag from
the block tree database in LoadBlockIndexDB. In practice this read is
done through the chainstate module's CompleteChainstateInitialization's
call to LoadBlockIndex. Since this is only done after the reindex option
is set already, it does not have an effect on it.
Make this clear by using the reindex option from the blockman opts which
is only controlled by the user.
GHA Windows images previously had multiple VC Build Tools installed,
which required specifying the `VCPKG_PLATFORM_TOOLSET_VERSION`
explicitly to avoid linker errors. This issue has been resolved as per
https://github.com/actions/runner-images/issues/9701.
fa52e13ee8 test: Remove struct.pack from almost all places (MarcoFalke)
fa826db477 scripted-diff: test: Use int.to_bytes over struct packing (MarcoFalke)
faf2a975ad test: Use int.to_bytes over struct packing (MarcoFalke)
faf3cd659a test: Normalize struct.pack format (MarcoFalke)
Pull request description:
`struct.pack` has many issues:
* The format string consists of characters that may be confusing and may need to be looked up in the documentation, as opposed to using easy to understand self-documenting code.
This lead to many test bugs, which weren't hit, which is fine, but still confusing. Ref: https://github.com/bitcoin/bitcoin/pull/29400, https://github.com/bitcoin/bitcoin/pull/29399, https://github.com/bitcoin/bitcoin/pull/29363, fa3886b7c6, ...
Fix all issues by using the built-in `int` helpers `to_bytes` via a scripted diff.
Review notes:
* For `struct.pack` and `int.to_bytes` the error behavior is the same, although the error messages are not identical.
* Two `struct.pack` remain. One for float serialization in a C++ code comment, and one for native serialization.
ACKs for top commit:
achow101:
ACK fa52e13ee8
rkrux:
tACK [fa52e13](fa52e13ee8)
theStack:
Code-review ACK fa52e13ee8
Tree-SHA512: ee80d935b68ae43d1654b047e84ceb80abbd20306df35cca848b3f7874634b518560ddcbc7e714e2e7a19241e153dee64556dc4701287ae811e26e4f8c57fe3e
This is an STL-like container that interface-wise looks like std::deque, but
is backed by a (fixed size, with vector-like capacity/reserve) circular buffer.
Belt-and suspenders after #30234. Self-assignment should be safe _and_
discouraged.
We used to opt out of this warning because something deep in our
serialization/byteswapping code could self-assign, but that doesn't appear to
be the case anymore.
This new test uses the `vExtraTxnForCompact` (`extra_txn`) vector of
optional orphan/conflicted/etc. transactions to provide a transaction
in a compact block that was not otherwise present in our mempool.
This also covers an improbable nullptr deref bug addressed in
bf031a517c (#29752) where the
`extra_txn` vec/circular-buffer was sometimes null-initialized and
not yet filled when dereferenced in `PartiallyDownloadedBlock::InitData`.
232928b58a build: no-longer allow GCC-10 in C++20 check (fanquake)
Pull request description:
Reverts part of fa67f096bd, now that we require a minimum of GCC 11.
See also:
https://github.com/bitcoin/bitcoin/pull/28349#issuecomment-1745143612.
ACKs for top commit:
maflcko:
utACK 232928b58a
theuni:
utACK 232928b58a
Tree-SHA512: 10e0adac2dd6e455aaf97ebfe73c7586430349fc27ac435bc6c0d99a4934a380398d14467aacd9cedf371345da291366b3ab2c3be7db5d97e21ad6212b2c7890
The helper assumes that the n and k values have to be provided as a
single byte push operation, which is only possible for values up to 16.
Fix that by passing the numbers directly to the CScript list, where it's
automatically converted to minimally-encoded pushes (see class
method `CScript.__coerce_instance`, branch `isinstance(other, int)`).
In case of 17..20, this means that the data-pushes are done with two
bytes using OP_PUSH1 (0x01), e.g. for n=20: 0x01,0x14
5f2c1d84e3 guix: show *_FLAGS variables in pre-build output (fanquake)
Pull request description:
For example:
```bash
# ADDITIONAL_GUIX_COMMON_FLAGS set in the ENV
ADDITIONAL_GUIX_ENVIRONMENT_FLAGS="--emulate-fhs" ./contrib/guix/guix-build
<snip>
INFO: Building f751991 for platform triple x86_64-linux-gnu:
...using reference timestamp: 1716905119
...running at most 10 jobs
...from worktree directory: '/bitcoin'
...bind-mounted in container to: '/bitcoin'
...in build directory: '/bitcoin/guix-build-f75199182133/distsrc-f75199182133-x86_64-linux-gnu'
...bind-mounted in container to: '/distsrc-base/distsrc-f75199182133-x86_64-linux-gnu'
...outputting in: '/bitcoin/guix-build-f75199182133/output/x86_64-linux-gnu'
...bind-mounted in container to: '/outdir-base/x86_64-linux-gnu'
ADDITIONAL FLAGS (if set)
ADDITIONAL_GUIX_COMMON_FLAGS: --no-substitutes
ADDITIONAL_GUIX_ENVIRONMENT_FLAGS: --emulate-fhs
ADDITIONAL_GUIX_TIMEMACHINE_FLAGS:
```
ACKs for top commit:
hebasto:
ACK 5f2c1d84e3.
Tree-SHA512: 85a6d508499b4ec1d6166343a1707b682d327b2fcfb2fb438571894478aac0062d21e1239b5092091ff98711c5c747151973c4f325a7a7c447d0e807166fcb07
4444de152f test: Set mocktime in p2p_disconnect_ban.py to avoid intermittent test failure (MarcoFalke)
fa6aa4027c test: Fix typos and use names args (MarcoFalke)
Pull request description:
Otherwise, the test may fail on slow hardware when running in valgrind.
Also, use named args for the absolute timepoint, while touching this file.
ACKs for top commit:
tdb3:
ACK for 4444de152f
AngusP:
re-ACK 4444de152f
Tree-SHA512: 660269c8dd18887d69b284f38656899caf028159ce83ddf921f3e9c080a5d0e663989f0e42b4baf4c4939f20f34da0e7e844dff9b7c91d0cab570c60958bd0e1
8801e319d5 refactor: remove unused `CKey::Negate` method (Sebastian Falbesoner)
Pull request description:
This method was introduced as a pre-requirement for the v2 transport protocol back then (see PR #14047, commit 463921bb), when it was still BIP151. With the replacement BIP324, this is not needed anymore, and it's also unlikely that for any other proposal we'd ever need to negate private keys at this abstraction level. I'd argue that this operation is usually something that should happen within a secp256k1 module (like e.g. done in MuSig2, Silent Payments...).
(If there is really demand in the future, it's also trivial to reintroduce the method.)
ACKs for top commit:
laanwj:
ACK 8801e319d5
sipa:
ACK 8801e319d5
achow101:
ACK 8801e319d5
Tree-SHA512: 7bc1566399635c5c6e4940a2724c865d5443eb190024379099330c023c516f1e4f423ed9e8c42bc93413b723a5464ec79d3f879f58c0e598fe24f495238df4ec
07aba8dd21 functional test: ensure confirmed utxo being sourced for 2nd chain (Greg Sanders)
Pull request description:
The test could fail/stop testing what we want if non-confirmed utxos become sourced through some internal change to `MiniWallet`; better to just fetch confirmed explicitly.
ACKs for top commit:
achow101:
ACK 07aba8dd21
ismaelsadeeq:
utACK 07aba8dd21
theStack:
ACK 07aba8dd21
Tree-SHA512: 66795fdf881139ed91bde0f8239a46bd9bc70bb311fa97c0e2b5537e1fd2a1fd36bf3a225fc77b9695deb835a9d6d29879aa1e05ea5054b9a33a400e199da014
2451a217dd test: addmultisigaddress, coverage for script size limits (furszy)
53302a0981 bugfix: addmultisigaddress, add unsupported operation for redeem scripts over 520 bytes (furszy)
9be6065cc0 test: coverage for 16-20 segwit multisig scripts (furszy)
9d9a91c4ea rpc: bugfix, incorrect segwit redeem script size used in signrawtransactionwithkey (furszy)
0c9fedfc45 fix incorrect multisig redeem script size limit for segwit (furszy)
f7a173b578 test: rpc_createmultisig, decouple 'test_sortedmulti_descriptors_bip67' (furszy)
4f33dbd8f8 test: rpc_createmultisig, decouple 'test_mixing_uncompressed_and_compressed_keys' (furszy)
25a81705d3 test: rpc_createmultisig, remove unnecessary checkbalances() (furszy)
b5a3289433 test: refactor, multiple cleanups in rpc_createmultisig.py (furszy)
3635d43268 test: rpc_createmultisig, remove manual wallet initialization (furszy)
Pull request description:
Fixing https://github.com/bitcoin/bitcoin/issues/28250#issuecomment-1674830104 and more.
Currently, redeem scripts longer than 520 bytes, which are technically valid under segwit rules, have flaws in the following processes:
1) The multisig creation process fails to deduce the output descriptor, resulting in the generation of an incorrect descriptor. Additionally, the accompanying user warning is also inaccurate.
2) The `signrawtransactionwithkey` RPC command fail to sign them.
3) The legacy wallet `addmultisigaddress` wrongly discards them.
The issue arises because most of these flows are utilizing the legacy spkm keystore, which imposes
the [p2sh max redeem script size rule](ded6873340/src/script/signingprovider.cpp (L160)) on all scripts. Which blocks segwit redeem scripts longer than
the max element size in all the previously mentioned processes (`createmultisig`, `addmultisigaddress`, and
`signrawtransactionwithkey`).
This PR fixes the problem, enabling the creation of multisig output descriptors involving more than 15 keys and
allowing the signing of these scripts, along with other post-segwit redeem scripts that surpass the 520-byte
p2sh limit.
Important note:
Instead of adding support for these longer redeem scripts in the legacy wallet, an "unsupported operation"
error has been added. The reasons behind this decision are:
1) The introduction of this feature brings about a compatibility-breaking change that requires downgrade
protection; older wallets would be unable to interact with these "new" legacy wallets.
2) Considering the ongoing deprecation of the legacy spkm, this issue provides another compelling
reason to transition towards descriptors.
Testing notes:
To easily verify each of the fixes, I decoupled the tests into standalone commits. So they can be
cherry-picked on top of master. Where `rpc_createmultisig.py` (with and without the `--legacy-wallet`
arg) will fail without the bugs fixes commits.
Extra note:
The initial commits improves the `rpc_createmultisig.py` test in many ways. I found this test very
antiquated, screaming for an update and cleanup.
ACKs for top commit:
pinheadmz:
ACK 2451a217dd
theStack:
Code-review ACK 2451a217dd
achow101:
ACK 2451a217dd
Tree-SHA512: 71794533cbd46b3a1079fb4e9d190d3ea3b615de0cbfa443466e14f05e4616ca90e12ce2bf07113515ea8113e64a560ad572bb9ea9d4835b6fb67b6ae596167f
d7290d662f fuzz: wallet, add target for Crypter (Ayush Singh)
Pull request description:
This PR adds fuzz coverage for `wallet/crypter`.
Motivation: Issue [27272](https://github.com/bitcoin/bitcoin/issues/27272#issue-1628327906)
I ran this for a long time with Sanitizers on and had no crashes; the average `exec/sec` also looks good to me. However, I would really appreciate it if some of the reviewers could try it on their machines too, and give their feedback.
ACKs for top commit:
maflcko:
utACK d7290d662f
achow101:
ACK d7290d662f
brunoerg:
utACK d7290d662f
Tree-SHA512: f5c496cabdd3263a7e1ad49eeff702725336f76bf19a82e5dbbead082e990889dd43c851d0d2d6ab740f44b8ec2aa06defd9ff6b02be68b5f8b4eaf963f88599
7f3f6c6dc8 refactor: replace hardcoded numbers (Lőrinc)
5676aec1e1 refactor: Model the bech32 charlimit as an Enum (josibake)
Pull request description:
Broken out from #28122
---
Bech32(m) was defined with a 90 character limit so that certain guarantees for error detection could be made for segwit addresses (see https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki#checksum-design).
However, there is nothing about the encoding scheme itself that requires a limit of 90 and in practice bech32(m) is being used without the 90 char limit (e.g. lightning invoices, silent payments). Further, increasing the character limit doesn't do away with error detection, it simply changes the guarantee.
The primary motivation for this change is for being able to parse BIP352 v0 silent payment addresses (see 622c7a98b9), which require up to 118 characters. In addition to BIP352, modeling the character limit as an enum allows us to easily support new address types that use bech32m and specify their own character limit.
ACKs for top commit:
paplorinc:
re-ACK 7f3f6c6dc8
achow101:
ACK 7f3f6c6dc8
theuni:
utACK 7f3f6c6dc8
Tree-SHA512: 9c793d657448c1f795093b9f7d4d6dfa431598f48d54e1c899a69fb2f43aeb68b40ca2ff08864eefeeb6627d4171877234b5df0056ff2a2b84415bc3558bd280
fa3169b073 rpc: Remove index-based Arg accessor (MarcoFalke)
Pull request description:
The index-based Arg accessor is redundant with the name-based one. It does not provide any benefit to the code reader, or otherwise, so remove it.
ACKs for top commit:
stickies-v:
re-ACK fa3169b073, addressed doc nits
achow101:
ACK fa3169b073
ryanofsky:
Code review ACK fa3169b073. One changes since last review are some documentation improvements
Tree-SHA512: f9da1c049dbf38c3b47a8caf8d24d195c2d4b88c7ec45a9ccfb78f1e39f29cb86869f84b308f6e49856b074c06604ab634c90eb89c9c93d2a8169e070aa1bd40
df6dc2aaae test: Assumeutxo: snapshots with less work should not be loaded (Hernan Marino)
Pull request description:
This PR adds a test which checks that snapshots with less accumulated work than the node's active chain, should not be loaded and return with an error. Although in a different context of discussion the missing test was detect in a thread in https://github.com/bitcoin/bitcoin/pull/29394 (see https://github.com/bitcoin/bitcoin/pull/29394#discussion_r1484122214)
ACKs for top commit:
maflcko:
utACK df6dc2aaae
kevkevinpal:
utACK [df6dc2a](df6dc2aaae)
achow101:
ACK df6dc2aaae
alfonsoromanz:
Re ACK df6dc2aaae. Make is successful and the test passes.
Tree-SHA512: 07a394b4b288cc8ad3f66ed4e70dcda468db18113e9442eb7215cf491768432d55efaaa5b79d633094917e05475a30f0c5e4f64f8f2da293ba306891b4485560
9013e2b97e Link to gen-bitcoin-conf.sh instead of bitcoin.conf placeholder (Epic Curious)
Pull request description:
Closes#30153.
This PR updates `doc/init.md` to mention generating an example bitcoin.conf instead of referencing the placeholder `share/examples/bitcoin.conf`. Also changes the code-formatted text to a markdown link.
## Background
- Two years ago, `share/examples/bitcoin.conf` was replaced with [a placeholder file](b483084d86). To see an example `bitcoin.conf`, the user now runs the `contrib/devtools/gen-bitcoin-conf.sh` script, which replaces the placeholder file with the parsed contents of `bitcoind --help`.
- The instructions in `init.md` about an example `bitcoin.conf` haven't changed significantly since they were [added almost 10 years ago](234bfbf6a5/doc/init.md (L39)). They should be updated to improve clarity.
ACKs for top commit:
edilmedeiros:
ACK 9013e2b97e
kevkevinpal:
reACK [9013e2b](9013e2b97e)
achow101:
ACK 9013e2b97e
stickies-v:
ACK 9013e2b97e
tdb3:
ACK for 9013e2b97e
Tree-SHA512: 5ac5ad672ad181d574e19e29c3727fb9e5373282444fae09b42d113d5c8915cb2829d496212638cdc4b70540b7e1794a751fcdc9539f956a594cddd70c8fd747
71aae72e1f test: test sendall does ancestor aware funding (ishaanam)
36757941a0 wallet, rpc: implement ancestor aware funding for sendall (ishaanam)
544131f3fb rpc, test: test sendall spends unconfirmed change and unconfirmed inputs when specified (ishaanam)
Pull request description:
This PR:
- Adds a functional test that `sendall` spends unconfirmed change
- Adds a functional test that `sendall` spends regular unconfirmed inputs when specified by user
- Adds ancestor aware funding to `sendall` by using `calculateCombinedBumpFee` and adjusting the effective value accordingly
- Adds a functional test for ancestor aware funding in `sendall`
ACKs for top commit:
S3RK:
ACK 71aae72e1f
achow101:
ACK 71aae72e1f
furszy:
ACK 71aae72e1f
Tree-SHA512: acaeb7c65166ce53123a1d6cb5012197202246acc02ef9f37a28154cc93afdbd77c25e840ab79bdc7e0b88904014a43ab1ddea79d5337dc310ea210634ab61f0
bd34dd85e7 Use `exact_target` shorthand in coinselector_tests (Murch)
7aa7e30441 Fold GetSelectionWaste() into ComputeAndSetWaste() (Murch)
Pull request description:
PR #26152 moved waste calculation into SelectionResult to be able to correct the waste score on basis of the bump_fee_group_discount for overlapping ancestries. This left two functions with largely overlapping purpose, where one was simply a wrapper of the other. This PR cleans up the overlap, and fixes the double-meaning of `change_cost` where the `GetChange()` function assumed that no change was created when `change_cost` was set to 0. This behavior was exploited in a bunch of tests, but is problematic, because a `change_cost` of 0 is permitted with custom settings for feerate and discard_feerate (i.e. when they’re both 0).
ACKs for top commit:
achow101:
ACK bd34dd85e7
furszy:
Code ACK bd34dd85e7
ismaelsadeeq:
Code Review ACK bd34dd85e7
Tree-SHA512: 83a2688d45d719dc61a64b5180fe136107faccf401a59df65245c05d701748a03e85ed56fde8c9b7ef39a3ab54374dd3718c559bda5b3f55dafedfd7fed25161
22d0f1a27e [fuzz] Avoid endless waiting in FuzzedSock::{Wait,WaitMany} (marcofleon)
a7fceda68b [fuzz] Make peeking through FuzzedSock::Recv fuzzer friendly (dergoegge)
865cdf3692 [fuzz] Use fuzzer friendly ConsumeRandomLengthByteVector in FuzzedSock::Recv (dergoegge)
Pull request description:
`FuzzedSock` has a few issues that block a fuzzer from making progress. See commit messages for details.
ACKs for top commit:
marcofleon:
Tested ACK 22d0f1a27e
brunoerg:
utACK 22d0f1a27e
Tree-SHA512: 2d66fc94ba58b6652ae234bd1dcd33b7d685b5054fe83e0cd624b053dd51519c23148f43a865ab8c8bc5fc2dc25e701952831b99159687474978a90348faa4c5
ffbc173ca1 depends: Update Boost download link (Hennadii Stepanov)
Pull request description:
The Boost has [migrated](https://github.com/boostorg/boost-tasks/pull/3) their downloads from from boostorg.jfrog.io to archives.boost.io.
So do we.
FWIW, the download speed at my location is much better :)
ACKs for top commit:
maflcko:
ACK ffbc173ca1
Tree-SHA512: 3f66675c390510ecfdacf2ac8af4a0a6bdbdf6fbfe01ed66fbe1b11bc9e935709e5492a754711cd788352c5853738fce755afd0c0480f36d0f8af31fcc108263
This method was introduced as a pre-requirement for the v2 transport
protocol back then (see PR #14047, commit 463921bb), when it was still
BIP151. With the replacement BIP324, this is not needed anymore, and
it's also unlikely that any other proposal would need to negate private
keys at this abstraction level.
(If there is really demand, it's trivial to reintroduce the method.)
3c08e11c3e doc: JSON-RPC request Content-Type is application/json (Luke Dashjr)
Pull request description:
Specify json content type in RPC examples.
Picks up #29946. Which needed rebasing and the commit message fixing,
ACKs for top commit:
laanwj:
ACK 3c08e11c3e
tdb3:
ACK for 3c08e11c3e
Tree-SHA512: 770bbbc0fb324cb63628980b13583cabf02e75079851850170587fb6eca41a70b01dcedaf1926bb6488eb9816a3cc6616fe8cee8c4b7e09aa39b7df5834ca0ec
e3249f2111 fuzz: add more coverage for `ScriptPubKeyMan` (brunoerg)
Pull request description:
This PR adds more coverage for `ScriptPubKeyMan`:
- Check `GetKey` and `HasPrivKey` after adding descriptor key.
- Cover `GetEndRange` and `GetKeyPoolSize`.
- Cover `MarkUnusedAddresses` with the scripts from ScriptPubKeys and `GetMetadata` with the destinations from them.
ACKs for top commit:
marcofleon:
Tested ACK e3249f2111. I ran the updated harness for ~9 hours on an empty corpus, generated a coverage report, and checked that the new functions mentioned were hit. Coverage of `scriptpubkeyman.cpp` increased.
murchandamus:
Tested ACK e3249f2111
Tree-SHA512: cfab91f6c8401174842e79209c0e9225c08f011fe9b41d0a58bcec716ae4545eaf803867f899ed7b5fbcefea45711f91894e36df082ba19732dd310cd9e61a79
For example:
```bash
ADDITIONAL_GUIX_COMMON_FLAGS set in the ENV
ADDITIONAL_GUIX_ENVIRONMENT_FLAGS="--emulate-fhs" ./contrib/guix/guix-build
<snip>
INFO: Building f75199182133 for platform triple x86_64-linux-gnu:
...using reference timestamp: 1716905119
...running at most 10 jobs
...from worktree directory: '/bitcoin'
...bind-mounted in container to: '/bitcoin'
...in build directory: '/bitcoin/guix-build-f75199182133/distsrc-f75199182133-x86_64-linux-gnu'
...bind-mounted in container to: '/distsrc-base/distsrc-f75199182133-x86_64-linux-gnu'
...outputting in: '/bitcoin/guix-build-f75199182133/output/x86_64-linux-gnu'
...bind-mounted in container to: '/outdir-base/x86_64-linux-gnu'
ADDITIONAL FLAGS (if set)
ADDITIONAL_GUIX_COMMON_FLAGS: --no-substitutes
ADDITIONAL_GUIX_ENVIRONMENT_FLAGS: --emulate-fhs
ADDITIONAL_GUIX_TIMEMACHINE_FLAGS:
```
Currently, when the FuzzedDataProvider of a FuzzedSock runs out of data,
FuzzedSock::Wait and WaitMany will simulate endless waiting as the
requested events are never simulated as occured.
Fix this by simulating event occurence when ConsumeBool() returns false
(e.g. when the data provider runs out).
Co-authored-by: dergoegge <n.goeggi@gmail.com>
FuzzedSock only supports peeking at one byte at a time, which is not
fuzzer friendly when trying to receive long data.
Fix this by supporting peek data of arbitrary length instead of only one
byte.
cbd4640ede build: remove --enable-lcov-branch-coverage (fanquake)
Pull request description:
This supports lcov `2.x` in the sense that we are no-longer hardcoding version specific options, and instead will use the `LCOV_OPTS` variable (which is the more flexible thing to do in any case). It's also quite likely that devs are already having to pass extra options to lcov `2.x`, given it's more stringent in terms of coverage generation and error checking. See this thread for an example: https://github.com/linux-test-project/lcov/issues/238.
Tested on one machine (LCOV 2.0, gcc 13.2) with:
```bash
./autogen.sh
./configure --enable-lcov CXXFLAGS="-fprofile-update=prefer-atomic" LCOV_OPTS="--rc branch_coverage=1 --ignore-errors mismatch"
make
make cov
<snip>
Processing file src/netaddress.cpp
lines=521 hit=480 functions=72 hit=72 branches=675 hit=499
Overall coverage rate:
lines......: 81.8% (79362 of 97002 lines)
functions......: 77.8% (10356 of 13310 functions)
branches......: 49.6% (130628 of 263196 branches)
```
and another machine (LCOV 2.1, Clang 18.1.3) with:
```bash
./autogen.sh
./configure --enable-lcov CC=clang CXX=clang++ LCOV_OPTS="--rc branch_coverage=1 --ignore-errors mismatch,inconsistent"
make
make cov
<snip>
Processing file src/util/strencodings.cpp
lines=315 hit=311 functions=38 hit=38 branches=425 hit=357
Overall coverage rate:
source files: 622
lines.......: 79.8% (70311 of 88132 lines)
functions...: 78.1% (13968 of 17881 functions)
branches....: 44.5% (157551 of 354317 branches)
Message summary:
101 warning messages:
count: 1
inconsistent: 100
3528 ignore messages:
inconsistent: 3528
```
Related to #28468.
ACKs for top commit:
theuni:
utACK cbd4640ede
hebasto:
ACK cbd4640ede, tested on Ubuntu 22.04.
Tree-SHA512: 94eb01e0e236a480052749f6107b1d0d2e4f6f70a8eefd55fa9ba3d2f72996c9e8a0f28340698b7ac82e7a71e9cf799b7a53ddb6e435e5e9795f5f98a18820f7
8defc182a3 scripted-diff: Replace nNextSweep with m_next_sweep (marcofleon)
0048680467 increase txorphan harness stability (marcofleon)
Pull request description:
This moves `nNextSweep` from being a static variable in `LimitOrphans` to being a member of the `TxOrphanage` class. This improves the stability of the `txorphan` fuzz harness, as each orphanage (created every iteration) now has its own value for `nNextSweep`.
ACKs for top commit:
maflcko:
utACK 8defc182a3
dergoegge:
Code review ACK 8defc182a3
glozow:
utACK 8defc182a3, I can rebase on this pretty easily
Tree-SHA512: 54d4a5074def764f6c895308b94e417662d2f21f157925421131745f22743907df59971f4ce545063658cd74ec133792cdd8df96ae3e69af8314e9b0ff899d48
a27e1ceb9f depends: consolidate dependency docs (fanquake)
Pull request description:
Adds missing `g++` for macOS. This is needed by Qt:
```bash
Configuring qt...
Creating qmake...
gmake[1]: Entering directory '/bitcoin/depends/work/build/arm64-apple-darwin/qt/5.15.14-4bca24c8f89/qtbase/qmake'
gmake[1]: g++: No such file or directory
gmake[1]: *** [Makefile:250: main.o] Error 127
```
`xz-utils` was also missing (but generally already installed), and is needed for the `.tar.xz` tarballs.
Remove `bsdmainutils`, as this is only needed by the main build system (for tests), and isn't needed to complete a depends build.
ACKs for top commit:
maflcko:
ACK a27e1ceb9f
Tree-SHA512: 720c31d4d4c9b86fda4aace405d528193714dd3e526f38d5b8a83e4b676a433b9c891f01d86d673be9ac848458eda8a89b0981003a42eaa6d97bacc2e914396a
0a3631fc35 depends: fix Qt macOS build with Clang 18 (fanquake)
b018bd779d depends: qt 5.15.14 (fanquake)
Pull request description:
Also adds a patch to Qts internal libpng, to fix compilation using Clang 18, when targetting macOS. I'd like to get this patched, so we can continue working on removing `FORCE_USE_SYSTEM_CLANG` (#30201); otherwise builds will be broken using the default Clang (`18`) on the current Ubuntu LTS (`24.04`).
With this PR, anyone using Ubuntu 24.04 should be able to `apt install clang llvm lld`, and then cross-compile for macOS using:
```bash
# clang --version
Ubuntu clang version 18.1.3 (1)
make -C depends HOST=arm64-apple-darwin FORCE_USE_SYSTEM_CLANG=1
./autogen.sh
CONFIG_SITE=/path/to/depends/arm64-apple-darwin/share/config.site ./configure
make
# file src/qt/bitcoin-qt
src/qt/bitcoin-qt: Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK|PIE|HAS_TLV_DESCRIPTORS>
```
ACKs for top commit:
TheCharlatan:
ACK 0a3631fc35
theuni:
utACK 0a3631fc35
hebasto:
ACK 0a3631fc35, a new patch indeed fixes cross-compiling on Ubuntu 24.04 with `FORCE_USE_SYSTEM_CLANG=1`.
Tree-SHA512: 711d321b1efbb1aeef802d1d7e72fff8f4e28aa2420d19df9db6f4449fc7d281e1d08ba242ce20122dfe21129e713bd59e7e6ade0b67d7271eea18b39ceb9283
Adds missing `g++` for macOS. This is needed by Qt:
```bash
Configuring qt...
Creating qmake...
gmake[1]: Entering directory '/bitcoin/depends/work/build/arm64-apple-darwin/qt/5.15.14-4bca24c8f89/qtbase/qmake'
gmake[1]: g++: No such file or directory
gmake[1]: *** [Makefile:250: main.o] Error 127
```
`xz-utils` was also missing (but generally already installed), and is
needed for the `.tar.xz` tarballs.
Remove bsdmainutils, as this is only needed by the main build system
(for tests), and isn't needed to complete a depends build.
It is expected to have some Bitcoin nodes unreachable some of the time.
A failure to connect to an IPv4 or IPv6 node is already properly logged
under category=net/severity=debug. Do the same when a connection fails
when using a SOCKS5 proxy. This could be either to an .onion address or
to an IPv4 or IPv6 address (via a Tor exit node).
Related: https://github.com/bitcoin/bitcoin/issues/29759
This removes the need to actually track misbehavior score (see further commit), because any
Misbehaving node will immediately hit the discouragement threshold.
With the Misbehavior score gone for non-connecting headers (see previous
commit), there is no need to only treat headers messages with up to 8
headers as potential BIP130 announcements. BIP130 does not specify such
a limit; it was purely a heuristic.
This misbehavior was originally intended to prevent bandwidth wastage due to
actually observed very broken (but likely non-malicious) nodes that respond
to GETHEADERS with a response unrelated to the request, triggering a request
cycle.
This has however largely been addressed by the previous commit, which causes
non-connecting HEADERS that are received while a GETHEADERS has not been
responded to, to be ignored, as long as they do not time out (2 minutes).
With that, the specific misbehavior is largely irrelevant (for inbound peers,
it is now harmless; for outbound peers, the eviction logic will eventually
kick them out if they're not keeping up with headers at all).
Since https://github.com/bitcoin/bitcoin/pull/25454 we keep track of the last
GETHEADERS request that was sent and wasn't responded to. So far, every incoming
HEADERS message is treated as a response to whatever GETHEADERS was last sent,
regardless of its contents.
This commit makes this tracking more accurate, by only treating HEADERS messages
which (1) are empty, (2) connect to our existing block header tree, or (3) are a
continuation of a low-work headers sync as responses that clear the "outstanding
GETHEADERS" state (m_last_getheaders_timestamp).
That means that HEADERS messages which do not satisfy any of the above criteria
will be ignored, not triggering a GETHEADERS, and potentially (for now, but see
later commit) increase misbehavior score.
4b7d984269 lint: add markdown hyperlink checker (willcl-ark)
Pull request description:
Potential followup to: #30025
This should prevent us reintroducing broken markdown links.
It does not test "online" (external) links, only those within this repo. Both relative and absolute links are parsed successfully if they resolve.
ACKs for top commit:
maflcko:
re-utACK 4b7d984269
davidgumberg:
reACK 4b7d984269
Tree-SHA512: 9bc40d700b73499c046bb76157bc139f32ec3850f64ef813bbf7f18f9c01a253abe6a857d6f559890165f2bd26e7742c05d86232cd9b8efb33ff85d735f4f095
Patch Qts internal libpng to resolve the failure.
I would like to have this patched, so we can continue working on the
removal of `FORCE_USE_SYSTEM_CLANG`. Otherwise builds will be broken using
the default clang (18) on the current Ubuntu LTS (24.04).
This supports lcov 2.x in the sense that we are no-longer hardcoding
version specific options, and instead will use the `LCOV_OPTS` variable
(which is the more correct/flexible thing to do in any case). It's also
quite likely that devs are already having to pass extra options to lcov
2.x, given it's more stringent in terms of coverage generation and error
checking. See this thread for an example:
https://github.com/linux-test-project/lcov/issues/238.
Added an example to the developer notes.
Tested on one machine (LCOV 2.0, gcc 13.2) with:
```bash
./autogen.sh
./configure --enable-lcov CXXFLAGS="-fprofile-update=prefer-atomic" LCOV_OPTS="--rc branch_coverage=1 --ignore-errors mismatch"
make
make cov
<snip>
Processing file src/netaddress.cpp
lines=521 hit=480 functions=72 hit=72 branches=675 hit=499
Overall coverage rate:
lines......: 81.8% (79362 of 97002 lines)
functions......: 77.8% (10356 of 13310 functions)
branches......: 49.6% (130628 of 263196 branches)
```
and another machine (LCOV 2.1, Clang 18.1.3) with:
```bash
./autogen.sh
./configure --enable-lcov CC=clang CXX=clang++ LCOV_OPTS="--rc branch_coverage=1 --ignore-errors mismatch,inconsistent"
make
make cov
<snip>
Processing file src/util/strencodings.cpp
lines=315 hit=311 functions=38 hit=38 branches=425 hit=357
Overall coverage rate:
source files: 622
lines.......: 79.8% (70311 of 88132 lines)
functions...: 78.1% (13968 of 17881 functions)
branches....: 44.5% (157551 of 354317 branches)
Message summary:
101 warning messages:
count: 1
inconsistent: 100
3528 ignore messages:
inconsistent: 3528
```
5deb0b024e build, test, doc: Temporarily remove Android-related stuff (Hennadii Stepanov)
Pull request description:
Previously, our Android builds were geared towards generating APKs, which relied on Qt. However, after migrating to C++20, compiling for Android became unfeasible due to Qt 5.15's compatibility limitations with NDK only up to r25, which includes an outdated embedded libc++ (see https://github.com/bitcoin/bitcoin/issues/29360).
All removed stuff will be reinstated after migrating the build system to CMake and upgrading Qt to version 6.x.
This PR makes possible a clean migration to the CMake-based build system as it removes code, which is not used at this moment.
ACKs for top commit:
vasild:
ACK 5deb0b024e
fanquake:
ACK 5deb0b024e - given none of this is currently tested/wont compile. Can be revisted in future.
Tree-SHA512: 3bc2ccfe881e11cc1d78c27acd6f1d86cfba86821ef3bb5eca2e80d978fdfa13659ec82284dcaadc507e2394524dea91d4b8f81d0030c1cef9708df8be76bf07
7c8abf3c20 bench: bugfix, properly release wallet before erasing directory (furszy)
Pull request description:
Simple fix for #29816.
Since the wallet is appended to the global `WalletContext` during
creation, merely calling `reset()` on the benchmark shared_pointer
is insufficient to destruct the wallet. This no destruction of the
wallet object results in keeping the db connection open, which
was causes the `fs::remove_all()` failure on Windows.
ACKs for top commit:
maflcko:
utACK 7c8abf3c20
kevkevinpal:
utACK [7c8abf3](7c8abf3c20)
hebasto:
re-ACK 7c8abf3c20, I agree with changes since my recent [review](https://github.com/bitcoin/bitcoin/pull/30122#pullrequestreview-2061694682).
Tree-SHA512: 279df65bea8f7aa02af0a2efed62dca9bf9b29cb748eb369c602d223e08a8a907dea7b1bffbd3dab91b1656c1d91b18a9a0534bc3f153bd751414b0e6230b3a4
9ddf39dd87 fuzz: Handle missing BDBRO errors (Ava Chow)
Pull request description:
Adds error messages that were not being handled. Also removes error messages that no longer exist.
Fixes#30166
ACKs for top commit:
dergoegge:
reACK 9ddf39dd87
TheCharlatan:
ACK 9ddf39dd87
Tree-SHA512: 2597536a1e5d030653dfcb02fd892f7492f5a091def787f6cbd421b8bca9544847684a498e9458ea99ae7de5a8a6d91532ff904d1e39222d324939d31d2eb3f0
949abebea0 [fuzz] Avoid collecting initialization coverage (dergoegge)
Pull request description:
Our coverage reports include coverage of initialization code, which can be misleading when trying to evaluate the coverage a fuzz harness achieves through fuzzing alone.
This PR proposes to make fuzz coverage reports more accurate by resetting coverage counters after initialization code has been run. This makes it easier to evaluate which code was actually reached through fuzzing (e.g. to spot fuzz blockers).
ACKs for top commit:
maflcko:
utACK 949abebea0
brunoerg:
nice, utACK 949abebea0
Tree-SHA512: c8579bda4f3d71d199b9331fbe6316fce375a906743d0bc216bb94958dc03fdc9a951ea50cfeb487494a75668ae3c16471a82f7e5fdd912d781dc29d063e2c5b
fa6d4891c7 refactor: Use type-safe time in txorphanage (MarcoFalke)
Pull request description:
This allows to remove manual conversions like multiplication by `60`, and uses a type-safe type instead of a raw `int64_t`.
ACKs for top commit:
epiccurious:
utACK fa6d4891c7.
dergoegge:
Code review ACK fa6d4891c7
brunoerg:
utACK fa6d4891c7
Tree-SHA512: c187d0e579b1131afcef8c901f5662c18ab867fa2a99fbb13b67bb1e10b2047128194bfef8329cde0d51e1c35d6227ae292b823968f37ea9422975e46e01846a
Currently, transport version is a global variable declared as
TRANSPORT_VERSION in v2_p2p.py. Making it an instance variable
would help in sending non empty transport version packets for
testing purposes. It might also help EncryptedP2PState be more
extensible in far future protocol upgrades.
Both `GetSelectionWaste()` and `ComputeAndSetWaste()` now are part of
`SelectionResult`. Instead of `ComputeAndSetWaste()` being a wrapper for
`GetSelectionWaste()`, we combine them to a new function
`RecalculateWaste()`.
As I was combining the logic of the two functions, I noticed that
`GetSelectionWaste()` was making the odd assumption that the
`change_cost` being set to zero means that no change is created.
However, if we build transactions at a feerate of zero with the
`discard_feerate` also set to zero, we'd organically have a
`change_cost` of zero, even when we create change on a transaction.
This commit cleans up this duplicate meaning of `change_cost` and relies
on `GetChange()` to figure out whether there is change on basis of the
`min_viable_change` and whatever is left after deducting fees.
Since this broke a bunch of tests that relied on the double-meaning of
`change_cost` a bunch of tests had to be fixed.
fac7298529 fuzz: Fix wallet_bdb_parser stdlib error matching (MarcoFalke)
Pull request description:
The stdlib error string is an implementation detail and can not be relied upon.
Ref: `libc++abi: terminating due to uncaught exception of type std::runtime_error: AutoFile::read: end of file: unspecified iostream_category error`
ACKs for top commit:
achow101:
ACK fac7298529
Tree-SHA512: 588acc71a05d97855d6bb65380411e8486692536434eadee7697de09f80b128ff2f90a31fd0e8384d084b554d2f3978efd076082e070e721cf05b07c94cc83b1
This adds a markdown hyperlink check task to the lint test_runner. It
relies on having the [`mlc`](https://crates.io/crates/mlc) binary found
on $PATH, but will fail with `success` if the binary is not found.
`mlc` is also added to the ci/04_install.sh script run by the
containerfile.
Note that broken markdown hyperlinks will be detected in untracked
markdown files found in a dirty working directory (including e.g.
.venv).
2fd34ba504 Add sanity checks for various ATMPArgs booleans (Greg Sanders)
20d8936d8b [refactor] make some members MemPoolAccept-wide (glozow)
cbbfe719b2 cpfp carveout is excluded in packages (glozow)
69f7ab05ba Add m_allow_sibling_eviction as separate ATMPArgs flag (Greg Sanders)
57ee3029dd Add description for m_test_accept (Greg Sanders)
Pull request description:
First few commits of https://github.com/bitcoin/bitcoin/pull/28984 to set the stage for the package RBF logic.
These refactors are preparation for evaluating an RBF in a multi-proposed-transaction context instead of only a single proposed transaction. Also, carveouts and sibling evictions only should work in single RBF cases so add logic to preclude multi-tx cases in the future.
No behavior changes aside from bailing earlier from failed carve-outs.
ACKs for top commit:
glozow:
reACK 2fd34ba504 via range-diff
sr-gi:
utACK [2fd34ba](2fd34ba504)
theStack:
re-ACK 2fd34ba504
Tree-SHA512: 5071c5b8d9b8d2c9faa278c8c4df31de288cb407a68e4d55544c588caff6c86160cce7825453549c6ed69e29d9ccb5ee2d4a518b18f563bfb12f2ced073fe42a
542e13b293 rpc: Enhance metadata of the dumptxoutset output (Fabian Jahr)
4d8e5edbaa assumeutxo: Add documentation on dumptxoutset serialization format (Fabian Jahr)
c14ed7f384 assumeutxo: Add test for changed coin size value (Fabian Jahr)
de95953d87 rpc: Optimize serialization disk space of dumptxoutset (Fabian Jahr)
Pull request description:
The second attempt at implementing the `dumptxoutset` space optimization as suggested in #25675. Closes#25675.
This builds on the work done in #26045, addresses open feedback, adds some further improvements (most importantly usage of compact size), documentation, and an additional test.
The [original snapshot at height 830,000](https://github.com/bitcoin/bitcoin/pull/29551) came in at 10.82 GB. With this change, the same snapshot is 8.94 GB, a reduction of 17.4%.
This also enhances the metadata of the output file and adds the following data to allow for better error handling and make future upgrades easier:
- A newly introduced utxo set magic
- A version number
- The network magic
- The block height
ACKs for top commit:
achow101:
ACK 542e13b293
TheCharlatan:
Re-ACK 542e13b293
theStack:
ACK 542e13b293
Tree-SHA512: 0825d30e5c3c364062db3c6cbca4e3c680e6e6d3e259fa70c0c2b2a7020f24a47406a623582040988d5c7745b08649c31110df4c10656aa25f3f27eb35843d99
84900ac34f doc: add release-notes-27064.md (Matthew Zipkin)
855dd8d592 system: use %LOCALAPPDATA% as default datadir on windows (Matthew Zipkin)
Pull request description:
Closes https://github.com/bitcoin/bitcoin/issues/2391
This PR changes the default datadir location on Windows from `C:\Users\Username\AppData\Roaming\Bitcoin` to `C:\Users\Username\AppData\Local\Bitcoin`. This change only applies to fresh installs. To preserve backwards compatibility, on startup we check for the existence of `C:\Users\Username\AppData\Roaming\Bitcoin\chainstate` and if it is there, we continue using the "Roaming" directory as the default datadir location.
[Note that in Windows 11 this change may be moot:](https://learn.microsoft.com/en-us/uwp/api/windows.storage.applicationdata.roamingfolder?view=winrt-22621)
> Roaming data and settings is no longer supported as of Windows 11. The recommended replacement is [Azure App Service](https://learn.microsoft.com/en-us/azure/app-service/). Azure App Service is widely supported, well documented, reliable, and supports cross-platform/cross-ecosystem scenarios such as iOS, Android and web. Settings stored here no longer roam (as of Windows 11), but the settings store is still available.
ACKs for top commit:
achow101:
ACK 84900ac34f
BenWestgate:
crACK 84900ac34f
hebasto:
re-ACK 84900ac34f, only addressed feedback since my recent [review](https://github.com/bitcoin/bitcoin/pull/27064#pullrequestreview-2028718273).
Tree-SHA512: 807c6e89571287e2c8f4934229aec91ef28e7d0a675234acf1b7d085c24c7b73a08b6e345fbfc9038e6239187b6b69c08490ddaa1c057de5ea975c4a000bba42
No change in behavior.
For single transaction acceptance, this is a simple refactor:
Workspace::m_all_conflicting
Workspace::m_conflicting_fees
Workspace::m_conflicting_size
Workspace::m_replaced_transactions
are now grouped under a new SubPackageState struct that is
a member of MemPoolAccept.
And local variables m_total_vsize and m_total_modified_fees are now
SubpackageState members so they can be accessed from
PackageMempoolChecks.
We want these to be package-wide variables because
- Transactions could conflict with the same tx (just not the same
prevout), or their conflicts could share descendants.
- We want to compare conflicts with the package fee rather than
individual transaction fee.
We reset these MemPoolAccept-wide fields for each subpackage
evaluation to not cause state leaking, similar to temporary
coins.
The behavior is not new, but this rule exits earlier than before.
Previously, a carve out could have been granted in PreChecks() but then
nullified in PackageMempoolChecks() when CheckPackageLimits() is called
with the default limits.
154b2b2296 [fuzz] V3_MAX_VSIZE and effective ancestor/descendant size limits (glozow)
a29f1df289 [policy] restrict all v3 transactions to 10kvB (glozow)
d578e2e354 [policy] explicitly require non-v3 for CPFP carve out (glozow)
Pull request description:
Opening for discussion / conceptual review.
We like the idea of a smaller maximum transaction size because:
- It lowers potential replacement cost (i.e. harder to do Rule 3 pinning via gigantic transaction)
- They are easier to bin-pack in block template production
- They equate to a tighter memory limit in data structures that are bounded by a number of transactions (e.g. orphanage and vExtraTxnForCompact). For example, the current memory bounds for orphanage is 100KvB * 100 = 40MB, and guaranteeing 1 tx per peer would require reserving a pretty large space.
History for `MAX_STANDARD_TX_WEIGHT=100KvB` (copied from https://github.com/bitcoin/bitcoin/pull/29873#issuecomment-2115459510):
- 2010-09-13 In 3df62878c3 satoshi added a 100kB (MAX_BLOCK_SIZE_GEN/5 with MBS_GEN = MAX_BLOCK_SIZE/2) limit on new transactions in CreateTransaction()
- 2013-02-04 https://github.com/bitcoin/bitcoin/pull/2273 In gavin gave that constant a name, and made it apply to transaction relay as well
Lowering `MAX_STANDARD_TX_WEIGHT` for all txns is not being proposed, as there are existing apps/protocols that rely on large transactions. However, it's been brought up that we should consider this for TRUCs (which is especially designed to avoid Rule 3 pinning).
This reduction should be ok because using nVersion=3 isn't standard yet, so this wouldn't break somebody's existing use case. If we find that this is too small, we can always increase it later. Decreasing would be much more difficult.
~[Expected size of a commitment transaction](https://github.com/lightning/bolts/blob/master/03-transactions.md#expected-weight-of-the-commitment-transaction) is within (900 + 172 * 483 + 224) / 4 = 21050vB~ EDIT: this is incorrect, but perhaps not something that should affect how we choose this number.
ACKs for top commit:
sdaftuar:
ACK 154b2b2296
achow101:
ACK 154b2b2296
instagibbs:
ACK 154b2b2296
t-bast:
ACK 154b2b2296
murchandamus:
crACK 154b2b2296
Tree-SHA512: 89392a460908a8ea9f547d90e00f5181de0eaa9d2c4f2766140a91294ade3229b3d181833cad9afc93a0d0e8c4b96ee2f5aeda7c50ad7e6f3a8320b9e0c5ae97
d7707d9843 rpc: avoid copying into UniValue (Cory Fields)
Pull request description:
These are the simple (and hopefully obviously correct) copies that can be moves instead.
This is a follow-up from https://github.com/bitcoin/bitcoin/pull/30094#issuecomment-2108751842
As it turns out, there are hundreds of places where we copy UniValues needlessly. It should be the case that moves are always preferred over copies, so there should be no downside to these changes.
willcl-ark, however, noticed that memory usage may increase in some cases. Logically this makes no sense to me. The only plausible explanation imo is that because the moves are faster, more ops/second occur in some cases.
This list of moves was obtained by changing the function signatures of the UniValue functions to accept only rvalues, then compiling and fixing them up one by one. There still exist many places where copies are being made. These can/should be fixed up, but weren't done here for the sake of doing the easy ones first.
I ran these changes through clang-tidy with `performance-move-const-arg` and `bugprone-use-after-move` and no bugs were detected (though that's obviously not to say it can be trusted 100%).
As stated above, there are still lots of other less trivial fixups to do after these including:
- Using non-const UniValues where possible so that moves can happen
- Refactoring code in order to be able to move a UniValue without introducing a use-after-move
- Refactoring functions to accept UniValues by value rather than by const reference
ACKs for top commit:
achow101:
ACK d7707d9843
ryanofsky:
Code review ACK d7707d9843. No changes since last review other than rebase. I agree benchmarks showing increased peak memory usage and RSS are surprising, but number of allocations is down as expected, and runtime is also decreased.
willcl-ark:
ACK d7707d9843
Tree-SHA512: 7f511be73984553c278186286a7d161a34b2574c7f5f1a0edc87c2913b4c025a0af5241ef9af2df17547f2e4ef79710aa5bbb762fc9472435781c0488dba3435
6629d1d0f8 test: improve robustness of connect_nodes() (furszy)
Pull request description:
Decoupled from #27837 because this can help other too, found it investigating a CI failure https://cirrus-ci.com/task/5805115213348864?logs=ci#L3200.
The `connect_nodes` function in the test framework relies on a stable number of peer
connections to verify that the new connection between the nodes is successfully established.
This approach is fragile, as any of the peers involved in the process can drop, lose, or
create a connection at any step, causing subsequent `wait_until` checks to stall indefinitely
even when the peers in question were connected successfully.
This commit improves the situation by using the nodes' subversion and the connection
direction (inbound/outbound) to identify the exact peer connection and perform the
checks exclusively on it.
ACKs for top commit:
stratospher:
reACK 6629d1d.
achow101:
ACK 6629d1d0f8
maflcko:
utACK 6629d1d0f8
AngusP:
re-ACK 6629d1d0f8
Tree-SHA512: 5f345c0ce49ea81b643e97c5cffd133e182838752c27592fcdeac14ad10919fb4b7ff38e289e42a7c3c638a170bd0d0b7a9cd493898997a2082a7b7ceef4aeeb
9f4ff1e965 windeploy: Renew certificate (Ava Chow)
Pull request description:
Renewed the Windows code signing certificate for another 3 years.
ACKs for top commit:
fanquake:
ACK 9f4ff1e965
glozow:
tested ACK 9f4ff1e965
Tree-SHA512: 6441aa55fa1aa4e532bbe1ce1ad8b9be8a197d8de0aa57a8b02d41681a433599f305eb0a519e90782cebe20f473322be519ff6398933f525c39cc3476e860283
1e54d61c46 test: add coverage for `mapped_as` from `getrawaddrman` (brunoerg)
8c2714907d net: rpc: return peer's mapped AS in getrawaddrman (brunoerg)
Pull request description:
This PR adds two new fields in `getrawaddrman` RPC: "mapped_as" and "source_mapped_as". These fields are used to return the ASN (Autonomous System Number) mapped to the peer and its source. With these informations we can have a better view of the bucketing logic with ASMap specially in projects like [addrman-observer](https://github.com/0xb10c/addrman-observer).
ACKs for top commit:
fjahr:
Code review ACK 1e54d61c46
virtu:
ACK [1e54d61](1e54d61c46)
0xB10C:
ACK 1e54d61c46
glozow:
ACK 1e54d61c46
Tree-SHA512: af86bcc7a2e69bebd3fa9eaa2e527e0758c44c0a958de7292514d5f99f8f01f5df3bae11400451268e0255f738ff3acccc77f48fe129937512f1e9d9963c4c5e
21b8a14d37 depends: Fetch miniupnpc sources from an alternative website (Hennadii Stepanov)
Pull request description:
The https://miniupnp.tuxfamily.org website is unavailable now.
ACKs for top commit:
achow101:
ACK 21b8a14d37
edilmedeiros:
utACK 21b8a14d37
theuni:
utACK 21b8a14d37
Tree-SHA512: 141427447d5f1902f17fe3774fadf0ac912f663d7fd6467dff42fd6131f6884906b77cf49bbe7387f118fd9e9d52af1d36778624f9f02237d2ba3387fc974be6
Test addresses are being mapped according to the ASMap
file provided properly. Compare the result of the `getrawaddrman`
RPC with the result from the ASMap Health Check.
Adjust the security check for:
ld64.lld: warning: Option `-allow_stack_execute' is not yet implemented.
ld64.lld: error: -fixup_chains is incompatible with -no_pie
and to account for the embedding of LLVMs version number.
a057869aa3 build: pass --with-ecmult-gen-kb=86 to secp256k1 (fanquake)
ca3d945dc6 Squashed 'src/secp256k1/' changes from d8311688bd..06bff6dec8 (fanquake)
Pull request description:
This includes changes from the 0.5.0 release: https://github.com/bitcoin-core/secp256k1/releases/tag/v0.5.0
> New function secp256k1_ec_pubkey_sort that sorts public keys using lexicographic (of compressed serialization) order.
> The implementation of the point multiplication algorithm used for signing and public key generation was changed, resulting in improved performance for those operations.
> The related configure option --ecmult-gen-precision was replaced with --ecmult-gen-kb (ECMULT_GEN_KB for CMake).
> This changes the supported precomputed table sizes for these operations. The new supported sizes are 2 KiB, 22 KiB, or 86 KiB (while the old supported sizes were 32 KiB, 64 KiB, or 512 KiB).
ACKs for top commit:
hebasto:
ACK a057869aa3, I've got a zero diff with my local branch, which reproduces the subtree update, and `ecmult gen table size = 86 KiB` in the configure summary.
jonasnick:
utACK a057869aa3
Tree-SHA512: 907012b0d7e0a6bd68b245c238e968f2318d8ac5de5ec9070245de8391c996eb5ec6428184d028f6f0f54d3b2f5a8292ad7081177e1c331397879505436dc38e
b3efb48673 protocol: make message types constexpr (Vasil Dimov)
2fa9de06c2 net: make the list of known message types a compile time constant (Vasil Dimov)
Pull request description:
Turn the `std::vector` to `std::array` because it is cheaper and allows us to have the number of the messages as a compile time constant: `ALL_NET_MESSAGE_TYPES.size()` which can be used in future code to build other `std::array`s with that size.
---
This change is part of https://github.com/bitcoin/bitcoin/pull/29418 but it makes sense on its own and would be good to have it, regardless of the fate of https://github.com/bitcoin/bitcoin/pull/29418. Also, if this is merged, that would reduce the size of https://github.com/bitcoin/bitcoin/pull/29418, thus the current standalone PR.
ACKs for top commit:
achow101:
ACK b3efb48673
jonatack:
ACK b3efb48673
maflcko:
utACK b3efb48673🎊
willcl-ark:
ACK b3efb48673
Tree-SHA512: 6d3860c138c64514ebab13d97ea67893e2d346dfac30a48c3d9bc769a1970407375ea4170afdb522411ced306a14a9af4eede99e964d1fb1ea3efff5d5eb57af
This carve out is intended to allow a second child under restricted
circumstances, but this topology is not allowed for v3 transactions.
As CPFP carve out does not explicitly require a second child to actually
exist, it has the effect of granting a free +10KvB descendant size limit
when a single child is enough to bust the descendant limit.
The 'connect_nodes' function in the test framework relies
on a stable number of peer connections to verify the new
connection between the nodes is successfully established.
This approach is fragile, as any of the peers involved in
the process can drop, lose, or create a connection at any
step, causing subsequent 'wait_until' checks to stall
indefinitely even when the peers in question are connected
successfully.
This commit improves the situation by using the nodes' subversion
and the connection direction (inbound/outbound) to identify the
exact peer connection and perform the checks exclusively on it.
d35ba1b3f1 util: avoid using thread_local variable that has a destructor (Vasil Dimov)
Pull request description:
Store the thread name in a `thread_local` variable of type `char[]` instead of `std::string`. This avoids calling the destructor when the thread exits. This is a workaround for
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=278701
For type-safety, return `std::string` from
`util::ThreadGetInternalName()` instead of `char[]`.
As a side effect of this change, we no longer store a reference to a `thread_local` variable in `CLockLocation`. This was dangerous because if the thread quits while the reference still exists (in the global variable `lock_data`, see inside `GetLockData()`) then the reference will become dangling.
ACKs for top commit:
laanwj:
Code review ACK d35ba1b3f1
hebasto:
re-ACK d35ba1b3f1.
theuni:
utACK d35ba1b3f1
Tree-SHA512: a2a3bc4401654d6e99db5b9c46a7051855f5a26886142298662e681b78dd581ff4c6bebe42f649b8e1fb8a78d569c6117302db2cd6362e884a22f2a5839b7d43
d51fbab4b3 wallet, test: Be able to always swap BDB endianness (Ava Chow)
0b753156ce test: Test bdb_ro dump of wallet without reset LSNs (Ava Chow)
c1984f1282 test: Test dumping dbs with overflow pages (Ava Chow)
fd7b16e391 test: Test dumps of other endian BDB files (Ava Chow)
6ace3e953f bdb: Be able to make byteswapped databases (Ava Chow)
d9878903fb Error if LSNs are not reset (Ava Chow)
4d7a3ae78e Berkeley RO Database fuzz test (TheCharlatan)
3568dce9e9 tests: Add BerkeleyRO to db prefix tests (Ava Chow)
70cfbfdadf wallettool: Optionally use BERKELEY_RO as format when dumping BDB wallets (Ava Chow)
dd57713f6e Add MakeBerkeleyRODatabase (Ava Chow)
6e50bee67d Implement handling of other endianness in BerkeleyRODatabase (Ava Chow)
cdd61c9cc1 wallet: implement independent BDB deserializer in BerkeleyRODatabase (Ava Chow)
ecba230979 wallet: implement BerkeleyRODatabase::Backup (Ava Chow)
0c8e728476 wallet: implement BerkeleyROBatch (Ava Chow)
756ff9b478 wallet: add dummy BerkeleyRODatabase and BerkeleyROBatch classes (Ava Chow)
ca18aea5c4 Add AutoFile::seek and tell (Ava Chow)
Pull request description:
Split from #26596
This PR adds `BerkeleyRODatabase` which is an independent implementation of a BDB file parser. It provides read only access to a BDB file, and can therefore be used as a read only database backend for wallets. This will be used for dumping legacy wallet records and migrating legacy wallets without the need for BDB itself.
Wallettool's `dump` command is changed to use `BerkeleyRODatabase` instead of `BerkeleyDatabase` (and `CWallet` itself) to demonstrate that this parser works and to test it against the existing wallettool functional tests.
ACKs for top commit:
josibake:
reACK d51fbab4b3
TheCharlatan:
Re-ACK d51fbab4b3
furszy:
reACK d51fbab4b3
laanwj:
re-ACK d51fbab4b3
theStack:
ACK d51fbab4b3
Tree-SHA512: 1e7b97edf223b2974eed2e9eac1179fc82bb6359e0a66b7d2a0c8b9fa515eae9ea036f1edf7c76cdab2e75ad994962b134b41056ccfbc33b8d54f0859e86657b
Replace ExpandHRP with a PreparePolynomialCoefficients function. Instead
of using a hardcoded value for the size of the array (90 in this case)
and a hardcoded value for the checksum, use the actual values vector and
define checksum size as a constexpr. Use the new CHECKSUM_SIZE
throughout instead 6.
Co-authored-by: Lőrinc <pap.lorinc@gmail.com>
8950053636 test: remove unneeded `-maxorphantx=1000` settings (Sebastian Falbesoner)
Pull request description:
It's unclear what the motivation for increasing the orphan pool is here, and it seems that this not needed at all. None of these tests involve orphan transactions explicitly, and if they would occur occasionally, there is no good reason to prefer a value of 1000 over the default of 100 (see DEFAULT_MAX_ORPHAN_TRANSACTIONS).
ACKs for top commit:
maflcko:
utACK 8950053636
edilmedeiros:
Tested ACK 8950053636
AngusP:
tACK 8950053636
glozow:
ACK 8950053636 From skimming the tests, it appears that none of these need a larger `-maxorphantx`.
Tree-SHA512: 81d4a4fb2ea92b97119f21cbc6c4b1240d863269932e6adf4982aead9726f20652523a4707add3ad38eb332d4452de41de6735265f22e62298f3b4b45de75a57
9365baa489 test: add conflicting topology test case (Greg Sanders)
Pull request description:
We want to ensure that even if topologies
that are acceptable are relaxed, like
removing package-not-child-with-unconfirmed-parents, that we don't end up accepting packages we shouldn't.
ACKs for top commit:
glozow:
reACK 9365baa489
rkrux:
reACK [9365baa](9365baa489)
Tree-SHA512: d58661064ca099ac0447c331a5020c74c0cdfe24259aa875592805bbd63de1bf23aa7ced9ff485fef90dc0602fcb997e631aaf1aa2e9805d2cf5f0e5c9b2f0e2
It is best to store all key origin information
(master key fingerprint and all derivation steps)
in the multisig descriptor. Being explicit with
this information should be beneficial if this approach
is used with other wallets/signers (whether hardware
or software). There is no harm including all of this
with xpubs (if anything it simplifies the test code)
and makes this example/docs more complete and safer
incase it is referenced by others.
It's unclear what the motivation for increasing the orphan pool is, and
it seems that this not needed at all. None of these tests involve orphan
transactions explicitly, and if they would occur occasionally, there is
no good reason to prefer a value of 1000 over the default of 100 (see
DEFAULT_MAX_ORPHAN_TRANSACTIONS).
b47bd95920 kernel: De-globalize fReindex (TheCharlatan)
Pull request description:
fReindex is one of the last remaining globals exposed by the kernel library, so move it into the blockstorage class to reduce the amount of global mutable state and make the kernel library a bit less awkward to use.
---
This pull request is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).
ACKs for top commit:
achow101:
ACK b47bd95920
ryanofsky:
Code review ACK b47bd95920. I rereviewed the whole PR, but the only change since last review was reverting the bugfix https://github.com/bitcoin/bitcoin/pull/29817#discussion_r1578327024 and make the change a pure refactoring.
mzumsande:
Code Review ACK b47bd95920
stickies-v:
ACK b47bd95920
Tree-SHA512: f7399d01f93bc0c0c7428fe95d19b9d29b4ed00a4f1deabca78fb0c4fecb434ec971e890feecb105938b5247c926850b1b7b4a4a9caa333a061e40777d0c8463
9408a04e42 tests, fuzz: use new NUMS_H const (josibake)
b946f8a4c5 crypto: add NUMS_H const (josibake)
Pull request description:
Broken out from #28122
---
[BIP341](https://github.com/bitcoin/bips/blob/master/bip-0341.mediawiki#constructing-and-spending-taproot-outputs) defines a NUMS point `H` as *H = lift_x(0x50929b74c1a04954b78b4b6035e97a5e078a5a0f28ec96d547bfee9ace803ac0)* which is [constructed](11af7015de/src/modules/rangeproof/main_impl.h (L16)) by taking the hash of the standard uncompressed encoding of the [secp256k1](https://www.secg.org/sec2-v2.pdf) base point G as X coordinate."
Add this as a constant so it can be used in our codebase. My primary motivation is BIP352 specifies a special case for when taproot spends use `H` as the internal key, but outside of BIP352 it seems generally useful to have `H` in the codebase, for testing or other use cases.
ACKs for top commit:
paplorinc:
re-ACK 9408a04e42
achow101:
ACK 9408a04e42
theStack:
Code-review ACK 9408a04e42
Tree-SHA512: ad84492f5d635c0cb05bd82546079ded7e5138e95361f20d8285a9ad6e69c10ee2cc3fe46e16b46ef03c4253c8bee1051911c6b91264c90c3b1ad33a824bff4b
Since the wallet is appended to the global WalletContext during
creation, merely calling 'reset()' on the benchmark shared_pointer
is insufficient to destruct the wallet. This no destruction of the
wallet results in the db connection remaining open, which was the
cause of the 'fs::remove_all()' error in Windows.
Co-Authored-By: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
Currently, we log the number of bytes of garbage when it is
generated. The log is a better fit for when the garbage
actually gets sent to the transport layer.
BerkeleyRODatabase is intended for use after BDB is removed, so it needs
to be able to read all of the records from a BDB file. Thus an
independent deserializer for BDB data files is implemented in it. This
deserializer is targeted towards the data files that Bitcoin Core
creates so it does not fully support all of BDB's features (e.g. other
database types, encryption, etc.).
Store the thread name in a `thread_local` variable of type `char[]`
instead of `std::string`. This avoids calling the destructor when
the thread exits. This is a workaround for
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=278701
For type-safety, return `std::string` from
`util::ThreadGetInternalName()` instead of `char[]`.
As a side effect of this change, we no longer store a reference
to a `thread_local` variable in `CLockLocation`. This was
dangerous because if the thread quits while the reference still
exists (in the global variable `lock_data`, see inside `GetLockData()`)
then the reference will become dangling.
e41667b720 blockstorage: Don't move cursor backwards in UpdateBlockInfo (Ryan Ofsky)
17103637c6 blockstorage: Rename FindBlockPos and have it return a FlatFilePos (Martin Zumsande)
d9e477c4dc validation, blockstorage: Separate code paths for reindex and saving new blocks (Martin Zumsande)
064859bbad blockstorage: split up FindBlockPos function (Martin Zumsande)
fdae638e83 doc: Improve doc for functions involved in saving blocks to disk (Martin Zumsande)
0d114e3cb2 blockstorage: Add Assume for fKnown / snapshot chainstate (Martin Zumsande)
Pull request description:
`SaveBlockToDisk` / `FindBlockPos` are used for two purposes, depending on whether they are called during reindexing (`dbp` set, `fKnown = true`) or in the "normal" case when adding new blocks (`dbp == nullptr`, `fKnown = false`).
The actual tasks are quite different
- In normal mode, preparations for saving a new block are made, which is then saved: find the correct position on disk (maybe skipping to a new blk file), check for available disk space, update the blockfile info db, save the block.
- during reindex, most of this is not necessary (the block is already on disk after all), only the blockfile info needs to rebuilt because reindex wiped the leveldb it's saved in.
Using one function with many conditional statements for this leads to code that is hard to read / understand and bug-prone:
- many code paths in `FindBlockPos` are conditional on `fKnown` or `!fKnown`
- It's not really clear what actually needs to be done during reindex (we don't need to "save a block to disk" or "find a block pos" as the function names suggest)
- logic that should be applied to only one of the two modes is sometimes applied to both (see first commit, or #27039)
#24858 and #27039 were recent bugs directly related to the differences between reindexing and normal mode, and in both cases the simple fix took a long time to be reviewed and merged.
This PR proposes to clean this code up by splitting out the reindex logic into a separate function (`UpdateBlockInfo`) which will be called directly from validation. As a result, `SaveBlockToDisk` and `FindBlockPos` only need to cover the non-reindex logic.
ACKs for top commit:
paplorinc:
ACK e41667b720
TheCharlatan:
Re-ACK e41667b720
ryanofsky:
Code review ACK e41667b720. Just improvements to comments since last review.
Tree-SHA512: a14ff9a0facf6b1e3c1cd724a2d19a79a25d4b48de64398fdd172671532a472bc10a20cbb64ac3a3e55814dcc877d0597a3e1699cabc4f9d9a86b439b6eaba20
There are no changes to behavior. Changes in this commit are all additions, and
are easiest to review using "git diff -U0 --word-diff-regex=." options.
Motivation for this change is to keep util functions with really generic names
like "Split" and "Join" out of the global namespace so it is easier to see
where these functions are defined, and so they don't interfere with function
overloading, especially since the util library is a dependency of the kernel
library and intended to be used with external code.
Add TransactionError to node namespace and include it directly instead of
relying on indirect include through common/messages.h
This is a followup to a previous commit which moved the TransactionError enum.
These changes were done in a separate followup just to keep the previous commit
more minimal and easy to review.
Move enum and message formatting functions to a common/messages header where
they should be more discoverable, and also out of the util library, so they
will not be a dependency of the kernel
The are no changes in behavior and no changes to the moved code.
Add separate PSBTError enum instead of reusing TransactionError enum for PSBT
operations, and drop unused error codes. The error codes returned by PSBT
operations and transaction broadcast functions mostly do not overlap, so using
an unified enum makes it harder to call any of these functions and know which
errors actually need to be handled.
Define PSBTError in the common library because PSBT functionality is
implemented in the common library and used by both the node (for rawtransaction
RPCs) and the wallet.
New node/types.h file is analagous to existing wallet/types.h and is a better
place to define simple node types that are shared externally with wallet and
gui code than the util library.
Motivation for this change is to completely remove util/error.h file currently
holding TransactionError in a followup commit.
Move miniscript / descriptor script parsing functions out of util library so
they are not a dependency of the kernel.
There are no changes to code or behavior.
This will help move the miniscript / descriptor parsing functions out of the
util library in an upcoming commit, so they are not exposed to libbitcoinkernel
applications. Moving the Split functions should also make them more
discoverable since they now close to related functions like Join.
The functions are moved verbatim without any changes.
Move HexStr and HexDigit functions from util to crypto. The crypto library does
not actually use these functions, but the consensus library does. The consensus
and util libraries not allowed to depend on each other, but are allowed to
depend on the cryto library, so the crypto library is a reasonable put these.
The consensus library uses HexStr and HexDigit in script.cpp, transaction.cpp,
and uint256.cpp.
The util library does not use HexStr but does use HexDigit in strencodings.cpp
to parse integers.
Move util/message to common/signmessage so it is named more clearly, and
because the util library is not supposed to depend on other libraries besides
the crypto library. The signmessage functions use CKey, CPubKey, PKHash, and
DecodeDestination functions in the consensus and common libraries.
Move chainparamsbase from util to common, because util library should not
depend on the common library and chainparamsbase uses the ArgsManager class in
common.
cbc6c440e3 doc: add comments and release-notes for JSON-RPC 2.0 (Matthew Zipkin)
e7ee80dcf2 rpc: JSON-RPC 2.0 should not respond to "notifications" (Matthew Zipkin)
bf1a1f1662 rpc: Avoid returning HTTP errors for JSON-RPC 2.0 requests (Matthew Zipkin)
466b90562f rpc: Add "jsonrpc" field and drop null "result"/"error" fields (Matthew Zipkin)
2ca1460ae3 rpc: identify JSON-RPC 2.0 requests (Matthew Zipkin)
a64a2b77e0 rpc: refactor single/batch requests (Matthew Zipkin)
df6e3756d6 rpc: Avoid copies in JSONRPCReplyObj() (Matthew Zipkin)
09416f9ec4 test: cover JSONRPC 2.0 requests, batches, and notifications (Matthew Zipkin)
4202c170da test: refactor interface_rpc.py (Matthew Zipkin)
Pull request description:
Closes https://github.com/bitcoin/bitcoin/issues/2960
Bitcoin Core's JSONRPC server behaves with a special blend of 1.0, 1.1 and 2.0 behaviors. This introduces compliance issues with more strict clients. There are the major misbehaviors that I found:
- returning non-200 HTTP codes for RPC errors like "Method not found" (this is not a server error or an HTTP error)
- returning both `"error"` and `"result"` fields together in a response object.
- different error-handling behavior for single and batched RPC requests (batches contain errors in the response but single requests will actually throw HTTP errors)
https://github.com/bitcoin/bitcoin/pull/15495 added regression tests after a discussion in https://github.com/bitcoin/bitcoin/pull/15381 to kinda lock in our RPC behavior to preserve backwards compatibility.
https://github.com/bitcoin/bitcoin/pull/12435 was an attempt to allow strict 2.0 compliance behind a flag, but was abandoned.
The approach in this PR is not strict and preserves backwards compatibility in a familiar bitcoin-y way: all old behavior is preserved, but new rules are applied to clients that opt in. One of the rules in the [JSON RPC 2.0 spec](https://www.jsonrpc.org/specification#request_object) is that the kv pair `"jsonrpc": "2.0"` must be present in the request. Well, let's just use that to trigger strict 2.0 behavior! When that kv pair is included in a request object, the [response will adhere to strict JSON-RPC 2.0 rules](https://www.jsonrpc.org/specification#response_object), essentially:
- always return HTTP 200 "OK" unless there really is a server error or malformed request
- either return `"error"` OR `"result"` but never both
- same behavior for single and batch requests
If this is merged next steps can be:
- Refactor bitcoin-cli to always use strict 2.0
- Refactor the python test framework to always use strict 2.0 for everything
- Begin deprecation process for 1.0/1.1 behavior (?)
If we can one day remove the old 1.0/1.1 behavior we can clean up the rpc code quite a bit.
ACKs for top commit:
cbergqvist:
re ACK cbc6c440e3
ryanofsky:
Code review ACK cbc6c440e3. Just suggested changes since the last review: changing uncaught exception error code from PARSE_ERROR to MISC_ERROR, renaming a few things, and adding comments.
tdb3:
re ACK for cbc6c440e3
Tree-SHA512: 0b702ed32368b34b29ad570d090951a7aeb56e3b0f2baf745bd32fdc58ef68fee6b0b8fad901f1ca42573ed714b150303829cddad4a34ca7ad847350feeedb36
fReindex is one of the last remaining globals exposed by the kernel
library, so move it into the blockstorage class to reduce the amount of
global mutable state and make the kernel library a bit less awkward to
use.
d0b047494c test: add GetAddedNodeInfo() CJDNS regression unit test (Jon Atack)
684da97070 p2p, bugfix: detect addnode cjdns peers in GetAddedNodeInfo() (Jon Atack)
Pull request description:
Addnode peers connected to us via the cjdns network are currently not detected by `CConnman::GetAddedNodeInfo()`, i.e. `fConnected` is always false. This causes the following issues:
- RPC `getaddednodeinfo` incorrectly shows them as not connected
- `CConnman::ThreadOpenAddedConnections()` continually retries to connect them
Fix the issue and add a unit regression test. Extracted from #28248. Suggest running the test with:
`./src/test/test_bitcoin -t net_peer_connection_tests -l test_suite`
ACKs for top commit:
mzumsande:
utACK d0b047494c
brunoerg:
crACK d0b047494c
pinheadmz:
ACK d0b047494c
Tree-SHA512: a4d81425f79558f5792585611f3fe8ab999b82144daeed5c3ec619861c69add934c2b2afdad24c8488a0ade94f5ce8112f5555d60a1ce913d4f5a1cf5dbba55a
06bff6dec8 Merge bitcoin-core/secp256k1#1528: tests: call `secp256k1_ecmult_multi_var` with a non-`NULL` error callback
4155e62fcc Merge bitcoin-core/secp256k1#1526: cmake: Fix `check_arm32_assembly` when using as subproject
9554362b15 tests: call secp256k1_ecmult_multi_var with a non-NULL error callback
9f4c8cd730 cmake: Fix `check_arm32_assembly` when using as subproject
7712a53061 Merge bitcoin-core/secp256k1#1524: check-abi: explicitly provide public headers
7d0bc0870f Merge bitcoin-core/secp256k1#1525: changelog: Correct 0.5.0 release date
d45d9b74bb changelog: Correct 0.5.0 release date
d7f6613dbb Merge bitcoin-core/secp256k1#1523: release cleanup: bump version after 0.5.0
2f05e2da4b release cleanup: bump version after 0.5.0
e3a885d42a Merge bitcoin-core/secp256k1#1522: release: prepare for 0.5.0
dd695563e6 check-abi: explicitly provide public headers
c0e4ec3fee release: prepare for 0.5.0
bb528cfb08 Merge bitcoin-core/secp256k1#1518: Add secp256k1_pubkey_sort
7d2591ce12 Add secp256k1_pubkey_sort
da515074e3 Merge bitcoin-core/secp256k1#1058: Signed-digit multi-comb ecmult_gen algorithm
4c341f89ab Add changelog entry for SDMC
a043940253 Permit COMB_BITS < 256 for exhaustive tests
39b2f2a321 Add test case for ecmult_gen recoded = {-1,0,1}
644e86de9a Reintroduce projective blinding
07810d9abb Reduce side channels from single-bit reads
a0d32b597d Optimization: use Nx32 representation for recoded bits
e03dcc44b5 Make secp256k1_scalar_get_bits support 32-bit reads
5005abee60 Rename scalar_get_bits -> scalar_get_bits_limb32; return uint32_t
6247f485b6 Optimization: avoid unnecessary doublings in precomputation
15d0cca2a6 Optimization: first table lookup needs no point addition
7a33db35cd Optimization: move (2^COMB_BITS-1)/2 term into ctx->scalar_offset
ed2a056f3d Provide 3 configurations accessible through ./configure
5f7be9f6a5 Always generate tables for current (blocks,teeth) config
fde1dfcd8d Signed-digit multi-comb ecmult_gen algorithm
486518b350 Make exhaustive tests's scalar_inverse(&x,&x) work
ab45c3e089 Initial gej blinding -> final ge blinding
aa00a6b892 Introduce CEIL_DIV macro and use it
git-subtree-dir: src/secp256k1
git-subtree-split: 06bff6dec8d038f7b4112664a9b882293ebc5178
141df0a288 crypto: disable asan for sha256_sse4 with clang and -O0 (Cory Fields)
Pull request description:
Clang is unable to compile the Transform function for that combination of options.
Fixes#29801.
ACKs for top commit:
achow101:
ACK 141df0a288
Tree-SHA512: d74fdac5840ad7524edfde069fb43ae75c31146e90ecc58bbc7912ff57a02b068547431b1766afeed782272c0b93b0b41a286c1cf26ec55ce332d94ce917d810
8d491ae9ec serialization: Add ParamsStream GetStream() method (Ryan Ofsky)
951203bcc4 net: Simplify ParamsStream usage (Ryan Ofsky)
e6794e475c serialization: Accept multiple parameters in ParamsStream constructor (Ryan Ofsky)
cb28849a88 serialization: Reverse ParamsStream constructor order (Ryan Ofsky)
83436d14f0 serialization: Drop unnecessary ParamsStream references (Ryan Ofsky)
84502b755b serialization: Drop references to GetVersion/GetType (Ryan Ofsky)
f3a2b52376 serialization: Support for multiple parameters (Ryan Ofsky)
Pull request description:
Currently it is only possible to attach one serialization parameter to a stream at a time. For example, it is not possible to set a parameter controlling the transaction format and a parameter controlling the address format at the same time because one parameter will override the other.
This limitation is inconvenient for multiprocess code since it is not possible to create just one type of stream and serialize any object to it. Instead it is necessary to create different streams for different object types, which requires extra boilerplate and makes using the new parameter fields a lot more awkward than the older version and type fields.
Fix this problem by allowing an unlimited number of serialization stream parameters to be set, and allowing them to be requested by type. Later parameters will still override earlier parameters, but only if they have the same type.
For an example of different ways multiple parameters can be set, see the new [`with_params_multi`](40f505583f/src/test/serialize_tests.cpp (L394-L410)) unit test.
This change requires replacing the `stream.GetParams()` method with a `stream.GetParams<T>()` method in order for serialization code to retrieve the desired parameters. The change is more verbose, but probably a good thing for readability because previously it could be difficult to know what type the `GetParams()` method would return, and now it is more obvious.
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
maflcko:
ACK 8d491ae9ec🔵
sipa:
utACK 8d491ae9ec
TheCharlatan:
ACK 8d491ae9ec
Tree-SHA512: 40b7041ee01c0372b1f86f7fd6f3b6af56ef24a6383f91ffcedd04d388e63407006457bf7ed056b0e37b4dec9ffd5ca006cb8192e488ea2c64678567e38d4647
cccddc03f0 Wallet encrypt on create, allow to navigate options (Hernan Marino)
Pull request description:
This fixes https://github.com/bitcoin-core/gui/issues/394.
It adds a "Go back" button to the "Confirm wallet encryption" window, allowing the users to change the password if they want to. It also adds a Cancel button to the "Wallet to be encrypted" window.
Prior to this change users had no option to alter the password, and were forced to either go ahead with wallet creation or cancel the whole process. Also, at the final window, they were shown a warning but with no option to cancel.
The new workflow for wallet encryption and creation is similar to the following:

ACKs for top commit:
alfonsoromanz:
Re-Tested ACK cccddc03f0
BrandonOdiwuor:
re-Tested ACK cccddc03f0
hebasto:
ACK cccddc03f0, tested on Ubuntu 24.04.
Tree-SHA512: d2856d75f75acbd7d51ede62b4abd317f6ed6a9b890fe0b73b63b921b4b3d61b49477e35dc74466a056a9e8c0c1598df7601111d36c57ef18fdfdf0b18f503e6
0fb17bf61a [log] updates in TxOrphanage (glozow)
b16da7eda7 [functional test] attackers sending mutated orphans (glozow)
6675f6428d [unit test] TxOrphanage handling of same-txid-different-witness txns (glozow)
8923edfc1f [p2p] allow entries with the same txid in TxOrphanage (glozow)
c31f148166 [refactor] TxOrphanage::EraseTx by wtxid (glozow)
efcc593017 [refactor] TxOrphanage::HaveTx only by wtxid (glozow)
7e475b9648 [p2p] don't query orphanage by txid (glozow)
Pull request description:
Part of #27463 in the "make orphan handling more robust" section.
Currently the main map in `TxOrphanage` is indexed by txid; we do not allow 2 transactions with the same txid into TxOrphanage. This means that if we receive a transaction and want to store it in orphanage, we'll fail to do so if a same-txid-different-witness version of the tx already exists in the orphanage. The existing orphanage entry can stay until it expires 20 minutes later, or until we find that it is invalid.
This means an attacker can try to block/delay us accepting an orphan transaction by sending a mutated version of the child ahead of time. See included test.
Prior to #28970, we don't rely on the orphanage for anything and it would be relatively difficult to guess what transaction will go to a node's orphanage. After the parent(s) are accepted, if anybody sends us the correct transaction, we'll end up accepting it. However, this is a bit more painful for 1p1c: it's easier for an attacker to tell when a tx is going to hit a node's orphanage, and we need to store the correct orphan + receive the parent before we'll consider the package. If we start out with a bad orphan, we can't evict it until we receive the parent + try the 1p1c, and then we'll need to download the real child, put it in orphanage, download the parent again, and then retry 1p1c.
ACKs for top commit:
AngusP:
ACK 0fb17bf61a
itornaza:
trACK 0fb17bf61a
instagibbs:
ACK 0fb17bf61a
theStack:
ACK 0fb17bf61a
sr-gi:
crACK [0fb17bf](0fb17bf61a)
stickies-v:
ACK 0fb17bf61a
Tree-SHA512: edcbac7287c628bc27036920c2d4e4f63ec65087fbac1de9319c4f541515d669fc4e5fdc30c8b9a248b720da42b89153d388e91c7bf5caf4bc5b3b931ded1f59
fa90ad23c0 ci: Roll test-each-commit Ubuntu (MarcoFalke)
fa6c82dd9b ci: Remove clang version pin in test-each-commit (MarcoFalke)
Pull request description:
Needed for https://github.com/bitcoin/bitcoin/pull/29077#issuecomment-2099704210
ACKs for top commit:
hebasto:
re-ACK fa90ad23c0.
Tree-SHA512: 753a3a77d967f308b5b5dd0bc7ea9f3268fc93c5ac978da3d79b85461cb1e994c6ac481888dc472b9a08be45ad0fad66ad3fda241a8955f999b7c2cb2a2b1f58
12d82817bf refactor: simplify `FormatSubVersion` using strprintf/Join (Sebastian Falbesoner)
Pull request description:
Rather than using std::ostringstream and manually joining the comments, use strprintf and our own `Join` helper.
ACKs for top commit:
maflcko:
utACK 12d82817bf
TheCharlatan:
tACK 12d82817bf
hebasto:
ACK 12d82817bf, I have reviewed the code and it looks OK.
tdb3:
ACK for 12d82817bf.
Tree-SHA512: b9b965c4416a4c0c8727e3c4b40da4be04b14067200220492e9bed4fa35c1907fb5cdec2a30052b9e762f71da3d3cf042f43c96ab1f2523df5bb9920b44ea2a5
b59a027d95 contrib: drop dead get_machine from test sym check (fanquake)
e6aba463ad contrib: use env_flags in get_arch (fanquake)
Pull request description:
This isn't an issue right now (because the get_arch check is simple), but becomes one as soon as we want to use `lld` for linking, and need LDFLAGS (otherwise we call `ld` and fail, see it's usage in #21778). So I've split this out for review. It also makes sense to use the same flags for all compilation in these checks.
Also drops some dead code in test-symbol-check.
ACKs for top commit:
TheCharlatan:
ACK b59a027d95
Tree-SHA512: d8afc4144815369aae63cf6dc6e983af46f208c7043d6ea5c9c811152649c256a8e67eb6864ea9d385d87b6b049fece07710a84b90da325da7fc3f05efcaacd6
cc67d33fda refactor: Simply include CTxMemPool::Options in CTxMemPool directly rather than duplicating definition (Luke Dashjr)
Pull request description:
Instead of duplicating mempool options two places, just include the Options struct directly on the CTxMemPool
ACKs for top commit:
achow101:
ACK cc67d33fda
kristapsk:
cr utACK cc67d33fda
jonatack:
ACK cc67d33fda
Tree-SHA512: 9deb5ea6f85eeb1c7e04536cded65303b0ec459936a97e4f257aff2c50b0984a4ddbf69a4651f48455b9c80200a1fd24e9c74926874fdd9be436bbbe406251ce
Previously, it was possible to move the cursor back to an older file
during reindex if blocks are enocuntered out of order during reindex.
This would mean that MaxBlockfileNum() would be incorrect, and
a wrong DB_LAST_BLOCK could be written to disk.
This improves the logic by only ever moving the cursor forward (if possible)
but not backwards.
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
The new name reflects that it is no longer called with existing blocks
for which the position is already known.
Returning a FlatFilePos directly simplifies the interface.
By calling SaveBlockToDisk only when we actually want to save a new
block to disk. In the reindex case, we now call UpdateBlockInfo
directly from validation.
This commit doesn't change behavior.
FindBlockPos does different things depending on whether the block is known
or not, as shown by the fact that much of the existing code is conditional on fKnown set or not.
If the block position is known (during reindex) the function only updates the block info
statistics. It doesn't actually find a block position in this case.
This commit removes fKnown and splits up these two code paths by introducing a separate function
for the reindex case when the block position is known.
It doesn't change behavior.
For JSON-RPC 2.0 requests we need to distinguish between
a missing "id" field and "id":null. This is accomplished
by making the JSONRPCRequest id property a
std::optional<UniValue> with a default value of
UniValue::VNULL.
A side-effect of this change for non-2.0 requests is that request which do not
specify an "id" field will no longer return "id": null in the response.
d4b17c7d46 kernel: Remove batchpriority from kernel library (TheCharlatan)
Pull request description:
The current usage of ScheduleBatchPriority is not transparent. Once the thread scheduling is changed, it remains unchanged for the remainder of the thread's lifetime. So move the call from `ImportBlocks` to the init code where it is clearer that its effect lasts for the entire lifetime of the thread.
Users of the kernel library might not expect `ImportBlocks` to have an influence on the thread it is called in. Particularly since it is only a compile time option and cannot be controlled at runtime. With this patch users of the kernel library can now freely choose their own scheduling policy.
This PR is easier reviewed with `git diff --color-moved-ws=ignore-all-space --color-moved=dimmed-zebra`
---
This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).
ACKs for top commit:
maflcko:
ACK d4b17c7d46📭
ryanofsky:
Code review ACK d4b17c7d46, just added suggested comment since last review
hebasto:
ACK d4b17c7d46, I have reviewed the code and it looks OK.
Tree-SHA512: cafedecd9affad58ddd7f30f68bba71291ca951bb186ff4b2da04b7f21f0b26e5e3143846d032b9e391bd5ce6c7466b97aa3758d2a85ebd7353eb8b69139641a
Avoid returning HTTP status errors for non-batch JSON-RPC 2.0 requests if the
RPC method failed but the HTTP request was otherwise valid. Batch requests
already did not return HTTP errors previously.
We want to ensure that even if topologies
that are acceptable are relaxed, like
removing package-not-child-with-unconfirmed-parents,
that we don't end up accepting packages we shouldn't.
Index by wtxid instead of txid to allow entries with the same txid but
different witnesses in orphanage. This prevents an attacker from
blocking a transaction from entering the orphanage by sending a mutated
version of it.
The current usage of ScheduleBatchPriority is not transparent. Once the
thread scheduling is changed, it remains unchanged for the remainder of
the thread's lifetime. So move the call from `ImportBlocks` to the init
code where it is clearer that its effect lasts for the entire lifetime
of the thread.
Users of the kernel library might not expect `ImportBlocks` to have an
influence on the thread it is called in. Particularly since it is only a
compile time option and cannot be controlled at runtime. With this patch
users of the kernel library can now choose their own scheduling policy.
019ad7327c depends: set RANLIB for CMake (fanquake)
43cfb428cb depends: set NM for CMake (fanquake)
1e4412b317 depends: set AR for CMake (fanquake)
Pull request description:
Needed for #21778. Should be more correct in any case.
ACKs for top commit:
theuni:
utACK 019ad7327c. I didn't test, but I tried this approach on a few deps and it seemed to work as expected.
TheCharlatan:
ACK 019ad7327c
Tree-SHA512: 78cc8981456f7476cafca0e40fcc569e474b92004c8024d1c4268b6aab53175074a06ab17ebded8d706bf0a7f77401642dd38bb7ce2e4b04abdcd149d3d69969
Implement ReadKey and HasKey of BerkeleyROBatch, and Next of BerkeleyROCursor.
Also adds the containers for records to BerkeleyRODatabase so that
BerkeleyROBatch will be able to access the records.
4a6d1d1e3b validation: don't clear cache on periodic flush (Andrew Toth)
Pull request description:
Since https://github.com/bitcoin/bitcoin/pull/17487 we no longer need to clear the coins cache when syncing to disk. A warm coins cache significantly speeds up block connection, and only needs to be fully flushed when nearing the `dbcache` limit.
Periodic flushes occur every 24 hours, which empties the cache and causes block connection to slow down. By keeping the cache through periodic flushes a node can run for several days with an increasingly hotter cache and connect blocks much more quickly. Now not only can setting a higher `dbcache` value be beneficial for IBD, it can also be beneficial for connecting blocks faster.
To benchmark in real world usage, I spun up 6 identical `t2.small` AWS EC2 instances, all running in the same region in the same VPC. I configured 2 instances to run master, 2 instances to run the change in this PR, and 2 instances to run the change in this PR but with `dbcache=1000`. All instances had `prune=5000` and a 20 GB `gp2` `EBS` volume. A 7th EC2 instance in the same VPC ran master and connected only to some trusted nodes in the outside network. Each of the 6 nodes under test only connected directly to this 7th instance. I manually pruned as much as possible and uploaded the same `blocks`, `chainstate` and `mempool.dat` to all instances. I started all 6 peers simultaneously at block height `835245` and ran them for over a week until block `836534`.
The results were much faster block connection times for this branch compared to master, and much faster for this branch with `dbcache=1000` compared to default `dbcache`.
| branch |speed |
|-----------:|----------:|
| master 1 | 1995.49ms/blk |
| master 2 | 2129.78ms/blk |
| branch default dbcache 1 | 1189.65ms/blk |
| branch default dbcache 2 | 1037.74ms/blk |
| branch dbcache=1000 1 | 393.69ms/blk |
| branch dbcache=1000 2 | 427.77ms/blk |
The log files of all 6 instances are [here](https://gist.github.com/andrewtoth/03c95033e7581d5dbc5be028639a1a91).
There is a lot of noise with the exact times of blocks being connected, so I plotted the rolling 20 block connect time averages. The large dots are the times where the cache is emptied. For the red master nodes, this happens every 24 hours. The blue branch nodes with default `dbcache` only filled up and emptied the caches once, which is seen in the middle. The green branch nodes with 1000 `dbcache` never emptied the cache. It is very clear from the chart that whenever the cache is emptied, connect block speed degrades significantly.

Also note that this still clears the cache for pruning flushes. Having frequent pruning flushes with a large cache that doesn't clear is less performant than the status quo https://github.com/bitcoin/bitcoin/pull/15265#issuecomment-458657451. See https://github.com/bitcoin/bitcoin/pull/28280.
ACKs for top commit:
sipa:
utACK 4a6d1d1e3b
achow101:
ACK 4a6d1d1e3b
brunoerg:
crACK 4a6d1d1e3b
Tree-SHA512: 05dbc677bc309bbcf89c52a6c5e853e2816b0ef0b5ee3719b30696df315a0427e244bb82da9ad828ec0e7ea8764552f8affe14c0184b52adf1909f5d8c1b4f9e
b77bad309e rpc: move UniValue in blockToJSON (willcl-ark)
Pull request description:
Fixes: #24542Fixes: #30052
Without explicitly declaring the move, these `UniValues` get copied, causing increased memory usage. Fix this by explicitly moving the `UniValue` objects.
Used by `rest_block` and `getblock` RPC.
ACKs for top commit:
maflcko:
review ACK b77bad309e
ismaelsadeeq:
ACK b77bad309e
TheCharlatan:
ACK b77bad309e
theuni:
utACK b77bad309e
hebasto:
ACK b77bad309e, I have reviewed the code and it looks OK.
BrandonOdiwuor:
ACK b77bad309e
Tree-SHA512: 767608331040f9cfe5c3568ed0e3c338920633472a1a50d4bbb47d1dc69d2bb11466d611f050ac8ad1a894b47fe1ea4d968cf34cbd44d4bb8d479fc5c7475f6d
58594c7040 fuzz: txorphan tests fixups (Sergi Delgado Segura)
Pull request description:
Motivated by https://github.com/bitcoin/bitcoin/pull/28970#discussion_r1576401327
Adds the following fixups in txorphan fuzz tests:
- Don't bond the output count of the created orphans to the number of available coins
- Allow duplicate inputs but don't store duplicate outpoints
Most significantly, this gets rid of the `duplicate_input` flag altogether, making the test easier to reason about. Notice how, under normal conditions, duplicate inputs would be caught by `MemPoolAccept::PreChecks`, however, no validations checks are run on the test before adding data to the orphanage (neither were they before this patch)
## Rationale
The way the test is currently written, duplicate inputs are allowed based on a random flag (`duplicate_input`). If the flag is unset, upon selecting an outpoint as input for a new transaction, the input is popped to prevent re-selection and later re-added to the collection (once all inputs have been picked). However, the re-addition to the collection is performed independently of whether the flag was set or not. This means that, if the flag is set, the selected inputs are duplicated which in turn makes these inputs more likely to be re-picked in the following iteration of the loop.
Additionally, both the input and output count of the transaction are bonded to the number of available outpoints. This makes sense for the former, but the latter shouldn't be.
ACKs for top commit:
maflcko:
utACK 58594c7040
glozow:
ACK 58594c7
instagibbs:
ACK 58594c7040
Tree-SHA512: e97cc2a43e388f87b64d2e4e45f929dd5b0dd85d668dd693b75e4c9ceea734cd7645952385d428208d07b70e1aafbec84cc2ec264a2e07d36fc8ba3e97885a8d
95897ff181 doc: removed help text saying that peers may not connect automatically (kevkevin)
Pull request description:
Introduced in https://github.com/bitcoin/bitcoin/pull/23542 and released in version 23.0 there has been significant time since this change (2 years).
This should be removed as it is no longer relevant
ACKs for top commit:
stickies-v:
ACK 95897ff181
tdb3:
ACK for 95897ff181.
vasild:
ACK 95897ff181
jonatack:
ACK 95897ff181
kristapsk:
ACK 95897ff181. According to https://bitnodes.io/dashboard/#user-agents stats, most nodes on the network are v23+.
Tree-SHA512: 9e35194f8a1e06f1447450af2ea30cdc43722665c2d2e4b7aa9a52afac5c1e83fed744742c836743a555cc180c90f9eebdc6637eba6190010d693eef9c5834f7
Without explicitly declaring the move, these UniValues get copied,
causing increased memory usage. Fix this by explicitly moving the
UniValue objects.
Used by `rest_block` and `getblock` RPC.
Bech32(m) was defined with a 90 character limit so that certain
guarantees for error detection could be made for segwit addresses.
However, there is nothing about the encoding scheme itself that requires
a limit and in practice bech32(m) has been used without the 90 char
limit (e.g. lightning invoices).
Further, increasing the character limit doesn't do away with error
detection, it simply lessons the guarantees.
Model charlimit as an Enum, so that if a different address scheme is
using bech32(m), the character limit for that address scheme can be
used, rather than always using the 90 charlimit defined for segwit
addresses.
upate comment
`ProduceSignature` already calls `VerifyScript` internally as last step in
order to check whether the signature data is complete. If and only if that is
the case, the `complete` field of the `SignatureData` is set accordingly and
there is no need then to verify the script after again, as we already know that
it would succeed.
This leads to a rough ~20% speed-up for `SignTransaction` for single-input
ECDSA or Taproot inputs, according to the `SignTransaction{ECDSA,Taproot}`
benchmarks.
671b7a3251 gui: fix create unsigned transaction fee bump (furszy)
Pull request description:
Fixes#810.
Not much to explain; we were requiring the wallet to be unlocked for the unsigned transaction creation process.
Fix this by moving the unlock wallet request to the signed transaction creation process.
ACKs for top commit:
pablomartin4btc:
tACK 671b7a3251
hebasto:
ACK 671b7a3251, tested on Ubuntu 24.04.
Tree-SHA512: 5b9ec5a1b91c014c05c83c63daaa8ba33f9dc1bfa930442315a0913db710df17a1b6bb4ad39f1419a7054f37ebedb7ad52e1c5d3d2fb444b1676162e89a4efd2
d1ed09a764 Bugfix: GUI: Help messages already have a trailing newline, so don't add an extra one (Luke Dashjr)
Pull request description:
Reviewing #29585, I noticed that `bitcoin-qt` adds an extra newline for `-help` and `-version` beyond the other binaries'.
ACKs for top commit:
hebasto:
ACK d1ed09a764, tested on Ubuntu 24.04.
Tree-SHA512: 15ee9d1060c2492bb3b04a0ac4cb53f7b959bbe32bce415793da0c221f1c963c8f2bb3996ea07d1a7c192bfc2e23be2cd7d4e5649c592eb3fc03906c2763f1aa
10c5275ba4 gui: don't permit port in proxy IP option (willcl-ark)
Pull request description:
Fixes: https://github.com/bitcoin-core/gui/issues/809
Previously it was possible through the GUI to enter an IP address:port into the "Proxy IP" configuration box. After the node was restarted the errant setting would prevent the node starting back up until manually removed from settings.json.
Prevent this with a simple check for ":" in the string. This is acceptable here in the GUI setting because we already fail on a hostname such as "http://x.x.x.x", so it won't cause false positives.
ACKs for top commit:
furszy:
utACK 10c5275ba4
hebasto:
ACK 10c5275ba4, tested on Ubuntu 24.04.
Tree-SHA512: ed83590630cf693680a3221f701ecd18dd08710a17b726dc4978a3a6e330a34fb77d73a4f710c01bcb3faf88b6604ff37bcdbb191ce1623348ca5b92fd6fe9a7
3bf00e1360 gui: debugwindow: update session ID tooltip (Marnix)
Pull request description:
When you have a v2 connection, there is always a session ID.
the _if any_ is a leftover from https://github.com/bitcoin-core/gui/pull/754, where the session ID property initially would always be displayed (transport v1 and v2).
So the session ID could be empty when you have a v1 connection.
As now the _Session ID_ property only is displayed for v2 connection, there will always be a session ID.
master

PR

Session ID not shown when transport v1

<!--
*** Please remove the following help text before submitting: ***
Pull requests without a rationale and clear improvement may be closed
immediately.
GUI-related pull requests should be opened against
https://github.com/bitcoin-core/gui
first. See CONTRIBUTING.md
-->
<!--
Please provide clear motivation for your patch and explain how it improves
Bitcoin Core user experience or Bitcoin Core developer experience
significantly:
* Any test improvements or new tests that improve coverage are always welcome.
* All other changes should have accompanying unit tests (see `src/test/`) or
functional tests (see `test/`). Contributors should note which tests cover
modified code. If no tests exist for a region of modified code, new tests
should accompany the change.
* Bug fixes are most welcome when they come with steps to reproduce or an
explanation of the potential issue as well as reasoning for the way the bug
was fixed.
* Features are welcome, but might be rejected due to design or scope issues.
If a feature is based on a lot of dependencies, contributors should first
consider building the system outside of Bitcoin Core, if possible.
* Refactoring changes are only accepted if they are required for a feature or
bug fix or otherwise improve developer experience significantly. For example,
most "code style" refactoring changes require a thorough explanation why they
are useful, what downsides they have and why they *significantly* improve
developer experience or avoid serious programming bugs. Note that code style
is often a subjective matter. Unless they are explicitly mentioned to be
preferred in the [developer notes](/doc/developer-notes.md), stylistic code
changes are usually rejected.
-->
<!--
Bitcoin Core has a thorough review process and even the most trivial change
needs to pass a lot of eyes and requires non-zero or even substantial time
effort to review. There is a huge lack of active reviewers on the project, so
patches often sit for a long time.
-->
ACKs for top commit:
vostrnad:
ACK 3bf00e1360
kristapsk:
ACK 3bf00e1360
jarolrod:
ACK 3bf00e1360
pablomartin4btc:
tACK 3bf00e1360
hebasto:
ACK 3bf00e1360.
Tree-SHA512: 4de0c56c070dc5d1cee73b629bdf3d1778c6d90d512337aa6cfd3eed4ce95cbcfbe5713e2942f6fc22907b2c4d9df7979ba8e9f91f7cc173b42699ea35113f96
7f5ac4520d build: swap otool for (llvm-)objdump (fanquake)
Pull request description:
This tool is used in GUI packaging on macOS, and also somewhat of a blocker for #21778. The main issue is that some distros don't really ship this tool in a standard ways, i.e Ubuntu only ships `llvm-otool` with a version suffix, i.e `llvm-otool-17`, which makes it hard to find and use. Rather than trying to deal with that mess, switch to using the equivalent LLVM tool (objdump), which is a drop-in replacement.
ACKs for top commit:
TheCharlatan:
ACK 7f5ac4520d
theuni:
ACK 7f5ac4520d. Tested `make deploy` on native macOS. Looks good.
hebasto:
ACK 7f5ac4520d.
Tree-SHA512: ac978043f14fb448010542a4a7ce8c6c74b4cbd90f83b4cb4d0bff55974010f10a70b5354f65b239a8bd961d7a3aca22ca165b42954ca87879b9e0524db5f879
Addnode (manual) peers connected to us via the cjdns network are currently not
detected by CConnman::GetAddedNodeInfo(), i.e. fConnected is always false.
This causes the following issues:
- RPC `getaddednodeinfo` incorrectly shows them as not connected
- CConnman::ThreadOpenAddedConnections() continually retries to connect them
96378fe734 Refactor: Remove ECC_Start and ECC_Stop from key header (TheCharlatan)
41eba5bd71 kernel: Remove key module from kernel library (TheCharlatan)
a08d2b3cb9 tools: Use ECC_Context helper in bitcoin-tx and bitcoin-wallet tools (Ryan Ofsky)
28905c1a64 test: Use ECC_Context helper in bench and fuzz tests (Ryan Ofsky)
538fedde1d common: Add ECC_Context RAII wrapper for ECC_Start/ECC_Stop (Ryan Ofsky)
Pull request description:
The key module's functionality is not used by the kernel library, but currently kernel users are still required to initialize the key module's `secp256k1_context_sign` global as part of the `kernel::Context` through `ECC_Start`. So move the `ECC_Start` call to the `NodeContext` ctor instead to completely remove the key module from the kernel library.
The gui tests currently keep multiple `NodeContext` objects in memory, so call `ECC_Stop` manually to avoid triggering an assertion on `ECC_Start`.
---
This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587). It removes a module from the kernel library.
ACKs for top commit:
achow101:
ACK 96378fe734
ryanofsky:
Code review ACK 96378fe734. Just suggested comment changes since last review.
theuni:
utACK 96378fe734
Tree-SHA512: 40be427e8e2c920c0e3ce64a9bdd90551be27a89af11440bfb6ab0dd3a1d1ccb7cf1f82383cd782818cd1bb44d5ae5d2161cf4d78d3127ce4987342007090bab
e912717ff6 test: add missing comparison of node1's mempool in MempoolPackagesTest (umiumi)
Pull request description:
#29941 Recreated a pull request because there was a conflict. Trying to resolve the conflict but the old one automatically closed.
Add missing comparison for TODO comments in `mempool_packages.py`
Also, notice that the ancestor size limits and descendant size limits actually implemented in #21800 , so I removed the todo for those two size limits.
ACKs for top commit:
kevkevinpal:
ACK [e912717](e912717ff6)
achow101:
ACK e912717ff6
alfonsoromanz:
Tested ACK e912717ff6. The code looks good to me and the test execution is successful.
rkrux:
tACK [e912717](e912717ff6)
Tree-SHA512: 8cb51746b0547369344c9ceef59599bfe9c91d424687af5e24dc6641f9e99fb433515d79c724e71fd3d5e02994f0cef623d3674367b8296b05c3c6fcdde282ef
fd6a7d3a13 test: use sleepy wait-for-log in reindex readonly (Matthew Zipkin)
Pull request description:
Also rename the busy wait-for-log method to prevent recurrence. See https://github.com/bitcoin/bitcoin/pull/27039#discussion_r1532578152
ACKs for top commit:
maflcko:
utACK fd6a7d3a13
achow101:
ACK fd6a7d3a13
tdb3:
ACK for fd6a7d3a13
rkrux:
ACK [fd6a7d3](fd6a7d3a13)
Tree-SHA512: 7ff0574833df1ec843159b35ee88b8bb345a513ac13ed0b72abd1bf330c454a3f9df4d927871b9e3d37bfcc07542b06ef63acef8e822cd18499adae8cbb0cda8
dd8fa86193 test: use tagged ephemeral MiniWallet instance in fill_mempool (Sebastian Falbesoner)
b2037ad4ae test: add MiniWallet tagging support to avoid UTXO mixing (Sebastian Falbesoner)
c8e6d08236 test: refactor: eliminate COINBASE_MATURITY magic number in fill_mempool (Sebastian Falbesoner)
4f347140b1 test: refactor: move fill_mempool to new module mempool_util (Sebastian Falbesoner)
Pull request description:
Different MiniWallet instances using the same mode (either ADDRESS_OP_TRUE, RAW_OP_TRUE or RAW_P2PK) currently always create and spend UTXOs with identical output scripts, which can cause unintentional tx dependencies (see e.g. the discussion in https://github.com/bitcoin/bitcoin/pull/29827#discussion_r1565443465). In order to avoid mixing of UTXOs between instances, this PR introduces the possibility to provide a MiniWallet tag name, that is used to derive a different internal key for the taproot construction, leading to a different P2TR output script. Note that since we use script-path spending and only the key-path is changed here, no changes in the MiniWallet spending logic are needed.
The new tagging option is then used in the `fill_mempool` helper to create an ephemeral wallet for the filling txs, as suggested in https://github.com/bitcoin/bitcoin/pull/29827#discussion_r1565964264. To avoid circular dependencies, `fill_mempool` is moved to a new module `mempool_util.py` first.
I'm still not sure if a generic word like "tag" is the right term for what this tries to achieve, happy to pick up better suggestions. Also, maybe passing a tag name is overkill and a boolean flag like "random_output_script" is sufficient?
ACKs for top commit:
glozow:
ACK dd8fa86193
achow101:
ACK dd8fa86193
rkrux:
tACK [dd8fa86](dd8fa86193)
brunoerg:
utACK dd8fa86193
Tree-SHA512: 5ef3558c3ef5ac32cfa79c8f751972ca6bceaa332cd7daac7e93412a88e30dec472cb041c0845b04abf8a317036d31ebddfc3234e609ed442417894c2bdeeac9
d53d848347 test: adds outbound eviction tests for non outbound-full-relay peers (Sergi Delgado Segura)
a8d9a0edc7 test: adds outbound eviction functional tests, updates comment in ConsiderEviction (Sergi Delgado Segura)
Pull request description:
## Motivation
While checking the outbound eviction code I realized a case was not considered within the comments, which in turn made me realize we had no functional tests for the outbound eviction case (when I went to check/add the test case).
This PR updates the aforementioned comment and adds functional tests to cover the outbound eviction logic, in addition to the existing unit tests found at `src/test/denialofservice_tests.cpp`.
ACKs for top commit:
davidgumberg:
reACK d53d848347
tdb3:
Re ACK for d53d848347
achow101:
ACK d53d848347
cbergqvist:
ACK d53d848347
Tree-SHA512: 633b84bb1229fe21e2f650c1beada33ca7f190b64eafd64df2266516d21175e5d652e019ff7114f00cb8bd19f5817dc19e65adf75767a88e24dc0842ce40c63e
6abe772a17 contrib: Add asmap-tool (Fabian Jahr)
Pull request description:
This adds `asmap.py` and `asmap-tool.py` from sipa's `nextgen` branch: https://github.com/sipa/asmap/tree/nextgen
The motivation is that we should maintain the tooling for de- and encoding asmap files within the bitcoin core repository because it is not possible to use an asmap file that is not encoded.
We already had an earlier version of `asmap.py` within the seeds contrib tools. The newer version only had a small amount of changes and is still compatible, so the old version is removed from contrib/seeds and the new version is made available to `makeseeds.py`.
ACKs for top commit:
virtu:
ACK [6abe772](6abe772a17)
0xB10C:
ACK 6abe772a17
achow101:
ACK 6abe772a17
brunoerg:
ACK 6abe772a17
Tree-SHA512: cc2a82ffa4eb46fa0ce4ca769dd82f8d0d2f37fc3652aa748eeb060e1142f9da4035008fe89433e2fd524a4dc153b7b9c085748944b49137b37009b0c0be8afb
b259b0e8d3 [Test] Assumeutxo: ensure failure when importing a snapshot twice (Alfonso Roman Zubeldia)
Pull request description:
I am getting familiar with the `assume_utxo` tests and I found that the scenario of trying to activate a snapshot twice is not covered. This test is to ensure failure when loading a snapshot if there is already a snapshot-based chainstate.
ACKs for top commit:
fjahr:
Code review ACK b259b0e8d3
kevkevinpal:
tACK [b259b0e](b259b0e8d3)
achow101:
ACK b259b0e8d3
rkrux:
tACK [b259b0e](b259b0e8d3)
Tree-SHA512: 3510861390d0e40cdad6861b728df04827a1b63e642f3d956aee66ed2770b1cb7e3aa3eb00c62eb9da0544703c943cc5296936c9ebfcac18c719741c354421bb
The key module's functionality is not used by the kernel library, but
currently kernel users are still required to initialize the key module's
`secp256k1_context_sign` global as part of the `kernel::Context` through
`ECC_Start`.
189d0da3f6 build, test: Remove unused `TIMEOUT` environment variable (Hennadii Stepanov)
Pull request description:
Setting the `TIMEOUT` environment variable has been a noop in both cases since its introduction.
It seems to have been inadvertently copy-pasted from existed code. For example, in commit d80e3cbece, it was needlessly copied from a valid case a few lines above for the `qa/pull-tester/run-bitcoind-for-test.sh` script.
ACKs for top commit:
maflcko:
utACK 189d0da3f6
edilmedeiros:
ACK 189d0da3f6
Tree-SHA512: 61111eba30e0c82a0220bea48eba451cd9caa68785b48ec8a91059ca5aadfaff2f6d2ccdc5aa737c5cefa33579cb735431bb9e94bda8fa047825d7bd28d542fb
78e52f663f doc: rpc: fix submitpackage examples (stickies-v)
1a875d4049 rpc: update min package size error message in submitpackage (stickies-v)
f9ece258aa doc: rpc: submitpackage takes sorted array (stickies-v)
17f74512f0 test: add bounds checking for submitpackage RPC (stickies-v)
Pull request description:
`submitpackage` requires the package to be topologically sorted with the child being the last element in the array, but this is not documented in the RPC method or the error messages.
Also sneaking in some other minor improvements that I found while going through the code:
- Informing the user that `package` needs to be an array of length between `1` and `MAX_PACKAGE_COUNT` is confusing when `IsChildWithPackage()` requires that the package size >= 2. Remove this check to avoid code duplication and sending a confusing error message.
- fixups to the `submitpackage` examples
ACKs for top commit:
fjahr:
re-ACK 78e52f663f
instagibbs:
ACK 78e52f663f
achow101:
ACK 78e52f663f
glozow:
utACK 78e52f663f
Tree-SHA512: a8845621bb1cbf784167fc7c82cb8ceb105868b65b26d3465f072d1c04ef3699e85a21a524ade805d423bcecbc34f7d5bff12f2c21cbd902ae1fb154193ebdc9
fKnown is true during reindex (and only then), which deletes
any existing snapshot chainstate. As a result, this function can never
be called wth fKnown set and a snapshot chainstate.
Add an Assume for this, and make the code initializing a blockfile cursor
for the snapshot conditional on !fKnown.
This is a preparation for splitting the reindex logic out of
FindBlockPos in the following commits.
75d27fefc7 net: reduce LOCK(cs_main) scope in ProcessGetBlockData (Andrew Toth)
613a45cd4b net: reduce LOCK(cs_main) scope in GETBLOCKTXN (Andrew Toth)
Pull request description:
Inspired by https://github.com/bitcoin/bitcoin/pull/11913 and https://github.com/bitcoin/bitcoin/pull/26308.
`cs_main` doesn't need to be locked while reading blocks. This removes the locks in `net_processing`.
ACKs for top commit:
sr-gi:
ACK [75d27fe](75d27fefc7)
achow101:
ACK 75d27fefc7
furszy:
ACK 75d27fefc with a non-blocking nit.
mzumsande:
Code Review ACK 75d27fefc7
TheCharlatan:
ACK 75d27fefc7
Tree-SHA512: 79b85f748f68ecfb2f2afd3267857dd41b8e76dd482c9c922037399dcbce7b1e5d4c708a4f5fd17c3fb6699b0d88f26a17cc1d92db115dd43c8d4392ae27cec4
98570fe29b test: add coverage for parsing cryptographically invalid pubkeys (Sebastian Falbesoner)
c740b154d1 rpc: use `HexToPubKey` helper for all legacy pubkey-parsing RPCs (Sebastian Falbesoner)
100e8a75bf rpc: check and throw specific pubkey parsing errors in `HexToPubKey` (Sebastian Falbesoner)
Pull request description:
Parsing legacy public keys can fail for three reasons (in this order):
- pubkey is not in hex
- pubkey has an invalid length (not 33 or 65 bytes for compressed/uncompressed, respectively)
- pubkey is crytographically invalid, i.e. is not on curve (`CPubKey.IsFullyValid()` check)
Many RPCs currently perform these checks manually with different error messages, even though we already have a `HexToPubKey` helper. This PR puts all three checks in this helper (the length check was done on the call-sites before), adds specific error messages for each case, and consequently uses it for all RPCs that parse legacy pubkeys. This leads to deduplicated code and also to more consistent and detailed error messages for the user.
Affected RPC calls are `createmultisig`, `addmultisigaddress`, `importpubkey`, `importmulti`, `fundrawtransaction`, `walletcreatefundedpsbt`, `send` and `sendall`.
Note that the error code (-5 a.k.a. `RPC_INVALID_ADDRESS_OR_KEY`) doesn't change in any of the causes, so the changes are not breaking RPC API compatibility. Only the messages are more specific.
The last commits adds test coverage for the cryptographically invalid (not-on-curve) pubkey case which wasn't exercised before.
ACKs for top commit:
stratospher:
tested ACK 98570fe.
davidgumberg:
ACK 98570fe29b
Eunovo:
Tested ACK 98570fe29b
achow101:
ACK 98570fe29b
Tree-SHA512: cfa474176e95b5b18f3a9da28fdd9e87195cd58994c1331198f2840925fff322fd323a6371feab74a1b32e4b9ea58a6dc732fa751b4cdd45402c1029af609ece
Setting the `TIMEOUT` environment variable has been a noop in both cases
since its introduction.
It seems to have been inadvertently copy-pasted from existing code. For
example, in commit d80e3cbece, it was
needlessly copied from a valid case a few line above for the
`qa/pull-tester/run-bitcoind-for-test.sh` script.
Similar to libtool, (llvm-)otool only exists with a version suffix
on some systems (Ubuntu), which makes it annoying to use/find. Avoid
this, by switching to objdump. Which is a drop-in replacement.
This is related to #21778, and the switchover to using vanilla LLVM for
macOS.
ee67bba76c test: added test coverage to loadtxoutset (kevkevin)
Pull request description:
The functional test coverage did not cover the rpc error of "Couldn't open file..." for loadtxoutset and this test adds coverage for it
This adds coverage to this line
https://github.com/bitcoin/bitcoin/blob/master/src/rpc/blockchain.cpp#L2777
ACKs for top commit:
maflcko:
ACK ee67bba76c
davidgumberg:
LGTM ACK ee67bba76c
rkrux:
ACK [ee67bba](ee67bba76c)
alfonsoromanz:
ACK ee67bba76c. Code looks good to me. I also ran `test/functional/feature_assumeutxo.py` to make sure all tests passes, including this one.
tdb3:
ACK for ee67bba76c
Tree-SHA512: 210a7eb928f625d2a8d9acb63ee83cb4aaec9c267e5a0c52ad219c2935466e2cdc68667e30ad29566e6060981587e5bec42805d296f6e60f9b3b13f3330575f2
fab179d102 ci: Exclude feature_init for now in valgrind task (MarcoFalke)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/30011
ACKs for top commit:
fanquake:
ACK fab179d102
Tree-SHA512: 5943a2abcec59253af8775e8ac7a120011a92cb66711b01a7e377a9302175d56c7de39ce028edc875b1584bf65458f92face2b0ee2028e84f4d3978d2cbafd0a
fb9f150759 gui: fix misleading signmessage error with segwit (willcl-ark)
Pull request description:
As described in https://github.com/bitcoin/bitcoin/issues/10542 (and numerous other places), message signing in Bitcoin Core does not support "signing with a segwit address" and likely will not in the foreseeable future, or at least until a new message-signing standard is agreed upon.
Therefore update the possibly misleading error message presented to the user in the GUI to detail more specifically the reason their message cannot be signed, in the case that a non P2PKH address is entered.
This change takes the [suggested wording](https://github.com/bitcoin/bitcoin/issues/10542#issuecomment-1960313569) from @adiabat.
Perhaps with this we can close https://github.com/bitcoin/bitcoin/issues/10542 ?
ACKs for top commit:
hebasto:
ACK fb9f150759.
Tree-SHA512: 5ba8d722ad3632dad2e0a2aa94b0f466b904e7885a247a5d26ebdfce54e3611090b103029d8dfce92adc49e50fe5f4830f687d867b4c56c3ea997e519b4e188d
fa09451f8e Add lint check for bitcoin-config.h include IWYU pragma (MarcoFalke)
dddd40ba82 scripted-diff: Add IWYU pragma keep to bitcoin-config.h includes (MarcoFalke)
Pull request description:
The `bitcoin-config.h` includes have issues:
* The header is incompatible with iwyu, because symbols may be defined or not defined. So the `IWYU pragma: keep` is needed to keep the include when a symbol is not defined on a platform. Compare the previous discussion in https://github.com/bitcoin/bitcoin/pull/29408#issuecomment-1948959711
* Guarding the includes by `HAVE_CONFIG_H` is verbose and brittle. Now that all build config dependencies have been removed from low level headers, the benefits are questionable, and the guard can be removed. The linter could also be tricked by guarding the include by `#if defined(HAVE_C0NFIG_H)` (`O` replaced by `0`). Compare the previous discussion in https://github.com/bitcoin/bitcoin/pull/29404#discussion_r1483189853 .
ACKs for top commit:
achow101:
ACK fa09451f8e
TheCharlatan:
ACK fa09451f8e
hebasto:
re-ACK fa09451f8e, only rebased since my recent [review](https://github.com/bitcoin/bitcoin/pull/29494#pullrequestreview-2028864535) (`timedata.cpp` removed in https://github.com/bitcoin/bitcoin/pull/29623).
Tree-SHA512: 47cb973f7f24bc625acc4e78683371863675d186780236d55d886cf4130e05a78bb04f1d731aae7088313b8e963a9677cc77cf518187dbd99d776f6421ca9b52
a68fed111b net: Fix misleading comment for Discover (laanwj)
7766dd280d net: Replace ifname check with IFF_LOOPBACK in Discover (laanwj)
Pull request description:
Checking the interface name is kind of brittle. In the age of network namespaces and containers, there is no reason a loopback interface can't be called differently.
Check for the `IFF_LOOPBACK` flag to detect loopback interface instead.
Also remove a misleading comment in Discover's doc comment.
ACKs for top commit:
sipa:
utACK a68fed111b
willcl-ark:
utACK a68fed111b
theuni:
utACK a68fed111b. Satoshi-era brittleness :)
Tree-SHA512: e2d7fc541f40f6a6af08286e7bcb0873ff55debdcd8b38b03f274897b673a6fb51d84d6c7241a02a9567ddf2645f50231d91bb1f55307ba7c6e68196c29b0edf
42fb5311b1 rpc: return warnings as an array instead of just a single one (stickies-v)
Pull request description:
The RPC documentation for `getblockchaininfo`, `getmininginfo` and `getnetworkinfo` states that "warnings" returns "any network and blockchain warnings". In practice, only a single warning (i.e. the latest one that is set) is returned, the other ones are ignored.
Fix that by returning all warnings as an array.
As a side benefit, clean up the GetWarnings() logic.
Since this PR changes the RPC result schema, I've added release notes. Users can temporarily revert to the old results by using `-deprecatedrpc=warnings`, until it's removed in a future version.
---
Some historical context from git log:
- when `GetWarnings` was introduced in 401926283a, it was used in the `getinfo` RPC, where only a [single error/warning was returned](401926283a (diff-7442c48d42cd5455a79915a0f00cce5e13359db46437a32b812876edb0a5ccddR250)) (similar to how it is now).
- later on, "warnings" RPC response fields were introduced, e.g. in ef2a3de25c, with the description [stating](ef2a3de25c (diff-1021bd3c74415ad9719bd764ad6ca35af5dfb33b1cd863c0be49bdf52518af54R411)) that it returned "any network warnings" but in practice still only a single warning was returned
ACKs for top commit:
achow101:
re-ACK 42fb5311b1
tdb3:
Re ACK for 42fb5311b1
TheCharlatan:
ACK 42fb5311b1
maflcko:
ACK 42fb5311b1🔺
Tree-SHA512: 4225ed8979cd5f030dec785a80e7452a041ad5703445da79d2906ada983ed0bbe7b15889d663d75aae4a77d92e302c93e93eca185c7bd47c9cce29e12f752bd3
Previously, our Android builds were geared towards generating APKs,
which relied on Qt. However, after migrating to C++20, compiling for
Android became unfeasible due to Qt 5.15's compatibility limitations
with NDK only up to r25, which includes an outdated embedded libc++.
All removed stuff will be reinstated after migrating the build system to
CMake and upgrading Qt to version 6.x."
7c69baf227 depends: pass verbose through to cmake based make (Max Edwards)
Pull request description:
While testing https://github.com/bitcoin/bitcoin/pull/29708 I was not able to enable verbose output to check which flags were being given to the compiler.
With this PR, running depends with V=1 will enable verbose output from makefiles generated by cmake.
How to test:
```shell
make -C depends libnatpmp V=1
```
ACKs for top commit:
hebasto:
ACK 7c69baf227. Tested using the folowing command:
fanquake:
ACK 7c69baf227
Tree-SHA512: 81cd1326e940c5f14cbde96735fd02b03c1150881ed88d1e8dfa9385dfa12284bfa2cdfe097ce5f43a726c1718afb76ae16f71552ab68c207d74fdc1f7bb46ae
9155b733e1 build, msvc: Compile test\fuzz\miniscript.cpp (Hennadii Stepanov)
Pull request description:
This PR resolves the remained point from the https://github.com/bitcoin/bitcoin/pull/29774#issuecomment-2028808614:
> What is the issue with the ... miniscript fuzz tests?
From the CI [log](https://github.com/bitcoin/bitcoin/actions/runs/8941546183/job/24562123707?pr=30031#step:29:234):
```
miniscript_script: succeeded against 721 files in 1s.
Run miniscript_script with args ['D:\\a\\bitcoin\\bitcoin\\src\\fuzz.exe', WindowsPath('D:/a/_temp/qa-assets/fuzz_seed_corpus/miniscript_script')]
miniscript_smart: succeeded against 1429 files in 2s.
Run miniscript_smart with args ['D:\\a\\bitcoin\\bitcoin\\src\\fuzz.exe', WindowsPath('D:/a/_temp/qa-assets/fuzz_seed_corpus/miniscript_smart')]
miniscript_stable: succeeded against 1871 files in 2s.
Run miniscript_stable with args ['D:\\a\\bitcoin\\bitcoin\\src\\fuzz.exe', WindowsPath('D:/a/_temp/qa-assets/fuzz_seed_corpus/miniscript_stable')]
miniscript_string: succeeded against 918 files in 3s.
Run miniscript_string with args ['D:\\a\\bitcoin\\bitcoin\\src\\fuzz.exe', WindowsPath('D:/a/_temp/qa-assets/fuzz_seed_corpus/miniscript_string')]
```
ACKs for top commit:
maflcko:
ACK 9155b733e1
TheCharlatan:
ACK 9155b733e1
Tree-SHA512: 967f199aac41733265532518ff7b1d881ba5a7bbde9f827d6a0b6d984c94a65b20d5854ce0ea247158eaa17b21d4c9f7d25c79bac17960788bacd2586112630b
This also changes behavior if ReadBlockFromDisk or
ReadRawBlockFromDisk fails. Previously, the node would crash
due to an assert. This has been replaced with logging the error,
disconnecting the peer, and returning early.
63317103c9 miniscript: make operator_mst consteval (Pieter Wuille)
Pull request description:
It seems modern compilers don't realize that all invocations of operator""_mst can be evaluated at compile time, despite the `constexpr` keyword.
Since C++20, we can force them to evaluate at compile time using `consteval`, turning all the miniscript type constants into actual compile-time constants.
This should give a nice but not very important speedup for miniscript logic, but it's also a way to start testing C++20 features.
ACKs for top commit:
hebasto:
re-ACK 63317103c9.
theuni:
utACK 63317103c9
Tree-SHA512: bdc9f1a6499b8bb3ca04f1a158c31e6876ba97206f95ee5718f50efd58b5b4e6b8867c07f791848430bfaa130b9676d8a68320b763cda9a340c75527acbfcc9e
976e5d8f7b test: Fix `test/streams_tests.cpp` compilation on SunOS / illumos (Hennadii Stepanov)
Pull request description:
On systems where `int8_t` is defined as `char`, the `{S,Uns}erialize(Stream&, signed char)` functions become undefined.
This PR resolves the issue by testing `{S,Uns}erialize(Stream&, int8_t)` instead.
No behavior change on systems where `int8_t` is defined as `signed char`, which is the case for most other systems.
Fixes https://github.com/bitcoin/bitcoin/issues/29884.
An alternative approach is mentioned in https://github.com/bitcoin/bitcoin/issues/29884#issuecomment-2058434577 as well.
ACKs for top commit:
maflcko:
lgtm ACK 976e5d8f7b
theuni:
ACK 976e5d8f7b. Nice to have the serialization concept actually tested :)
Tree-SHA512: 1033863e584fa8e99a281b236fa01fc919f610a024bcec792116762e28c1c16ee481bd01325c3a0ca9dd9d753176aa63bd9ac7e08a9bbce772db2949d06f6e61
f0e22be69a build: no-longer disable WARN_CXXFLAGS when CXXFLAGS is set (fanquake)
b088062e68 ci: remove -Wdocumentation from -Werror in multiprocess CI (fanquake)
bec6a88fbc ci: remove -Warray-bounds from -Werror for win64 (fanquake)
7469ac7032 ci: disable -Werror=maybe-uninitialized for Windows builds (fanquake)
Pull request description:
Now that `CXXFLAGS` are [back in user control](https://github.com/bitcoin/bitcoin/pull/24391), I don't think there's a
reason to no-longer use our warning flags when `CXXFLAGS` has been
overriden (this includes, by default, when building from depends).
Anyone can suppress warnings from third-party code by
passing the relevant `-Wno-` options in `CXXFLAGS`.
Closes: #18092.
ACKs for top commit:
maflcko:
utACK f0e22be69a🍡
hebasto:
ACK f0e22be69a.
theuni:
ACK f0e22be69a. It'll be nice to have this fixed.
TheCharlatan:
ACK f0e22be69a
Tree-SHA512: dcef4bd4a57bab6f586ca015fad725e7a38bf24b7a08808a74d8c8ca47cf68c5fca7b04ed38649a047c6929fb708e2c97f2000fc46d5a8d25da49951c5bb0f66
Due to a bug in the legacy wallet, the p2sh maximum script size limit is also imposed
on 'p2sh-segwit' and 'bech32' redeem scripts.
Although redeem scripts over MAX_SCRIPT_ELEMENT_SIZE bytes are technically valid for
segwit output types, we don't want to enable this feature in legacy wallets for the
following reasons:
1) It introduces a compatibility-breaking change requiring downgrade protection; older
wallets would be unable to interact with these "new" legacy wallets.
2) Considering the ongoing deprecation of the legacy spkm, this issue adds another
good reason to transition towards descriptors.
This exercises the bug fixed by previous commits, where
we were unable to generate and sign for segwit redeem scripts
(in this case multisig redeem scripts) longer than 520 bytes.
and also, this adds coverage for legacy 15-15 multisig script
generation and signing.
The process currently fails to sign redeem scripts that are longer than
520 bytes. Even when it shouldn't. The 520 bytes redeem scripts limit
is a legacy p2sh rule, and not a segwit limitation.
Segwit redeem scripts are not restricted by the script item size limit.
The reason why this occurs, is the usage of the same keystore used by
the legacy spkm. Which contains blockage for any redeem scripts longer
than the script item size limit.
The multisig script generation process currently fails when
the user exceeds 15 keys, even when it shouldn't. The maximum
number of keys allowed for segwit redeem scripts (p2sh-segwit
and bech32) is 20 keys.
This is because the redeem script placed in the witness is not
restricted by the item size limit.
The reason behind this issue is the utilization of the legacy
p2sh redeem script restrictions on segwit ones. Redeem scripts
longer than 520 bytes are blocked from being inserted into the
keystore, which causes the signing process and the descriptor
inference process to fail.
This occurs because the multisig generation flow uses the same
keystore as the legacy spkm (FillableSigningProvider), which
contains the 520-byte limit.
And also, simplified the test a bit by re-using the already existing 'wallet_multi'
(instead of creating a new one). Plus, removed the 'is_bdb_compiled()' calls
which were there basically to check if the test has the wallet compiled or not.
The function exists merely to check that the node2's wallet
received the transactions created during all the 'do_multisig()'
calls.
It was created as a standalone function because 'getbalance()'
only returns something when transactions are confirmed. So,
the rationale on that time was to have a method mining blocks
to confirm the recently created transactions to be able to
check the incoming balance.
This is why we have the "moved" class field.
This change removes all the hardcoded amounts and verifies
node2 balance reception directly inside 'do_multisig()'.
Cleaning up the test in the following ways:
* Generate priv-pub key pairs used for testing only once (instead of doing it 4 times).
* Simplifies 'wmulti' wallet creation, load and unload process.
* Removes confusing class members initialized and updated inside a nested for-loop.
* Simplifies do_multisig() outpoint detection:
The outpoint index information is already contained in MiniWallet's
`send_to` return value dictionary as "sent_vout".
Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
ffc674595c Replace remaining "520" magic numbers with MAX_SCRIPT_ELEMENT_SIZE (Jon Atack)
Pull request description:
Noticed these while reviewing BIPs yesterday.
It would be clearer and more future-proof to refer to their constant name.
ACKs for top commit:
instagibbs:
ACK ffc674595c
sipa:
ACK ffc674595c
achow101:
ACK ffc674595c
glozow:
ACK ffc674595c, agree it's clearer for these comments to refer to the greppable name of the limit rather than the number
Tree-SHA512: 462afc1c64543877ac58cb3acdb01d42c6d08abfb362802f29f3482d75401a2a8adadbc2facd222a9a9fefcaab6854865ea400f50ad60bec17831d29f7798afe
It seems modern compilers don't realize that all invocations of operator""_mst
can be evaluated at compile time, despite the constexpr keyword.
Since C++20, we can force them to evaluate at compile time, turning all the
miniscript type constants into actual compile-time constants.
It appears that MSVC does not support consteval operator"" when used inside
certain expressions. For the few places where this happens, define a
constant outside the operator call.
Co-Authored-By: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
bd2de7ac59 refactor, test: Always initialize pointer (Hennadii Stepanov)
Pull request description:
This change fixes MSVC warning [C4703](https://learn.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4703).
All `DisableSpecificWarnings` dropped from `test_bitcoin.vcxproj` as all remained are inherited from `common.init.vcxproj`.
Required to simplify warning suppression porting to the CMake-based build system.
ACKs for top commit:
maflcko:
utACK bd2de7ac59
sipsorcery:
utACK bd2de7ac59.
ryanofsky:
Code review ACK bd2de7ac59
Tree-SHA512: 006db041d3c3697a77d9df14de86cf7c8a10804b45789df01268b2236cf6452e77dc57e89f5d5a6bc26d4b5cd483f0722d6035649c8a523b57954bb1fc810d0c
As described in #10542 (and numerous other places), message signing in
Bitcoin Core only supports message signing using P2PKH addresses, at
least until a new message-signing standard is agreed upon.
Therefore update the possibly-misleading error message presented to the
user in the GUI to detail more specifically the reason their message
cannot be signed, in the case that a non P2PKH address is entered.
Now that CXXFLAGS are back in user control, I don't think there's a
reason to no-longer use our warning flags when CXXFLAGS has been
overriden (this includes when building from depends).
Anyone can suppress warnings from third-party code by
passing the relevant `-Wno-` options in CXXFLAGS.
Fixes: #18092.
This produces false positives, such as:
```bash
torcontrol.cpp: In static member function ‘static void TorControlConnection::readcb(bufferevent*, void*)’:
torcontrol.cpp:94:28: error: ‘result’ may be used uninitialized [-Werror=maybe-uninitialized]
94 | self->message.code = ToIntegral<int>(s.substr(0, 3)).value_or(0);
| ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./netaddress.h:18,
from ./torcontrol.h:11,
from torcontrol.cpp:6:
./util/strencodings.h:184:7: note: ‘result’ was declared here
184 | T result;
| ^~~~~~
cc1plus: all warnings being treated as errors
make[2]: *** [Makefile:11088: libbitcoin_node_a-torcontrol.o] Error 1
```
b50d127a77 refactor: Make 64-bit shift explicit (Hennadii Stepanov)
Pull request description:
This PR fixes MSVC warning [C4334](https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-3-c4334) in the fuzzing code. Similar to https://github.com/bitcoin/bitcoin/pull/26252.
All `DisableSpecificWarnings` dropped from `fuzz.vcxproj` as all remained are inherited from `common.init.vcxproj`.
Required to simplify warning suppression porting to the CMake-based build system.
ACKs for top commit:
maflcko:
utACK b50d127a77
sipsorcery:
utACK b50d127a77
Tree-SHA512: 18f6082b4234506ad2f9df54e577031b97cdf9f7ef64cad4162f275660716ab73587a97d3af0f778dfd48d2751d8676b5d3381d0aa837fcc60a09704473a9209
ec1f1abfef test:Validate UTXO snapshot with coin_height > base_height & amount > money_supply (jrakibi)
Pull request description:
### Ensure snapshot loading fails for coins exceeding base height
**Objective**: This test verifies that snapshot loading is correctly rejected for coins with a height greater than the base height.
**Update**:
- Added `test_invalid_snapshot_wrong_coin_code` to `feature_assumeutxo.py`.
- The test artificially sets a coin's height above 299 in a snapshot and checks for load failure.
- Edit: Added a test case for outputs whose amounts surpass the MAX_MONEY supply limit.
This implementation addresses the request for enhancing `assumeutxo` testing as outlined in issue #28648
---
**Edit: This is an explanation on how I arrive at content values: b"\x84\x58" and b"\xCA\xD2\x8F\x5A"**
You can use this tool to decode the utxo snapshot https://github.com/jrakibi/utxo-live
Here’s an overview of how it’s done:
The serialization format for a UTXO in the snapshot is as follows:
1. Transaction ID (txid) - 32 bytes
2. Output Index (outnum)- 4 bytes
3. VARINT (code) - A varible-length integer encoding the height and whether the transaction is a coinbase. The format of this VARINT is (height << 1) | coinbase_flag.
4. VARINT (amount_v) - A variable-length integer that represents a compressed format of the output amount (in satoshis).
For the test cases mentioned:
* **`b"\x84\x58"`** - This value corresponds to a VARINT representing the height and coinbase flag. Once we decode this code, we can extract the height and coinbase using `height = code_decoded >> 1` and `coinbase = code_decoded & 0x01`. In our case, with code_decoded = 728, it results in `height = 364` and `coinbase = 0`.
* **`b"\xCA\xD2\x8F\x5A"`** - This byte sequence represents a compressed amount value. The decompression function takes this value and translates it into a full amount in satoshis. In our case, the decompression of this amount translates to a number larger than the maximum allowed value of coins (21 million BTC)
ACKs for top commit:
fjahr:
re-ACK ec1f1abfef
maflcko:
ACK ec1f1abfef👑
achow101:
ACK ec1f1abfef
Tree-SHA512: 42b36fd1d76e9bc45861028acbb776bd2710c5c8bff2f75c751ed505995fbc1d4bc698df3be24a99f20bcf6a534615d2d9678fb3394162b88133eaec88ca2120
8b52e7f628 update comments in cpp-subprocess (check_output references) (Sebastian Falbesoner)
97f159776e remove unused method `Popen::kill` from cpp-subprocess (Sebastian Falbesoner)
908c51fe4a remove commented out code in cpp-subprocess (Sebastian Falbesoner)
ff79adbe05 remove unused templates from cpp-subprocess (Sebastian Falbesoner)
Pull request description:
This PR removes remaining code that is unused within the cpp-subprocess module (templates and commented out code). Happy to add more removals if anyone finds more unused parts. Note that there are some API functions of the `Popen` class that we don't use, e.g. `wait()`, `pid()`, `poll()`, `kill()`, but they sound IMHO common enough to be useful in the future, so not sure how deep we should go there.
ACKs for top commit:
fjahr:
Code review ACK 8b52e7f628
achow101:
ACK 8b52e7f628
hebasto:
ACK 8b52e7f628.
Tree-SHA512: 14c1cd2216185d941923f06fdc7acbeed66cd87e2691d9a352f7309b3e07fe4877b580f598a2e4106f9c48395ed6de00a0bfb5d3c3af9c4624d1956a0f543e99
fa9be2f795 lint: [doc] Clarify Windows line endings (CR LF) not to be used (MarcoFalke)
Pull request description:
It has been this case since the linter was introduced years ago. Given a misunderstanding (https://github.com/bitcoin/bitcoin/pull/28074#issuecomment-2088028856), clarify the docs.
ACKs for top commit:
brunoerg:
ACK fa9be2f795
Tree-SHA512: be714db9df533e0962ed84102ffdb72717902949b930d58cf5a79cba36297f6b2b1f75e65a2c1f46bcb8e2f4ad5d025f3d15210f468a5ec9631a580b74f923ea
5195baa600 depends: fix miniupnpc snprintf usage on Windows (fanquake)
3c2d440f14 depends: switch miniupnpc to CMake (Cory Fields)
f5618c79d9 depends: add upstream CMake patch to miniupnpc (fanquake)
6866b571ab depends: miniupnpc 2.2.7 (fanquake)
Pull request description:
This picks up one of the changes from #29232, which is a switch to building miniupnpc with CMake. It includes an update to the most recent version of miniupnpc (2.2.7), which means we can drop one patch from that commit, and includes a new patch for a change I've upstreamed https://github.com/miniupnp/miniupnp/pull/721, as well as some suggestions from the previous PR.
ACKs for top commit:
theuni:
ACK 5195baa600.
TheCharlatan:
utACK 5195baa600
Tree-SHA512: 5b27e132cd5eed285e9be34c8b96893417d92a1ae55c99345c9a89e1c1c5e40e4bc840bc061b879758b2b11fcb520cd98c3da985c1e153f2e5380cf63efe2d69
e504b1fa1f test: Add test case for spending bare multisig (Brandon Odiwuor)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/29113
ACKs for top commit:
ajtowns:
ACK e504b1fa1f ; LGTM and just checking the 1-of-3 case seems fine
maflcko:
utACK e504b1fa1f
achow101:
ACK e504b1fa1f
willcl-ark:
reACK e504b1fa1f
Tree-SHA512: 641a12599efa34e1a3eb65b125318df326628fef3e6886410ea9e63a044664fad7bcad46d1d6f41ddc59630746b9963cedb569c2682b5940b32b9225883da8f2
Adds the following fixups in txorphan fuzz tests:
- Don't bond the output count of the created orphans based on the number of available coins
- Allow duplicate inputs, when applicable, but don't store duplicate outpoints
Rationale
---------
The way the test is currently written, duplicate inputs are allowed based on a random flag (`duplicate_input`).
If the flag is unset, upon selecting an outpoint as input for a new transaction, the input is popped to prevent re-selection,
and later re-added to the collection (once all inputs have been picked). However, the re-addition to the collection is performed independently of whether the flag was set or not.
This means that, if the flag is set, the selected inputs are duplicated which in turn makes these inputs more likely to be re-picked in the following iteration of the loop.
Additionally, both the input and output count of the transaction and bonded to the number of available outpoints. This makes sense for the former, but the latter shouldn't be.
The RPC documentation for `getblockchaininfo`, `getmininginfo` and
`getnetworkinfo` states that "warnings" returns "any network and
blockchain warnings". In practice, only a single warning is returned.
Fix that by returning all warnings as an array.
As a side benefit, cleans up the GetWarnings() logic.
1ea8674316 [doc] update release-process.md and backports section of CONTRIBUTING (glozow)
Pull request description:
While doing various release process things for the first time, I noticed some of our docs are outdated and/or confusing.
ACKs for top commit:
achow101:
ACK 1ea8674316
Tree-SHA512: 4ad10d4ce2c33fe15cb02599353107bb72ecb867aefc6c120cfd5cdea42aa8fa3783f9e0218c2f3815f030e0694cc8fb24011ce88358a0206cb07416a256a962
774359b4a9 build, msvc: Compile `test\fuzz\bitdeque.cpp` (Hennadii Stepanov)
85f50a46c5 refactor: Fix "error C2248: cannot access private member" on MSVC (Hennadii Stepanov)
Pull request description:
This PR resolves one point from the https://github.com/bitcoin/bitcoin/pull/29774#issuecomment-2028808614:
> What is the issue with the bitdeque... ?
ACKs for top commit:
maflcko:
lgtm ACK 774359b4a9
sipa:
utACK 774359b4a9
achow101:
ACK 774359b4a9
dergoegge:
utACK 774359b4a9
Tree-SHA512: dba5c0217b915468af08475795437a10d8e8dedfadeb319f36d9b1bf54a91a8b2c61470a6047565855276c2bc8589c7776dc19237610b65b57cc841a303de8b3
2179e2c320 doc: i2p: improve `-i2pacceptincoming` mention (brunoerg)
Pull request description:
In i2p documentation, it says that "the first time Bitcoin Core connects to the I2P router,
it automatically generates a persistent I2P address and its corresponding private key by
default _**or if `-i2pacceptincoming=1` is set**_". This is weird, because `-i2pacceptincoming=1`
by itself does not have any effect. Moreover, `-i2pacceptincoming` is 1 by default anyway.
ACKs for top commit:
laanwj:
This documentation change is correct and makes the documentation slightly shorter, thus easier to read. ACK 2179e2c320
davidgumberg:
ACK 2179e2c320
achow101:
ACK 2179e2c320
byaye:
ACK 2179e2c320
Tree-SHA512: 18a6a627343fb0aa824029d99df8a232153ba288ce94ec8c5da25693885237381fba505ea1e71c756b2a611243a302d319ca7ae03b526020cd6588710fc2ac17
82f41d76f1 Added seednode prioritization message to help output (tdb3)
3120a4678a Gives seednode priority over dnsseed if both are provided (Sergi Delgado Segura)
Pull request description:
This is a follow-up of #27577
If both `seednode` and `dnsseed` are provided, the node will start a race between them in order to fetch data to feed the `addrman`.
This PR gives priority to `seednode` over `dnsseed` so if some nodes are provided as seeds, they can be tried before defaulting to the `dnsseeds`
ACKs for top commit:
davidgumberg:
untested reACK 82f41d76f1
itornaza:
tested re-ACK 82f41d76f1
achow101:
ACK 82f41d76f1
cbergqvist:
ACK 82f41d76f1
Tree-SHA512: 4e39e10a7449af6cd9b8f9f6878f846b94bca11baf89ff2d4fbcd4f28293978a6ed71a3a86cea36d49eca891314c834e32af93f37a09c2cc698a878f84d31c62
c6be144c4b Remove timedata (stickies-v)
92e72b5d0d [net processing] Move IgnoresIncomingTxs to PeerManagerInfo (dergoegge)
7d9c3ec622 [net processing] Introduce PeerManagerInfo (dergoegge)
ee178dfcc1 Add TimeOffsets helper class (stickies-v)
55361a15d1 [net processing] Use std::chrono for type-safe time offsets (stickies-v)
038fd979ef [net processing] Move nTimeOffset to net_processing (dergoegge)
Pull request description:
[An earlier approach](1d226ae1f9/) in #28956 involved simplifying and refactoring the network-adjusted time calculation logic, but this was eventually [left out](https://github.com/bitcoin/bitcoin/pull/28956#issuecomment-1904214370) of the PR to make it easier for reviewers to focus on consensus logic changes.
Since network-adjusted time is now only used for warning/informational purposes, cleaning up the logic (building on @dergoegge's approach in #28956) should be quite straightforward and uncontroversial. The main changes are:
- Previously, we would only calculate the time offset from the first 199 outbound peers that we connected to. This limitation is now removed, and we have a proper rolling calculation. I've reduced the set to 50 outbound peers, which seems plenty.
- Previously, we would automatically use the network-adjusted time if the difference was < 70 mins, and warn the user if the difference was larger than that. Since there is no longer any automated time adjustment, I've changed the warning threshold to ~~20~~ 10 minutes (which is an arbitrary number).
- Previously, a warning would only be raised once, and then never again until node restart. This behaviour is now updated to 1) warn to log for every new outbound peer for as long as we appear out of sync, 2) have the RPC warning toggled on/off whenever we go in/out of sync, and 3) have the GUI warn whenever we are out of sync (again), but limited to 1 messagebox per 60 minutes
- no more globals
- remove the `-maxtimeadjustment` startup arg
Closes#4521
ACKs for top commit:
sr-gi:
Re-ACK [c6be144](c6be144c4b)
achow101:
reACK c6be144c4b
dergoegge:
utACK c6be144c4b
Tree-SHA512: 1063d639542e882186cdcea67d225ad1f97847f44253621a8c4b36c4d777e8f5cb0efe86bc279f01e819d33056ae4364c3300cc7400c087fb16c3f39b3e16b96
e518a8bf8a [functional test] opportunistic 1p1c package submission (glozow)
87c5c524d6 [p2p] opportunistically accept 1-parent-1-child packages (glozow)
6c51e1d7d0 [p2p] add separate rejections cache for reconsiderable txns (glozow)
410ebd6efa [fuzz] break out parent functions and add GetChildrenFrom* coverage (glozow)
d095316c1c [unit test] TxOrphanage::GetChildrenFrom* (glozow)
2f51cd680f [txorphanage] add method to get all orphans spending a tx (glozow)
092c978a42 [txpackages] add canonical way to get hash of package (glozow)
c3c1e15831 [doc] restore comment about why we check if ptx HasWitness before caching rejected txid (glozow)
6f4da19cc3 guard against MempoolAcceptResult::m_replaced_transactions (glozow)
Pull request description:
This enables 1p1c packages to propagate in the "happy case" (i.e. not reliable if there are adversaries) and contains a lot of package relay-related code. See https://github.com/bitcoin/bitcoin/issues/27463 for overall package relay tracking.
Rationale: This is "non-robust 1-parent-1-child package relay" which is immediately useful.
- Relaying 1-parent-1-child CPFP when mempool min feerate is high would be a subset of all package relay use cases, but a pretty significant improvement over what we have today, where such transactions don't propagate at all. [1]
- Today, a miner can run this with a normal/small maxmempool to get revenue from 1p1c CPFP'd transactions without losing out on the ones with parents below mempool minimum feerate.
- The majority of this code is useful for building more featureful/robust package relay e.g. see the code in #27742.
The first 2 commits are followups from #29619:
- https://github.com/bitcoin/bitcoin/pull/29619#discussion_r1523094034
- https://github.com/bitcoin/bitcoin/pull/29619#discussion_r1519819257
Q: What makes this short of a more full package relay feature?
(1) it only supports packages in which 1 of the parents needs to be CPFP'd by the child. That includes 1-parent-1-child packages and situations in which the other parents already pay for themselves (and are thus in mempool already when the package is submitted). More general package relay is a future improvement that requires more engineering in mempool and validation - see #27463.
(2) We rely on having kept the child in orphanage, and don't make any attempt to protect it while we wait to receive the parent. If we are experiencing a lot of orphanage churn (e.g. an adversary is purposefully sending us a lot of transactions with missing inputs), we will fail to submit packages. This limitation has been around for 12+ years, see #27742 which adds a token bucket scheme for protecting package-related orphans at a limited rate per peer.
(3) Our orphan-handling logic is somewhat opportunistic; we don't make much effort to resolve an orphan beyond asking the child's sender for the parents. This means we may miss packages if the first sender fails to give us the parent (intentionally or unintentionally). To make this more robust, we need receiver-side logic to retry orphan resolution with multiple peers. This is also an existing problem which has a proposed solution in #28031.
[1]: see this writeup and its links 02ec218c78/bip-0331.mediawiki (propagate-high-feerate-transactions)
ACKs for top commit:
sr-gi:
tACK e518a8bf8a
instagibbs:
reACK e518a8bf8a
theStack:
Code-review ACK e518a8bf8a📦
dergoegge:
light Code review ACK e518a8bf8a
achow101:
ACK e518a8bf8a
Tree-SHA512: 632579fbe7160cb763bbec6d82ca0dab484d5dbbc7aea90c187c0b9833b8d7c1e5d13b8587379edd3a3b4a02a5a1809020369e9cd09a4ebaf729921f65c15943
6a8b2befea refactor: Avoid copying util::Result values (Ryan Ofsky)
834f65e824 refactor: Drop util::Result operator= (Ryan Ofsky)
Pull request description:
This PR just contains the first two commits of #25665.
It disables copying of `util::Result` objects because unnecessary copies are inefficient and not possible after #25665, which makes `util::Result` object move-only.
It disables the assignment operator and replaces it with an `Update()` method, because #25665 adds more information to `util::Result` objects (warning and error messages and failure values) and having an assignment operator that overwrites data instead of merging it would make it easy to accidentally erase existing information while trying to assign new information.
ACKs for top commit:
stickies-v:
re-ACK 6a8b2befea
achow101:
ACK 6a8b2befea
furszy:
re-ACK 6a8b2befea
Tree-SHA512: 3f21af9031d50d6c68cca69133de03080f69b1ddcf8b140bdeb762069f14645209b2586037236d15b6ebd8973af0fbefd7e83144aeb7b84078a4cb4df812f984
f8a141c2da test: Don't rely on incentive incompatible replacement in mempool_accept_v3.py (Suhas Daftuar)
Pull request description:
In the sibling eviction test, we're currently testing that a transaction with ancestor feerate (and mining score) of 179 s/b is able to replace a transaction with ancestor feerate (and mining score) of 300 s/b, due to a shortcoming in our current RBF rules.
In preparation for fixing our RBF rules to not allow such replacements, fix the test by bumping the fee of the replacement to be a bit higher.
ACKs for top commit:
glozow:
ACK f8a141c2da
instagibbs:
ACK f8a141c2da
Tree-SHA512: 0babe60be2f41634301e434fedb7abc765daaa37c2c280acb569eaf02a793369d81401ab02b8ae1689bda4872f475bd4c2f48cae4a54a61ece20db0a014e23ac
cc15c5bfd1 fuzz: don't allow adding duplicate transactions to the mempool (Suhas Daftuar)
Pull request description:
Filter duplicate transaction ids from being added to the mempool in the `partially_downloaded_block` fuzz target.
I think a prerequisite for calling `CTxMemPool::addUnchecked` should be that the underlying txid doesn't already exist in the mempool (otherwise `addUnchecked` would need a way to return failure, which we don't currently have).
ACKs for top commit:
glozow:
utACK cc15c5bfd1 makes sense to me
maflcko:
lgtm ACK cc15c5bfd1
brunoerg:
ACK cc15c5bfd1
dergoegge:
utACK cc15c5bfd1
Tree-SHA512: 85f84ce405aba584e6d00391515f0a86c5648ce8b2da69036e50a6c1f6833d050d09b1972cc5ffbe7c4edb3e5f7f965ef34bd839deeddac27a889cc8d2e53b8f
2e266f33b5 depends: Fix build of Qt for 32-bit platforms (laanwj)
Pull request description:
The 32 to 64-bit `time_t` transition causes a build failure in the built-in zlib about conflicting `_TIME_BITS` and `_FILE_OFFSET_BITS`.
Note that zlib doesn't use `time_t` at all, so it is a false alarm.
Take the following patch from upstream zlib:
a566e156b3.patch
Closes#29980.
ACKs for top commit:
hebasto:
re-ACK 2e266f33b5.
fanquake:
ACK 2e266f33b5 - at some point qt's open source 5.15.x branch will catch up to where they bumped the internal zlib to >= 1.3 (which contains this change), and we'll be able to drop this patch. Checked that it fixes the build issue in the interim.
Tree-SHA512: b297aed8b299c671ff439b5b7b410832ff5004fd9b13c3b4a5fb5bde9dcf24a5eda08cd0a39565ae0641d9533711142bdc2889a32d343b9c4b41bfac24f0ca28
30a6c99935 rpc: access some args by name (stickies-v)
bbb31269bf rpc: add named arg helper (stickies-v)
13525e0c24 rpc: add arg helper unit test (stickies-v)
Pull request description:
Adds string overloads for the `RPCHelpMan::Arg` and `RPCHelpMan::MaybeArg` helpers to be able to access RPC arguments by name instead of index number. Especially in RPCs with a large number of parameters, this can be quite helpful.
Example usage:
```cpp
const auto action{self.Arg<std::string>("action")};
```
Most of the LoC is adding test coverage and documentation updates. No behaviour change.
An alternative approach to #27788 with significantly less overhaul.
ACKs for top commit:
fjahr:
Code review ACK 30a6c99935
maflcko:
ACK 30a6c99935🥑
ryanofsky:
Code review ACK 30a6c99935. Nice change! Implementation is surprisingly simple and additional unit test coverage is welcome, too.
Tree-SHA512: 4904f5f914fe1d421d32f60edb7c5a028c8ea0f140a2f207a106b4752d441164e073066a6bf2e17693f859fe847815a96609d3cf521e0ac4178d8cd09362ea3d
fae0db555c refactor: Use chrono type for g_mock_time (MarcoFalke)
fa382d3dd0 test: Add missing Assert(mock_time_in >= 0s) to SetMockTime (MarcoFalke)
Pull request description:
Seems odd to have the assert in the *deprecated* function, but not in the other.
Fix this by adding it to the other, and by inlining the deprecated one.
Also, use chrono type for the global mocktime variable.
ACKs for top commit:
davidgumberg:
crACK fae0db555c
stickies-v:
ACK fae0db555c
Tree-SHA512: 630c2917422ff2a7fa307114f95f22ad3c205429ffe36e67f0b2650733e40c876289c1aecebe882a9123d3106db7606bd6eff067ed6e2ecb95765984d3fe8612
fa1964c5b8 build: Bump clang minimum supported version to 15 (MarcoFalke)
Pull request description:
Most supported operating systems ship with clang-15 (or later), so bump the minimum to that and allow new code to drop workarounds for previous clang bugs.
For reference:
* https://packages.debian.org/bookworm/clang-15
* https://packages.ubuntu.com/jammy/clang-15
* CentOS-like 8/9 Stream: All Clang versions from 15 to 17
* FreeBSD 12/13: All Clang versions from 15 to 16
* OpenSuse Tumbleweed ships with https://software.opensuse.org/package/clang (`clang17`); No idea about OpenSuse Leap
On operating systems where the clang version is not shipped by default, the user would have to use GCC, or install clang in a different way. For example:
* https://packages.debian.org/bullseye/g++ (g++-10)
* https://packages.ubuntu.com/focal/g++-10
* https://apt.llvm.org/, or nix, or guix, or compile clang from source, ...
ACKs for top commit:
hebasto:
ACK fa1964c5b8.
fanquake:
ACK fa1964c5b8 - oss-fuzz LLVM will either be bumped globally tomorrow, or we'll land our own bump.
Tree-SHA512: b34234025b471de740480c269449891ebb95a0d9ccca67a355ff6568068bfcf1e8b104e8c13a8c0df07dbc2044dc6f03958063dc572dc4faf830bfe73466d55d
The 32 to 64-bit time_t transition causes a build failure in the built-in
zlib about conflicting _TIME_BITS and _FILE_OFFSET_BITS.
Note that zlib doesn't use time_t at all, so it is a false alarm.
Take the following patch from upstream zlib:
a566e156b3.patch
Closes#29980.
b8e084b978 guix: remove no-longer-used bzip2 (fanquake)
bd6e1d6718 depends: switch qrencode to .tar.gz (fanquake)
4a9b71b900 depends: switch libxcb_util_wm to .tar.gz (fanquake)
fad989852d depends: switch libxcb_util_render to .tar.gz (fanquake)
ce28cb31b4 depends: switch libxcb_util_keysyms to .tar.gz (fanquake)
00a6896346 depends: switch libxcb_util_image to .tar.gz (fanquake)
8e9190c6aa depends: switch libxcb_util to .tar.gz (fanquake)
b845029d46 depends: switch xproto to .tar.gz (fanquake)
5996c30384 depends: switch libXau to .tar.gz (fanquake)
e7a8dd5931 depends: switch fontconfig to .tar.gz (fanquake)
58c423def3 depends: switch boost to .tar.gz (fanquake)
Pull request description:
This moves packages in depends that use `.tar.bzip2` to `.tar.gz` (which is what we use for our own release tarballs). Doing so means we can drop `bzip2` from our Guix release env. You can observe that Guix building master without it would currently fail:
```diff
diff --git a/contrib/guix/manifest.scm b/contrib/guix/manifest.scm
index 8f13c642d3..96818c7748 100644
--- a/contrib/guix/manifest.scm
+++ b/contrib/guix/manifest.scm
@@ -499,7 +499,6 @@ inspecting signatures in Mach-O binaries.")
moreutils
;; Compression and archiving
tar
- bzip2
gzip
xz
;; Build tools
```
`FORCE_DIRTY_WORKTREE=1 ./contrib/guix/guix-build`
```bash
Extracting boost...
/sources/boost_1_81_0.tar.bz2: OK
tar (child): lbzip2: Cannot exec: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
```
Guix Build:
```bash
8f6959d01ae972bae1340dfaf18753607152eca9844e6d8c4fa128314a4ba762 guix-build-b8e084b9781e/output/aarch64-linux-gnu/SHA256SUMS.part
3c9c1cc000e3e6b7c2853c9d530c9afa1c880a43e7ab4c766aaa88283ff0908c guix-build-b8e084b9781e/output/aarch64-linux-gnu/bitcoin-b8e084b9781e-aarch64-linux-gnu-debug.tar.gz
f45fbece697b450538aded11f568e92b2af391e873e113c3038d022eff41688f guix-build-b8e084b9781e/output/aarch64-linux-gnu/bitcoin-b8e084b9781e-aarch64-linux-gnu.tar.gz
08295d770c11b2057206f98aaf4123007c7475bd942840d048f5f9d5efec1ce1 guix-build-b8e084b9781e/output/arm-linux-gnueabihf/SHA256SUMS.part
0a0db6967168019b8b890ec4d31b3a87a88c4956b703938ec4447d514cfc231e guix-build-b8e084b9781e/output/arm-linux-gnueabihf/bitcoin-b8e084b9781e-arm-linux-gnueabihf-debug.tar.gz
3d1538e8bf4edfb66a4875198dfa90b79dcfe44eb9c4e76e47d73a18175c838a guix-build-b8e084b9781e/output/arm-linux-gnueabihf/bitcoin-b8e084b9781e-arm-linux-gnueabihf.tar.gz
87e7805155dbed3bd64763f199ea63843ed8c4eb37873753c7e60b0b42565eaf guix-build-b8e084b9781e/output/arm64-apple-darwin/SHA256SUMS.part
fa33590296aeae2b738b023a4cbf2de4a4e06662a5f7d407c251a8af714bd587 guix-build-b8e084b9781e/output/arm64-apple-darwin/bitcoin-b8e084b9781e-arm64-apple-darwin-unsigned.tar.gz
32b8fbbdf240f9f08e44c7bb0a8ea2e8a40537e59ec2231cf6635edc6592f226 guix-build-b8e084b9781e/output/arm64-apple-darwin/bitcoin-b8e084b9781e-arm64-apple-darwin-unsigned.zip
d176f3b7c8140c8dfde03bd87fd5abd4a89b497ba11fa6849bc92a33cb621a07 guix-build-b8e084b9781e/output/arm64-apple-darwin/bitcoin-b8e084b9781e-arm64-apple-darwin.tar.gz
5273b17087e3565ab042a7989cfba71cf1629331d0267137d7eccabee1a06a13 guix-build-b8e084b9781e/output/dist-archive/bitcoin-b8e084b9781e.tar.gz
b84a9180181994a6a17a1c2a4701f8ba5a82654233d5a8afcf596d28dd8b3924 guix-build-b8e084b9781e/output/powerpc64-linux-gnu/SHA256SUMS.part
fd3396f6b64425a31b5a3565ab4d8a1c1668c291349a0f9e9b8904dad04ee24c guix-build-b8e084b9781e/output/powerpc64-linux-gnu/bitcoin-b8e084b9781e-powerpc64-linux-gnu-debug.tar.gz
73cb4bd2a67934c93ea8e3f3bc04b8917627ec09d75c151bb01977bba97522c8 guix-build-b8e084b9781e/output/powerpc64-linux-gnu/bitcoin-b8e084b9781e-powerpc64-linux-gnu.tar.gz
15938e7f0f71303b96566d60e3b255816e7fd70d628601e592e1d6840eb8d2a1 guix-build-b8e084b9781e/output/riscv64-linux-gnu/SHA256SUMS.part
408b4973865e3a77be833438f71181fd88acd0490127257b3667309e8421030e guix-build-b8e084b9781e/output/riscv64-linux-gnu/bitcoin-b8e084b9781e-riscv64-linux-gnu-debug.tar.gz
a5c02144ffb79cfa0179ff0d7ac0f81192ef1d3b1acfad334adf486e50b776bb guix-build-b8e084b9781e/output/riscv64-linux-gnu/bitcoin-b8e084b9781e-riscv64-linux-gnu.tar.gz
de904843d8bb8601a2d763701ebb929e61b447e01040267a12149a2902489535 guix-build-b8e084b9781e/output/x86_64-apple-darwin/SHA256SUMS.part
414cb3cf3fa10b9a3cda47e98858222f01fdd164371dd54761642e6793099849 guix-build-b8e084b9781e/output/x86_64-apple-darwin/bitcoin-b8e084b9781e-x86_64-apple-darwin-unsigned.tar.gz
6ce43d7f007bf17eca16d3ee48190318e09aacd82c5396f9565e6345ec6bd2fa guix-build-b8e084b9781e/output/x86_64-apple-darwin/bitcoin-b8e084b9781e-x86_64-apple-darwin-unsigned.zip
24eba9c0dd1312a68c2b2a800cc915595e343c0ead982b6cbe025abe7a7bff19 guix-build-b8e084b9781e/output/x86_64-apple-darwin/bitcoin-b8e084b9781e-x86_64-apple-darwin.tar.gz
2869a01ce847298950a91b3b8514bc8fa39cc274a8e9cd4f68f4f038c1bb3040 guix-build-b8e084b9781e/output/x86_64-linux-gnu/SHA256SUMS.part
3f63e1d3b19b640d3994074b344d595bcd6fca420a1a8669b63b4ad22978308b guix-build-b8e084b9781e/output/x86_64-linux-gnu/bitcoin-b8e084b9781e-x86_64-linux-gnu-debug.tar.gz
ccc3eb8eb56c1596981e81c8c95cadee3db799ed69b0cd1fb1e102da10adacfb guix-build-b8e084b9781e/output/x86_64-linux-gnu/bitcoin-b8e084b9781e-x86_64-linux-gnu.tar.gz
1ff6dab6dcde9ddbbe407cca02119c4a5d545034c91389a1c647020902b7b40e guix-build-b8e084b9781e/output/x86_64-w64-mingw32/SHA256SUMS.part
a91c2247fd9d886e3f3ada551c0a4f9f7ffc4874e07ea5ab9de14f2743b9b8c7 guix-build-b8e084b9781e/output/x86_64-w64-mingw32/bitcoin-b8e084b9781e-win64-debug.zip
6fbc8d5df571fd535990370009bdfcbb37b9697c33446a08eadb1279ba6e4649 guix-build-b8e084b9781e/output/x86_64-w64-mingw32/bitcoin-b8e084b9781e-win64-setup-unsigned.exe
38f7a981fd2999c1e138860e1ddc183dafec090d867e37f5ab5c2d48ad4ef9ee guix-build-b8e084b9781e/output/x86_64-w64-mingw32/bitcoin-b8e084b9781e-win64-unsigned.tar.gz
88aca0a40a64a289617aad060a9ccf8c78bc6a201470720d8caf48d793d5207f guix-build-b8e084b9781e/output/x86_64-w64-mingw32/bitcoin-b8e084b9781e-win64.zip
```
ACKs for top commit:
laanwj:
This is fully expected (no weird changes from dropping bzip2 from the build env). ACK b8e084b978
Tree-SHA512: 7da9a75a3ff7fa0c9ff464e3a82f5b1d0cfdd28d5de049c910142179f7e1211c922b705361844c7029ce9baaa8e97e8016b454d2e4eee98e31fae1379674fbe2
Remove obsolete `check_output` references in the comments and remove
the numbering of the Popen API methods, as they don't seem to provide a
value and just make diffs larger for future changes.
Checking the interface name is kind of brittle. In the age of network
namespaces and containers, there is no reason a loopback interface can't
be called differently.
Check for the `IFF_LOOPBACK` flag to detect loopback interface instead.
97a4ad5713 build, msvc: Drop duplicated `common\url.cpp` source file (Hennadii Stepanov)
Pull request description:
After https://github.com/bitcoin/bitcoin/pull/29904, the `common\url.cpp` source file is included into the `SOURCE_FILES` by the `msvc-autogen.py` script.
Removes a compiler [warning](https://github.com/bitcoin/bitcoin/actions/runs/8853698173/job/24315127236#step:20:1776):
```
url.obj : warning LNK4006: "class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > __cdecl UrlDecode(class std::basic_string_view<char,struct std::char_traits<char> >)" (?UrlDecode@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$basic_string_view@DU?$char_traits@D@std@@@2@@Z) already defined in common_url.obj; second definition ignored [D:\a\bitcoin\bitcoin\build_msvc\libbitcoin_common\libbitcoin_common.vcxproj]
```
ACKs for top commit:
fanquake:
ACK 97a4ad5713
Tree-SHA512: 294955d6e6940b48a429e2302fb456706a5c62515d479398036b40716ee6b722535876adeb9b988ddb8fc942fabc39fe358c50eff0baaae92bd24bbeb4362885
f19f0a2e5a test: Run framework unit tests in parallel (tdb3)
Pull request description:
Functional test framework unit tests are currently run prior to all other functional tests.
This PR enables execution of the test framework unit tests in parallel with the functional tests, rather than before the functional tests, saving runtime and more efficiently using available cores.
This is a follow up to https://github.com/bitcoin/bitcoin/pull/29470#issuecomment-1962313977
### New behavior:
1) When running all tests, the framework unit tests are run in parallel with the other tests (unless explicitly skipped with `--exclude`). This parallelization introduces marginal time savings when running all tests, depending on the machine used. As an example, a 2-3% time savings (9 seconds) was observed on a machine using `--jobs=18` (with 18 available cores).
2) When running specific functional tests, framework unit tests are now skipped by default. Framework unit tests can be added by including `feature_framework_unit_tests.py` in the list of specific tests being executed. The rationale for skipping by default is that if the tester is running specific functional tests, there is a conscious decision to focus testing, and choosing to run all tests (where unit tests are run by default) would be a next step.
3) The `--skipunit` option is now removed since unit tests are parallelized (they no longer delay other tests). Unit tests are treated equally as functional tests.
### Implementation notes:
Since `TextTestRunner` can be noisy (even with verbosity=0, and therefore trigger job failure through the presence of non-failure stderr output), the approach taken was to send output to stdout, and forward test result (as determined by `TestResult` returned). This aligns with the previous check for unit test failure (`if not result.wasSuccessful():`).
This approach was tested by inserting `self.assertEquals(True, False)` into test_framework/address.py and seeing specifics of the failure reported.
```
135/302 - feature_framework_unit_tests.py failed, Duration: 0 s
stdout:
.F
======================================================================
FAIL: test_bech32_decode (test_framework.address.TestFrameworkScript.test_bech32_decode)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/dev/myrepos/bitcoin/test/functional/test_framework/address.py", line 228, in test_bech32_decode
self.assertEqual(True, False)
AssertionError: True != False
----------------------------------------------------------------------
Ran 2 tests in 0.003s
FAILED (failures=1)
stderr:
```
There was an initial thought to parallelize the execution of the unit tests themselves (i.e. run the 12 unit test files in parallel), however, this is not anticipated to further reduce runtime meaningfully and is anticipated to add unnecessary complexity.
ACKs for top commit:
maflcko:
ACK f19f0a2e5a🌽
achow101:
ACK f19f0a2e5a
kevkevinpal:
Approach ACK f19f0a2e5a
Tree-SHA512: ab9f82c30371b2242bc7a263ea0e25d35e68e2ddf223d2a55498ad940d1e5b73bba76cce8b264d71e2ed31b753430d8ef8d57efc1e4fd9ced7fb845e27f4f47e
This makes it similar to -checkaddrman and -checkmempool, which
also allow to run the check occasionally instead of always / never.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
by not saving all indexes in a std::multimap, but only
those that are not part of the best header chain.
The indexes of the best header chain are stored in a vector,
which, in the typical case of a mostly linear chain with
a few forks, results in a much smaller multimap, and increases
performance noticeably for long chains.
This does not change the actual consistency checks that are being
performed for each index, just the way the block index tree is
stored and traversed.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Peer protection is only given to outbound-full-relay peers. Add a negative
test to check that other type of outbound peers are not given protection under
the circumstances that outbound-full-relay would
It's very hard to randomly construct a transaction that would be the
parent of an existing orphanage tx. For functions like
AddChildrenToWorkSet and GetChildren that take orphan parents, use a tx
that was previously constructed.
Copying util::Result values is less efficient than moving them because they
allocate memory and contain strings. Also this is needed to avoid compile
errors in https://github.com/bitcoin/bitcoin/pull/25722 which adds a
std::unique_ptr member to util::Result which implicity disables copying.
fd81a37239 net: attempts to connect to all resolved addresses when connecting to a node (Sergi Delgado Segura)
Pull request description:
This is a follow-up of #28155 motivated by https://github.com/bitcoin/bitcoin/pull/28155#discussion_r1362677038
## Rationale
Prior to this, when establishing a network connection via `CConnman::ConnectNode`, if the connection needed address resolution, a single address would be picked at random from the resolved addresses and our node would try to connect to it. However, this would lead to the behavior of `ConnectNode` being unpredictable when the address was resolved to various ips (e.g. the address resolving to IPv4 and IPv6, but we only support one of them).
This patches the aforementioned behavior by going over all resolved IPs until a valid one is found or until we
exhaust them.
ACKs for top commit:
mzumsande:
re-ACK fd81a37239 (just looked at diff, only small logging change)
achow101:
ACK fd81a37239
vasild:
ACK fd81a37239
Tree-SHA512: fa1ebc5c84fe61dd0a7fe1113ae2d594a75ad661c43ed8984a31fc9bc50f166b2759b0d8d84ee5dc247691eff78c8156fac970af797bbcbf67492eec0353fb58
`util::Result` objects are aggregates that can hold multiple fields with
different information. Currently Result objects can only hold a success value
of an arbitrary type or a single bilingual_str error message. In followup PR
https://github.com/bitcoin/bitcoin/pull/25722, Result objects may be able to
hold both success and failure values of different types, plus error and warning
messages.
Having a Result::operator= assignment operator that completely erases all
existing Result information before assigning new information is potentially
dangerous in this case. For example, code that looks like it is assigning a
warning value could erase previously-assigned success or failure values.
Conversely, code that looks like it is just assigning a success or failure
value could erase previously assigned error and warning messages.
To prevent potential bugs like this, disable Result::operator= assignment
operator.
It is possible in the future we may want to re-enable operator= in limited
cases (such as when implicit conversions are not used) or add a Replace() or
Reset() method that mimicks default operator= behavior. Followup PR
https://github.com/bitcoin/bitcoin/pull/25722 also adds a Result::Update()
method providing another way to update an existing Result object.
Co-authored-by: stickies-v <stickies-v@protonmail.com>
65951e0418 index: race fix, lock cs_main while 'm_synced' is subject to change (Ryan Ofsky)
Pull request description:
Fixes#29831 and #29863. Thanks to Marko for the detailed description of the issue.
The race occurs because a block could be connected and its event signaled in-between reading the 'next block' and setting `m_synced` during the index initial synchronization. This is because `cs_main` is not locked through the process of determining the final index sync state.
To address the issue, the `m_synced` flag set has been moved under `cs_main` guard.
ACKs for top commit:
fjahr:
Code review ACK 65951e0418
achow101:
ACK 65951e0418
ryanofsky:
Code review ACK 65951e0418
Tree-SHA512: 77286e22de164a27939d2681b7baa6552eb75e99c541d3b9631f4340d7dd01742667c86899b6987fd2d97799d959e0a913a7749b2b69d9e50505128cd3ae0e69
9adf949d2a contrib: rpcauth.py - Add new option (-j/--json) to output text in json format (bstin)
Pull request description:
This is a simple change to rpcauth.py utility in order to output as json instead raw text.
This is beneficial because integrating json output is simpler with multiple different forms of automation and tooling
ACKs for top commit:
maflcko:
ACK 9adf949d2a
achow101:
ACK 9adf949d2a
willcl-ark:
tACK 9adf949d2a
tdb3:
ACK for 9adf949d2a
Tree-SHA512: 2cdc3b2071fbe4fb32a84ce42ee8ad216cff96ed82aaef58daeb3991953ac137ae42d6898a7fdb6cbd1800e1f61ff8d292f0b150eaebdd2a3fd9d37ed7450787
3e9c736a26 test: fix accurate multisig sigop count (BIP16), add unit test (Sebastian Falbesoner)
Pull request description:
In the course of reviewing #29589 I noticed the following buggy call-site of `CScriptOp.decode_op_n` in the CScript's `GetSigOpCount` method:
4cc99df44a/test/functional/test_framework/script.py (L591-L593)
This should be `lastOpcode` rather than `opcode`. The latter is either OP_CHECKMULTISIG or OP_CHECKMULTISIGVERIFY at this point, so `decode_op_n` would result in an error. Also, in `CScript.raw_iter`, we have to return the op as `CScriptOp` type instead of a bare integer, otherwise we can't call the decode method on it. To prevent this in the future, add some simple unit tests for `GetSigOpCount`.
Note that this was unnoticed, as the code part was never hit so far in the test framework.
ACKs for top commit:
achow101:
ACK 3e9c736a26
Christewart:
ACK 3e9c736a26
rkrux:
tACK [3e9c736](3e9c736a26)
hernanmarino:
tACK 3e9c736a26
Tree-SHA512: 51647bb6d462fbd101effd851afdbd6ad198c0567888cd4fdcac389a9fb4bd3d7e648095c6944fd8875d36272107ebaabdc62d0e2423289055588c12294d05a7
c4f857cc30 test: Extends wait_for_getheaders so a specific block hash can be checked (Sergi Delgado Segura)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/18614
Previously, `wait_for_getheaders` would check whether a node had received **any** getheaders message. This implied that, if a test needed to check for a specific block hash within a headers message, it had to make sure that it was checking the desired message. This normally involved having to manually clear `last_message`. This method, apart from being too verbose, was error-prone, given an undesired `getheaders` would make tests pass.
This adds the ability to check for a specific block_hash within the last `getheaders` message.
ACKs for top commit:
achow101:
ACK c4f857cc30
BrandonOdiwuor:
crACK c4f857cc30
cbergqvist:
ACK c4f857cc30
stratospher:
tested ACK c4f857c. went through all getheaders messages sent in the tests and checked that it's the one we want.
Tree-SHA512: afc9a31673344dfaaefcf692ec2ab65958c3d4c005f5f3af525e9960f0622d8246d5311e59aba06cfd5c9e0ef9eb90a7fc8e210f030bfbe67b897c061efdeed1
992c714451 common: Don't terminate on null character in UrlDecode (Fabian Jahr)
099fa57151 scripted-diff: Modernize name of urlDecode function and param (Fabian Jahr)
8f39aaae41 refactor: Remove hooking code for urlDecode (Fabian Jahr)
650d43ec15 refactor: Replace libevent use in urlDecode with our own code (Fabian Jahr)
46bc6c2aaa test: Add unit tests for urlDecode (Fabian Jahr)
Pull request description:
Fixes#29654 (as a side-effect)
Removing dependencies is a general goal of the project and the xz backdoor has been an additional wake up call recently. Libevent shows many of the same symptoms, few maintainers and slow releases. While libevent can not be removed completely over night we should start removing it’s usage where it's possible, ideally with the end goal to removing it completely.
This is a pretty easy win in that direction. The [`evhttp_uridecode` function from libevent](e0a4574ba2/http.c (L3542)) we were using in `urlDecode` could be easily emulated in fewer LOC. This also ports the [applicable test vectors over from libevent](https://github.com/libevent/libevent/blob/master/test/regress_http.c#L3430).
ACKs for top commit:
achow101:
ACK 992c714451
theStack:
Code-review ACK 992c714451
maflcko:
ACK 992c714451👈
stickies-v:
ACK 992c714451
Tree-SHA512: 78f76ae7ab3b6710eab2aaac20f55eb0da7803e057eaa6220e865f328666a5399ef1a479702aaf630b2f974ad3aa15e2b6adac9c11bc8c3d4be21e8af1667fea
3bf4f8db66 lint: scripted-diff verification also requires GNU grep (Sjors Provoost)
Pull request description:
I noticed while trying to verify all historical `scripted-diff:` commits on macOS that some scripts require GNU sed.
For example 0d6d2b650d uses `git grep --perl-regexp`.
ACKs for top commit:
hernanmarino:
cr ACK 3bf4f8db66
maflcko:
utACK 3bf4f8db66
achow101:
ACK 3bf4f8db66
alfonsoromanz:
Tested ACK 3bf4f8db66
kristapsk:
cr utACK 3bf4f8db66
Tree-SHA512: 09a060ab1bafad03df60d0f20c3dd1451850868dbd66ea38b18178b6230c1f06cf48622db82d9c51422d5689962ee0cd7aae0a31f84bd6d878215e6d73c1d47e
9381052194 doc: Bash is needed in gen_id and is not installed on FreeBSD by default (Hennadii Stepanov)
Pull request description:
On FreeBSD 14.0, in the `depends` directory:
- without `bash`:
```
$ gmake print-bdb_build_id_long
env: bash: No such file or directory
env: bash: No such file or directory
bdb_build_id_long=bdb-4.8.30-4b0c6f8e95251b9c6731844fc34111c04b75fd9f15c671d6e34f2a4d014ec1be-release
$ gmake print-final_build_id
env: bash: No such file or directory
env: bash: No such file or directory
final_build_id=722b2d3e264
```
- with `bash`:
```
$ gmake print-bdb_build_id_long
bdb_build_id_long=bdb-4.8.30-4b0c6f8e95251b9c6731844fc34111c04b75fd9f15c671d6e34f2a4d014ec1be-release 1ed47cefe468014c79dedb275cf921f44ab28d91dd56bf94712409b81326d765
$ gmake print-final_build_id
final_build_id=7b4f9aaa683
```
ACKs for top commit:
vasild:
ACK 9381052194
kristapsk:
ACK 9381052194
alfonsoromanz:
ACK 9381052194
Tree-SHA512: da3f3469ac416518180194f09fb054fb352a2793848fb9a7982439de08244ff6149a7f449ad21fcdf0e9bd79b6949a91751f9cc35833953d2b6a35cea5c6ae21
1a9aa8d4ee build: better scope usage of -Wl,-headerpad_max_install_names (fanquake)
3bee51427a build: don't use install_name_tool for macOS deploy when cross-compiling (fanquake)
78b6b5c485 build: don't pass strip to macOS deploy if cross-compiling (fanquake)
Pull request description:
Neither of these tools are actually used when we are cross-compiling for macOS. They are used when we have to adjust non-static libs during a deploy after building on a macOS machine. Simplies #29739 (will be rebased on top).
Guix (aarch64):
```bash
8f29bce75d7f574306a0e38d793e0e4e145b547a4b9e5a755a54976121d8ac41 guix-build-5afd3c302051/output/arm64-apple-darwin/SHA256SUMS.part
9ba01fe46be715adcbe80f39dc7dbe449f32ca9d9b660da698f933aef3e6d80b guix-build-5afd3c302051/output/arm64-apple-darwin/bitcoin-5afd3c302051-arm64-apple-darwin-unsigned.tar.gz
37719437e951449341d0e10dcc4afe93e955d59de5312ce6351e1fa01b4927ac guix-build-5afd3c302051/output/arm64-apple-darwin/bitcoin-5afd3c302051-arm64-apple-darwin-unsigned.zip
06a79fc871dcd4290f5f7e7e9de19a5a535203d20279f4555d1c319d07abe2d0 guix-build-5afd3c302051/output/arm64-apple-darwin/bitcoin-5afd3c302051-arm64-apple-darwin.tar.gz
98d2b8b37197dcad36a04eb2f3ff2130b859220a17b83a4186a78dcf0af4eafd guix-build-5afd3c302051/output/dist-archive/bitcoin-5afd3c302051.tar.gz
df63ff44ef41565ff13ce6dde5485173a18d5866ebc316df86f9ebd91fda18f5 guix-build-5afd3c302051/output/x86_64-apple-darwin/SHA256SUMS.part
28362ce9e80d5e78db198efa5f89434fbe76ca91df5fde7455da4d50ceb8523a guix-build-5afd3c302051/output/x86_64-apple-darwin/bitcoin-5afd3c302051-x86_64-apple-darwin-unsigned.tar.gz
534745b679eb9e8e408dd251a6bf0829e62e12f7a41772b8a57a044ded14208c guix-build-5afd3c302051/output/x86_64-apple-darwin/bitcoin-5afd3c302051-x86_64-apple-darwin-unsigned.zip
f53d0c9a1bb83d548c7d274c7d39653a3989fb1b4efec49e73dd1cac7c92074c guix-build-5afd3c302051/output/x86_64-apple-darwin/bitcoin-5afd3c302051-x86_64-apple-darwin.tar.gz
```
ACKs for top commit:
TheCharlatan:
ACK 1a9aa8d4ee
Tree-SHA512: 0aa77ea4d6dc45c226806bb1758b6aa7e8ca17f91045bab4fc6891af7b9de476211cd5692c11cb9d5bcf59744fd86a2534812a77fe304ae10c3518e08fc412be
3c1ae3ee33 depends: switch libnatpmp to CMake (Cory Fields)
72ba7b5d26 depends: libnatpmp f2433bec24ca3d3f22a8a7840728a3ac177f94ba (fanquake)
Pull request description:
This picks up one of the changes from https://github.com/bitcoin/bitcoin/pull/29232, which is a switch to building libnatpmp with CMake. It includes an update to the most recent version of libnatpmp (f2433bec24), which includes (https://github.com/miniupnp/libnatpmp/pull/43).
From an initial look I couldn't find any significant difference between the Autotools and CMake produced libs.
ACKs for top commit:
m3dwards:
ACK 3c1ae3ee33
hebasto:
ACK 3c1ae3ee33.
TheCharlatan:
ACK 3c1ae3ee33
Tree-SHA512: 1dd9d9933a5fceb9f8c4e1d68cd5cb4456a10a6dd27a6f6316f14493f9d2efad981ef8be9570c09ca82d45163aebd7f4cb2b2449989ec6084268ddba9a564c83
The previous behavior was the result of casting the result returned from the libevent function evhttp_uridecode to std:string but this was probably not intended.
The point of this was to be able to build bitcoin-tx and bitcoin-wallet without libevent, see #18504.
Now that we use our own implementation of urlDecode this is not needed anymore.
Co-authored-by: stickies-v <stickies-v@protonmail.com>
b22901dfa9 Avoid explicitly computing diagram; compare based on chunks (Pieter Wuille)
Pull request description:
This merges the `BuildDiagramFromChunks` and `CompareFeeRateDiagram` introduced in #29242 into a single `CompareChunks` function, which operates on sorted chunk data rather than diagrams, instead computing the diagram on the fly.
This avoids the need for the construction of an intermediary diagram object, and removes the slightly arbitrary "all diagrams must start at (0, 0)" requirement.
Not a big deal, but I think the result is a bit cleaner and not really more complicated.
ACKs for top commit:
glozow:
reACK b22901d
instagibbs:
reACK b22901dfa9
Tree-SHA512: ca37bdf61d9a9cb5435f4da73e97ead33bf65828ad9af49b87336b1ece70db8ced1c21f517fc6eb6d616311c91f3da75ecae6b9bd42547133e3a3c5320b7816d
08f756bd37 Replace locale-dependent `std::strerror` with `SysErrorString` (Hennadii Stepanov)
d8e4ba4d05 refactor: Rename `subprocess.hpp` to follow our header name conventions (Hennadii Stepanov)
Pull request description:
This PR renames the header `*.hpp` --> `*.h` and adjusts the header guard name, which makes it available for processing by linters.
Fixed the following linter warning:
```
The locale dependent function strerror(...) appears to be used:
src/util/subprocess.h: std::runtime_error( err_msg + ": " + std::strerror(err_code) )
Unnecessary locale dependence can cause bugs that are very tricky to isolate and fix. Please avoid using locale-dependent functions if possible.
Advice not applicable in this specific case? Add an exception by updating the ignore list in /bitcoin/test/lint/lint-locale-dependence.py
^---- failure generated from lint-locale-dependence.py
```
ACKs for top commit:
TheCharlatan:
ACK 08f756bd37
Tree-SHA512: 57a2f01c20eb9552481e428a4969bd59e9ada9f784fe1a45cb62aa9c9152c8e950d336854f45af0e2e5dc7c7b2a1fb216c8f832e3d6ccfb457ad71b6e423231e
08ff17d142 ci: disable _FORTIFY_SOURCE with MSAN (fanquake)
Pull request description:
By undefining `_FORTIFY_SOURCE` we can drop`--disable-hardening`.
ACKs for top commit:
maflcko:
lgtm ACK 08ff17d142
hernanmarino:
utACK 08ff17d142 . Relevant CI test seems to be working OK.
Tree-SHA512: 948fd075aa648a7e34c37376fb913074ebc07d1c3cb0737d5fcbe7eac0b35c4152139773e4515ccb80f2d11b1ced6c6984da1757c2bcf8dd90e8ff6f664dae8e
4d8d21320e sign: don't assume we are parsing a sane Miniscript (Antoine Poinsot)
Pull request description:
The script provided for signature might be externally provided, for instance by way of 'finalizepsbt'. Therefore the script might be ill-crafted, so don't assume pubkeys are always 32 bytes.
Thanks to Niklas for finding this.
FIxes https://github.com/bitcoin/bitcoin/issues/29851.
ACKs for top commit:
achow101:
ACK 4d8d21320e
furszy:
ACK 4d8d21320e with a small nuance that could be tackled in a follow-up by someone else (or never).
Tree-SHA512: 29b7948b56e6dc05eac1014d684f2129ab1d19cb1e5d304216c826b7057c0e1d84ceb18731b91124b680e17d90e38de9f9a5526e4f6ecc3ea816881a6599bb47
970cbc3172 doc: Suggest only necessary Qt packages for installation on OpenBSD (Hennadii Stepanov)
Pull request description:
The currently suggested `qt5` installs many unneeded dependencies, for example, `qtsensors`, `qtspeech` etc.
Similar to https://github.com/bitcoin/bitcoin/pull/29932.
ACKs for top commit:
theStack:
re-ACK 970cbc3172
Tree-SHA512: c680a4f88c27cae145822f30d9856fe76b856c6b11babe652c7171f4322dbe7af8725cccf8b45295067690ed98aa3f6329a8b65a59f31348854638ac375372d1
fa6ab0d020 rpc: Reword SighashFromStr error message (MarcoFalke)
Pull request description:
Put quotes around the parameter. In theory, `std::quoted` should be used, but that seems overkill.
This should avoid error messages such as `A valid sighash parameter is not a valid sighash parameter. (code -8)`.
Also, it should fix fuzz false positives when searching for internal bugs in the `rpc` fuzz target. For example, `ZGVzY3JpcHRvcnByb2Nlc3Nwc2J0XP9ce1tdXOVJbnRlcm5hbCBidWcgZGV0ZWN0ZWQAXQ0AHfcAXQ1p7TJv`.
ACKs for top commit:
dergoegge:
ACK fa6ab0d020
brunoerg:
utACK fa6ab0d020
Tree-SHA512: e2c0cc0126de61873a863af38b7b0a23d2dadd596ca0418dae2ad091e8acfb6a9d657c376d59187bb008989dc78c6b44fe518590e5217e4049a867b220c9fb18
fb4cc5f423 netbase: clean up Proxy logging (Matthew Zipkin)
Pull request description:
Follow up to #27375 and see https://github.com/bitcoin/bitcoin/pull/29649#issuecomment-2057456834
This removes an extra log message when we can't connect to our own proxy, and another when the proxy is invalid.
## Before #27375 if proxy is unreachable
```
2024-04-15T17:54:51Z connect() to 127.0.0.1:9999 failed after wait: Connection refused (61)
2024-04-15T17:54:52Z connect() to 127.0.0.1:9999 failed after wait: Connection refused (61)
2024-04-15T17:54:52Z connect() to 127.0.0.1:9999 failed after wait: Connection refused (61)
2024-04-15T17:54:53Z connect() to 127.0.0.1:9999 failed after wait: Connection refused (61)
2024-04-15T17:54:53Z connect() to 127.0.0.1:9999 failed after wait: Connection refused (61)
```
## After #27375 if unix proxy is unreachable:
```
2024-04-15T17:54:03Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
2024-04-15T17:54:03Z Cannot connect to socket for /Users/matthewzipkin/Desktop/tor
2024-04-15T17:54:04Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
2024-04-15T17:54:04Z Cannot connect to socket for /Users/matthewzipkin/Desktop/tor
2024-04-15T17:54:04Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
2024-04-15T17:54:04Z Cannot connect to socket for /Users/matthewzipkin/Desktop/tor
2024-04-15T17:54:05Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
2024-04-15T17:54:05Z Cannot connect to socket for /Users/matthewzipkin/Desktop/tor
```
## After this PR:
```
2024-04-15T18:18:51Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
2024-04-15T18:18:51Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
2024-04-15T18:18:52Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
2024-04-15T18:18:52Z connect() to /Users/matthewzipkin/Desktop/tor failed: No such file or directory (2)
```
ACKs for top commit:
tdb3:
CR ACK for fb4cc5f423
laanwj:
ACK fb4cc5f423
Tree-SHA512: f07b9f7f2ea9f4bc01780c09f0b076547108294a1fa7d158a0dd48d6d7351569e461e5cccf232b7b1413ce2e3679668e523e5a7c89cd58c909da76d3dcbc34de
6f5954acac ci: Drop no longer needed `-I` flag in "tidy" task (Hennadii Stepanov)
Pull request description:
As title says.
ACKs for top commit:
maflcko:
lgtm ACK 6f5954acac
Tree-SHA512: 4d1cf6eb93bb4b9c51637d182e1fa585263fd998077ec66f34909d69cea070f78fe706a01cca1704dbd1c80465270f874d68972e470506b6b2032dc8bfc20ccc
Reorganize functional test framework unit tests to run in parallel
with other functional tests.
The option `skipunit` is removed, since unit tests no longer delay
functional test execution.
Unit tests are run by default when running all tests, and can be
run explicitly with `feature_framework_unit_tests.py` when running
a subset of tests.
4357158c47 wallet: return and display signer error (Sjors Provoost)
dc55531087 wallet: compare address returned by displayaddress (Sjors Provoost)
6c1a2cc09a test: use h marker for external signer mock (Sjors Provoost)
Pull request description:
* HWI returns the requested address: as a sanity check, we now compare that to what we expected
* external signer documentation now reflects that HWI alternatives must implement this check
* both RPC and GUI will now return an error text, rather than just fail (the GUI even failed silently in some cases)
ACKs for top commit:
brunoerg:
ACK 4357158c47
achow101:
ACK 4357158c47
Tree-SHA512: 4f56edf3846745c8e7d08ef55cf29e8bb468256457149377c5f02da097931f9ca0c06bdbd856dc2385cde4fd11e4dc3b634c5a48814ff27f5562c8a25d43da93
e30e8625bb test: remove duplicated ban test (brunoerg)
Pull request description:
Test the ban list is preserved through restart has been done by both `rpc_setban` and `p2p_disconnect_ban`. Since `p2p_disconnect_ban` does it in a more elegant way, we can keep only it and remove the other one.
bf1b6383db/test/functional/p2p_disconnect_ban.py (L74-L110)
ACKs for top commit:
achow101:
ACK e30e8625bb
tdb3:
ACK for e30e8625bb.
hernanmarino:
tested ACK e30e8625bb
BrandonOdiwuor:
ACK e30e8625bb
alfonsoromanz:
ACK e30e8625bb
Tree-SHA512: e89624f23011e6ffd76c31b2933b8386711e1d2c03366d6b3ea850484a4fd571f69971cdbc75ce2f546d541cb3fc7f4d495a5a011217d879746414e3286ac111
55b13ecd2e doc: explain what the wallet password does (Brandon Odiwuor)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/18085
Add a document explaining what the wallet `passphrase` does
ACKs for top commit:
fjahr:
ACK 55b13ecd2e
achow101:
ACK 55b13ecd2e
Tree-SHA512: 6cc621269fe51ba7bd5d859427906a520e989bebcb274b08017c9900a4d781a165fc62dea46ed45aa48239cacd3c1f6eb4dd0c916be940d680a03266109716b0
13adbf733f remove unneeded environment option from cpp-subprocess (Sebastian Falbesoner)
2088777ba0 remove unneeded cwd option from cpp-subprocess (Sebastian Falbesoner)
03ffb09c31 remove unneeded bufsize option from cpp-subprocess (Sebastian Falbesoner)
79c3036373 remove unneeded close_fds option from cpp-subprocess (Sebastian Falbesoner)
62db8f8e5a remove unneeded session_leader option from cpp-subprocess (Sebastian Falbesoner)
80d008c66d remove unneeded defer_spawn option from cpp-subprocess (Sebastian Falbesoner)
cececad7b2 remove unneeded preexec function option from cpp-subprocess (Sebastian Falbesoner)
633e45b2e2 remove unneeded shell option from cpp-subprocess (Sebastian Falbesoner)
Pull request description:
The newly introduced cpp-subprocess library provides a good number of options for the `Popen` class:
0de63b8b46/src/util/subprocess.hpp (L1009-L1020)
Some of them are either not fully implemented (`shell`, missing an implementation on Windows), implemented in an ugly way (e.g. using "Impoverished, meager, needy, truly needy version of type erasure" for `preexec_func` according to the author's own words) or simply unlikely to be ever needed for our external signer use-case (`defer_spawn`). Instead of maintaining incomplete and/or unneeded code, I'd suggest to get rid of it and only keep support for options if there is a strong reason for it.
ACKs for top commit:
achow101:
ACK 13adbf733f
hebasto:
re-ACK 13adbf733f.
Tree-SHA512: 8270da27891cb659da2ef6062a23f4b86331859b15ac27b79ae7433b14f5bd7efaba621f2b3ba1953708d0f38377a8bd23ef1cc0f28b9c152ac8958dd9eec6b0
6d91cb781c test: add unit tests for `calculate_input_weight` (Sebastian Falbesoner)
f81fad5e0f test: introduce and use `calculate_input_weight` helper (Sebastian Falbesoner)
Pull request description:
Rather than manually estimating an input's weight by adding up all the involved components (fixed-size skeleton, compact-serialized lengths, and the actual scriptSig / witness stack items) we can simply take use of the serialization classes `CTxIn` / `CTxInWitness` instead, to achieve the same with significantly less code.
The new helper is used in the functional tests rpc_psbt.py and wallet_send.py, where the previous manual estimation code was
duplicated. Unit tests are added in the second commit.
ACKs for top commit:
kevkevinpal:
tACK [6d91cb7](6d91cb781c)
QureshiFaisal:
tACK [6d91cb7](6d91cb781c)
achow101:
ACK 6d91cb781c
AngusP:
tACK 6d91cb781c
rkrux:
tACK [6d91cb7](6d91cb781c)
Tree-SHA512: 04424e4d94d0e13745a9c11df2dd3697c98552bbb0e792c4af67ecbb66060adc3cc0cefc202cdee2d9db0baf85b8bedf2eb339ac4b316d986b5f10f6b70c5a33
You can use this tool to decode the utxo snapshot https://github.com/jrakibi/utxo-live
Here’s an overview of how it’s done:
The serialization forma for a UTXO in the snapshot is as follows:
1. Transaction ID (txid) - 32 bytes
2. Output Index (outnum)- 4 bytes
3. VARINT (code) - A varible-length integer encoding the height and whether the transaction is a coinbase. The format of this VARINT is (height << 1) | coinbase_flag.
4. VARINT (amount_v) - A variable-length integer that represents a compressed format of the output amount (in satoshis).
For the test cases mentioned:
* b"\x84\x58" - This value corresponds to a VARINT representing the height and coinbase flag. Once we decode this code, we can extract the height and coinbase using height = code_decoded >> 1 and coinbase = code_decoded & 0x01. In our case, with code_decoded = 728, it results in height = 364 and coinbase = 0.
* b"\xCA\xD2\x8F\x5A" - This byte sequence represents a compressed amount value. The decompression function takes this value and translates it into a full amount in satoshis. In our case, the decompression of this amount translates to a number larger than the maximum allowed value of coins (21 million BTC)
test:Validate UTXO snapshot with coin_height > base_height & amount > money_supply
test:Validate UTXO snapshot with coin_height > base_height & amount > money_supply
fa6c300a99 test: Fix intermittent timeout in p2p_tx_download.py (MarcoFalke)
Pull request description:
Currently the test passes, but may fail during shutdown, because blocks and transactions are synced with `NUM_INBOUND` * `self.num_nodes` peers, which may take a long time.
There is no need for this test to have this amount of inbounds.
So avoid the extraneous inbounds to speed up the test and avoid the intermittent test failures.
ACKs for top commit:
instagibbs:
ACK fa6c300a99
fjahr:
Thanks, ACK fa6c300a99
achow101:
ACK fa6c300a99
theStack:
ACK fa6c300a99
Tree-SHA512: 0a480fd1db293ed8571ae629557cf81d5a79ec883e9e635f22c8a7cf48427161249ad2180b66c67661306f696c977b8e06ad520bd11911f119c9c95b3ffc9134
c3e632b441 Bugfix: bitcoin-cli: Check length of peer.transport_protocol_type (Luke Dashjr)
Pull request description:
"v" would dereference beyond the string length, and "v10" would show as '1'
Turn both of these cases into a blank, like anything else unexpected currently is.
ACKs for top commit:
sipa:
utACK c3e632b441.
hernanmarino:
utACK c3e632b441
alfonsoromanz:
ACK c3e632b441
achow101:
ACK c3e632b441
Tree-SHA512: f641e4412521adae7c8c8e1f268bdaaa223d9048d8286e3df4b13905faaa0d601155ce581cd649f760cab2acc4122356fa94a44714f1f190845552100105eda0
The script provided for signature might be externally provided, for
instance by way of 'finalizepsbt'. Therefore the script might be
ill-crafted, so don't assume pubkeys are always 32 bytes.
Thanks to Niklas for finding this.
f2e3662e57 net: Decrease nMaxIPs when learning from DNS seeds (laanwj)
Pull request description:
Limit number of IPs learned from a single DNS seed to 32, to prevent the results from one DNS seed from dominating AddrMan. Note that the number of results from a UDP DNS query is bounded to 33 already, but it is possible for it to use TCP where a larger number of results can be returned.
Closes#16070.
ACKs for top commit:
Sjors:
utACK f2e3662e57
achow101:
ACK f2e3662e57
1440000bytes:
utACK f2e3662e57
mzumsande:
utACK f2e3662e57
Tree-SHA512: 3f108c2baba7adfedb8019daaf60aa00e628b38d3942e1319c7183a4683670be01929ced9e6372c8e983c902e8633f81fbef12d7cdcaadd7f77ed729c1019942
6b02c11d66 test: Fix intermittent issue in p2p_handshake.py (stratospher)
Pull request description:
When establishing outbound connections [`TestNode` --------> `P2PConnection`], `P2PConnection` listens for a single connection from `TestNode` on a [port which is fixed based on `p2p_idx`](312f54278f/test/functional/test_framework/p2p.py (L746)).
If we reuse the same port when disconnecting and establishing connections again, we might hit this scenario where:
- disconnection is done on python side for `P2PConnection`
- disconnection not complete on c++ side for `TestNode`
- we're trying to establish a new connection on same port again
Prevent this scenario from happening by ensuring disconnection on c++ side for TestNode as well.
One way to reproduce this on master would be adding a sleep statement before disconnection happens on c++ side.
```diff
diff --git a/src/net.cpp b/src/net.cpp
index e388f05b03..62507d1f39 100644
--- a/src/net.cpp
+++ b/src/net.cpp
@@ -2112,6 +2112,7 @@ void CConnman::SocketHandlerConnected(const std::vector<CNode*>& nodes,
if (!pnode->fDisconnect) {
LogPrint(BCLog::NET, "socket closed for peer=%d\n", pnode->GetId());
}
+ std::this_thread::sleep_for(std::chrono::milliseconds(1000));
pnode->CloseSocketDisconnect();
}
else if (nBytes < 0)
```
ACKs for top commit:
maflcko:
lgtm ACK 6b02c11d66
mzumsande:
Tested ACK 6b02c11d66
BrandonOdiwuor:
Tested ACK 6b02c11d66
theStack:
Tested ACK 6b02c11d66
glozow:
ACK 6b02c11d66
Tree-SHA512: 69509edb61ba45739fd585b6cc8a254f412975c124a5b5a52688288ecaaffd264dd76019b8290cc34c26c3ac2dfe477965ee5a11d7aabdd8e4d2a75229a4a068
21d0e6c7b7 doc: release notes for PR 27679 (Matthew Zipkin)
791dea204e test: cover unix sockets in zmq interface (Matthew Zipkin)
c87b0a0ff4 zmq: accept unix domain socket address for notifier (Matthew Zipkin)
Pull request description:
This is a follow-up to https://github.com/bitcoin/bitcoin/pull/27375, allowing ZMQ notifications to be published to a UNIX domain socket.
Fortunately, libzmq handles unix sockets already, all we really have to do to support it is allow the format in the actual option.
[libzmq](https://libzmq.readthedocs.io/en/latest/zmq_ipc.html) uses the prefix `ipc://` as opposed to `unix:` which is [used by Tor](https://gitlab.torproject.org/tpo/core/tor/-/blob/main/doc/man/tor.1.txt?ref_type=heads#L1475) and now also by [bitcoind](a85e5a7c9a/doc/release-notes-27375.md (L5)) so we need to switch that internally.
As far as I can tell, [LND](d20a764486/zmq.go (L38)) supports `ipc://` and `unix://` (notice the double slashes).
With this patch, LND can connect to bitcoind using unix sockets:
Example:
*bitcoin.conf*:
```
zmqpubrawblock=unix:/tmp/zmqsb
zmqpubrawtx=unix:/tmp/zmqst
```
*lnd.conf*:
```
bitcoind.zmqpubrawblock=ipc:///tmp/zmqsb
bitcoind.zmqpubrawtx=ipc:///tmp/zmqst
```
ACKs for top commit:
laanwj:
Code review ACK 21d0e6c7b7
tdb3:
crACK for 21d0e6c7b7. Changes lgtm. Will follow up with some testing within the next few days as time allows.
achow101:
ACK 21d0e6c7b7
guggero:
Tested and code review ACK 21d0e6c7b7
Tree-SHA512: ffd50222e80dd029d903e5ddde37b83f72dfec1856a3f7ce49da3b54a45de8daaf80eea1629a30f58559f4b8ded0b29809548c0638cd1c2811b2736ad8b73030
016ed248ba fuzz: explicitly cap the vsize of RBFs for diagram checks (Greg Sanders)
Pull request description:
In master we are hitting a case where vsize transactions much larger than max standard size are causing an overflow in not-yet-exposed RBF diagram checking code: https://github.com/bitcoin/bitcoin/pull/29757#issuecomment-2049220195
`ConsumeTxMemPoolEntry` is creating entries with tens of thousands of sigops cost, causing the resulting RBFs to be "overly large".
To fix this I cause the fuzz test to stop adding transactions to the mempool when we reach a potential overflow of `int32_t`.
ACKs for top commit:
glozow:
ACK 016ed248ba
marcofleon:
ACK 016ed248ba. I ran libFuzzer on `package_rbf` on the current master branch until the overflow was encountered. Then I built the PR branch and ran the fuzzer using the crash input.
Tree-SHA512: b3ffc98d2c4598eb3010edd58b9370aab1441aafbb1044c83b2b90c17dfe9135b8de9dba475dd0108863c1ffedede443cd978e95231a41cf1f0715629197fa51
60ca5d5508 test: p2p: add test for rejected tx request logic (`m_recent_rejects` filter) (Sebastian Falbesoner)
e9dc511a7e fixup: get all utxos up front in fill_mempool, discourage wallet mixing (glozow)
Pull request description:
Motivated by the discussion in #28970 (https://github.com/bitcoin/bitcoin/pull/28970#discussion_r1553911167), this PR adds test coverage for the logic around the `m_recent_rejects` filter, in particular that the filter is cleared after a new block comes in:
f0794cbd40/src/net_processing.cpp (L2199-L2206)
As expected, the second part of the test fails if the following patch is applied:
```diff
diff --git a/src/net_processing.cpp b/src/net_processing.cpp
index 6996af38cb..5cb1090e70 100644
--- a/src/net_processing.cpp
+++ b/src/net_processing.cpp
@@ -2202,7 +2202,7 @@ bool PeerManagerImpl::AlreadyHaveTx(const GenTxid& gtxid)
// or a double-spend. Reset the rejects filter and give those
// txs a second chance.
hashRecentRejectsChainTip = m_chainman.ActiveChain().Tip()->GetBlockHash();
- m_recent_rejects.reset();
+ //m_recent_rejects.reset();
}
const uint256& hash = gtxid.GetHash();
```
I'm still not sure in which file this test fits best, and if there is already test coverage for the first part of the test somewhere. Happy for any suggestions.
ACKs for top commit:
maflcko:
ACK 60ca5d5508🍳
glozow:
code review ACK 60ca5d5508
instagibbs:
ACK 60ca5d5508
Tree-SHA512: 9cab43858e8f84db04a708151e6775c9cfc68c20ff53096220eac0b2c406f31aaf9223e8e04be345e95bf0a3f6dd15efac50b0ebeb1582a48a4560b3ab0bcba5
If we reuse the same port when disconnecting and establishing connections
again, we might hit this scenario:
- disconnection is done on python side for P2PConnection
- disconnection is not complete on c++ side for TestNode
- we're trying to establish a new connection on same port again
Prevent this scenario from happening by ensuring disconnection on c++
side for TestNode as well.
- Mention which directories contain the respective unsigned tarballs
- Clarify that bitcoin.conf might not need to be updated
- Specify where to put historical release notes if there is already
something in release-notes.md
- Clarify what exactly is the problem with running guix-codesign more
than once
- Correct number: 6 codesigned attestations are needed before uploading
binaries
- Remove scp command which is outdated
- Remove server path which is outdated
- Specify that translations update should happen before branch-off, not
before each release candidate
- Mention that you should notify lists when RCs are available
- Put "Archive the release notes" as a separate step, since creating the
github release has a dependency on it.
- Put bitcoincore.org website updates as a separate step, since
updating packaging repos may have a dependency on it.
- Update "bitcoin-dev mailing list" to "bitcoin-dev group"
- Document that maintainers should create PRs to collect backports
- Remove section about not uploading `*-debug` files, reader should
upload all build artifacts.
- Torrent is created automatically, so delete instructions.
- Mention that server also generates ots file automatically.
Prior to this, when establishing a network connection via CConnman::ConnectNode,
if the connection needed address resolution, a single address would be picked
at random from the resolved addresses and our node will try to connect to it. However,
this would lead to the behavior of ConnectNode being unpredictable when the address
was resolved to various ips (e.g. the address resolving to IPv4 and IPv6, but we only
support one of them).
This patches the aforementioned behavior by going over all resolved IPs until we find one
we can connect to or until we exhaust them.
On systems where `int8_t` is defined as `char`, the
`{S,Uns}erialize(Stream&, signed char)` functions become undefined.
This change resolves the issue by testing
`{S,Uns}erialize(Stream&, int8_t)` instead.
No behavior change on systems where `int8_t` is defined as
`signed char`, which is the case for most other systems.
c6d1b8de89 gui: change example address from legacy (P2PKH) to bech32m (P2TR) (Sebastian Falbesoner)
Pull request description:
Legacy addresses are less and less common these days and not recommended to use, so it seems senseful to also reflect that in the example addresses and update to the most recent address / output type (bech32m / P2TR). Also, as I couldn't see any value in computing these at runtime, they are pre-generated. This was done with the following Python script, executed in `./test/functional` (it's also included in the commit body, though without the she-bang):
```python
#!/usr/bin/env python3
from test_framework.segwit_addr import CHARSET, decode_segwit_address, encode_segwit_address
from test_framework.messages import sha256
output_key = sha256(b'bitcoin dummy taproot output key')
for network, hrp in [('mainnet', 'bc'), ('signet', 'tb'), ('testnet', 'tb'), ('regtest', 'bcrt')]:
dummy_address = encode_segwit_address(hrp, 1, output_key)
while decode_segwit_address(hrp, dummy_address) != (None, None):
last_char = CHARSET[(CHARSET.index(dummy_address[-1]) + 1) % 32]
dummy_address = dummy_address[:-1] + last_char
print(f'{network:7} example address: {dummy_address}')
```
Note that the last bech32 character is modified in order to make the checksum fail.
master (mainnet):

PR (mainnet):

ACKs for top commit:
maflcko:
lgtm ACK c6d1b8de89
pablomartin4btc:
tACK c6d1b8de89
Tree-SHA512: a53c267a3e0d29b9c41bf043b123e7152fbf297e2322d74ce047ba2582b54768187162d462cc334e91a84874731c2e0793726ad44d9970c10ecfe70a1d4f3f1c
05416422d3 refactor: Avoid implicit-integer-sign-change in processNewTransaction (MarcoFalke)
321f105d08 refactor: Avoid implicit-signed-integer-truncation-or-sign-change in FreedesktopImage (MarcoFalke)
6d8eecd33a refactor: Avoid implicit-integer-sign-change in createTransaction (MarcoFalke)
Pull request description:
This is allowed by the language. However, the `integer` sanitizer complains about it. Thus, fix it, so that the `integer` sanitizer can be used in the future to catch unintended sign changes.
Fixes#805.
ACKs for top commit:
pablomartin4btc:
tACK 05416422d3
hebasto:
ACK 05416422d3, I have reviewed the code and it looks OK.
Tree-SHA512: eaa941479bd7bee196eb8b31d93b8e1db122410cf62e8ec4cbbec35cfd14cc766081c3df5dd14a228e21ad2678d8b8ba0d2649e5934c994a90ae96d8b264b4ce
This ensures that the index does not miss any 'new block' signals
occurring in-between reading the 'next block' and setting 'm_synced'.
Because, if this were to happen, the ignored blocks would never be
indexed, thus stalling the index forever.
0244416aac security: restrict abis in bitcoind.service (Charlie)
Pull request description:
[As noted here](https://www.freedesktop.org/software/systemd/man/systemd.exec.html#MemoryDenyWriteExecute=), it's a good idea to pair `MemoryDenyWriteExecute=true` with `SystemCallArchitectures=native` because `MemoryDenyWriteExecute` can be circumvented in some operating systems which support multiple ABIs like x86/x86-64.
This helps restrict the possible application binary interfaces (ABIs) that can be used when running bitcoind through systemd, reducing the attack surface area.
ACKs for top commit:
laanwj:
ACK 0244416aac . This is a sensible security feature.
0xB10C:
ACK 0244416aac
Tree-SHA512: 77a35b0674d8d67d857cd20ae1b8cd011f82d6f5ed21bc106cbe45bfa937e786ddc1bf7261e3bdb8c289df1224e91658760905d2c8f37cc4c6506ef8037ad158
24b67fa9f6 doc: Add example of mixing private and public keys in descriptors (Anton A)
Pull request description:
closes: #27414
ACKs for top commit:
achow101:
ACK 24b67fa9f6
alfonsoromanz:
Re ACK 24b67fa9f6
Tree-SHA512: 8c063f23199ac0ff35909f786a5b0de1b4a9b15d1e93bdcdac10cb4bd2002c12e99b6fb1c2e56d16971e7622b67d910b79088429df92c48279be2d7797049911
dd3e0fa125 build: Fix false positive `CHECK_ATOMIC` test for clang-15 (Hennadii Stepanov)
Pull request description:
On the master branch @ 0de63b8b46, a building `bitcoind` with clang-15 for `i686-pc-linux-gnu` fails to link:
```
CXXLD bitcoind
/usr/bin/ld: libbitcoin_wallet.a(libbitcoin_wallet_a-wallet.o): in function `std::remove_volatile<double>::type std::__atomic_impl::load<double>(double const*, std::memory_order)':
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:948: undefined reference to `__atomic_load'
/usr/bin/ld: libbitcoin_wallet.a(libbitcoin_wallet_a-wallet.o): in function `void std::__atomic_impl::store<double>(double*, std::remove_volatile<double>::type, std::memory_order)':
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: libbitcoin_wallet.a(libbitcoin_wallet_a-wallet.o): in function `void std::__atomic_impl::store<double>(double*, std::remove_volatile<double>::type, std::memory_order)':
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: libbitcoin_wallet.a(libbitcoin_wallet_a-wallet.o): in function `std::remove_volatile<double>::type std::__atomic_impl::load<double>(double const*, std::memory_order)':
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:948: undefined reference to `__atomic_load'
/usr/bin/ld: libbitcoin_wallet.a(libbitcoin_wallet_a-wallet.o): in function `void std::__atomic_impl::store<double>(double*, std::remove_volatile<double>::type, std::memory_order)':
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: libbitcoin_wallet.a(libbitcoin_wallet_a-backup.o): in function `void std::__atomic_impl::store<double>(double*, std::remove_volatile<double>::type, std::memory_order)':
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: libbitcoin_wallet.a(libbitcoin_wallet_a-backup.o):/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: more undefined references to `__atomic_store' follow
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
due to false positive `CHECK_ATOMIC` test in the `configure` script.
This PR fixes this test.
ACKs for top commit:
maflcko:
review ACK dd3e0fa125
fanquake:
ACK dd3e0fa125
Tree-SHA512: b60acf8d83fc84cc3280d95028395d341ed9ed2fcf38ae0a101d50aa19cc35540e9763aa36668c4dc1e1bc7e1f33dbda0e662df39c9e414a284ef91d7fc55fba
1ae5b208d3 test: fix intermittent failure in p2p_compactblocks_hb.py (Martin Zumsande)
Pull request description:
Fixes#29860
As a result of node1 receiving a block, it sends out SENDCMPCT messages to some of its peers to update the high-bandwidth status. We need to wait until those are received and processed by the peers to avoid intermittent failures. Before, we'd only wait until all peers have synced with the new block (within `generate`) which is not sufficient.
I could reproduce the failure by adding a `std::this_thread::sleep_for(std::chrono::milliseconds(1000));` sleep to the [net_processing code](c7567d9223/src/net_processing.cpp (L3763)) that processes `NetMsgType::SENDCMPCT`.
ACKs for top commit:
instagibbs:
ACK 1ae5b208d3
alfonsoromanz:
Tested ACK 1ae5b208d3
glozow:
ACK 1ae5b208d3
Tree-SHA512: 47c29616e73a5e0ff966fc231e4f672c1a6892511e5c10a3905b30ad6b2a3d1267fa0a88bd8f64b523fe580199d22a43545c84e361879e5096483152065c4b9a
fa4c69669e test: Fix failing univalue float test (MarcoFalke)
Pull request description:
Currently the test may fail for some compilers, because `1e-8` may not be possible to represent exactly/consistently.
```
$ ./src/univalue/test/object
object: univalue/test/object.cpp:424: void univalue_readwrite(): Assertion `v.read("0.00000000000000000000000000000000000001e+30 ") && v.get_real() == 1e-8' failed.
Aborted (core dumped)
```
Fixes https://github.com/bitcoin/bitcoin/pull/27256#discussion_r1567356943
ACKs for top commit:
laanwj:
ACK fa4c69669e
stickies-v:
ACK fa4c69669e , thanks for fixing!
Tree-SHA512: dea4f4f843381d5e8ffaa812b2290a11e081b29f8777d041751c4aa9942e60f1f8d2d1a652d9a52b41dec470a490c9fe26ca9bc762dd593c3521b328a8af2826
05da2460db guix: remove gcc-toolchain static from Windows build (fanquake)
Pull request description:
The libs in this dir are the following:
```bash
ls /gnu/store/2vnbkrdin4rrf7ygnr80mlcglin4qqa4-gcc-toolchain-12.3.0-static/lib/
libanl.a libc.a libdl.a libm.a
libBrokenLocale.a libcrypt.a libg.a libmcheck.a
libpthread.a librt.a
libresolv.a libutil.a
```
These do not need to be propogated into the Windows build environment.
Guix Build (aarch64):
```bash
450c0c4f45f9cb7ed7fc2ef6e7557b6a23004b82c951399da3b7635e8451a076 guix-build-05da2460db89/output/dist-archive/bitcoin-05da2460db89.tar.gz
5df68ab18636090c387bc90297356d0e148b02931d3a99c0f6d33cd268aa072b guix-build-05da2460db89/output/x86_64-w64-mingw32/SHA256SUMS.part
13e979f60d9296aa11081fbbb360404da9fbb797bb4663ed2d1189d800659b4f guix-build-05da2460db89/output/x86_64-w64-mingw32/bitcoin-05da2460db89-win64-debug.zip
d1cc78437a96f012a59af7c757bef592f163559e523d45014d7804d0be29a8b8 guix-build-05da2460db89/output/x86_64-w64-mingw32/bitcoin-05da2460db89-win64-setup-unsigned.exe
33a9cfd4475677646bb32c9c45c25bd796ca5adb126590bf556d4e6f9592c676 guix-build-05da2460db89/output/x86_64-w64-mingw32/bitcoin-05da2460db89-win64-unsigned.tar.gz
5d2ee251668d3d31bf378826ab06f98542dd20926cdee2df5c3315e11222a519 guix-build-05da2460db89/output/x86_64-w64-mingw32/bitcoin-05da2460db89-win64.zip
```
Somewhat similar to #29673.
ACKs for top commit:
laanwj:
ACK 05da2460db
hebasto:
ACK 05da2460db,
Tree-SHA512: bf514a726a22e2bfae4de645b10d90a66fe090971340c4299aeb9b2ff9cf536ca6cfed274d312ea5d5a172775cbda6db0e609492ec603f5aee55c8de81462cc0
b7ba60f81a test: add coverage for -reindex and assumeutxo (Martin Zumsande)
e57f951805 init, validation: Fix -reindex option with an existing snapshot (Martin Zumsande)
Pull request description:
In c711ca186f logic was introduced that `-reindex` and `-reindex-chainstate` will delete the snapshot chainstate.
This doesn't work currently, instead of deleting the snapshot chainstate the node crashes with an assert (this can be triggered by applying the added test commit on master).
Fix this, and another bug that would prevent the new active chainstate from having a mempool after `-reindex` has deleted the snapshot (also covered by the test).
ACKs for top commit:
fjahr:
re-ACK b7ba60f81a
hernanmarino:
crACK b7ba60f81a . Good fix
BrandonOdiwuor:
re-ACK b7ba60f81a
byaye:
Tested ACK b7ba60f81a
Tree-SHA512: c168f36997d7677d590af37b10427870f5d30123abf1c76032a16661e486735373bfa7e049e6aca439526fbcb6d619f970bf9d042196c851bf058a75a32fafdc
As a result of node1 receiving a block, it sends out
SENDCMPCT messages to its peers to update the status.
We need to wait until those are received and
processed by the peers to avoid intermittent failures.
Both RPC and GUI now render a useful error message instead of (silently) failing.
Replace bool with util::Result<void> to clarify that this either succeeds or returns an error message.
It should never be a nullopt when the transaction result is valid -
Assume() this is the case. However, as a belt-and-suspenders just in
case it is nullopt, use an empty list.
c2e0489b71 [rpc, bugfix] Enforce maximum value for setmocktime (dergoegge)
Pull request description:
The maximum value for our mocktime must be representable in nanoseconds, otherwise we end up with negative values returned from `NodeClock::now()`.
Found through fuzzing:
```
$ echo "c2V0bW9ja3RpbWVcZTptYf9w/3NldG3///////////////9p////ZP///ymL//////89////Nv9L////////LXkBAABpAA==" | base64 --decode > rpc-8cab9148ab4418ebd1923c213e9d3fe9c9b49b39.crash
$ FUZZ=rpc ./src/test/fuzz/fuzz rpc-8cab9148ab4418ebd1923c213e9d3fe9c9b49b39.crash
fuzz_libfuzzer: util/time.cpp:28: static NodeClock::time_point NodeClock::now(): Assertion `ret > 0s' failed.
```
ACKs for top commit:
maflcko:
re-ACK c2e0489b71
brunoerg:
crACK c2e0489b71
glozow:
ACK c2e0489b71
Tree-SHA512: d7e237ca37bedd74a6b085fb6e726a142705371044c77488f593f35afe70aeca756fdba86920294b1d322c7a9b2cde9ce4e1b7d410a6ccc1fd7c6f3a6e77200a
92f88a9629 doc: fixup NAT-PMP help doc (fanquake)
02395edca9 init: remove redundant upnp #ifdef (fanquake)
Pull request description:
This is a very belated followup to #26896 (which removed the configure options for setting the upnp and natpmp runtime default) and corrects the `-help` docs for `-upnp` and `-natpmp`.
ACKs for top commit:
davidgumberg:
ACK 92f88a9629
hernanmarino:
ACK 92f88a9629
Tree-SHA512: 795dc8a8703bf322b5831d845de85f2428ee0dd45d3064b48ff47d147147381af26c0a9d00c596db12009b254763844b209989daf4e7470d20e8a1753b640966
fadf7e90dc ci: Bump s390x to ubuntu:24.04 (MarcoFalke)
Pull request description:
Now that most other CI tasks are rolled to 24.04, roll this one as well.
Also, re-enable feature_init.
ACKs for top commit:
fanquake:
ACK fadf7e90dc - finished running after ~5 hours.
Tree-SHA512: d69f40227ba982eaaccc1e144ba05d3c8c4a096ac401077df3db5cf3276d1e146a438e4e0237dda876b3644fc7f0accd724a19e3c1cdb94c30c679899137280e
4722b7c715 build: remove minisketch clz check (fanquake)
1eea10a6d2 Squashed 'src/minisketch/' changes from a571ba20f9..3472e2f5ec (fanquake)
Pull request description:
https://github.com/sipa/minisketch/pull/81 will fix#29799.
Minor build cleanups after https://github.com/sipa/minisketch/pull/80.
ACKs for top commit:
dergoegge:
utACK 4722b7c715
hebasto:
ACK 4722b7c715, I have verified the subtree update and reviewed the build system changes. Both look OK.
Tree-SHA512: eabd82e5a13cc4f32155319df97368f2e8c93320a4265b6c372efcb1ea4e756f6693df7c02498c8ea989ccd376a20277fa110c66d0754cb9bca5e54d18e0a965
3472e2f5ec Merge sipa/minisketch#81: Avoid overflowing shift by special casing inverse of 1
653d8b2e26 Avoid overflowing shift by special casing inverse of 1
33b7c200b9 Merge sipa/minisketch#80: Add c++20 version of CountBits
4a48f31a37 Merge sipa/minisketch#83: ci: Fix "s390x (big-endian)" task
82b6488acb Add c++20 version of CountBits
0498084d31 ci: Fix "s390x (big-endian)" task
71709dca9e Merge sipa/minisketch#82: ci: Fix `x86_64-w64-mingw32` task
9e6127fa98 Merge sipa/minisketch#74: Avoid >> above type width in BitWriter
ed420bc170 ci: Fix `x86_64-w64-mingw32` task
fe1040f227 Drop -Wno-shift-count-overflow compile flag
154bcd43bd Avoid >> above type width in BitWriter
67b87acdb6 Merge sipa/minisketch#78: ci: Update macOS image for CI
7de7250416 ci: Update macOS image for CI
83d812ea9f Merge sipa/minisketch#73: ci: Use correct variable to designate C++ compiler
e051a7d690 ci: Install wine32 package for Windows tests
2d2c695d78 build: Drop unused `CC` variable
1810fcbd11 ci: Use correct variable to designate C++ compiler
022b959049 Merge sipa/minisketch#77: Add missing include
08443c4892 Add missing include
git-subtree-dir: src/minisketch
git-subtree-split: 3472e2f5ec75ace39ce9243af6b3fee233a67492
A common issue that our fuzzers keep finding is that outpoints don't
exist in the non witness utxos. Instead of trying to track this down and
checking in various individual places, do the check early during
deserialization.
4ba1d0b553 fuzz: Add coverage for client_maxfeerate (Greg Sanders)
91d7d8f22a AcceptMultipleTransactions: Fix workspace client_maxfeerate (Greg Sanders)
f3aa5bd5eb fill_mempool: assertions and docsctring update (Greg Sanders)
a3da63e8fe Move fill_mempool to util function (Greg Sanders)
73b68bd8b4 fill_mempool: remove subtest-specific comment (Greg Sanders)
Pull request description:
Bug causes an `Assume()` failure due to the expectation that the individual result should be invalid when done over `submitpackage` via rpc.
Bug introduced by https://github.com/bitcoin/bitcoin/pull/28950 , and I discovered it rebasing https://github.com/bitcoin/bitcoin/pull/28984 since it's easier to hit in that test scenario.
Tests in place were only checking `AcceptSingleTransaction`-level checks due to package evaluation only triggering when minfee is too high for the parent transaction.
Added test along with fix, moving the fill_mempool utility into a common area for re-use.
ACKs for top commit:
glozow:
reACK 4ba1d0b553
theStack:
ACK 4ba1d0b553
ismaelsadeeq:
re-ACK 4ba1d0b553 via [diff](4fe7d150eb..4ba1d0b553)
Tree-SHA512: 3729bdf7f25d04e232f173ccee04ddbb2afdaafa3d04292a01cecf58fb11b3b2bc133e8490277f1a67622b62d17929c242dc980f9bb647896beea4332ee35306
Limit number of IPs learned from a single DNS seed to 32, to prevent the results from
one DNS seed from dominating AddrMan. Note that the number of results from a UDP DNS query is
bounded to 33 already, but it is possible for it to use TCP where a potentially enormous
number of results can be returned.
Closes#16070.
This helper class is an alternative to CMedianFilter, but without a
lot of the special logic and exceptions that we needed while it was
still used for consensus.
95c594f4e9 depends: remove no longer needed patch for Boost::Process (Sebastian Falbesoner)
Pull request description:
As Boost::Process has been replaced by cpp-subprocess (PR #28981), this patch touches an unused code part and is hence not needed anymore.
ACKs for top commit:
hebasto:
ACK 95c594f4e9, I have reviewed the code and it looks OK.
fanquake:
ACK 95c594f4e9
Tree-SHA512: 0309b826f8c260e4180624f17302e51329fc4bd7a5431997d6d27d468dd5f7dbcd9db6a742efaba33ba30dbe361830eb1446fdbec927505ccf42412f9211934e
d5a715536e build: remove boost::process dependency for building external signer support (Sebastian Falbesoner)
70434b1c44 external_signer: replace boost::process with cpp-subprocess (Sebastian Falbesoner)
cc8b9875b1 Add `cpp-subprocess` header-only library (Hennadii Stepanov)
Pull request description:
Closes https://github.com/bitcoin/bitcoin/issues/24907.
This PR is based on **theStack**'s [work](https://github.com/bitcoin/bitcoin/issues/24907#issuecomment-1466087049).
The `subprocess.hpp` header has been sourced from the [upstream repo](https://github.com/arun11299/cpp-subprocess) with the only modification being the removal of convenience functions, which are not utilized in our codebase.
Windows-related changes will be addressed in subsequent follow-ups.
ACKs for top commit:
achow101:
reACK d5a715536e
Sjors:
re-tACK d5a715536e
theStack:
Light re-ACK d5a715536e
fanquake:
ACK d5a715536e - with the expectation that this code is going to be maintained as our own. Next PRs should:
Tree-SHA512: d7fb6fecc3f5792496204190afb7d85b3e207b858fb1a75efe483c05260843b81b27d14b299323bb667c990e87a07197059afea3796cf218ed8b614086bd3611
Add a close (X) button to the Peers Detail panel.
Reuse the same icon used in the Console Tab.
The close button deselects the peer highlighted
in the PeerTableView and hides the detail panel.
This PR addresses issue #485:
Co-authored-by: @w0xlt <w0xlt@users.noreply.github.com>
a71eadf66b Change MAC_OSX macro to __APPLE__ in crypto package (Lőrinc)
Pull request description:
Split out from https://github.com/bitcoin/bitcoin/pull/29450#issuecomment-2044405345 to avoid the uncertainties and simplify review.
ACKs for top commit:
theuni:
ACK a71eadf66b
fanquake:
ACK a71eadf66b
Tree-SHA512: b6a7bd7ca95585dd9110cefe7c1213f4a1a72bdfc88670abf4a0d9a8bbc12e093544524adce46aa9ca714c472d417f74ca4a678af682ed2488053059434eaa02
a8203e9412 refactor: Simplify `extra_txn` to be a vec of CTransactionRef instead of a vec of pair<Wtxid, CTransactionRef> (AngusP)
c3c18433ae refactor: Use typesafe Wtxid in compact block encoding message, instead of ambiguous uint256. (AngusP)
Pull request description:
The first commit replaces `uint256` with typesafe `Wtxid` (or `Txid`) types introduced in #28107.
The second commit then simplifies the extra tx `vector` to just be of `CTransactionRef`s instead of a `std::pair<Wtxid, CTransactionRef>`, as it's easy to get a `Wtxid` from a transaction ref.
ACKs for top commit:
glozow:
ACK a8203e9412
dergoegge:
ACK a8203e9412
Tree-SHA512: b4ba1423a8059f9fc118782bd8a668213d229c822f22b01267de74d6ea97fe4f2aad46b5da7c0178ecc9905293e9a0eafba1d75330297c055e27fd53c8c8ebfd
47cedee776 fuzz: Introduce `BITCOINFUZZ` environment variable (Hennadii Stepanov)
1573e9a11e fuzz, refactor: Deduplicate fuzz binary path creation (Hennadii Stepanov)
Pull request description:
These changes are split from https://github.com/bitcoin/bitcoin/pull/29774 and can be beneficial on their own.
The new `BITCOINFUZZ` environment variable complements the already existing set of variables used by tests: b5d21182e5/test/functional/test_framework/test_framework.py (L238-L243)
ACKs for top commit:
maflcko:
lgtm ACK 47cedee776
davidgumberg:
utACK 47cedee776
Tree-SHA512: 45809cfd13dc4a45c44cc433184352e84726cb95bea80fd8f581c59a0b8b0a5495260ff66922f9c57c38adbdbdd102439238f370fd49d6ea27a241a5e6249895
03b87a3e64 Drop Windows Socket dependency for `randomenv.cpp` (Hennadii Stepanov)
Pull request description:
This change drops a dependency on the ws2_32 library for our libbitcoinkernel by switching to [`GetComputerName`](https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-getcomputernamew) function.
ACKs for top commit:
sipsorcery:
utACK 03b87a3e64.
laanwj:
Code review ACK 03b87a3e64.
fanquake:
ACK 03b87a3e64
Tree-SHA512: a4abd5499176634d5f3fbf4e794a7504c40232fb73bd7f41955fbfb2cc7c44bc7ea4518c5203836e52f552c30414c6c3e1b24f0922641dbf1c8377981c0ffaf0
d4e36ae80d test: Update --tmpdir doc string to say directory must not exist (kevkevin)
Pull request description:
The error message given if passing an existing dir to --tmpdir is confusing so this makes it clear that the directory must not already exist
This change is motivated by this comment https://github.com/bitcoin/bitcoin/pull/29335#issuecomment-1960913020
ACKs for top commit:
maflcko:
lgtm ACK d4e36ae80d
davidgumberg:
ACK d4e36ae80d
Tree-SHA512: fb31fd079767abbf94076615817943f35f5c9262fc97e65c631a18d33b3a343fe6a2d151613256e632d2b372ab2de0435f4712309b4a77ed3c663fd93a7dcdd1
5efebc0edb depends: add the new LLVM debug macro (fanquake)
Pull request description:
`LIBCXX_HARDENING_MODE` is the new macro, the previous one was removed in LLVM 18.
See https://libcxx.llvm.org/Hardening.html.
Required before https://github.com/google/oss-fuzz/pull/11725 will do anything (with the bump to 18.x).
Seems reasonable to do now that almost all our test infra is using LLVM 18.
ACKs for top commit:
theuni:
ACK 5efebc0edb
Tree-SHA512: 43078eeb5940c55ef4f95c72682f8a372dcd3eb97956b3114149c16d9f59b067a999b2aab7f34ffb57eab191524514408e2bba154ff4a6ea0cd6ec4d119c5d18
4f273ab436 Change Luke Dashjr seed to dashjr-list-of-p2p-nodes.us (Luke Dashjr)
Pull request description:
To avoid issues with DNS blacklisting, I've setup a separate domain for my DNS seed.
(This time, without a potentially alarming name)
ACKs for top commit:
kevkevinpal:
Concept ACK [4f273ab](4f273ab436), name looks good to me
petertodd:
ACK 4f273ab436
mzumsande:
ACK 4f273ab436
fanquake:
ACK 4f273ab436
Tree-SHA512: 689698e3c735df3ed0c2756a9d4adb5644bb9d8a6954e23d66bfa9d94ee10954f77fb241d9593f750054d731aa1532368a0fc8277884f6c2a98ac47cd0bdeeb7
78407b99ed [clang-tidy] Enable the misc-no-recursion check (dergoegge)
Pull request description:
Recursion is a frequent source of stack overflow bugs. Secondly, introduction of recursion can be non-obvious.
This PR proposes to use the clang-tidy `misc-no-recursion` check to make introduction of new recursion obvious. We don't make use of recursion a lot in our code base but there are a few places that need suppressions anyway (mostly the descriptor and univalue/rpc code).
ACKs for top commit:
stickies-v:
ACK 78407b99ed
TheCharlatan:
Re-ACK 78407b99ed
fanquake:
ACK 78407b99ed
Tree-SHA512: 34126d704c46086fe7371906ca852c25ced1dbd5fcfd85bf623810cd171a797569a92a33c7e26b8dc01c30c7bbf81aa326718926e8354585091411989a4edb14
The libs in this dir are the following:
```bash
ls /gnu/store/2vnbkrdin4rrf7ygnr80mlcglin4qqa4-gcc-toolchain-12.3.0-static/lib/lib
libanl.a libc.a libdl.a libm.a
libBrokenLocale.a libcrypt.a libg.a libmcheck.a
libpthread.a librt.a
libresolv.a libutil.a
```
These do not need to be propogated into the Windows build environment.
93fae5ae7c test: remove immediate tx relay workaround in wallet_groups.py (Sebastian Falbesoner)
Pull request description:
Reverts commit ab4efad51b (PR #26970). This workaround is not needed anymore, as since #27114 the test sets the noban permission for both in- and outbound connections via the `noban_tx_relay` setting, and we don't have to rely on this topology hack anymore. See commit c985eb854c (kudos to brunoerg!).
Can be tested by executing `$ time ./test/functional/wallet_groups.py` both on master and PR and verifying that the execution time is roughly equal.
ACKs for top commit:
maflcko:
lgtm ACK 93fae5ae7c
brunoerg:
utACK 93fae5ae7c
Tree-SHA512: b949fd05b4308815ba02d0ee4d1318f642b930288dd03223f46db7db745177af1c070bc7058743ac27963c5ad90564089867cc12f31fee94812a16919c353bab
61641e2466 ci: remove --with-asm usage (secp256k1) (fanquake)
c7efee591a ci: use LLVM 18.1.3 in MSAN jobs (fanquake)
Pull request description:
Bumps LLVM to `18.1.3`:
* Includes https://github.com/llvm/llvm-project/pull/86201, which is useful as it removes the need to (possibly) apply a work around when running the CI locally.
Drops `--with-asm=no` (only being passed to secp256k1) from the MSAN CI. New MSAN annotations were pulled in as part of #29803.
ACKs for top commit:
maflcko:
lgtm ACK 61641e2466
hebasto:
ACK 61641e2466.
Tree-SHA512: da51c9f08a9aacb9dd936c47ef47777a8c84234e4df5b9776647ac94ebe88084b5e7b8182af90cfa01ae183072f6ce5915b73825f66b2567214ab270b2ff7837
49c0b8b228 test: Bump timeouts in feature_index_prune and wallet_importdescriptors (Christopher Bergqvist)
Pull request description:
Timeout issues where encountered when running functional tests with `--jobs=16 --extended`.
Note that running `--extended` without `--jobs=16` does not trigger the issues.
Tested under NixOS on a Xeon CPU with 16 logical cores.
(A few tests are skipped locally as I haven't enabled BPF and a few other things).
## Measurements
Line in `feature_index_prune.py` took 101.6s, 96.6s, 103.0s across 3 runs on my machine.
Default limit is 60, suggested to increase limit to 150 seconds.
Line in the `wallet_importdescriptors.py --descriptors` took 5.4s, 5.7s, 6.0s across 3 runs.
Suggested to increase from 5 to 10 seconds.
## Logs
Output slightly modified by separate change that lets code run past given timeouts and the provides more information - "Took 101.6 seconds to complete, 69.4% over the given limit.".
<details>
<summary>
Click to expand.
</summary>
### feature_index_prune.py
```
52/305 - feature_index_prune.py failed, Duration: 250 s
stdout:
2024-04-01T22:25:24.010000Z TestFramework (INFO): PRNG seed is: 990421162716295219
2024-04-01T22:25:24.014000Z TestFramework (INFO): Initializing test directory /mnt/tmp/test_runner_₿_🏃_20240402_002516/feature_index_prune_302
2024-04-01T22:25:24.913000Z TestFramework (INFO): check if we can access blockfilters and coinstats when pruning is enabled but no blocks are actually pruned
2024-04-01T22:26:48.417000Z TestFramework (INFO): prune some blocks
2024-04-01T22:26:48.460000Z TestFramework (INFO): check if we can access the tips blockfilter and coinstats when we have pruned some blocks
2024-04-01T22:26:48.483000Z TestFramework (INFO): check if we can access the blockfilter and coinstats of a pruned block
2024-04-01T22:26:59.175000Z TestFramework (INFO): make sure trying to access the indices throws errors
2024-04-01T22:27:50.422000Z TestFramework (INFO): prune exactly up to the indices best blocks while the indices are disabled
2024-04-01T22:27:52.596000Z TestFramework (INFO): make sure that we can continue with the partially synced indices after having pruned up to the index height
2024-04-01T22:29:34.242000Z TestFramework.utils (ERROR): wait_until() failed. Predicate: '''
self.wait_until(lambda: self.nodes[1].getindexinfo() == expected_stats)#, timeout=150)
'''
2024-04-01T22:29:34.244000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/home/chris/Documents/Code/bitcoin-core/test/functional/test_framework/test_framework.py", line 132, in main
self.run_test()
File "/home/chris/Documents/Code/bitcoin-core/test/functional/feature_index_prune.py", line 117, in run_test
self.sync_index(height=1500)
File "/home/chris/Documents/Code/bitcoin-core/test/functional/feature_index_prune.py", line 34, in sync_index
self.wait_until(lambda: self.nodes[1].getindexinfo() == expected_stats)#, timeout=150)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/chris/Documents/Code/bitcoin-core/test/functional/test_framework/test_framework.py", line 780, in wait_until
return wait_until_helper_internal(test_function, timeout=timeout, timeout_factor=self.options.timeout_factor)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/chris/Documents/Code/bitcoin-core/test/functional/test_framework/util.py", line 305, in wait_until_helper_internal
raise AssertionError(m)
AssertionError: Predicate '''
self.wait_until(lambda: self.nodes[1].getindexinfo() == expected_stats)#, timeout=150)
''' not true after 60 seconds. Took 101.6 seconds to complete, 69.4% over the given limit.
2024-04-01T22:29:34.298000Z TestFramework (INFO): Stopping nodes
2024-04-01T22:29:34.511000Z TestFramework (WARNING): Not cleaning up dir /mnt/tmp/test_runner_₿_🏃_20240402_002516/feature_index_prune_302
2024-04-01T22:29:34.511000Z TestFramework (ERROR): Test failed. Test logging available at /mnt/tmp/test_runner_₿_🏃_20240402_002516/feature_index_prune_302/test_framework.log
2024-04-01T22:29:34.511000Z TestFramework (ERROR):
2024-04-01T22:29:34.512000Z TestFramework (ERROR): Hint: Call /home/chris/Documents/Code/bitcoin-core/test/functional/combine_logs.py '/mnt/tmp/test_runner_₿_🏃_20240402_002516/feature_index_prune_302' to consolidate all logs
2024-04-01T22:29:34.512000Z TestFramework (ERROR):
2024-04-01T22:29:34.512000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024-04-01T22:29:34.512000Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
2024-04-01T22:29:34.512000Z TestFramework (ERROR):
stderr:
53/305 - p2p_blockfilters.py passed, Duration: 130 s
```
### wallet_importdescriptors.py --descriptors
```
297/305 - wallet_importdescriptors.py --descriptors failed, Duration: 76 s
stdout:
2024-04-01T22:48:27.663000Z TestFramework (INFO): PRNG seed is: 8528678505617325332
2024-04-01T22:48:27.664000Z TestFramework (INFO): Initializing test directory /mnt/tmp/test_runner_₿_🏃_20240402_004231/wallet_importdescriptors_98
2024-04-01T22:48:28.021000Z TestFramework (INFO): Setting up wallets
2024-04-01T22:48:28.100000Z TestFramework (INFO): Mining coins
2024-04-01T22:48:29.714000Z TestFramework (INFO): Import should fail if a descriptor is not provided
2024-04-01T22:48:29.725000Z TestFramework (INFO): Should import a p2pkh descriptor
2024-04-01T22:48:29.740000Z TestFramework (INFO): Test can import same descriptor with public key twice
2024-04-01T22:48:29.760000Z TestFramework (INFO): Test can update descriptor label
2024-04-01T22:48:29.785000Z TestFramework (INFO): Internal addresses cannot have labels
2024-04-01T22:48:29.788000Z TestFramework (INFO): Internal addresses should be detected as such
2024-04-01T22:48:29.854000Z TestFramework (INFO): Should not import a p2sh-p2wpkh descriptor without checksum
2024-04-01T22:48:29.855000Z TestFramework (INFO): Should not import a p2sh-p2wpkh descriptor that has range specified
2024-04-01T22:48:29.858000Z TestFramework (INFO): Should not import a p2sh-p2wpkh descriptor and have it set to active
2024-04-01T22:48:29.860000Z TestFramework (INFO): Should import a (non-active) p2sh-p2wpkh descriptor
2024-04-01T22:48:29.984000Z TestFramework (INFO): Should import a 1-of-2 bare multisig from descriptor
2024-04-01T22:48:30.002000Z TestFramework (INFO): Should not treat individual keys from the imported bare multisig as watchonly
2024-04-01T22:48:30.005000Z TestFramework (INFO): Ranged descriptors cannot have labels
2024-04-01T22:48:30.014000Z TestFramework (INFO): Private keys required for private keys enabled wallet
2024-04-01T22:48:30.027000Z TestFramework (INFO): Ranged descriptor import should warn without a specified range
2024-04-01T22:48:30.065000Z TestFramework (INFO): Should not import a ranged descriptor that includes xpriv into a watch-only wallet
2024-04-01T22:48:30.070000Z TestFramework (INFO): Should not import a descriptor with hardened derivations when private keys are disabled
2024-04-01T22:48:30.108000Z TestFramework (INFO): Verify we can only extend descriptor's range
2024-04-01T22:48:30.364000Z TestFramework (INFO): Check we can change descriptor internal flag
2024-04-01T22:48:30.536000Z TestFramework (INFO): Key ranges should be imported in order
2024-04-01T22:48:30.708000Z TestFramework (INFO): Check we can change next_index
2024-04-01T22:48:30.838000Z TestFramework (INFO): Check imported descriptors are not active by default
2024-04-01T22:48:30.870000Z TestFramework (INFO): Check can activate inactive descriptor
2024-04-01T22:48:30.903000Z TestFramework (INFO): Check can deactivate active descriptor
2024-04-01T22:48:30.924000Z TestFramework (INFO): Verify activation state is persistent
2024-04-01T22:48:30.973000Z TestFramework (INFO): Should import a descriptor with a WIF private key as spendable
2024-04-01T22:48:30.987000Z TestFramework (INFO): Test can import same descriptor with private key twice
2024-04-01T22:48:32.173000Z TestFramework (INFO): Test that multisigs can be imported, signed for, and getnewaddress'd
2024-04-01T22:48:43.803000Z TestFramework (INFO): Multisig with distributed keys
2024-04-01T22:48:48.895000Z TestFramework (INFO): We can create and use a huge multisig under P2WSH
2024-04-01T22:49:05.628000Z TestFramework (INFO): Under P2SH, multisig are standard with up to 15 compressed keys
2024-04-01T22:49:20.258000Z TestFramework (INFO): Amending multisig with new private keys
2024-04-01T22:49:23.306000Z TestFramework (INFO): Combo descriptors cannot be active
2024-04-01T22:49:23.313000Z TestFramework (INFO): Descriptors with no type cannot be active
2024-04-01T22:49:23.348000Z TestFramework (INFO): Test importing a descriptor to an encrypted wallet
2024-04-01T22:49:43.957000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/home/chris/Documents/Code/bitcoin-core/test/functional/test_framework/test_framework.py", line 132, in main
self.run_test()
File "/home/chris/Documents/Code/bitcoin-core/test/functional/wallet_importdescriptors.py", line 691, in run_test
with self.nodes[0].assert_debug_log(expected_msgs=["Rescan started from block 0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206... (slow variant inspecting all blocks)"], timeout=5):#10):
File "/nix/store/rac8pxbi1vapwrlqzbrkycbyg521djzw-python3-3.11.6/lib/python3.11/contextlib.py", line 144, in __exit__
next(self.gen)
File "/home/chris/Documents/Code/bitcoin-core/test/functional/test_framework/test_node.py", line 493, in assert_debug_log
self._raise_assertion_error(f'Expected messages "{expected_msgs}" found too late, took {now - start:.1f} seconds, {((now - start) / (time_end - start)) - 1:.1%} over the given limit. Log:\n\n{print_log}\n\n')
File "/home/chris/Documents/Code/bitcoin-core/test/functional/test_framework/test_node.py", line 188, in _raise_assertion_error
raise AssertionError(self._node_msg(msg))
AssertionError: [node 0] Expected messages "['Rescan started from block 0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206... (slow variant inspecting all blocks)']" found too late, took 5.4 seconds, 8.9% over the given limit. Log:
- 2024-04-01T22:49:33.066512Z [http] [httpserver.cpp:306] [http_request_cb] [http] Received a POST request for /wallet/encrypted_wallet from 127.0.0.1:47658
- 2024-04-01T22:49:33.066668Z [httpworker.0] [rpc/request.cpp:187] [parse] [rpc] ThreadRPCServer method=importdescriptors user=__cookie__
- 2024-04-01T22:49:33.070999Z [httpworker.0] [wallet/sqlite.cpp:57] [TraceSqlCallback] [/mnt/tmp/test_runner_₿_🏃_20240402_004231/wallet_importdescriptors_98/node0/regtest/wallets/encrypted_wallet/wallet.dat] SQLite Statement: INSERT INTO main VALUES(?, ?)
- 2024-04-01T22:49:33.071061Z [httpworker.0] [wallet/sqlite.cpp:57] [TraceSqlCallback] [/mnt/tmp/test_runner_₿_🏃_20240402_004231/wallet_importdescriptors_98/node0/regtest/wallets/encrypted_wallet/wallet.dat] SQLite Statement: DELETE FROM main WHERE key = ?
- 2024-04-01T22:49:33.071137Z [httpworker.0] [wallet/sqlite.cpp:57] [TraceSqlCallback] [/mnt/tmp/test_runner_₿_🏃_20240402_004231/wallet_importdescriptors_98/node0/regtest/wallets/encrypted_wallet/wallet.dat] SQLite Statement: BEGIN TRANSACTION
- 2024-04-01T22:49:33.074190Z [httpworker.0] [wallet/sqlite.cpp:57] [TraceSqlCallback] [/mnt/tmp/test_runner_₿_🏃_20240402_004231/wallet_importdescriptors_98/node0/regtest/wallets/encrypted_wallet/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?)
- 2024-04-01T22:49:33.075564Z [httpworker.0] [wallet/sqlite.cpp:57] [TraceSqlCallback] [/mnt/tmp/test_runner_₿_🏃_20240402_004231/wallet_importdescriptors_98/node0/regtest/wallets/encrypted_wallet/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?)
...<thousands of almost identical lines>...
- 2024-04-01T22:49:38.416139Z [httpworker.0] [wallet/sqlite.cpp:57] [TraceSqlCallback] [/mnt/tmp/test_runner_₿_🏃_20240402_004231/wallet_importdescriptors_98/node0/regtest/wallets/encrypted_wallet/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?)
- 2024-04-01T22:49:38.416528Z [httpworker.0] [wallet/sqlite.cpp:57] [TraceSqlCallback] [/mnt/tmp/test_runner_₿_🏃_20240402_004231/wallet_importdescriptors_98/node0/regtest/wallets/encrypted_wallet/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?)
- 2024-04-01T22:49:38.427946Z [httpworker.0] [wallet/sqlite.cpp:57] [TraceSqlCallback] [/mnt/tmp/test_runner_₿_🏃_20240402_004231/wallet_importdescriptors_98/node0/regtest/wallets/encrypted_wallet/wallet.dat] SQLite Statement: COMMIT TRANSACTION
- 2024-04-01T22:49:38.429778Z [httpworker.0] [wallet/sqlite.cpp:57] [TraceSqlCallback] [/mnt/tmp/test_runner_₿_🏃_20240402_004231/wallet_importdescriptors_98/node0/regtest/wallets/encrypted_wallet/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?)
- 2024-04-01T22:49:38.429916Z [httpworker.0] [wallet/sqlite.cpp:57] [TraceSqlCallback] [/mnt/tmp/test_runner_₿_🏃_20240402_004231/wallet_importdescriptors_98/node0/regtest/wallets/encrypted_wallet/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?)
- 2024-04-01T22:49:38.430001Z [httpworker.0] [wallet/wallet.h:933] [WalletLogPrintf] [encrypted_wallet] Setting spkMan to active: id = c6149b35399517457b0b1d8ccdd7efda25a2f20fc7f8167adda8e79b10e260b7, type = legacy, internal = false
- 2024-04-01T22:49:38.430134Z [httpworker.0] [wallet/wallet.h:933] [WalletLogPrintf] [encrypted_wallet] RescanFromTime: Rescanning last 329 blocks
- 2024-04-01T22:49:38.430170Z [httpworker.0] [wallet/wallet.h:933] [WalletLogPrintf] [encrypted_wallet] Rescan started from block 0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206... (slow variant inspecting all blocks)
- 2024-04-01T22:49:38.441914Z [httpworker.0] [wallet/scriptpubkeyman.h:258] [WalletLogPrintf] [encrypted_wallet] MarkUnusedAddresses: Detected a used keypool item at index 4000, mark all keypool items up to this item as used
2024-04-01T22:49:44.029000Z TestFramework (INFO): Stopping nodes
2024-04-01T22:49:44.132000Z TestFramework (WARNING): Not cleaning up dir /mnt/tmp/test_runner_₿_🏃_20240402_004231/wallet_importdescriptors_98
2024-04-01T22:49:44.132000Z TestFramework (ERROR): Test failed. Test logging available at /mnt/tmp/test_runner_₿_🏃_20240402_004231/wallet_importdescriptors_98/test_framework.log
2024-04-01T22:49:44.132000Z TestFramework (ERROR):
2024-04-01T22:49:44.133000Z TestFramework (ERROR): Hint: Call /home/chris/Documents/Code/bitcoin-core/test/functional/combine_logs.py '/mnt/tmp/test_runner_₿_🏃_20240402_004231/wallet_importdescriptors_98' to consolidate all logs
2024-04-01T22:49:44.133000Z TestFramework (ERROR):
2024-04-01T22:49:44.133000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024-04-01T22:49:44.133000Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
2024-04-01T22:49:44.133000Z TestFramework (ERROR):
stderr:
Remaining jobs: [feature_pruning.py, feature_dbcrash.py, feature_assumeutxo.py, rpc_scantxoutset.py, feature_coinstatsindex.py, p2p_node_network_limited.py --v1transport, p2p_node_network_limited.py --v2transport, feature_config_args.py]
298/305 - p2p_node_network_limited.py --v1transport passed, Duration: 24 s
```
</details>
## Related
Almost identical timeout in `feature_index_prune.py` in #27091 on MacOS, and for `wallet_importdescriptors.py --descriptors` in #27282 on Alpine & CI.
ACKs for top commit:
maflcko:
lgtm ACK 49c0b8b228
tdb3:
ACK for 49c0b8b228
itornaza:
approach ACK 49c0b8b228
BrandonOdiwuor:
crACK 49c0b8b228
Tree-SHA512: f62ade74701588d76bfe838b7e7bbda1db38fd98688fd5d13c2c008064027add2ee9d053dee602d84919fab4c9bf53183c31819d94a6174066f237d0f6a62086
2d1819455c crypto: chacha20: always use our fallback timingsafe_bcmp rather than libc's (Cory Fields)
Pull request description:
Looking at libc sources, apple and openbsd implementations match our naive fallback. Only FreeBSD (and only x86_64) seems to [implement an optimized version](https://github.com/freebsd/freebsd-src/blob/main/lib/libc/amd64/string/timingsafe_bcmp.S).
It's not worth the hassle of using a platform-specific function for such little gain.
Additionally, as mentioned below, this is the only case outside of sha2 that requires an autoconf check, and I have upcoming PRs to remove the sha2 ones.
Apple's [impl is unoptimized](https://opensource.apple.com/source/Libc/Libc-1244.1.7/string/FreeBSD/timingsafe_bcmp.c.auto.html).
As-is [OpenBSD's impl](https://github.com/openbsd/src/blob/master/lib/libc/string/timingsafe_bcmp.c).
Relevant IRC conversation with sipa:
> \<cfields\> sipa: chacha20poly1305.cpp uses libc's timingsafe_bcmp when possible. But looking around at apple/freebsd/openbsd, I don't see any impl that doesn't use the naive implementation that matches our fallback...
> \<cfields\> is there any reason to belive there's an optimized impl somewhere that we're actually hitting?
> \<cfields\> asking because after cleaning up sha2, timingsafe_bcmp is the last autoconf check that remains in all of crypto. It'd make life easy if we could just always use our internal one.
> \<cfields\> *all of crypto/
> \<sipa\> cfields: let's get rid of the dependency then
> \<sipa\> it's a trivial function
> \<sipa\> and if we need it for some platforms, no real reason not to use it on all
After the above discusstion, I did end up finding the x86_64-optimized FreeBSD impl, but I don't think that's all that significant.
ACKs for top commit:
sipa:
utACK 2d1819455c
fanquake:
ACK 2d1819455c
TheCharlatan:
ACK 2d1819455c
theStack:
ACK 2d1819455c
Tree-SHA512: b9583e19ac2f77c5d572aa5b95bc4b53669d5717e5708babef930644980de7c5d06a9c7decd5c2b559d70b8597328ecfe513375e3d8c3ef523db80012dfe9266
All `CTransactionRef` have `.GetWitnessHash()` that returns a cached `const Wtxid` (since fac1223a56),
so we don't need to pass transaction refs around with their IDs as they're easy to get from a ref.
Reverts commit ab4efad51b (PR #26970).
This workaround is not needed anymore, as since #27114 the test sets
the noban permission for both in- and outbound connections via the
`noban_tx_relay` setting, and we don't have to rely on these topology
hacks anymore. See commit c985eb854c.
The `BITCOINFUZZ` environment variable allows to override the default
path to the fuzz binary.
It complements the already existing set of variables used by tests:
- BITCOIND
- BITCOINCLI
- BITCOINUTIL
- BITCOINWALLET
561a650e0f test: Fix debug recommendation in argsman_tests (Fabian Jahr)
Pull request description:
There are recommendations in the `argsman_tests` comments on how to re-run and debug a test failure to see if it reflects an expected or unexpected change. The command tries to run a test in `util_tests` but this is in `argsman_tests` so the command doesn't work with just copy+paste. I didn't investigate further but I suspect that these tests were moved between files.
ACKs for top commit:
fanquake:
ACK 561a650e0f
Tree-SHA512: b3bb94ba1635c9455149b455f2b30ee37a8067a6242339531ab54d428177a288da29a4a10702652305eb34aa7638f51dad35fa6b0e7b74617e445327b8c4c053
a3485af67d ci: Drop duplicated compiler flags (Hennadii Stepanov)
Pull request description:
On the master branch @ 0d509bab45, it is easy to check the _"Options used to compile and link"_ section in the `configure` script output and observe duplicated compiler flags.
This PR cleans such cases up.
ACKs for top commit:
maflcko:
re-ACK a3485af67d
fanquake:
ACK a3485af67d - no-longer a change in behaviour.
Tree-SHA512: 7e644fcfad7be48af3b18edd2994c0c78a21ac3f9fff497724be80f74c9e859d156de15ca4024c5c50d1080435576ce63402b48aba5c2fd556e2ed7e318e0e34
fa9f36baba build: Remove HAVE_GMTIME_R (MarcoFalke)
fa72dcbfa5 refactor: FormatISO8601* without gmtime* (MarcoFalke)
fa2c486afc Revert "time: add runtime sanity check" (MarcoFalke)
Pull request description:
Now that the `ChronoSanityCheck` has passed for everyone with C++17 and is guaranteed by C++20 to always pass, remove it.
Also, remove `gmtime_r` and `gmtime_s` and replace them with `year_month_day`+`hh_mm_ss` from C++20.
ACKs for top commit:
sipa:
utACK fa9f36baba
fanquake:
ACK fa9f36baba - more std lib & even less stuff to port.
Tree-SHA512: a9e7e805b757b7dade0bcc3f95273a7dc4f68622630d74838339789dd203ad7542d36b2e090a93b2bc5a7ecc383207dd7ec82c68147108bdac7ce44f088c8c9a
Looking at apple/freebsd/openbsd sources, their implementations match our naive
fallback. It's not worth the hassle of using a platform-specific function for
no gain.
007ea322a6 depends: switch to building libqrencode with CMake (fanquake)
884330c0a5 guix: make cmake-minimal a global requirement (fanquake)
Pull request description:
Switch to building libqrencode with CMake. Note that upstream (https://github.com/fukuchi/libqrencode) hasn't seen any activity for ~4 years, so the odds of getting anything upstream seems low, but I've made two minor changes to the source here, which I will PR in any case.
From an initial look I couldn't find any significant difference between the Autotools and CMake produced libs. As part of this change we move cmake-minimal in Guix into the global package set.
ACKs for top commit:
TheCharlatan:
ACK 007ea322a6
Tree-SHA512: c784f790ddea958082c8ae96d3744bdf99331a8799765f9d44f00861b8e2cfcab1a88a3d64af5b10e51a8d5938d55eb6a3d271790b565e50492a39d00dc0e30f
Rather than manually estimating an input's weight by adding up all the
involved components (fixed-size skeleton, compact-serialized lengths,
and the actual scriptSig / witness stack items) we can simply take use
of the serialization classes `CTxIn` / `CTxInWitness` instead, to
achieve the same with significantly less code.
The new helper is used in the functional tests rpc_psbt.py and
wallet_send.py, where the previous manual estimation code was
duplicated.
Fixes: #809
Previously it was possible through the GUI to enter an IP address:port
into the "Proxy IP" configuration box. After the node was restarted the
errant setting would prevent the node starting back up until manually
removed from settings.json.
bbe82c116eFix#29767, set m_synced = true after Commit() (nanlour)
Pull request description:
I think this problem https://github.com/bitcoin/bitcoin/issues/29767#issue-2216373048 is because of
in BaseIndex::Sync
61de64df67/src/index/base.cpp (L163-L168)
Setup m_synced = true; before Commit();
So this may cause a race condition window to BaseIndex::BlockConnected
61de64df67/src/index/base.cpp (L271-L274)
So i try to fix it with move m_synced = true after Commit().
Also see comment of Sync():
61de64df67/src/index/base.h (L151-L156)
I am a newcomer interested in Bitcoin, trying to become a member of the Bitcoin Core development team. Please give me some feedback if you could, as I may be doing something wrong. Thank you!
ACKs for top commit:
fjahr:
Code review ACK bbe82c116e
ryanofsky:
Code review ACK bbe82c116e
Tree-SHA512: 89a09498a232c87ef1e083d4cc4ed9bb15f045ad0624d5d150a87187b2b8a48a41137974dbc7ea5c37f73da90742c43259f5aa7f84b4179eb8d62033e44fa479
Previously, `wait_for_getheaders` would check whether a node had received **any**
getheaders message. This implied that, if a test needed to check for a specific block
hash within a headers message, it had to make sure that it was checking the desired message.
This normally involved having to manually clear `last_message`. This method, apart from being
too verbose, was error prone, given an undesired `getheaders` would make tests pass.
This adds the ability to check for a specific block_hash within the last `getheaders` message.
73d92309d7 guix: use GCC 11 for macOS builds (fanquake)
Pull request description:
Note that this is just the native compiler, which is used to build the toolchain we use to build the actual binaries.
Partially motivated by #29091, where it could now be a bit confusing if we are explicitly using GCC 10 in our release toolchain, when our minimum required is 11 (this can't be bumped to 12 due to build issues with native tools).
At the same time, remove `gcc-toolchain "static"` from the macOS build env.
ACKs for top commit:
hebasto:
ACK 73d92309d7.
Tree-SHA512: 31392290b327cc0e19498cf053b7c9eb19e70295933d650b29b29589356ad455d35b6addcdaae702a9635513c07070fb17d61bcb48445d3cb1a9d4a93aa6ddf3
fac012c726 ci: Temporarily disable bpfcc-tools (MarcoFalke)
Pull request description:
This works around package install errors, such as https://github.com/bitcoin/bitcoin/runs/23354020361. Should be possible to reproduce locally via `apt update && apt install bpfcc-tools` on noble:
```
python3-bpfcc : Depends: libbpfcc (>= 0.29.1+ds-1ubuntu4) but it is not going to be installed
ACKs for top commit:
hebasto:
ACK fac012c726, I have reviewed the code, it looks OK. And CI is green.
TheCharlatan:
ACK fac012c726
Tree-SHA512: 369e89bc0fbf4d75455c7c047ae904849267f82e647eeadd7ee6045393bd9812a5e682a58eb8d2ccd8c89a030e6665f054596fdcd0708f591ed4c693398125d1
3cb80febb8 guix: Remove another leftover from #29648 (Hennadii Stepanov)
Pull request description:
It was overlooked in bitcoin/bitcoin#29787.
ACKs for top commit:
TheCharlatan:
ACK 3cb80febb8
Tree-SHA512: c4eae65ffa0a79f4d57ba07730effee6aeff9d9625bc00a4534ffe46d3a16ae56bc8753e3fec93d7ff81ea7be39662282c631861a21ea8a9dc5d31b79acb231d
84fbf9b284 depends: remove -g from sqlite debug flags (fanquake)
eef51afc6a depends: add -g to DEBUG=1 flags (fanquake)
Pull request description:
Add `-g` to the base DEBUG=1 flags in depends.
Avoids the need to specify it per-package.
More alignment with `--enable-debug` behaviour in configure.
We also want to align the optimization flags, currently -O1 vs -O0, however that can be it's own PR.
ACKs for top commit:
theuni:
ACK 84fbf9b284
Tree-SHA512: 1ee98ba0c13e4b80bb87632658b4f53ce49c73e0e7712990c30da60deca4a349a744232f2d78f243dee9a07f5b9b70f9c2c4ae34082c34ae51b37b054fac61fd
2de2ea2ff6 build, depends: Fix `libmultiprocess` cross-compilation (Hennadii Stepanov)
Pull request description:
On the master branch @ 3b12fc7bcd, the following command fails:
```
$ make -C depends libmultiprocess HOST=arm64-apple-darwin MULTIPROCESS=1
...
[100%] Linking CXX executable mpgen
...
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
...
```
This PR prevents building all default targets that include `mpgen`, which expectedly fails to link when cross-compiling.
ACKs for top commit:
ryanofsky:
Code review ACK 2de2ea2ff6
fanquake:
ACK 2de2ea2ff6 - I checked that this fixes the macOS cross-compilation issue. I'm assuming these packages are also likely to change further in the (near) future, given the changes going in upstream: https://github.com/chaincodelabs/libmultiprocess/pulls?q=is%3Apr+is%3Aclosed.
Tree-SHA512: 563551afbe483c923b52c6171f9d73bcc30bc4febd821b5abfe8aadb2ac601b94c2d10a73746ace3710d9f0afa4798eb090e77ccb1ae66a819495912802d91c9
The developer mailing list was migrated to Google Groups in February 2024
as announced in https://groups.google.com/g/bitcoindev/c/aewBuV6k-LI.
The archives maintained by the Linux Foundation stopped updating in December
2024. Thus, we point to the new archive maintained by gnusha.org.
The codebase refers to old discussions linked to the Linux Foundation archives.
Since all links are still active to this date, we keep them as they are.
See #29782.
fd8527a20e guix: remove errant leftover from #29648 (fanquake)
Pull request description:
We no longer build a lib, so a non-existent dir is causing builds to fail.
ACKs for top commit:
josibake:
ACK fd8527a20e
hebasto:
ACK fd8527a20e.
TheCharlatan:
ACK fd8527a20e
Tree-SHA512: 9175a0de3f95f56939b3eaa3e89dca2cfae4996bcd84ef6b8e2872672bef39cb0550c9f4a79475d887eb8fac92c15dfa8c352648ff167d54a0b736978412226c
Timeout issues where encountered when running functional tests with `--jobs=16 --extended`.
Line in `feature_index_prune.py` took 101.6s, 96.6s, 103.0s across 3 runs on my machine, default limit is 60.
Line in the `wallet_importdescriptors.py --descriptors` took 5.4s, 5.7s, 6.0s across 3 runs.
b0344c219a logging: remove unused BCLog::UTIL (Vasil Dimov)
d3b3af9034 log: deduplicate category names and improve logging.cpp (Vasil Dimov)
Pull request description:
The code in `logging.cpp` needs to:
* Get the category name given the flag (e.g. `BCLog::PRUNE` -> `"prune"`)
* Get the flag given the category name (e.g. `"prune"` -> `BCLog::PRUNE`)
* Get the list of category names sorted in alphabetical order
Achieve this by using the proper std containers. The result is
* less code (the diff of the first commit is +62 / -129)
* faster code (to linear search and no copy+sort)
* more maintainable code (the categories are no longer duplicated in `LogCategories[]` and `LogCategoryToStr()`)
This behavior is preserved:
`BCLog::NONE` -> `""` (lookup by `LogCategoryToStr()`)
`""` -> `BCLog::ALL` (lookup by `GetLogCategory("")`)
---
Also remove unused `BCLog::UTIL`.
---
These changes (modulo the `BCLog::UTIL` removal) are part of https://github.com/bitcoin/bitcoin/pull/29415 but they make sense on their own and would be good to have them, regardless of the fate of https://github.com/bitcoin/bitcoin/pull/29415. Also, if this is merged, that would reduce the size of https://github.com/bitcoin/bitcoin/pull/29415, thus the current standalone PR.
ACKs for top commit:
davidgumberg:
crACK b0344c219a
pinheadmz:
ACK b0344c219a
ryanofsky:
Code review ACK b0344c219a. Nice cleanup! Having to maintain multiple copies of the same mapping seemed messy and a like a possible footgun. I checked old and new mappings in both directions and confirmed no behavior should be changing.
Tree-SHA512: 57f87a090932f9b33dc8e075d1855dba9b71a3243a0758511745483dec2d9c46d3b532eadab297e78164c9b7caba370986ee380696a45f0778a841082f8e21a7
d32346c39e [doc] add historical release notes for 26.1 (glozow)
Pull request description:
Need for github release
ACKs for top commit:
fanquake:
ACK d32346c39e - looks like just a newline difference.
Tree-SHA512: 567647bdab8b5a8405b43682e3a0f86282eb0b866c83423a7866fef872307a7ca88d564604f1cfe53d6ce48407aa6cf88bdf18b35c16c959e2a36062bff39a50
eff19fa1c8 build, macos: Drop unused `osx_volname` target (Hennadii Stepanov)
Pull request description:
The `osx_volname` makefile target was introduced in https://github.com/bitcoin/bitcoin/pull/7192 and was used to pass the package name to Gitian scripts as a content of the `osx_volname` file.
With the current Guix scripts, the `osx_volname` file is never read. Therefore, its creation might be omitted.
My Guix builds:
```
x86_64
5e2d254e207d53784621c8df331c9bf4a969da667d185992402f48a5ac49f563 guix-build-eff19fa1c8d7/output/arm64-apple-darwin/SHA256SUMS.part
089dba70685893aca5e7c8ce1d53a07380e87ca50eda8b3a2a75aeaeb1d28e48 guix-build-eff19fa1c8d7/output/arm64-apple-darwin/bitcoin-eff19fa1c8d7-arm64-apple-darwin-unsigned.tar.gz
390c57197c6ab4aefdde1c665d5e4ebdfb4ae5e553f8f93b017f2fad1093d110 guix-build-eff19fa1c8d7/output/arm64-apple-darwin/bitcoin-eff19fa1c8d7-arm64-apple-darwin-unsigned.zip
e1edde7ca28bf26aea8d956b1d3c1725a475f2a9c148f5c36b651db4b814091c guix-build-eff19fa1c8d7/output/arm64-apple-darwin/bitcoin-eff19fa1c8d7-arm64-apple-darwin.tar.gz
d0096ea73a5f75cc4d3cef4ef1761ae3e48c8a63aff918f07371c5c88896e4e6 guix-build-eff19fa1c8d7/output/dist-archive/bitcoin-eff19fa1c8d7.tar.gz
51b4affb9fd6f8aea05b7d25d29f017d0a0a145395f457caa14b9af9646b035b guix-build-eff19fa1c8d7/output/x86_64-apple-darwin/SHA256SUMS.part
b1df081ecf636a92754e673e5388d1d988653d4646f0b0446a4c9f14d865a265 guix-build-eff19fa1c8d7/output/x86_64-apple-darwin/bitcoin-eff19fa1c8d7-x86_64-apple-darwin-unsigned.tar.gz
62e09926029d176da950d3e3db7ff8ae6cbe4c0b2ea17b084fc1d28565f91475 guix-build-eff19fa1c8d7/output/x86_64-apple-darwin/bitcoin-eff19fa1c8d7-x86_64-apple-darwin-unsigned.zip
477dcb2382cbd447bd88a3b644b4bd736f5b67d66d42cb73fe31ffc153d3e181 guix-build-eff19fa1c8d7/output/x86_64-apple-darwin/bitcoin-eff19fa1c8d7-x86_64-apple-darwin.tar.gz
```
ACKs for top commit:
Empact:
ACK eff19fa1c8
Sjors:
tACK eff19fa1c8
TheCharlatan:
ACK eff19fa1c8
Tree-SHA512: 29714be5c58caa07b3eb99846d71bb83366dade769af022059bb3c499878adcd34cdf03b006c5da561291f373ccc59abdb83c925057ec0049465eaa8dd6ef4e1
61560d5e93 test: makes timeout a forced named argument in tests methods that use it (Sergi Delgado Segura)
Pull request description:
This makes calls to such methods more explicit and less error-prone.
Motivated by https://github.com/bitcoin/bitcoin/pull/29736#discussion_r1540654057
ACKs for top commit:
maflcko:
lgtm ACK 61560d5e93
brunoerg:
ACK 61560d5e93
BrandonOdiwuor:
crACK 61560d5e93
AngusP:
ACK 61560d5e93
stratospher:
tested ACK 61560d5.
Tree-SHA512: 8d6ec3fe1076c868ddbd3050f3c242dbd83cc123f560db3d3b0ed74968e6050dc9ebf4e7c716af9cc1b290c97d736c2fc2ac936b0b69ebdbceed934dae7d55d9
It's useful to be able to seek to a specific position in a file. Allow
AutoFile to seek by using fseek.
It's also useful to be able to get the current position in a file.
Allow AutoFile to tell by using ftell.
2eb5175de8 test: fix StopIteration exception in p2p_node_network_limited.py (furszy)
Pull request description:
Fixes#29731
The `next()` call throws an exception if the default parameter is omitted and the iterator is exhausted.
Fix it by providing a default value.
The failure can be tested by commenting out lines 90 and 91 in the test (the `self.connect_nodes(2, 0)`). Since there is no connection, the node in question retrieves a single element in the 'getchaintips()' call. This scenario without the fix, aborts the test right away, throwing an `StopIteration` exception, and with the fix, the test properly waits until the timeout (`wait_until()` call).
ACKs for top commit:
maflcko:
review ACK 2eb5175de8
brunoerg:
crACK 2eb5175de8
BrandonOdiwuor:
crACK 2eb5175de8
tdb3:
Tested ACK for 2eb5175de8.
Tree-SHA512: b0873eb4d3334146fd250cd2cd23add3e744877033c8bfa4eb8dff36633100604adf49dd7846856ddfa88c9768663f095db705c00eef3641618df8fc13f8c2c5
80f8b92f4f remove libbitcoinconsensus (fanquake)
Pull request description:
This was deprecated in `v27.0`, for removal in `v28.0`. See discussion in PR #29189.
ACKs for top commit:
theuni:
Concept ACK and light review ACK 80f8b92f4f. My only hesitation here is that (afaics?) there's now nothing keeping undesired features like threading or globals from working their way into the interpreter in future commits.
m3dwards:
Concept ACK 80f8b92f4f
TheCharlatan:
ACK 80f8b92f4f
hebasto:
ACK 80f8b92f4f, I have reviewed the code and it looks OK.
Tree-SHA512: 17a62118aeb088f2695c892bb32794dfea3061e3cb7d9e8e9f1c06c3ff6f63a7587fa532e37edbb91fbc5a19b12c9a0f8e05fa9e8864aa07f92665375d847e80
fa75220ac5 ci: Use clang-18 in asan/fuzz/tsan task (MarcoFalke)
fad23a0646 ci: Bump clang+llvm in i686_multiprocess task (MarcoFalke)
Pull request description:
Use clang-18, which comes with bugfixes and sanitizer upgrades.
ACKs for top commit:
fanquake:
ACK fa75220ac5
Tree-SHA512: da251eb55d3a7e537110b03fb0fc10fc6e2f4406d55bdb33bdc91459daabfd3b8d1d20859a66718bb1b09126e11495f6584c52504a85eb09bde9eafafdaf0b0b
430f319f73 depends: qt 5.15.13 (fanquake)
Pull request description:
I need to try and patch around Qts buildsystem to keep #21778 moving along (the issue being that even when you tell Qt to build using Clang on Linux, it still calls out to GCC, breaking our ability to have a macOS release build env that doesn't have a GCC toolchain installed, and thus no `ld` binary).
Before trying to patch Qt any further, update to the latest LTS release, and update the current patch set.
ACKs for top commit:
TheCharlatan:
Re-ACK 430f319f73
hebasto:
re-ACK 430f319f73.
Tree-SHA512: 11122c04d63f4ec79e5d7945e1686eaf308cb1f98c3ff8bc52e265d7353cd4f1a532f4d78d55b43183245aeebf93cc1092087d9fe83e42eefa8a69ad9017d2da
6c2990416e ci: Pull in qtbase5-dev instead of seperate low-level libraries (laanwj)
a3c6a13cb2 doc: Suggest installing dev packages for debian/ubuntu qt5 build (laanwj)
Pull request description:
Pretty much all library packages were renamed in the 64-bit time_t migration to add `t64` (even on 64-bit platforms).
Instead of complicating the doc with conditional package names, suggest installing the `-dev` packages which still have the same name, and besides that, are the right way to go about it as they contain the "user facing" C++ headers needed to build against Qt5. They pull in the necessary library packages through dependencies.
For Fedora, devel packages are already suggested.
This affects Ubuntu 24.04 and Debian Testing.
ACKs for top commit:
maflcko:
lgtm ACK 6c2990416e
hebasto:
ACK 6c2990416e.
Tree-SHA512: dae21b7d08fdb221b7b72c323fdaaa6d1a8b014f90e24d2beae64b1ae229fdbeb93d726a61e826447813e52a3cb9775426aefc6c44a4c5ccc541afabb89cb135
601edd8ee8 ci: use codespell 2.2.6 (fanquake)
52fa0d285f doc: fix some typos (crazeteam)
b5ed13a240 doc: Fix typos (RoboSchmied)
Pull request description:
Combines the recent PRs to fix typos so they can be merged.
ACKs for top commit:
brunoerg:
crACK 601edd8ee8
tdb3:
crACK 601edd8ee8
kristapsk:
cr utACK 601edd8ee8
Tree-SHA512: d054b1dad1336d6b9291cc5d5252d4debf6424a993d4edd6a97d7c15055a7fc48a333d30967f72e7dc9c6c1d9a9038ca8bb5e219c529f4c2365ea48404a508d0
ee1b9b231a CalculateFeerateDiagramsForRBF: update misleading description of old diagram contents (Greg Sanders)
a9d42b9aa5 CompareFeerateDiagram: short-circuit comparison when detected as incomparable (Greg Sanders)
cebcced65e remove erroneous CompareFeerateDiagram comment about slope (Greg Sanders)
a0376e1061 unit test: clarify unstated assumption for calc_feerate_diagram_rbf chunking (Greg Sanders)
890cb015f3 s/effected/affected/ (Greg Sanders)
d9391ec095 CalculateFeerateDiagramsForRBF: remove size tie-breaking from chunking conflicts (Greg Sanders)
b684d82d7e fuzz: Add more invariant checks for package_rbf (Greg Sanders)
2a3ada8b21 fuzz: finer grained ImprovesFeerateDiagram check on error result (Greg Sanders)
c377ae9ba0 unit test: improve ImprovesFeerateDiagram coverage with one less vb case (Greg Sanders)
d2bf923eb1 unit test: make calc_feerate_diagram_rbf less brittle (Greg Sanders)
defe023f6e fuzz: add PrioritiseTransaction coverage in diagram checks (Greg Sanders)
216d5ff162 unit test: add coverage showing priority affects diagram check results (Greg Sanders)
a80d80936a unit test: add CheckConflictTopology case for not the only child (Greg Sanders)
69bd18ca80 unit test: check tx4 conflict error message (Greg Sanders)
c0c37f07eb unit test: have CompareFeerateDiagram tested with diagrams both ways (Greg Sanders)
b62e2c0fa5 ImprovesFeerateDiagram: Spelling fix and removal of unused diagram vectors (Greg Sanders)
bb42402945 doc: fix comment about non-existing CompareFeeFrac (Greg Sanders)
Pull request description:
Follow-ups to https://github.com/bitcoin/bitcoin/pull/29242
ACKs for top commit:
glozow:
ACK ee1b9b231a, reviewed the changes and package_rbf fuzzer seems to run fine
murchandamus:
crACK ee1b9b231a
ismaelsadeeq:
Code review ACK ee1b9b231a
willcl-ark:
ACK ee1b9b231a
Tree-SHA512: 8399fe12064fb49b0e4c73258968b57be1d9c2e35701b2d3b0bb67e2e4052e44216358238f92508e4697d0fb6176518d5b885474054d3deda242f669e99262a7
746b6d8839 test: Add test for createwalletdescriptor (Ava Chow)
2402b63062 wallet: Test upgrade of pre-taproot wallet to have tr() descriptors (Ava Chow)
460ae1bf67 wallet, rpc: Add createwalletdescriptor RPC (Ava Chow)
8e1a475062 wallet: Be able to retrieve single key from descriptors (Ava Chow)
85b1fb19dd wallet: Add GetActiveHDPubKeys to retrieve xpubs from active descriptors (Ava Chow)
73926f2d31 wallet, descspkm: Refactor wallet descriptor generation to standalone func (Andrew Chow)
54e74f46ea wallet: Refactor function for single DescSPKM setup (Andrew Chow)
3b09d0eb7f tests: Test for gethdkeys (Ava Chow)
5febe28c9e wallet, rpc: Add gethdkeys RPC (Ava Chow)
66632e5c24 wallet: Add IsActiveScriptPubKeyMan (Ava Chow)
fa6a259985 desc spkm: Add functions to retrieve specific private keys (Ava Chow)
fe67841464 descriptor: Be able to get the pubkeys involved in a descriptor (Ava Chow)
ef6745879d key: Add constructor for CExtKey that takes CExtPubKey and CKey (Ava Chow)
Pull request description:
This PR adds a `createwalletdescriptor` RPC which allows users to add new automatically generated descriptors to their wallet, e.g. to upgrade a 0.21.x wallet to contain a taproot descriptor. This RPC takes 3 arguments: the output type to create a descriptor for, whether the descriptor will be internal or external, and the HD key to use if the user wishes to use a specific key. The HD key is an optional parameter. If it is not specified, the wallet will use the key shared by the active descriptors, if they are all single key. For most users in the expected upgrade scenario, this should be sufficient. In more advanced cases, the user must specify the HD key to use.
Currently, specified HD keys must already exist in the wallet. To make it easier for the user to know, `gethdkeys` is also added to list out the HD keys in use by all of the descriptors in the wallet. This will include all HD keys, whether we have the private key, for it, which descriptors use it and their activeness, and optionally the extended private key. In this way, users with more complex wallets will be still be able to get HD keys from their wallet for use in other scenarios, and if they want to use `createwalletdescriptor`, they can easily get the keys that they can specify to it.
See also https://github.com/bitcoin/bitcoin/pull/26728#issuecomment-1866961865
ACKs for top commit:
Sjors:
re-utACK 746b6d8839
furszy:
ACK 746b6d8
ryanofsky:
Code review ACK 746b6d8839, and this looks ready to merge. There were various suggested changes since last review where main change seems to be switching `gethdkeys` output to use normalized descriptors (removing hardened path components).
Tree-SHA512: f2849101e6fbf1f59cb031eaaaee97af5b1ae92aaab54c5716940d210f08ab4fc952df2725b636596cd5747b8f5beb1a7a533425bc10d09da02659473516fbda
Pretty much all library packages were renamed in the 64-bit time_t
migration to add `t64` (even on 64-bit platforms).
Instead of complicating the doc with conditional package names, suggest
installing the `-dev` packages which still have the same name, and
besides that, are the right way to go about it as they contain the
"user facing" C++ headers needed to build against Qt5.
For Fedora, devel packages are already suggested.
This affects Ubuntu 24.04 and Debian Testing.
4d5b55735b log: renamed disk to file so wording was more accurate (kevkevin)
b9f04be870 mempool: Log added for dumping mempool transactions to disk (kevkevin)
Pull request description:
Sometimes when shutting off bitcoind it can take a while to dump the mempool transaction onto the disk so
this change adds additional logging to the `DumpMempool` method in `kernel/mempool_persist.cpp`
Motivated by https://github.com/bitcoin/bitcoin/pull/29227 this change
- adds a single new line for the amount of transactions being dumped and the amount of memory being dumped to file
This is in response to https://github.com/bitcoin/bitcoin/pull/29227#issuecomment-1893375082
The logs will now look like this
```
2024-02-09T23:41:52Z DumpAnchors: Flush 2 outbound block-relay-only peer addresses to anchors.dat completed (0.02s)
2024-02-09T23:41:52Z scheduler thread exit
2024-02-09T23:41:52Z Writing 29 mempool transactions to file...
2024-02-09T23:41:52Z Writing 0 unbroadcast transactions to file.
2024-02-09T23:41:52Z Dumped mempool: 0.000s to copy, 0.022s to dump, 0.015 MB dumped to file
2024-02-09T23:41:52Z Flushed fee estimates to fee_estimates.dat.
2024-02-09T23:41:53Z Shutdown: done
```
ACKs for top commit:
maflcko:
cr-ACK 4d5b55735b
glozow:
reACK 4d5b557
Tree-SHA512: 049191e140d00c1ea57debe0138f1c9eb0f9bb0ef8138e2568e6d89e64f45a5d5853ce3b9cc0b28566aab97555b47ddfb0f9199fc8cea6b81e53f50592d5ae6a
The `next()` call throws an exception if the default parameter is omitted and the iterator is exhausted.
Fix it by providing a default value.
The failure can be tested by commenting out lines 90 and 91 in the test (the `self.connect_nodes(2, 0)``).
Since there is no connection, the node in question retrieves a single element in the 'getchaintips()' call.
This scenario without the fix, aborts the test right away, throwing an StopIteration exception, and with
the fix, the test properly waits until the timeout (wait_until() call).
I need to try and patch around Qts buildsystem to keep #21778 moving
along (the issue being that even when you tell Qt to build using
Clang on Linux, it still calls out to GCC, breaking our ability to have
a macOS release build env that doesn't have a GCC toolchain installed,
and thus no ld binary).
Before trying to patch Qt any further, update to the latest LTS
release, and update the current patch set.
5952292133 wallet, rpc: show mempool conflicts in `gettransaction` result (ishaanam)
54e07ee22f wallet: track mempool conflicts (ishaanam)
d64922b590 wallet refactor: use CWalletTx member functions to determine tx state (ishaanam)
ffe5ff1fb6 scripted-diff: wallet: s/TxStateConflicted/TxStateBlockConflicted (ishaanam)
180973a941 test: Add tests for wallet mempool conflicts (ishaanam)
Pull request description:
The `mempool_conflicts` variable is added to `CWalletTx`, it is a set of txids of txs in the mempool conflicting with the wallet tx or a wallet tx's parent. This PR only changes how mempool-conflicted txs are dealt with in memory.
`IsSpent` now returns false for an output being spent by a mempool conflicted transaction where it previously returned true.
A txid is added to `mempool_conflicts` during `transactionAddedToMempool`. A txid is removed from `mempool_conflicts` during `transactionRemovedFromMempool`.
This PR also adds a `mempoolconflicts` field to the `gettransaction` wallet RPC result.
Builds on #27145
Second attempt at #18600
ACKs for top commit:
achow101:
ACK 5952292133
ryanofsky:
Code review ACK 5952292133. Just small suggested changes since last review
furszy:
ACK 59522921
Tree-SHA512: 615779606723dbb6c2e302681d8e58ae2052ffee52d721ee0389746ddbbcf4b4c4afacf01ddf42b6405bc6f883520524186a955bf6b628fe9b3ae54cffc56a29
b7e7e727ab depends: fix mingw-w64 Qt DEBUG=1 build (fanquake)
Pull request description:
The issue is that compilation is done with `x86_64-w64-mingw32-g++-posix`, but then linking is done with `x86_64-w64-mingw32-g++`.
I'm guessing this has been broken since #24131 (01d1845a80), but have not checked.
Fixes#29734.
Unblocks #29527 (`DEBUG=1` builds can be tested).
ACKs for top commit:
hebasto:
ACK b7e7e727ab, tested on Ubuntu 22.04 with the [installed](https://github.com/bitcoin/bitcoin/issues/29734#issuecomment-2022852344) `g++-mingw-w64-x86-64` package.
TheCharlatan:
ACK b7e7e727ab
Tree-SHA512: 9e24e84046c0489c20971bb9c68d1a643c233837193c184f61bff79dfc8d7397a5c5526ac1a205ad423920f2589559cd01cb104ceb7f89515bb6421510d82ca9
Upstream repo: https://github.com/arun11299/cpp-subprocess
Commit: 4025693decacaceb9420efedbf4967a04cb028e7
The "Convenience Functions" section is unused in our codebase, so it has
been removed.
fa22a438fa ci: Print tsan errors to stderr (MarcoFalke)
Pull request description:
This fixes a bug introduced in https://github.com/bitcoin/bitcoin/pull/27667
All sanitizers print their errors to stderr, except for tsan, which prints to a file and expects the file to be read.
Fix this by not using a log file in any sanitizer.
ACKs for top commit:
dergoegge:
utACK fa22a438fa
Tree-SHA512: 15dca57932a21bda145335fab6367bbf2ae67b25e0b7b61044d2c06ab7a8db3a452f057f6656b81a031726375b7bb238f5ced18ab8894f005e7ab254c7d1ef06
The issue is that compilation is done with `x86_64-w64-mingw32-g++-posix`,
but then linking is done with `x86_64-w64-mingw32-g++`.
I'm guessing this has been broken since #24131
(01d1845a80), but have not checked.
Fixes#29734.
Unblocks #29527 (now DEBUG=1 builds can be tested).
Test the ban list is preserved through restart has been
done by both `rpc_setban` and `p2p_disconnect_ban`.
Since `p2p_disconnect_ban` does it in a more elegant
way, we can keep only it and remove the duplicated one.
7850c5fe20 guix: build GCC with --enable-standard-branch-protection (fanquake)
Pull request description:
This is one change extracted from #24123 (which now produces fully BTI & PAC enabled bins), which will mean that everything in depends, for Guix builds, is compiled using `-mbranch-protection=standard`.
Turning this on by default, is similar to what we already do with `--enable-default-ssp`, `--enable-default-pie` etc.
See: https://gcc.gnu.org/install/specific.html#aarch64-x-x
> To enable Branch Target Identification Mechanism and Return Address Signing by default at configure time use the `--enable-standard-branch-protection` option.
> This is equivalent to having `-mbranch-protection=standard` during compilation. This can be explicitly disabled during compilation by passing the `-mbranch-protection=none` option which turns off all types of branch protections.
ACKs for top commit:
TheCharlatan:
ACK 7850c5fe20
Tree-SHA512: 18f898da27021bab502e708ea5fa9b325352f8f6e23d9488a2a0feda87e0af2ac0e4f87b3af9ad6a9a37bbfc99ab0285de4f0bdc174dcd38163d92c122e958e2
7295986778 Unit tests for CalculateFeerateDiagramsForRBF (Greg Sanders)
b767e6bd47 test: unit test for ImprovesFeerateDiagram (Greg Sanders)
7e89b659e1 Add fuzz test for FeeFrac (Greg Sanders)
4d6528a3d6 fuzz: fuzz diagram creation and comparison (Greg Sanders)
e9c5aeb11d test: Add tests for CompareFeerateDiagram and CheckConflictTopology (Greg Sanders)
588a98dccc fuzz: Add fuzz target for ImprovesFeerateDiagram (Greg Sanders)
2079b80854 Implement ImprovesFeerateDiagram (Greg Sanders)
66d966dcfa Add FeeFrac unit tests (Greg Sanders)
ce8e22542e Add FeeFrac utils (Greg Sanders)
Pull request description:
This is a smaller piece of https://github.com/bitcoin/bitcoin/pull/28984 broken off for easier review.
Up to date explanation of diagram checks are here: https://delvingbitcoin.org/t/mempool-incentive-compatibility/553
This infrastructure has two near term applications prior to cluster mempool:
1) Limited Package RBF(https://github.com/bitcoin/bitcoin/pull/28984): We want to allow package RBF only when we know it improves the mempool. This narrowly scoped functionality allows use with v3-like topologies, and will be expanded at some point post-cluster mempool when diagram checks can be done efficiently against bounded cluster sizes.
2) Replacement for single tx RBF(in a cluster size of up to two) against conflicts of up to cluster size two. `ImprovesFeerateDiagram` interface will have to change for this use-case, which is a future direction to solve certain pins and improve mempool incentive compatibility: https://delvingbitcoin.org/t/ephemeral-anchors-and-mev/383#diagram-checks-fix-this-3
And longer-term, this would be the proposed way we would compute incentive compatibility for all conflicts, post-cluster mempool.
ACKs for top commit:
sipa:
utACK 7295986778
glozow:
code review ACK 7295986778
murchandamus:
utACK 7295986778
ismaelsadeeq:
Re-ACK 7295986778
willcl-ark:
crACK 7295986778
sdaftuar:
ACK 7295986778
Tree-SHA512: 79593e5a087801c06f06cc8b73aa3e7b96ab938d3b90f5d229c4e4bfca887a77b447605c49aa5eb7ddcead85706c534ac5eb6146ae2396af678f4beaaa5bea8e
a8bfc3dea1 test: add coverage for bech32m in `wallet_keypool_topup` (brunoerg)
Pull request description:
0dcac51049 added coverage for all keypool addresses types in `wallet_keypool_topup` (4y ago). Now we have bech23m, so this PR adds it.
ACKs for top commit:
achow101:
ACK a8bfc3dea1
marcofleon:
ACK a8bfc3dea1. Definitely a more straightfoward addition to the test. Reviewed the code, built the PR branch and ran all functional tests without issues.
furszy:
utACK a8bfc3dea
Tree-SHA512: aa830b723a7a54b23744f9fb3cf5214452c4ffc8e3bbe0e8bd980bdf902e61c3dd2fd57361b82c5c0c5224aa0774158daf34b6b2188edda0a971f82111976051
This didn't work for two reasons:
1.) GetSnapshotCoinsDBPath() was used to retrieve the path.
This requires coins_views to exist, but the initialisation only happens later
(in CompleteChainstateInitialization) so the node hits an assert in
CCoinsViewDB& CoinsDB() and crashes.
2.) The snapshot was already activated, so it has the mempool attached.
Therefore, the mempool needs to be transferred back to the ibd
chainstate before deleting the snapshot chainstate.
76045bb9d6 depends: always set CMAKE_POSITION_INDEPENDENT_CODE=ON (fanquake)
d04623678c depends: always set CMAKE_INSTALL_LIBDIR=lib/ (fanquake)
Pull request description:
Set `CMAKE_INSTALL_LIBDIR=lib/` and `CMAKE_POSITION_INDEPENDENT_CODE=ON` globally in depends, rather than per-package. `CMAKE_INSTALL_LIBDIR=lib/` is needed to override the annoying [`GNUInstallDirs`](https://cmake.org/cmake/help/latest/module/GNUInstallDirs.html) `lib` vs `lib64` behaviour, and we always want PIC code. The PIC commit is the counterpart to the same Autotools change in #29488. I'm PRing these commits as I have a CMake branch building on top, and want to avoid adding the same workarounds to every package we are going to touch, but these can go in separately as the build should be tested for existing packages (i.e multiprocess).
ACKs for top commit:
hebasto:
re-ACK 76045bb9d6.
theuni:
utACK 76045bb9d6. Both changes make sense to me, and both can be overridden if needed, though I can't imagine we'd need to.
Tree-SHA512: 655a0b6b7ee5a5820f52e8e919ef03fc216d29f13f3904f72b64ce57436510e073c903039488d5740535c56e1f6221267229238c5231de5f8467d238fd562578
416b9d9427 correct '-dbcache' to '-prune' (Ben Westgate)
Pull request description:
This looks like a typo, '-prune' is doubled not the '-dbcache' by my understanding.
ACKs for top commit:
Sjors:
ACK 416b9d9427
fjahr:
ACK 416b9d9427
Tree-SHA512: d954a8976f6e43e0af9bc2ba89fec1a1ccc90483cc005af3d2810379bd6bea7dbd87766af71c2a45b3d751a3f682f0f67a4525ec350bf6163bc4bb88fab6e4cb
fa1146d01b lint: Fix COMMIT_RANGE issues (MarcoFalke)
Pull request description:
`COMMIT_RANGE` has problems on forks or local branches:
* When `LOCAL_BRANCH` is set, it assumes the presence of a `master` branch, and that the `master` branch is up-to-date. Both of which may be false. (See also discussion in https://github.com/bitcoin/bitcoin/pull/29274#discussion_r1504226422)
* When `COMMIT_RANGE` isn't set in `lint-git-commit-check.py`, and `--prev-commits` isn't set either, it has the same (broken) assumptions.
Fix all issues by simply assuming a merge commit exists. This allows to drop `LOCAL_BRANCH`. It also allows to drop `SKIP_EMPTY_NOT_A_PR`, because scripts will already skip an empty range. Finally, it allows to drop `--prev-commits n`, because one can simply say `COMMIT_RANGE='HEAD~n..HEAD'` to achieve the same.
ACKs for top commit:
Sjors:
tACK fa1146d01b
Tree-SHA512: f1477a38267fd4fdb8d396211a5d6bed5f418798c7edaba43487957aaf726cd45244ccf15187b3dd896d398fa1df3fe0a37323e49cf44d60a2018786ed41e5ba
11ee058ef5 tidy: remove C compiler check (fanquake)
c3a4ea1971 tidy: set CMAKE_CXX_STANDARD to 20 (fanquake)
5b690aeb15 tidy: remove terminfo TODO (fanquake)
24410e560a tidy: set minimum CMake to 3.22 (fanquake)
Pull request description:
See https://github.com/hebasto/bitcoin/pull/123 for the minimum version bump.
ACKs for top commit:
hebasto:
re-ACK 11ee058ef5.
Tree-SHA512: 94a508ea24bf17919961bbdbc2e9d17658858e179c3b2017d5932557af32530d9d6aab197453aa5444f5478c417129c5a8e39522ff82bafac0d5a6966c7246d3
Rather than potentially having to set this per-package, set it globally,
as this should always be what we want. Without doing this, changes in
later commits will have to add this per-package.
Similar to https://github.com/bitcoin/bitcoin/pull/29488, which is the
Autotools equivalent.
Rather than setting this per package, set it globally, as this is always
what we want. Without doing this, later commit will have to add the same
doc + change to more packages.
e037c4fe09 depends: always configure with --with-pic (fanquake)
Pull request description:
We currently do this sporadically. Not only amongst packages, but across OS's, i.e sometimes it's done for BSDs/Android, and sometimes not.
Configure with `--with-pic` globally instead. I think this generally makes more sense, and should not have any downsides.
See related discussion in https://github.com/bitcoin/bitcoin/pull/28846#discussion_r1399123100.
ACKs for top commit:
hebasto:
ACK e037c4fe09.
Tree-SHA512: efc743ff92f9f99f3ac16514e98363ad395c6f956cd4be7e785b5c573685baf7fcd68c51d6a705ee8761fc676eb045b7e61676595be0eb0f70f34e99174cddc0
The addpeeraddress calls can fail due to collisions. As we are using a
deteministic addrman, they won't fail with the current bucket/position
calculation. However, if the calculation is changed, they might collide
and fail silently causing tests using `seed_addrman()` to fail.
Assert that the addpeeraddress calls are successful.
824f47294a node: Use log levels in noui_ThreadSafeMessageBox (TheCharlatan)
ddc7872c08 node: Make translations of fatal errors consistent (TheCharlatan)
Pull request description:
The extra `bilingual_str` argument of the fatal error notifications and `node::AbortNode()` is often unused and when used usually contains the same string as the message argument. It also seems to be confusing, since it is not consistently used for errors requiring user action. For example some assumeutxo fatal errors require the user to do something, but are not translated.
So simplify the fatal error and abort node interfaces by only passing a translated string. This slightly changes the fatal errors displayed to the user.
ACKs for top commit:
stickies-v:
re-ACK 824f47294a
maflcko:
ACK 824f47294a🔎
achow101:
ACK 824f47294a
hebasto:
re-ACK 824f47294a.
Tree-SHA512: 2868ee7b045fe7f3ac582ce5039141b398480b7627734976201dafaaef7544b8461635a7292fee4a7f32ff1bfc26f9bd4d0c292dca424ba42fb7fc4483d7ce8d
99954f914f test: fix test to ensure hidden RPC is present in detailed help (stratospher)
0d01f6f0c6 test: remove unused mocktime in test_addpeeraddress (0xb10c)
6205466512 rpc: "addpeeraddress tried" return error on failure (0xb10c)
Pull request description:
When trying to add an address to the IP address manager tried table, it's first added to the new table and then moved to the tried table. Previously, adding a conflicting address to the address manager's tried table with test-only `addpeeraddress tried=true` RPC would return `{ "success": true }`. However, the address would not be added to the tried table, but would remain in the new table. This caused, e.g., issue #28964.
This is fixed by new returning `{ "success": false, "error": "..." }` for failed tried table additions. Since the address remaining in the new table can't be removed (the address manager interface does not support removing addresses at the moment and adding this seems to be a bigger effort), an error message is returned. This indicates to a user why the RPC failed and allows accounting for the extra address in the new table. This is done in the functional test for the `getrawaddrman` RPC.
Fixes#28964
ACKs for top commit:
achow101:
ACK 99954f914f
stratospher:
reACK 99954f9. 🚀
brunoerg:
utACK 99954f914f
Tree-SHA512: 2f1299410c0582ebc2071271ba789a8abed905f9a510821f77afbcf2a555ec31397578ea55cbcd162fb828be27afedd3246c7b13ad8883f2f745bb8e04364a76
032a597482 test: make p2p_handshake robust against timeoffset warnings (stickies-v)
Pull request description:
The new `p2p_handshake` test requires that limited nodes are not peered with when the node's system time exceeds ~ 24h of the node's chaintip timestamp, as per [`PeerManagerImpl::GetDesirableServiceFlags`](2ffaa92702/src/net_processing.cpp (L1717)).
By patching this test to modify the timestamp of the chaintip as opposed to mocking the node's system time, we make it resilient to future commits where the node raises a warning if it detects its system time is too much out of sync with its outbound peers.
Resolves a silent merge conflict in https://github.com/bitcoin/bitcoin/pull/29623, that is changing the warning behaviour when significant time differences with outbound peers are detected, [failing the test as it's currently in master](https://cirrus-ci.com/task/6553996884705280?logs=ci#L4666).
Considerations/alternatives I've thought of:
- could add `self.setup_clean_chain = True` to `self.set_test_params()`, to avoid creating a new tip with a (much) older date, but it doesn't seem to matter?
- could avoid using `setmocktime` altogether and instead use [`create_block`](2ffaa92702/test/functional/test_framework/blocktools.py (L68)) instead, but that seems like it'll be a lot more verbose and I don't think it's worth it?
Big thanks to theStack for his time in discussing this with me offline.
ACKs for top commit:
maflcko:
lgtm ACK 032a597482
theStack:
ACK 032a597482
brunoerg:
crACK 032a597482
BrandonOdiwuor:
Code Review ACK 032a597482
Tree-SHA512: 407564754a100bc9252f5737182de2e111993944ec9a0463a4a43195ce98cd1119de982c8fe5f7531ddb56603043812bf7bf2163a780d30b6df03a072c3308c3
fa4d98b3c8 Avoid divide-by-zero in header sync logs when NodeClock is behind (MarcoFalke)
fa58550317 refactor: Modernize header sync logs (MarcoFalke)
Pull request description:
The log may be confusing, when the NodeClock is behind the current header tip.
Fix it, by assuming the NodeClock is never behind the current header tip.
ACKs for top commit:
sipa:
utACK fa4d98b3c8
sr-gi:
tACK [fa4d98b](fa4d98b3c8)
achow101:
ACK fa4d98b3c8
tdb3:
ACK fa4d98b3c8
Tree-SHA512: 3c5aee4030af387695918c5238012c972ebf850b52e956b5f74590cd7fd4eff0b3e593d411e3eb2a0bb12294af8dc6fbe320f90e4c261399b65a404ff3c3cbd9
The test requires that limited nodes are not peered with when
the node's system time exceeds ~ 24h of the node's chaintip
timestamp, as per PeerManagerImpl::GetDesirableServiceFlags.
By patching this test to modify the timestamp of the chaintip as
opposed to mocking the node's system time, we make it resilient
to future commits where the node raises a warning if it detects
its system time is too much out of sync with its outbound peers.
See https://github.com/bitcoin/bitcoin/pull/29623
To enable Branch Target Identification Mechanism and Return
Address Signing by default at configure time use the
`--enable-standard-branch-protection` option.
This is equivalent to having `-mbranch-protection=standard` during
compilation. This can be explicitly disabled during compilation
by passing the `-mbranch-protection=none` option which turns off
all types of branch protections.
See:
https://gcc.gnu.org/install/specific.html#aarch64-x-x
The extra `bilingual_str` argument of the fatal error notifications and
`node::AbortNode()` is often unused and when used usually contains the
same string as the message argument. It also seems to be confusing,
since it is not consistently used for errors requiring user action. For
example some assumeutxo fatal errors require the user to do something,
but are not translated.
So simplify the fatal error and abort node interfaces by only passing a
translated string. This slightly changes the fatal errors displayed to
the user.
Also de-duplicate the abort error log since it is repeated in noui.cpp.
cf5faf73c9 guix: bump time-machine to dc4842797bfdc5f9f3f5f725bf189c2b68bd6b5a (fanquake)
Pull request description:
This includes a commit to fix building LLVM 17 on riscv64, see https://git.savannah.gnu.org/cgit/guix.git/commit/?id=4e26331a5ee87928a16888c36d51e270f0f10f90.
Followup to discussion in https://github.com/bitcoin/bitcoin/pull/28880#issuecomment-1843313196.
If you don't have riscv64 hardware, this can be tested with the following:
```bash
# observe failure when cross-compiling using our current time-machine
guix time-machine --commit=d5ca4d4fd713a9f7e17e074a1e37dda99bbb09fc -- build --target=riscv64-linux-gnu llvm
....
riscv64-linux-gnu-ld: CMakeFiles/dsymutil.dir/dsymutil.cpp.o: undefined reference to symbol '__atomic_fetch_and_1@@LIBATOMIC_1.0'
riscv64-linux-gnu-ld: /gnu/store/i4ga0pnr1b74bir2bjyp8mcrrbsvk7d3-gcc-cross-riscv64-linux-gnu-11.3.0-lib/riscv64-linux-gnu/lib/libatomic.so.1:
error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
# build success when using the new time-machine
guix time-machine --commit=dc4842797bfdc5f9f3f5f725bf189c2b68bd6b5a -- build --target=riscv64-linux-gnu llvm
....
grafting '/gnu/store/7y0j0y8jaz4mjx2nz0y42wdnxxjp6id6-llvm-17.0.6-opt-viewer' -> '/gnu/store/8xvahrrjscbprh6cjj0qp5bm9mm78wwa-llvm-17.0.6-opt-viewer'...
grafting '/gnu/store/bjhw648bz7ijd2p9hgzzdbw1q8hpagk8-llvm-17.0.6' -> '/gnu/store/x50qi8i2ywgpx6azv4k55ms0w5xjxxg5-llvm-17.0.6'...
successfully built /gnu/store/q9xvk8gzzvb4dxfzf6yi5164zd0d1vj2-llvm-17.0.6.drv
```
Also includes at least:
Linux Headers 6.1.67 -> 6.1.80
ACKs for top commit:
TheCharlatan:
ACK cf5faf73c9
hebasto:
ACK cf5faf73c9, tested on x86_64 hardware as described in the PR description.
Tree-SHA512: b49d4f90effeec666b12b5447a24c90315b82675cfc166bc1230ac173134bab6b277fc7e064bbb75e990275165b2b27d88e4ec1cdeea4750541ec6443cb50f41
f65b0f6401 index: Move last_locator_write_time and logging to end of threadsync loop (Fabian Jahr)
Pull request description:
In the index sync thread, when initializing an index for the first time, stop callng BaseIndex::Commit when m_best_block_index is null, to avoid a spurious "failed to commit" error from that function. This error started happening in commit 7878f97bf1 from https://github.com/bitcoin/bitcoin/pull/25494 and was reported by pstratem in https://github.com/bitcoin/bitcoin/pull/26903 with an alternate fix.
ACKs for top commit:
achow101:
ACK f65b0f6401
ryanofsky:
Code review ACK f65b0f6401. Just moved log "Syncing" log line since last commit to avoid having to call now() twice.
furszy:
ACK f65b0f6401
TheCharlatan:
ACK f65b0f6401
Tree-SHA512: afa8f05786318d36346d167ff53ea0b3bc8abdb0ad04465d199dc3eb91e9f837369e24fcb7e24b5757b02d698ec504e61da6ac365eaf006c874fc07a424a7e20
Behavior changes are:
- if a tx has a mempool conflict, the wallet will not attempt to
rebroadcast it
- if a txo is spent by a mempool-conflicted tx, that txo is no
longer considered spent
9d9a7458a2 assumeutxo: Remove BLOCK_ASSUMED_VALID flag (Ryan Ofsky)
ef174e9ed2 test: assumeutxo snapshot block CheckBlockIndex crash test (Ryan Ofsky)
0391458d76 test: assumeutxo stale block CheckBlockIndex crash test (Ryan Ofsky)
ef29c8b662 assumeutxo: Get rid of faked nTx and nChainTx values (Ryan Ofsky)
9b97d5bbf9 doc: Improve comments describing setBlockIndexCandidates checks (Ryan Ofsky)
0fd915ee6b validation: Check GuessVerificationProgress is not called with disconnected block (Ryan Ofsky)
63e8fc912c ci: add getchaintxstats ubsan suppressions (Ryan Ofsky)
f252e687ec assumeutxo test: Add RPC test for fake nTx and nChainTx values (Ryan Ofsky)
Pull request description:
The `PopulateAndValidateSnapshot` function introduced in f6e2da5fb7 from #19806 has been setting fake `nTx` and `nChainTx` values that can show up in RPC results (https://github.com/bitcoin/bitcoin/issues/29328) and make `CBlockIndex` state hard to reason about, because it is difficult to know whether the values are real or fake.
Revert to previous behavior of setting `nTx` and `nChainTx` to 0 when the values are unknown, instead of faking them. Also drop no-longer needed `BLOCK_ASSUMED_VALID` flag.
Dropping the faked values also fixes assert failures in the `CheckBlockIndex` `(pindex->nChainTx == pindex->nTx + prev_chain_tx)` check that could happen previously if forked or out-of-order blocks before the snapshot got submitted while the snapshot was being validated. The PR includes two commits adding tests for these failures and describing them in detail.
Compatibility note: This change could cause new `-checkblockindex` failures if a snapshot was loaded by a previous version of Bitcoin Core and not fully validated, because fake `nTx` values will have been saved to the block index. It would be pretty easy to avoid these failures by adding some compatibility code to `LoadBlockIndex` and changing `nTx` values from 1 to 0 when they are fake (when `(pindex->nStatus & BLOCK_VALID_MASK) < BLOCK_VALID_TRANSACTIONS`), but a little simpler not to worry about being compatible in this case.
ACKs for top commit:
Sjors:
re-ACK 9d9a7458a2
achow101:
ACK 9d9a7458a2
mzumsande:
Tested ACK 9d9a7458a2
maflcko:
ACK 9d9a7458a2🎯
Tree-SHA512: b1e1e2731ec36be30d5f5914042517219378fc31486674030c29d9c7488ed83fb60ba7095600f469dc32f0d8ba79c49ff7706303006507654e1762f26ee416e0
dfcef536d0 blockstorage: do not flush block to disk if it is already there (Matthew Zipkin)
Pull request description:
Closes https://github.com/bitcoin/bitcoin/issues/2039
When reindexing from flat-file block storage there is no need to write anything back to disk, since the block data is already there. This PR skips flushing to disk those blocks that already have a known position in the datastore. Skipping this means that users can write-protect the `blk` files on disk which may be useful for security or even safely sharing that data between multiple bitcoind instances.
`FindBlockPos()` may also flush the undo data file, but again this is skipped if the corresponding block position is known, like during the initial stage of a reindex when block data is being indexed. Once the block index is complete the validation mechanism will call `ConnectBlock()` which will save undo data at that time.
The call stack looks like this:
```
init()
ThreadImport() <-- process fReindex flag
LoadExternalBlockFile()
AcceptBlock()
SaveBlockToDisk()
FindBlockPos()
FlushBlockFile() <-- unnecessary if block is already on disk
```
A larger refactor of this part of the code was started by mzumsande here: https://github.com/mzumsande/bitcoin/tree/202207_refactor_findblockpos including this fix, reviewers can let me know if the changes should be combined.
ACKs for top commit:
sipa:
utACK dfcef536d0
mzumsande:
re-ACK dfcef536d0
achow101:
ACK dfcef536d0
furszy:
Rebase diff ACK dfcef53.
Tree-SHA512: 385c5ac1288b325135398d0ddd3ab788fa98cc0ca19bd2474c74039f2ce70d5088c1d1c9d4dd10aefcbd4c757767ec5805d07ba8cee9289a66f96e6f9eaa5279
99afb9d15a refactor: init, simplify index shutdown code (furszy)
0faafb57f8 index: decrease ThreadSync cs_main contention (furszy)
f1469eb454 index: cache last block filter header (furszy)
a6756ecdb2 index: blockfilter, decouple header lookup into its own function (furszy)
331f044e3b index: blockfilter, decouple Write into its own function (furszy)
bcbd7eb8d4 bench: basic block filter index initial sync (furszy)
Pull request description:
Work decoupled from #26966 per request.
The aim is to remove an unnecessary disk read operation that currently takes place with every new arriving block (or scanned block during background sync). Instead of reading the last filter header from disk merely to access its hash for constructing the next filter, this work caches it, occupying just 32 more bytes in memory.
Also, reduces `cs_main` lock contention during the index initial sync process. And, simplifies the indexes initialization and shutdown procedure.
Testing Note:
To compare the changes, added a pretty basic benchmark in the second commit. Alternatively, could also test the changes by timing the block filter sync from scratch on any network; start the node with `-blockfilterindex` and monitor the logs until the syncing process finish.
Local Benchmark Results:
*Master (c252a0fc0f):
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 132,042,516.60 | 7.57 | 0.3% | 7.79 | `BlockFilterIndexSync`
*PR (43a212cfdac6c64e82b601c664443d022f191520):
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 126,915,841.60 | 7.88 | 0.6% | 7.51 | `BlockFilterIndexSync`
ACKs for top commit:
Sjors:
re-ACK 99afb9d15a
achow101:
ACK 99afb9d15a
TheCharlatan:
Re-ACK 99afb9d15a
andrewtoth:
ACK 99afb9d15a
Tree-SHA512: 927daadd68f4ee1ca781a89519539b895f5185a76ebaf525fbc246ea8dcf40d44a82def00ac34b188640802844b312270067f1b33e65a2479e06be9169c616de
faecf3a7e6 ci: Bump msan to llvm-18 (MarcoFalke)
Pull request description:
Last one: https://github.com/bitcoin/bitcoin/pull/28476
ACKs for top commit:
fanquake:
ACK faecf3a7e6 - There is now a 18.1.2, but given it doesn't fix the instrumenting in libunwind, we don't need that here. I've tested that both jobs are now working on both arches.
Tree-SHA512: 489c0b343bdc732687131317a570f3efbb18a3f548736d739da90d1a1e784df1dbb208c2da8a2a7740f27f961a841c477487a14c4d59910368f651225f0779b2
2f23987849 test: p2p: check limited peers desirability (depending on best block depth) (Sebastian Falbesoner)
c4a67d396d test: p2p: check disconnect due to lack of desirable service flags (Sebastian Falbesoner)
405ac819af test: p2p: support disconnect waiting for `add_outbound_p2p_connection` (Sebastian Falbesoner)
Pull request description:
This PR adds missing test coverage for disconnecting peers which don't offer the desirable service flags in their VERSION message:
5f3a0574c4/src/net_processing.cpp (L3384-L3389)
This check is relevant for the connection types "outbound-full-relay", "block-relay-only" and "addr-fetch" (see `CNode::ExpectServicesFromConn(...)`). Feeler connections always disconnect, which is also tested here.
In lack of finding a proper file where this test would fit in, I created a new one. Happy to take suggestions there.
ACKs for top commit:
davidgumberg:
reACK 2f23987849
itornaza:
tested ACK 2f23987849
fjahr:
re-utACK 2f23987849
cbergqvist:
re ACK 2f23987849
stratospher:
tested ACK 2f23987. 🚀
Tree-SHA512: cf75d9d4379d0f34fa1e13152e6a8d93cd51b9573466ab3a2fec32dc3e1ac49b174bd1063cae558bc736b111c8a6e7058b1b57a496df56255221bf367d29eb5d
6e873df347 serfloat: improve/simplify tests (Pieter Wuille)
b45f1f5658 serfloat: do not test encode(bits)=bits anymore (Pieter Wuille)
Pull request description:
Closes#28941.
Our current tests for serfloat verify two distinct properties:
1. Whether they roundtrip `double`->`uint64_t`->`double` (excluding NaN values) on all systems.
2. Whether on systems with a typical floating point unit that encoding matches the hardware representation, as before v22.0, we would dump the hardware representation directly to disk and we wanted to retain compatibility with that.
#28941 seems to show that the second property doesn't always hold, but just for "subnormal" numbers (below $2^{-1021}$). Since we don't care about encoding these numbers, we could exclude such subnormal numbers from the hardware-identical representation test, but this PR goes further and just drops the second property entirely, as I don't think we care about edge-case compatibility with pre-v22.0 code for fee_estimates.dat (the only place it is used).
ACKs for top commit:
glozow:
ACK 6e873df347
fanquake:
ACK 6e873df347 - It's not as much of a priority, but I think we could still backport this.
Tree-SHA512: e18ceee0753a7ee7e999fdfa10b014dc5bb67b6ef79522a0f8c76b889adcfa785772fc26ed7559bcb5a09a9938e243bb54eedd9549bc59080a2c8090155e2267
current check to make sure that detailed help for hidden RPC
is displayed won't work because the assertion isn't sufficient.
Even if unknown RPCs are passed, RPC names would still be present
in node.help().
Drops the mocktime added in fa4c6836c9.
Setting the mocktime in test_addpeeraddress() isn't needed
anymore as it doesn't leak into test_getrawaddrman() anymore
(since 2cc8ca19f4).
test_getrawaddrman() clear's the addrman and sets it's own
mocktime.
When trying to add an address to the IP address manager tried table,
it's first added to the new table and then moved to the tried table.
Previously, adding a conflicting address to the address manager's
tried table with test-only `addpeeraddress tried=true` RPC would
return `{ "success": true }`. However, the address would not be added
to the tried table, but would remain in the new table. This caused,
e.g., issue 28964.
This is fixed by returning `{ "success": false, "error":
"failed-adding-to-tried" }` for failed tried table additions. Since
the address remaining in the new table can't be removed (the address
manager interface does not support removing addresses at the moment
and adding this seems to be a bigger effort), an error message is
returned. This indicates to a user why the RPC failed and allows
accounting for the extra address in the new table.
Also:
To check the number of addresses in each addrman table,
the addrman checks were re-run and the log output of this check
was asserted. Ideally, logs shouldn't be used as an interface
in automated tests. To avoid asserting the logs, use the getaddrmaninfo
and getrawaddrman RPCs (which weren't implemented when the test was added).
Removing the "getnodeaddress" calls would also remove the addrman checks
from the test, which could reduce the test coverage. To avoid this,
these are kept.
fae70ba00d ci: Better tidy errors (MarcoFalke)
Pull request description:
Currently tidy errors are not nice, because the user may have to scroll up to see them in a large block of text. See for example (before) https://github.com/bitcoin/bitcoin/runs/19670551485
Fix that by `tee`ing the output to a file and summarizing the errors in the end again. See for example (after): https://github.com/bitcoin/bitcoin/runs/22647850662
ACKs for top commit:
hebasto:
ACK fae70ba00d, logs with errors look cleaner.
TheCharlatan:
ACK fae70ba00d
Tree-SHA512: dcaea557fed40089409d16ce2cbaa8a9cfbf047f601d5daadfee0823b0eed7badc12d803addc0b7b6bb3f1eaf5c787fccb2488475d32c4efd80835f386f761dd
432a542e27 test: fix intermittent failures with test=addrman (Martin Zumsande)
Pull request description:
The `nKey` of the addrman is generated the first time the node is started with an empty `peers.dat`. Therefore, restarting a node or turning it off and on again won't make a previously non-deterministic addrman deterministic.
This could lead to intermittent failures in `feature_asmap.py` and `rpc_net.py`
Fixes#29634
ACKs for top commit:
kevkevinpal:
ACK [432a542](432a542e27)
stratospher:
Tested ACK 432a542e27.
brunoerg:
crACK 432a542e27
0xB10C:
ACK 432a542e27
Tree-SHA512: a8e284baeb0be2df7284b8a2792cb9edc9e2d5b877a3b29ab7277ffdb75b17efa58a4d42576441eb493cd518e7c5ffdb05597b27e42b5001cf1a80e78bb04c83
626f8e398e fuzz: actually test garbage >64b in p2p transport test (Pieter Wuille)
Pull request description:
This fixes an oversight from #28196: in the `p2p_transport_bidirectional_v2` fuzz test, when the desired garbage length is over 64 bytes, the code would actually use garbage length 0. Fix this.
ACKs for top commit:
instagibbs:
ACK 626f8e398e
brunoerg:
crACK 626f8e398e
Tree-SHA512: f6346367adb10464b6c9d20aef43625531d2a4d8110887ad03214b8c1907b83560f2dd5b5415e2180a40b4cd276d51881b32b60c740471b5c6bb218aa19848d8
38f70ba6ac RPC: Add maxfeerate and maxburnamount args to submitpackage (Greg Sanders)
Pull request description:
Resolves https://github.com/bitcoin/bitcoin/issues/28949
I couldn't manage to do it very cleanly outside of (sub)package evaluation itself, since it would change the current interface very heavily. Instead I threaded through the max fee argument and used that directly via ATMPArgs. From that perspective, this is somewhat a reversion from https://github.com/bitcoin/bitcoin/pull/19339. In a post-cluster mempool world, these checks could be consolidated to right after the given (ancestor) package is linearized/chunked, by just checking the feerate of the top chunk and rejecting the submission entirely if the top chunk is too high.
The implication here is that subpackages can be submitted to the mempool prior to hitting this new fee-based error condition.
ACKs for top commit:
ismaelsadeeq:
Re-ACK 38f70ba6ac👍🏾
glozow:
ACK 38f70ba6ac with some non-blocking nits
murchandamus:
LGTM, code review ACK 38f70ba6ac
Tree-SHA512: 38212aa9de25730944cee58b0806a3d37097e42719af8dd7de91ce86bb5d9770b6f7c37354bf418bd8ba571c52947da1dcdbb968bf429dd1dbdf8715315af18f
64722e4359 ci: Drop `--enable-c++20` option (Hennadii Stepanov)
Pull request description:
This option has ceased to exist since https://github.com/bitcoin/bitcoin/pull/28349.
ACKs for top commit:
maflcko:
ACK 64722e4359
Tree-SHA512: bd392c331f775605615e1b236682269b83a1e6363a4d3f09c4d8d54495cf3d22973a921ebf6b8a9f813ba6c024d3324761f3291aaf7f473995f5eaa4c195bc43
Add a test for a CheckBlockIndex crash that would happen before previous
"assumeutxo: Get rid of faked nTx and nChainTx values" commit.
The crash was an assert failure in the (pindex->nChainTx == pindex->nTx +
prev_chain_tx) check that would previously happen if the snapshot block was
submitted after loading the snapshot and downloading a few blocks after the
snapshot. In that case ReceivedBlockTransactions() previously would overwrite
the nChainTx value of the submitted snapshot block with a fake value based on
the previous block, so the (pindex->nChainTx == pindex->nTx + prev_chain_tx)
check would later fail on the first block after the snapshot. This test was
originally posted by Martin Zumsande <mzumsande@gmail.com> in
https://github.com/bitcoin/bitcoin/pull/29370#issuecomment-1974096225
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
Add a test for a CheckBlockIndex crash that would happen before previous
"assumeutxo: Get rid of faked nTx and nChainTx values" commit.
The crash was an assert failure in the (pindex->nChainTx == pindex->nTx +
prev_chain_tx) check that would previously happen if a snapshot was loaded, and
a block was submitted which forked from the chain before the snapshot block and
after the last downloaded background chain block. This block would not be
marked assumed-valid because it would not be an ancestor of the snapshot, and
it would have nTx set, nChainTx unset, and prev->nChainTx set with a fake
value, so the assert would fail. After the fix, prev->nChainTx is unset instead
of being set to a fake value, so the assert succeeds. This test was originally
posted by maflcko in
https://github.com/bitcoin/bitcoin/issues/29261#issuecomment-1918947945
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
The `PopulateAndValidateSnapshot` function introduced in
f6e2da5fb7 from #19806 has been setting fake
`nTx` and `nChainTx` values that can show up in RPC results (see #29328) and
make `CBlockIndex` state hard to reason about, because it is difficult to know
whether the values are real or fake.
Revert to previous behavior of setting `nTx` and `nChainTx` to 0 when the
values are unknown, instead of faking them.
This commit fixes at least two assert failures in the (pindex->nChainTx ==
pindex->nTx + prev_chain_tx) check that would happen previously. Tests for
these failures are added separately in the next two commits.
Compatibility note: This change could result in -checkblockindex failures if a
snapshot was loaded by a previous version of Bitcoin Core and not fully
validated, because fake nTx values will have been saved to the block index. It
would be pretty easy to avoid these failures by adding some compatibility code
to `LoadBlockIndex` and changing `nTx` values from 1 to 0 when they are fake
(when `(pindex->nStatus & BLOCK_VALID_MASK) < BLOCK_VALID_TRANSACTIONS`), but a
little simpler not to worry about being compatible in this case.
Add ubsan suppressions for integer overflows in the getchaintxstats RPC.
getchainstatstx line "int nTxDiff = pindex->nChainTx - past_block.nChainTx" can
trigger ubsan integer overflows when assumeutxo snapshots are loaded, from
subtracting unsigned values and assigning the result to a signed int.
The overflow behavior probably exists in current code but is hard to trigger
because it would require calling getchainstatstx at the right time with
specific parameters as background blocks are being downloaded. But the overflow
behavior becomes easier to trigger in the upcoming commit removing fake
nChainTx values, so a suppression needs to be added before then for CI to pass.
getchainstatstx should probably be improved separately in another PR to not
need this suppression, and handle edge cases and missing nChainTx values more
carefully.
636c9862cf ci: Bump `TIDY_LLVM_V` (Hennadii Stepanov)
Pull request description:
This PR switches to the latest [IWYU 0.22](https://github.com/include-what-you-use/include-what-you-use/releases/tag/0.22), which is compatible with Clang 18.
ACKs for top commit:
fanquake:
ACK 636c9862cf
Tree-SHA512: 78ce89244c5e487dd1be8b4bd2ca6f06d19b04b78289ebc21985110574053545dcce5eb622edf2bede2cf7bb58360170e976d30a4484a127d34dd17b1c604e9c
fa5844f06d Remove unused g++-10 workaround (MarcoFalke)
fa8409e760 build: Bump g++ minimum supported version to 11 (MarcoFalke)
Pull request description:
This drops support for vanilla Ubuntu Focal 20.04 and Debian (Oldstable) Bullseye, compiling from source. Users on those operating systems would have to stick with a pre-compiled release, a previous release branch of Bitcoin Core, upgrade their system, compile their own compiler, or use a non-vanilla PPA or package manager.
Otherwise, g++-11 is offered by common distributions:
* https://packages.ubuntu.com/jammy/g++ (`g++-11`)
* https://packages.debian.org/bookworm/g++ (`g++-12`)
* FreeBSD 12/13 ships with g++ 12
* CentOS-like 9 ships with g++ 11
* OpenSuse Tumbleweed ships with g++ 13 https://software.opensuse.org/package/gcc13-c++ (No idea about OpenSuse Leap)
ACKs for top commit:
TheCharlatan:
ACK fa5844f06d
fanquake:
ACK fa5844f06d
Tree-SHA512: fc72d3a53956a0a4a6475ebf56b5fce76c3c4c793ed8e774327cad2b0f307d2d1c8aeafe2a414a7eb51f8de6d4bb78d30b8f60bf6e383234079851e72015c6e3
This new function takes the populated sets of
direct and all conflicts computed in the current
mempool, assuming the replacements are a single
chunk, and computes a diagram check.
The diagram check only works against cluster
sizes of 2 or less, and fails if it encounters
a different topology.
Co-authored-by: Suhas Daftuar <sdaftuar@chaincode.com>
5555395c15 lint: Use git --no-pager to print any output in one go (MarcoFalke)
fa5729436c lint: Fix lint-whitespace issues (MarcoFalke)
Pull request description:
The lint check has many issues:
* It uses `COMMIT_RANGE`, which is brittle code, apparently making it harder to run the CI locally, or self-hosted. See https://github.com/bitcoin/bitcoin/pull/29274#discussion_r1457739319
* The result depends on `COMMIT_RANGE`, or the number of commits passed to the script, which can cause false negatives or false positives.
* It is based on the diff output, parsing it, and printing it again, which is brittle as well.
* The output does not include line number, making it harder to act on a lint error.
Fix all issues by removing the script and replacing it with a simple call to `git grep -I --line-number ...`.
ACKs for top commit:
TheCharlatan:
Re-ACK 5555395c15
Tree-SHA512: 71ea8b6382af064beb72fb17f21a0ae9e9238c97e7fa43c2ec353fd1dd73a7bbd696ba0f0a9f65d1eff7c86cbf6cc104a992cb5450a3d50f122955e835270065
We currently do this sporadically. Not only amongst packages, but across
OS's, i.e sometimes it's done for BSDs/Android, and sometimes not.
Configure with `--with-pic` globally instead. I think this generally
makes more sense, and should not have any downsides.
See related discussion in
https://github.com/bitcoin/bitcoin/pull/28846#discussion_r1399123100.
This includes a commit to fix building LLVM 17 on riscv64, see
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=4e26331a5ee87928a16888c36d51e270f0f10f90.
Followup to discussion in
https://github.com/bitcoin/bitcoin/pull/28880#issuecomment-1843313196.
If you don't have riscv64 hardware, this can be tested with the
following:
```bash
guix time-machine --commit=d5ca4d4fd713a9f7e17e074a1e37dda99bbb09fc -- build --target=riscv64-linux-gnu llvm
....
riscv64-linux-gnu-ld: CMakeFiles/dsymutil.dir/dsymutil.cpp.o: undefined reference to symbol '__atomic_fetch_and_1@@LIBATOMIC_1.0'
riscv64-linux-gnu-ld: /gnu/store/i4ga0pnr1b74bir2bjyp8mcrrbsvk7d3-gcc-cross-riscv64-linux-gnu-11.3.0-lib/riscv64-linux-gnu/lib/libatomic.so.1:
error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
guix time-machine --commit=dc4842797bfdc5f9f3f5f725bf189c2b68bd6b5a -- build --target=riscv64-linux-gnu llvm
....
grafting '/gnu/store/7y0j0y8jaz4mjx2nz0y42wdnxxjp6id6-llvm-17.0.6-opt-viewer' -> '/gnu/store/8xvahrrjscbprh6cjj0qp5bm9mm78wwa-llvm-17.0.6-opt-viewer'...
grafting '/gnu/store/bjhw648bz7ijd2p9hgzzdbw1q8hpagk8-llvm-17.0.6' -> '/gnu/store/x50qi8i2ywgpx6azv4k55ms0w5xjxxg5-llvm-17.0.6'...
successfully built /gnu/store/q9xvk8gzzvb4dxfzf6yi5164zd0d1vj2-llvm-17.0.6.drv
```
bf264e0598 test: check_mempool_result negative feerate (kevkevin)
Pull request description:
Adds test coverage in `mempool_accept.py` to check if a negative `maxfeerate` is input into `check_mempool_result`
Asserts "Amount out of range" error message and `-3` error code
Motivated by this [comment](https://github.com/bitcoin/bitcoin/pull/29434/files#r1491112250)
ACKs for top commit:
maflcko:
lgtm ACK bf264e0598
brunoerg:
nice, utACK bf264e0598
davidgumberg:
Looks great, ACK bf264e0598
Tree-SHA512: 58931b774cc887c616f2fd91af3ee65cc5db55acd8e2875c76de448c80bd4e020b057c5f4f85556431377f0d0e7553771fb285d1ec20cf64f64ec92a47776b78
10d56530e0 guix: temporarily disable powerpcle taget (fanquake)
001412a4d2 guix: use GCC 12.3.0 (fanquake)
ce54330cf6 ci: use Debian Bookworm (GCC 12) for ARM ci job (fanquake)
0da6451c58 ci: use Debian Bookworm (GCC 12) for win64 job (fanquake)
Pull request description:
Switch to using [GCC `12.3.0`](https://gcc.gnu.org/gcc-12/) to build release binaries.
Temporarily disables the `powerpc64le-linux-gnu` target due to non-determinism issues when building across `aarch64` and `x86_64`. Trying to fix the non-determinism was going to require trying to selectively disable optimization flags, which is already not ideal (and didn't fix all issues), and the migration to GCC 12 as our release compiler is now the blocker for multiple other (c++20 and similar) changes, so leaving this blocked on the `powerpc64le` binaries does not seem like a good tradeoff.
ACKs for top commit:
TheCharlatan:
ACK 10d56530e0
Tree-SHA512: 401bbaaf2b72c795a06a24875ffd666151b41bae8f45bda10526ff4f6b59782704246afc6585f6b849021cbff8a7b861961d139bffe45536aaaeb3952b72ae57
0831b54dfc test: simplify test_runner.py (tdb3)
Pull request description:
Implements the simplifications to test_runner.py proposed by sipa in PR #23995.
Remove the num_running variable as it can be implied by the length of the jobs list.
Remove the i variable as it can be implied by the length of the test_results list.
Instead of counting results to determine if finished, make the queue object itself
responsible (by looking at running jobs and jobs left).
ACKs for top commit:
mzumsande:
re-ACK 0831b54
davidgumberg:
reACK 0831b54dfc
marcofleon:
re-ACK 0831b54dfc
Tree-SHA512: e5473e68d49cd779b29d97635329283ae7195412cb1e92461675715ca7eedb6519a1a93ba28d40ca6f015d270f7bcd3e77cef279d9cd655155ab7805b49638f1
c70e4fc9a3 netbase: remove unnecessary log message (Matthew Zipkin)
Pull request description:
This is a follow-up to #27375 that removes a spammy non-debug-level log message we don't need.
See https://github.com/bitcoin/bitcoin/pull/27375#issuecomment-1994498888
ACKs for top commit:
fanquake:
ACK c70e4fc9a3 - thanks. Merging this now because it's swamping non-debug logs. i.e:
Tree-SHA512: 837682860abdf740fce5dc88c8599e066660cf16b4cab1473391eb51ad538ae52d236ecd3543df866e2a2165870397a8bf21ad9f5125ed8212a3fe207d615553
1ec6684b08 test: Add test for loadtxoutset when headers are not synced (Fabian Jahr)
2bc1ecfaa9 test: Remove unnecessary sync_blocks in assumeutxo tests (Fabian Jahr)
Pull request description:
It adds a test for the change to `loadtxoutset` made in #29345. Before that change the test doesn't fail right away but times out after 10 minutes.
Also removes a `sync_blocks` call that didn't seem to do anything valuable.
ACKs for top commit:
achow101:
ACK 1ec6684b08
pablomartin4btc:
tACK 1ec6684b08
BrandonOdiwuor:
ACK 1ec6684b08
theStack:
ACK 1ec6684b08
Tree-SHA512: 1337decdf91e4a4f7213fcf8ace1d705e5c1422e0ac3872a59b5be9c33e743850cb8f5f7474750a534952eefd5cfe43fe85a54efb9bc0e47515136a2903676e5
28287cfbe1 test: add script compression coverage for not-on-curve P2PK outputs (Sebastian Falbesoner)
Pull request description:
This PR adds unit test coverage for the script compression functions `{Compress,Decompress}Script` in the special case of uncompressed P2PK outputs (scriptPubKey: OP_PUSH65 <0x04 ....> OP_CHECKSIG) with [pubkeys that are not fully valid](44b05bf3fe/src/pubkey.cpp (L297-L302)), i.e. where the encoded point is not on the secp256k1 curve. For those outputs, script compression is not possible, as the y coordinate of the pubkey can't be recovered (see also call-site of `IsToPubKey`):
44b05bf3fe/src/compressor.cpp (L49-L50)
Likewise, for a compressed script of an uncompressed P2PK script (i.e. compression ids 4 and 5) where the x coordinate is not on the curve, decompression fails:
44b05bf3fe/src/compressor.cpp (L122-L129)
Note that the term "compression" is used here in two different meanings (though they are related), which might be a little confusing. The encoding of a pubkey can either be compressed (33-bytes with 0x02/0x03 prefixes) or uncompressed (65-bytes with 0x04 prefix). On the other hand there is also compression for whole output scripts, which is used for storing scriptPubKeys in the UTXO set in a compact way (and also for the `dumptxoutset` result, accordingly). P2PK output scripts with uncompressed pubkeys get compressed by storing only the x-coordinate and the sign as a prefix (0x04 = even, 0x05 = odd). Was diving deeper into the subject while working on https://github.com/bitcoin/bitcoin/pull/27432, where the script decompression of uncompressed P2PK needed special handling (see also https://github.com/bitcoin/bitcoin/issues/24628#issuecomment-1108798536).
Trivia: as of now (block 801066), there are 13 uncompressed P2PK outputs in the UTXO set with a pubkey not on the curve (which obviously means they are unspendable).
ACKs for top commit:
achow101:
ACK 28287cfbe1
tdb3:
ACK for 28287cfbe1.
cbergqvist:
ACK 28287cf!
marcofleon:
Nicely done, ACK 28287cfbe1. Built the PR branch, ran the unit and functional tests, everything passed.
Tree-SHA512: 777b6c3065654fbfa1ce94926f4cadb91a9ca9dc4dd4af6008ad77bd1da5416f156ad0dfa880d26faab2e168bf9b27e0a068abc9a2be2534d82bee61ee055c65
And thread the feerate value through ProcessNewPackage to
reject individual transactions that exceed the given
feerate. This allows subpackage processing, and is
compatible with future package RBF work.
07cd510ffe [refactor] consolidate invalid MempoolAcceptResult processing (glozow)
9353aa4066 [refactor] consolidate valid MempoolAcceptResult processing (glozow)
Pull request description:
Every time we try to `ProcessTransaction` (i.e. submit a tx to mempool), we use the result to update a few net processing data structures. For example, after a failure, the {wtxid, txid, both, neither} (depending on reason) should be cached in `m_recent_rejects` so we don't try to download/validate it again.
There are 2 current places and at least 1 future place where we need to process `MempoolAcceptResult`:
- In the `ProcessMessage` logic after receiving and validating a tx
- In `ProcessOrphanTx` where we retry orphans
- With #28970, after processing a package of transactions, we should do these updates for each tx in the package.
Consolidate this code so it isn't repeated in 2 places and so we can reuse it in a future PR.
ACKs for top commit:
instagibbs:
ACK 07cd510ffe
achow101:
ACK 07cd510ffe
dergoegge:
Code review ACK 07cd510ffe
TheCharlatan:
ACK 07cd510ffe
Tree-SHA512: c4e74cb65e4f52882fca52e6682efa5dcf1562d98418454e09be256ffd026caae642a90aa5b9cccaae214be240d6f4be9d87b516953b2ee69a655f16ea569ed9
567cec9a05 doc: add release notes and help text for unix sockets (Matthew Zipkin)
bfe5192891 test: cover UNIX sockets in feature_proxy.py (Matthew Zipkin)
c65c0d0163 init: allow UNIX socket path for -proxy and -onion (Matthew Zipkin)
c3bd43142e gui: accomodate unix socket Proxy in updateDefaultProxyNets() (Matthew Zipkin)
a88bf9dedd i2p: construct Session with Proxy instead of CService (Matthew Zipkin)
d9318a37ec net: split ConnectToSocket() from ConnectDirectly() for unix sockets (Matthew Zipkin)
ac2ecf3182 proxy: rename randomize_credentials to m_randomize_credentials (Matthew Zipkin)
a89c3f59dc netbase: extend Proxy class to wrap UNIX socket as well as TCP (Matthew Zipkin)
3a7d6548ef net: move CreateSock() calls from ConnectNode() to netbase methods (Matthew Zipkin)
74f568cb6f netbase: allow CreateSock() to create UNIX sockets if supported (Matthew Zipkin)
bae86c8d31 netbase: refactor CreateSock() to accept sa_family_t (Matthew Zipkin)
adb3a3e51d configure: test for unix domain sockets (Matthew Zipkin)
Pull request description:
Closes https://github.com/bitcoin/bitcoin/issues/27252
UNIX domain sockets are a mechanism for inter-process communication that are faster than local TCP ports (because there is no need for TCP overhead) and potentially more secure because access is managed by the filesystem instead of serving an open port on the system.
There has been work on [unix domain sockets before](https://github.com/bitcoin/bitcoin/pull/9979) but for now I just wanted to start on this single use-case which is enabling unix sockets from the client side, specifically connecting to a local Tor proxy (Tor can listen on unix sockets and even enforces strict curent-user-only access permission before binding) configured by `-onion=` or `-proxy=`
I copied the prefix `unix:` usage from Tor. With this patch built locally you can test with your own filesystem path (example):
`tor --SocksPort unix:/Users/matthewzipkin/torsocket/x`
`bitcoind -proxy=unix:/Users/matthewzipkin/torsocket/x`
Prep work for this feature includes:
- Moving where and how we create `sockaddr` and `Sock` to accommodate `AF_UNIX` without disturbing `CService`
- Expanding `Proxy` class to represent either a `CService` or a UNIX socket (by its file path)
Future work:
- Enable UNIX sockets for ZMQ (https://github.com/bitcoin/bitcoin/pull/27679)
- Enable UNIX sockets for I2P SAM proxy (some code is included in this PR but not tested or exposed to user options yet)
- Enable UNIX sockets on windows where supported
- Update Network Proxies dialog in GUI to support UNIX sockets
ACKs for top commit:
Sjors:
re-ACK 567cec9a05
tdb3:
re ACK for 567cec9a05.
achow101:
ACK 567cec9a05
vasild:
ACK 567cec9a05
Tree-SHA512: de81860e56d5de83217a18df4c35297732b4ad491e293a0153d2d02a0bde1d022700a1131279b187ef219651487537354b9d06d10fde56225500c7e257df92c1
Remove the num_running variable as it can be implied by the
length of the jobs list.
Remove the i variable as it can be implied by the length of the
test_results list.
Instead of counting results to determine if finished, make the
queue object itself responsible (by looking at running jobs and
jobs left).
Originally proposed by @sipa in PR #23995.
Co-authored-by: Pieter Wuille <pieter@wuille.net>
The nKey of the addrman is generated the first time the node is
started. Therefore, restarting a node or turning it off and on
again won't make a previously non-deterministic addrman
deterministic.
Co-authored-by: 0xb10c <b10c@b10c.me>
d0e6564240 log: Remove error() reference (Fabian Jahr)
Pull request description:
Mini-followup to #29236 that was just merged. Removes a reference to `error()` that was missed in a comment.
ACKs for top commit:
ryanofsky:
Code review ACK d0e6564240. Just dropped LogPrintf reference since last review
stickies-v:
ACK d0e6564240
Empact:
ACK d0e6564240
Tree-SHA512: 8abe4895951013c2ceca9a57743aacabaf8af831d07eee9ae8372c121c16e88b7226f0e537200c3464792e19ac7e03b57ba0be31f43add8802753972b0aefc48
e710cefd57 rest: read raw block in rest_block and deserialize for json (Andrew Toth)
95ce0783a6 rpc: read raw block in getblock and deserialize for verbosity > 0 (Andrew Toth)
0865ab8712 test: check more details on zmq raw block response (Andrew Toth)
38265cc14e zmq: read raw block with ReadRawBlockFromDisk (Andrew Toth)
da338aada7 blockstorage: check nPos in ReadRawBlockFromDisk before seeking back (Andrew Toth)
Pull request description:
For the `getblock` endpoint with `verbosity=0`, the `rest_block` REST endpoint for `bin` and `hex`, and zmq `NotifyBlock` we don't have to deserialize the block since we're just sending the raw data. This PR uses `ReadRawBlockFromDisk` instead of `ReadBlockFromDisk` to serve these requests, and only deserializes for `verbosity > 0` and `json` REST requests. See benchmarks in https://github.com/bitcoin/bitcoin/pull/26684.
Benchmarked using ApacheBench. Requesting block 750,000 in binary 10k times on a single core (set `-rest=1` in config):
`ab -n 10000 -c 1 "http://127.0.0.1:8332/rest/block/0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e.bin"`
On master, mean time 15ms.
On this branch, mean time 1ms.
For RPC
```
echo '{"jsonrpc": "1.0", "id": "curltest", "method": "getblock", "params": ["0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e", 0]}' > /tmp/data.json
ab -p /tmp/data.json -n 1000 -c 1 -A user:password "http://127.0.0.1:8332/"
```
On master, mean time 32ms
On this branch, mean time 13ms
ACKs for top commit:
achow101:
re-ACK e710cefd57
Tree-SHA512: 4cea13c7b563b2139d041b1fdcfdb793c8cc688654ae08db07e7ee6b875c5e582b8185db3ae603abbfb06d2164724f29205774620b48c493726b991999af289e
0a533613fb docs: add release notes for #27114 (brunoerg)
e6b8f19de9 test: add coverage for whitelisting manual connections (brunoerg)
c985eb854c test: add option to speed up tx relay/mempool sync (brunoerg)
66bc6e2d17 Accept "in" and "out" flags to -whitelist to allow whitelisting manual connections (Luke Dashjr)
8e06be347c net_processing: Move extra service flag into InitializeNode (Luke Dashjr)
9133fd69a5 net: Move `NetPermissionFlags::Implicit` verification to `AddWhitelistPermissionFlags` (Luke Dashjr)
2863d7dddb net: store `-whitelist{force}relay` values in `CConnman` (brunoerg)
Pull request description:
Revives #17167. It allows whitelisting manual connections. Fixes#9923
Since there are some PRs/issues around this topic, I'll list some motivations/comments for whitelisting outbound connections from them:
- Speed-up tx relay/mempool sync for testing purposes (my personal motivation for this) - In #26970, theStack pointed out that we whitelist peers to speed up tx relay for fast mempool synchronization, however, since it applies only for inbound connections and considering the topology `node0 <--- node1 <---- node2 <--- ... <-- nodeN`, if a tx is submitted from any node other than node0, the mempool synchronization can take quite long.
- https://github.com/bitcoin/bitcoin/pull/29058#issuecomment-1865155764 - "Before enabling -v2transport by default (which I'd image may happen after https://github.com/bitcoin/bitcoin/pull/24748) we could consider a way to force manual connections to be only-v1 or even only-v2 (disabling reconnect-with-v1). A possibility could be through a net permission flag, if https://github.com/bitcoin/bitcoin/pull/27114 makes it in."
- https://github.com/bitcoin/bitcoin/pull/17167#issuecomment-1168606032 - "This would allow us to use https://github.com/bitcoin/bitcoin/pull/25355 when making outgoing connections to all nodes, except to whitelisted ones for which we would use our persistent I2P address."
- Force-relay/mempool permissions for a node you intentionally connected to.
ACKs for top commit:
achow101:
ACK 0a533613fb
sr-gi:
re-ACK [0a53361](0a533613fb)
pinheadmz:
ACK 0a533613fb
Tree-SHA512: 97a79bb854110da04540897d2619eda409d829016aafdf1825ab5515334b0b42ef82f33cd41587af235b3af6ddcec3f2905ca038b5ab22e4c8a03d34f27aebe1
Note that for speed this commit also removes the proof of work and
signet signature checks before returning the block in getblock.
It is assumed if a block is stored it will be valid.
Note that for speed this commit also removes the proof of work and
signet signature checks before returning the block in getblock.
It is assumed if a block is stored it will be valid.
ReadRawBlockFromDisk assumes a non-null pos that has an nPos >= 8.
This simple check makes the function safer to call in the future,
so callers don't need to worry about causing UB if the pos is null.
Retain native GCC 10 toolchain for macOS, to prevent compile failures in
native tools (this will be removed entirely when we tansition to LLD).
Update the vmov-alignment patch, for changes in GCC 12.
1342a31f3a [functional test] sibling eviction (glozow)
5fbab37859 [unit test] sibling not returned from SingleV3Checks if 1p2c or 3gen (glozow)
170306728a [policy] sibling eviction for v3 transactions (glozow)
b5d15f764f [refactor] return pair from SingleV3Checks (glozow)
Pull request description:
When we receive a v3 transaction that would bust a mempool transaction's descendant limit, instead of rejecting the new tx, consider replacing the other descendant if it is much higher feerate (using existing RBF criteria to assess that it's more incentive compatible and to avoid DoS).
Delving post with more background and motivation: https://delvingbitcoin.org/t/sibling-eviction-for-v3-transactions/472
ACKs for top commit:
sdaftuar:
ACK 1342a31f3a
achow101:
ACK 1342a31f3a
instagibbs:
ACK 1342a31f3a
Tree-SHA512: dd957d49e51db78758f566c49bddc579b72478e371275c592d3d5ba097d20de47a6c81952045021b99d82a787f5b799baf16dd0ee0e6de90ba12e21e275352be
Only NextSyncBlock requires cs_main lock. The
other function calls like Commit or Rewind will
lock or not cs_main internally when they need it.
Avoiding keeping cs_main locked when Commit() or
Rewind() write data to disk.
115c283516 ci: add print of powershell version to win64 job (Max Edwards)
Pull request description:
Extraction of just printing powershell version from closed PR: https://github.com/bitcoin/bitcoin/pull/29581
See https://github.com/bitcoin/bitcoin/pull/29581#issuecomment-1984212990 for the cause of a CI failure which was a powershell update.
This PR will make it easier to notice in the future that PS has changed.
ACKs for top commit:
hebasto:
ACK 115c283516. We still use PowerShell in some steps of the "Win64 native" CI job.
Tree-SHA512: 4c7ba9df4f0a98491120326f05e877a995f43a387fe9bbd193549b32f5a4488f85f83e472c9277db457110a7deda04f08832fe6e8129aff4b0b7278be23d4e35
acc06bc91f ci, macos: Use `--break-system-packages` with Homebrew's python (Hennadii Stepanov)
ae5f72027f ci: Add workaround for Homebrew's python link error (Hennadii Stepanov)
Pull request description:
Homebrew [promoted](https://github.com/Homebrew/homebrew-core/pull/150390) `python@3.12` to the default `python3`. Now, our "macOS native" CI job is facing the following issues:
1. Installing `qt@5` [requires](https://github.com/bitcoin/bitcoin/actions/runs/8216848118/job/22471875454#step:4:51) re-installing `python@3.12`:
```
==> Fetching dependencies for qt@5: readline, python@3.12 and gettext
```
2. Re-installing `python@3.12` [fails](https://github.com/bitcoin/bitcoin/actions/runs/8216848118/job/22471875454#step:4:127) due to symbolic link conflicts on macOS `x86_64`:
```
==> Pouring python@3.12--3.12.2_1.ventura.bottle.tar.gz
Error: The `brew link` step did not complete successfully
```
3. Homebrew's `python@3.12` is marked as externally managed (according to PEP 668), necessitating different approaches for installing Python packages.
This pull request resolves all the issues mentioned above.
ACKs for top commit:
m3dwards:
reACK acc06bc91f to get the CI passing again.
Tree-SHA512: 82cf72bff328f1e0725342431ac14ad4bae7a758186d97db6c7a558e4b661dcbf3fabe536978e26e709c5f6f7f5c11aac46642634c6685f1291592d8d825ad87
fa39151394 refactor: Remove unused error() (MarcoFalke)
fad0335517 scripted-diff: Replace error() with LogError() (MarcoFalke)
fa808fb749 refactor: Make error() return type void (MarcoFalke)
fa1d624348 scripted-diff: return error(...); ==> error(...); return false; (MarcoFalke)
fa9a5e80ab refactor: Add missing {} around error() calls (MarcoFalke)
Pull request description:
`error(...)` has many issues:
* It is often used in the context of `return error(...)`, implying that it has a "fancy" type, creating confusion with `util::Result/Error`
* `-logsourcelocations` does not work with it, because it will pretend the error happened inside of `logging.h`
* The log line contains `ERROR: `, as opposed to `[error]`, like for other errors logged with `LogError`.
Fix all issues by removing it.
ACKs for top commit:
fjahr:
re-utACK fa39151394
stickies-v:
re-ACK fa39151394, no changes since 4a903741b0
ryanofsky:
Code review ACK fa39151394. Just rebase since last review
Tree-SHA512: ec5bb502ab0d3733fdb14a8a00762638fce0417afd8dd6294ae0d485ce2b7ca5b1efeb50fc2cd7467f6c652e4ed3e99b0f283b08aeca04bbfb7ea4f2c95d283a
8aff3fd292 depends: don't use -h with touch on OpenBSD (fanquake)
Pull request description:
Should fix#29447.
ACKs for top commit:
theStack:
Tested ACK 8aff3fd292
hebasto:
ACK 8aff3fd292, tested on OpenBSD 7.1 by running the following commands twice and observing the same output:
Tree-SHA512: c054f80d347600617b21d5a7051315d43ebf858088a28f9b4bd43515f16f957d8033857a194f50556a6f0c67a8afbc2a50e143a477fbb4ef2d36e6365976b82f
2cc8ca19f4 [test] Use deterministic addrman in addrman info tests (stratospher)
a897866109 [test] Restart a node with empty addrman (stratospher)
71c19915c0 [test] Use deterministic addrman in addpeeraddress test (stratospher)
7b868e6b67 Revert "test: avoid non-determinism in asmap-addrman test" (stratospher)
69e091f3e1 [init] Create deterministic addrman in tests using -test=addrman (stratospher)
be25ac3092 [init] Remove -addrmantest command line arg (stratospher)
802e6e128b [init] Add new command line arg for use only in functional tests (stratospher)
Pull request description:
An address is placed in a `[bucket,position]` in the addrman table (new table or tried table) using the `addpeeraddress` RPC. This `[bucket,position]` is calculated using `nKey`(and other metrics) for the addrman which is chosen randomly during every run.
Supposing there are 2 addresses to be placed in an addrman table. During every test run, a different `[bucket,position]` would be calculated for each address.These calculated `[bucket,position]` could even be the same for the 2 addresses in some test runs and result in collisions in the addrman. We wouldn't be able to predict when the collisions are going to happen because we can't predict the `nKey` value which is chosen at random. This can cause flaky tests.
Because of these non deterministic collisions, we are limited in what we can do to test addrman functionality. Currently in our tests don't add a second address to prevent these collisions from happening - we only keep 1 address in the new table and 1 address in the tried table. See https://github.com/bitcoin/bitcoin/pull/26988#discussion_r1091145647, https://github.com/bitcoin/bitcoin/pull/23084, [#22831(comment)](https://github.com/bitcoin/bitcoin/pull/22831/files#r708302639).
This PR lets us create a deterministic addrman with fixed `nKey` so that we can know the `[bucket,position]` collisions beforehand, safely add more addresses in an addrman table and write more extensive tests.
ACKs for top commit:
amitiuttarwar:
ACK 2cc8ca19f4
achow101:
ACK 2cc8ca19f4
0xB10C:
ACK 2cc8ca19f4
mzumsande:
Code Review ACK 2cc8ca19f4
Tree-SHA512: 8acd9bdfe7de1eb44d22373bf13533d8ecf602df966fdd5b8b78afcd8cc35a286c95d2712f67a89473a0d68dded7d38f5599f6e4bf95a6589475444545bfb189
Adds a new boolean parameter `wait_for_disconnect` to the
`add_outbound_p2p_connection` method. If set, the node under
test is checked to disconnect immediately after receiving the
version message (same logic as for feeler connections).
ecc036c5d6 ci: add --v2transport to an existing CI job (Martin Zumsande)
3a25a575f0 test: ignore --v2transport for older versions instead of asserting (Martin Zumsande)
547aacff08 test: add -v1transport option and use it in test_runner (Martin Zumsande)
Pull request description:
This suggests a strategy to run the functional tests with both v1 and v2 transport in the CI.
**Status Quo:**
There is both the global `--v2transport` option for the `test_runner.py` (not enabled by default), plus the possibility to specify `--v2transport` for particular tests, which is used for a handful of tests. Currently, when running `test_runner.py --v2transport`, these tests are run twice with the same `--v2transport` configuration, as has been noted in https://github.com/bitcoin/bitcoin/pull/29358#discussion_r1485626063, which is wasteful.
**Suggested Change:**
Fix this by adding a `--v1transport` option and using it in `test_runner.py`, so that irrespective of the global `--v2transport` flag, the tests that run twice use v1 in one run and v2 in the other.
Also add `--v2transport` to one CI task (`multiprocess, i686, DEBUG`).
This means, that for each CI task, the majority of functional tests will run once using the global `--v2transport` option if provided, while a few selected tests will always run two times, once with `v1` and once with `v2`.
**Rationale:**
A simpler alternative would have been to remove all test-specific `--v2transport` commands from `test_runner.py` and just enable `--v2transport` option for a few CI tasks. I didn't do that because it would have meant that v2 would never be running in the CI for some platforms, and also be run a lot less locally by users and devs (who would have to actively enable the `--v2transport` option).
ACKs for top commit:
tdb3:
ACK for ecc036c5d6.
achow101:
ACK ecc036c5d6
stratospher:
ACK ecc036c.
vasild:
ACK ecc036c5d6
Tree-SHA512: 375b2293d49991f2fbd8e1bb646c0034004a09cee36063bc32996b721323eb19a43d7b2f36b3f9a3fdca846d74f48d8f1387565c03ef5d34b3481d2a0fe1d328
a951dba3a9 wallet: default wallet migration, modify inconvenient backup filename (furszy)
Pull request description:
Fixes#29584
On default legacy wallets, the backup filename starts with an "-" due to the wallet name being empty. This is inconvenient for systems who treat what follows the initial "-" character as flags.
Note:
As the user can freely set the wallet name to anything, we could also guard the backup filename against other inconvenient characters in the future (we need to be careful here, because the wallet name could also be a path).
ACKs for top commit:
achow101:
ACK a951dba3a9
brunoerg:
utACK a951dba3a9
vasild:
ACK a951dba3a9
Tree-SHA512: 6347bb12cfb50526a4baad96e4f1df9d82b493f79f0a0f7e0a1c8335a86a1e8e147c7b7f95cec6ede6f4507506a7eaf7972bd35131a2d5ed4cbbf38d94f0a9ca
This fixes the log output when -logsourcelocations is used.
Also, instead of 'ERROR:', the log will now say '[error]', like other
errors logged with LogError.
-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended 's! error\("([^"]+)"! LogError("\1\\n"!g' $( git grep -l ' error(' ./src/ )
-END VERIFY SCRIPT-
This is needed for the next commit.
-BEGIN VERIFY SCRIPT-
# Separate sed invocations to replace one-line, and two-line error(...) calls
sed -i --regexp-extended 's!( +)return (error\(.*\);)!\1\2\n\1return false;!g' $( git grep -l 'return error(' )
sed -i --null-data --regexp-extended 's!( +)return (error\([^\n]*\n[^\n]*\);)!\1\2\n\1return false;!g' $( git grep -l 'return error(' )
-END VERIFY SCRIPT-
c5b5843d8f test: avoid requesting blocks beyond limited peer threshold (furszy)
2f6a05512f p2p: sync from limited peer, only request blocks below threshold (furszy)
73127722a2 refactor: Make FindNextBlocks friendlier (furszy)
Pull request description:
Even when the node believes it has IBD completed, need to avoid
requesting historical blocks from network-limited peers.
Otherwise, the limited peer will disconnect right away.
The simplest scenario could be a node that gets synced, drops
connections, and stays inactive for a while. Then, once it re-connects
(IBD stays completed), the node tries to fetch all the missing blocks
from any peer, getting disconnected by the limited ones.
Note:
Can verify the behavior by cherry-picking the test commit alone on
master. It will fail there.
ACKs for top commit:
achow101:
ACK c5b5843d8f
vasild:
ACK c5b5843d8f
mzumsande:
Code Review ACK c5b5843d8f
pinheadmz:
ACK c5b5843d8f
Tree-SHA512: 9e550698bc6e63cc587b2b988a87d0ab555a8fa188c91c3f33287f8201d77c28b373331845356ad86f17bb21c15950b6466bc1cafd0ce8139d70364cb71c2ad2
a3badf75f6 tests: Provide more helpful assert_equal errors (Anthony Towns)
Pull request description:
In the functional tests, we often compare dicts with assert_equal, but the output makes it very hard to tell exactly which entry in the dicts don't match when there are a lot of entries and only minor differences. Change the output to make it clearer.
ACKs for top commit:
achow101:
ACK a3badf75f6
vasild:
ACK a3badf75f6
brunoerg:
utACK a3badf75f6
josibake:
ACK a3badf75f6
BrandonOdiwuor:
Code Review ACK a3badf75f6
Tree-SHA512: 1d4b4a3b2e2e28ab09f10b41b04b52b37f64e0d8a54e2306f37de0c3eb3299a7ad4ba225b9efa67057a75e90d008a17385c810a32c9b212d240be280c2dcf2e5
Deduplicate code that exists in both tx processing and ProcessOrphanTx.
Additionally, this can be reused in a function that handles multiple
MempoolAcceptResults from package submission.
Deduplicate code that exists in both tx processing and ProcessOrphanTx.
Additionally, this can be reused in a function that handles multiple
MempoolAcceptResults from package submission.
d27e2d87b9 test: test_bitcoin: allow -testdatadir=<datadir> (Larry Ruane)
Pull request description:
This backward-compatible change would help with code review, testing, and debugging. When `test_bitcoin` runs, it creates a working or data directory within `/tmp/test_common_Bitcoin\ Core/`, named as a long random (hex) string.
This small patch does three things:
- If the (new) argument `-testdatadir=<datadir>` is given, use `<datadir>/test_temp/<test-name>/datadir` as the working directory
- When the test starts, remove `<datadir>/test_temp/<test-name>/datadir` if it exists from an earlier run (currently, it's presumed not to exist due to the long random string)
- Don't delete the working directory at the end of the test if a custom data directory is being used
Example usage, which will remove, create, use `/somewhere/test_temp/getarg_tests/boolarg`, and leave it afterward:
```
$ test_bitcoin --run_test=getarg_tests/boolarg -- -testdatadir=/somewhere
Running 1 test case...
Test directory (will not be deleted): "/somewhere/test_temp/getarg_tests/boolarg/datadir"
*** No errors detected
$ ls -l /somewhere/test_temp/getarg_tests/boolarg/datadir
total 8
drwxrwxr-x 2 larry larry 4096 Feb 22 10:28 blocks
-rw-rw-r-- 1 larry larry 1273 Feb 22 10:28 debug.log
```
(A relative pathname also works.)
This change affects only `test_bitcoin`; it could also be applied to `test_bitcoin-qt` but that's slightly more involved so I'm skipping that for now.
The rationale for this change is that, when running the test using the debugger, it's often useful to watch `debug.log` as the test runs and inspect some of the other files (I've looked at the generated `blknnnn.dat` files for example). Currently, that requires figuring out where the test's working directory is since it changes on every test run. Tests can be run with `-printtoconsole=1` to show debug logging to the terminal, but it's nice to keep `debug.log` continuously open in an editor, for example.
Even if not using a debugger, it's sometimes helpful to see `debug.log` and other artifacts after the test completes.
Similar functionality is already possible with the functional tests using the `--tmpdir=` and `--nocleanup` arguments.
ACKs for top commit:
davidgumberg:
ACK d27e2d87b9
tdb3:
re-ACK for d27e2d87b9
achow101:
ACK d27e2d87b9
cbergqvist:
ACK d27e2d87b95b7982c05b4c88e463cc9626ab9f0a! (Already did some testing with `fs::remove()` to make sure it was compatible with the `util::Lock/UnlockDirectory` implementation).
marcofleon:
ACK d27e2d87b9. I ran all the tests with my previous open file limit and no errors were detected. Also ran some individual tests with no, relative, and absolute paths and everything looks good.
furszy:
ACK d27e2d8
Tree-SHA512: a8f535f34a48b6699cb440f97f5562ec643f3bfba4ea685768980b871fc8b6e1135f70fc05dbe19aa2c8bacb1ddeaff212d63473605a7422ff76332b3a6b1f68
Adds a new boolean parameter `expect_success` to the
`add_p2p_connection` method. If set, the node under
test doesn't wait for connection to be established
and is useful for testing scenarios when disconnection
is expected.
Without this parameter, intermittent test failures can happen
if the disconnection happens before wait_until for is_connected
is hit inside `add_p2p_connection`.
Co-Authored-By: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
Early key response test is a special kind of test which requires
modified v2 handshake functions. More such tests can be added
where v2 handshake functions send incorrect garbage terminator,
excess garbage bytes etc.. Hence, rename p2p_v2_earlykey.py to a
general test file name - p2p_v2_misbehaving.py.
random_bitflip function (used in signature tests prior to this
commit) can be used in p2p_v2_misbehaving test to generate wrong
garbage terminator, wrong garbage bytes etc..
So, move the function to util.
5b358cdd1a i2p: log connection was refused due to arbitrary port (brunoerg)
Pull request description:
For I2P, we do not try to connect if port is != 0. However, we do not have anything that indicates it or any error when trying to connect with port != 0. This PR adds a log for it. Also, it improves the functional test. With this log we can ensure the reason we won't connect is the port, in the current test, we cannot ensure it.
ACKs for top commit:
jonatack:
ACK 5b358cdd1a
epiccurious:
re-ACK 5b358cdd1a.
achow101:
ACK 5b358cdd1a
kristapsk:
re-ACK 5b358cdd1a
vasild:
ACK 5b358cdd1a
Tree-SHA512: 027245afa771c9295fff0bfd17c251dca4a9f4c739e5773922de3c030a65ef05d96291edcbdeeaa50ba3add61f75f28d8c00be503e03fc33d3491d1956fc549f
d5228efb53 kernel: Remove dependency on CScheduler (TheCharlatan)
06069b3913 scripted-diff: Rename MainSignals to ValidationSignals (TheCharlatan)
0d6d2b650d scripted-diff: Rename SingleThreadedSchedulerClient to SerialTaskRunner (TheCharlatan)
4abde2c4e3 [refactor] Make MainSignals RAII styled (TheCharlatan)
84f5c135b8 refactor: De-globalize g_signals (TheCharlatan)
473dd4b97a [refactor] Prepare for g_signals de-globalization (TheCharlatan)
3fba3d5dee [refactor] Make signals optional in mempool and chainman (TheCharlatan)
Pull request description:
By defining a virtual interface class for the scheduler client, users of the kernel can now define their own event consuming infrastructure, without having to spawn threads or rely on the scheduler design.
Removing `CScheduler` also allows removing the thread and exception modules from the kernel library.
To make the `CMainSignals` class easier to use from a kernel library perspective, remove its global instantiation and adopt RAII practices.
Renames `CMainSignals` to `ValidationSignals`, which more accurately describes its purpose and scope.
Also make the `ValidationSignals` in the `ChainstateManager` and CTxMemPool` optional. This could be useful in the future for using or testing these classes without having to instantiate any form of signal handling.
---
This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587). It improves the kernel API and removes two modules from the kernel library.
ACKs for top commit:
maflcko:
re-ACK d5228efb53🌄
ryanofsky:
Code review ACK d5228efb53. Just comment change since last review.
vasild:
ACK d5228efb53
furszy:
diff ACK d5228ef
Tree-SHA512: e93a5f10eb6182effb84bb981859a7ce750e466efd8171045d8d9e7fe46e4065631d9f6f533c5967c4d34c9bb7d7a67e9f4593bd4c5b30cd7b3bbad7be7b331b
On default legacy wallets, the backup filename starts with an "-" due
to the wallet name being empty. This is inconvenient for systems who
treat what follows the initial "-" character as flags.
8e17f00a5d build, msvc: Cleanup `bitcoin_config.h.in` (Hennadii Stepanov)
Pull request description:
This PR mirrors changes from https://github.com/bitcoin/bitcoin/pull/29263 into the MSVC build system.
ACKs for top commit:
fanquake:
ACK 8e17f00a5d
Tree-SHA512: b8e5cca015ff112c2969a60436524e97007ff2c559b3c12425d0549af694b16248311cc3e7c33f798bc095a679933641496836bb846eee6a2a377956ef53f56e
738a53720e [fuzz] Apply fuzz env (suppressions, etc.) when fetching harness list (dergoegge)
Pull request description:
The fuzz test runner does not add the UBSan suppressions when fetching the harness list. We can observe this in CI as lots of UBSan errors prior to the harnesses actually executing: https://api.cirrus-ci.com/v1/task/5678606140047360/logs/ci.log
```
+ test/fuzz/test_runner.py -j10 -l DEBUG /ci_container_base/ci/scratch/qa-assets/fuzz_seed_corpus/ --empty_min_time=60
/usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/string_view:578:38: runtime error: unsigned integer overflow: 12 - 23 cannot be represented in type 'size_type' (aka 'unsigned long')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/string_view:578:38 in
/usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/string_view:578:33: runtime error: implicit conversion from type 'size_type' (aka 'unsigned long') of value 18446744073709551605 (64-bit, unsigned) to type 'const difference_type' (aka 'const long') changed the value to -11 (64-bit, signed)
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /usr/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/string_view:578:33 in
crypto/sha256.cpp:75:57: runtime error: left shift of 1359893119 by 26 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior crypto/sha256.cpp:75:57 in
crypto/sha256.cpp:75:79: runtime error: left shift of 1359893119 by 21 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior crypto/sha256.cpp:75:79 in
crypto/sha256.cpp:75:101: runtime error: left shift of 1359893119 by 7 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior crypto/sha256.cpp:75:101 in
crypto/sha256.cpp:82:47: runtime error: unsigned integer overflow: 2968370640 + 2483695512 cannot be represented in type 'uint32_t' (aka 'unsigned int')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior crypto/sha256.cpp:82:47 in
crypto/sha256.cpp:74:57: runtime error: left shift of 1779033703 by 30 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior crypto/sha256.cpp:74:57 in
crypto/sha256.cpp:74:79: runtime error: left shift of 1779033703 by 19 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior crypto/sha256.cpp:74:79 in
crypto/sha256.cpp:74:101: runtime error: left shift of 1779033703 by 10 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior crypto/sha256.cpp:74:101 in
crypto/sha256.cpp:83:29: runtime error: unsigned integer overflow: 3458249854 + 980412007 cannot be represented in type 'uint32_t' (aka 'unsigned int')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior crypto/sha256.cpp:83:29 in
crypto/sha256.cpp:82:21: runtime error: unsigned integer overflow: 528734635 + 4228187651 cannot be represented in type 'uint32_t' (aka 'unsigned int')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior crypto/sha256.cpp:82:21 in
crypto/sha256.cpp:84:7: runtime error: unsigned integer overflow: 1013904242 + 3720769133 cannot be represented in type 'uint32_t' (aka 'unsigned int')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior crypto/sha256.cpp:84:7 in
crypto/sha256.cpp:85:12: runtime error: unsigned integer overflow: 3720769133 + 2654153126 cannot be represented in type 'uint32_t' (aka 'unsigned int')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior crypto/sha256.cpp:85:12 in
crypto/sha256.cpp:82:33: runtime error: unsigned integer overflow: 4165002546 + 1259303586 cannot be represented in type 'uint32_t' (aka 'unsigned int')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior crypto/sha256.cpp:82:33 in
crypto/sha256.cpp:125:50: runtime error: unsigned integer overflow: 3835390401 + 1367343104 cannot be represented in type 'unsigned int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior crypto/sha256.cpp:125:50 in
crypto/sha256.cpp:77:58: runtime error: left shift of 1367343104 by 15 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
...
```
To fix this we simply apply the usual fuzz env variables (that apply the suppressions) when fetching the harness list as well.
ACKs for top commit:
ismaelsadeeq:
Tested ACK 738a53720e
fanquake:
ACK 738a53720e
Tree-SHA512: befebaeb4ee5f2eddca67fc6dc69e997c6a250ea54844e5e6e93d1f6a13be49364a3ace31eaa942b02dcf73612af29ec4ace86c9eb7567b92f6f5dc3ea14dc11
Refactors the helper functions in the test to provide more
fine-grained control over RPC requests and responses than
the usual AuthProxy methods.
No change in test behavior, the same RPC requests are made.
Specifying this argument overrides the path location for test_bitcoin;
it becomes <datadir>/test_common_Bitcoin Core/<testname>/datadir. Also,
this directory isn't removed after the test completes. This can make it
easier for developers to study the results of a test (see the state of
the data directory after the test runs), and also (for example) have an
editor open on debug.log to monitor it across multiple test runs instead
of having to re-open a different pathname each time.
Example usage (note the "--" is needed):
test_bitcoin --run_test=getarg_tests/boolarg -- \
-testdatadir=/somewhere/mydatadir
This will create (if necessary) and use the data directory:
/somewhere/mydatadir/test_common_Bitcoin Core/getarg_tests/boolarg/datadir
Co-authored-by: furszy <mfurszy@protonmail.com>
8b7630cb1f build: ignore deprecated-declaration warnings in objc++ macOS code (fanquake)
bd8f0354ba build: Add missed definition for `AM_OBJCXXFLAGS` (Hennadii Stepanov)
Pull request description:
These come from GUI code, and haven't/aren't being fixed, see discussion in https://github.com/bitcoin-core/gui/issues/112. For now, just ignore them entirely. Note that this only applies to ObjCXX code, so will not hide any relevant warnings coming from C or CXX code (and they would be unlikely in any case).
Alternative to https://github.com/bitcoin/bitcoin/pull/29362 (which disables all compiler warnings, for macOS builds in the CI). This PR includes one commit from that PR.
Relevant output:
```bash
qt/macnotificationhandler.mm:27:9: warning: 'NSUserNotification' is deprecated: first deprecated in macOS 11.0 - All NSUserNotifications API should be replaced with UserNotifications.frameworks API [-Wdeprecated-declarations]
NSUserNotification* userNotification = [[NSUserNotification alloc] init];
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSUserNotification.h:24:12: note: 'NSUserNotification' has been explicitly marked deprecated here
@interface NSUserNotification : NSObject <NSCopying> {
^
qt/macnotificationhandler.mm:27:50: warning: 'NSUserNotification' is deprecated: first deprecated in macOS 11.0 - All NSUserNotifications API should be replaced with UserNotifications.frameworks API [-Wdeprecated-declarations]
NSUserNotification* userNotification = [[NSUserNotification alloc] init];
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSUserNotification.h:24:12: note: 'NSUserNotification' has been explicitly marked deprecated here
@interface NSUserNotification : NSObject <NSCopying> {
^
qt/macnotificationhandler.mm:30:11: warning: 'NSUserNotificationCenter' is deprecated: first deprecated in macOS 11.0 - All NSUserNotifications API should be replaced with UserNotifications.frameworks API [-Wdeprecated-declarations]
[[NSUserNotificationCenter defaultUserNotificationCenter] deliverNotification: userNotification];
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSUserNotification.h:118:12: note: 'NSUserNotificationCenter' has been explicitly marked deprecated here
@interface NSUserNotificationCenter : NSObject {
^
3 warnings generated.
```
ACKs for top commit:
hebasto:
re-ACK 8b7630cb1f.
Tree-SHA512: 2f1fec97d9aa46aa23989d9fb283fc574dff9dc8f44847bb273b8fcf942f56f64c6d93dfcd7af8fbb52bf152e0fe76818118ce416d8cec5de852c32b6697a243
These come from GUI code, and haven't/aren't being fixed, see discussion
in https://github.com/bitcoin-core/gui/issues/112. For now, just ignore
them entirely. Note that this only applies to ObjCXX code, so will not
hide any relevant warnings coming from C or CXX code (and they would be
unlikely in any case).
Alternative to #29362, which disables all compiler warnings, for macOS
builds in the CI.
Relevant output:
```bash
qt/macnotificationhandler.mm:27:9: warning: 'NSUserNotification' is deprecated: first deprecated in macOS 11.0 - All NSUserNotifications API should be replaced with UserNotifications.frameworks API [-Wdeprecated-declarations]
NSUserNotification* userNotification = [[NSUserNotification alloc] init];
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSUserNotification.h:24:12: note: 'NSUserNotification' has been explicitly marked deprecated here
@interface NSUserNotification : NSObject <NSCopying> {
^
qt/macnotificationhandler.mm:27:50: warning: 'NSUserNotification' is deprecated: first deprecated in macOS 11.0 - All NSUserNotifications API should be replaced with UserNotifications.frameworks API [-Wdeprecated-declarations]
NSUserNotification* userNotification = [[NSUserNotification alloc] init];
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSUserNotification.h:24:12: note: 'NSUserNotification' has been explicitly marked deprecated here
@interface NSUserNotification : NSObject <NSCopying> {
^
qt/macnotificationhandler.mm:30:11: warning: 'NSUserNotificationCenter' is deprecated: first deprecated in macOS 11.0 - All NSUserNotifications API should be replaced with UserNotifications.frameworks API [-Wdeprecated-declarations]
[[NSUserNotificationCenter defaultUserNotificationCenter] deliverNotification: userNotification];
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSUserNotification.h:118:12: note: 'NSUserNotificationCenter' has been explicitly marked deprecated here
@interface NSUserNotificationCenter : NSObject {
^
3 warnings generated.
```
7ab54397f8 seeds: Update testnet seeds (Ava Chow)
34a233b6d8 seeds: Update mainnet seeds (Ava Chow)
9701bc435f makeseeds: Check i2p seeds too (Ava Chow)
a8ec9eede4 makeseeds: Update PATTERN_AGENT (Ava Chow)
Pull request description:
The ipv4 and ipv6 seeds are updated from sipa's crawler, as outlined in contrib/seeds/README.md. The onion and i2p seeds are pulled from my node's addrman using `getrawaddrman` and then a connection was made to each node to retrieve the current service flags, block height, and user agent string before filtering through makeseeds.py. The CJDNS nodes were not updated as my node is not connected to that network.
makeseeds.py is also updated for more recent user agent strings as well as being able to handle i2p addresses.
Also updated the testnet seeds.
ACKs for top commit:
fanquake:
ACK 7ab54397f8
Tree-SHA512: 5edba63d51116e5d9a8ae23561ba5a311f4df88c555c60b2d7a6066e63f8cdfd256be7dac9acea4b370879d0d3c3a4b55328c15de4284b5f0d86e6cac2e5ba9b
53ffd5a410 docs: Fix broken reference to CI setup in test/lint/README.md (naiyoma)
Pull request description:
The current [reference](https://github.com/bitcoin/bitcoin/blob/master/test/ci/lint/04_install.sh
) for CI setup in /test/lint#readme returns a 404.
ACKs for top commit:
fanquake:
ACK 53ffd5a410
Tree-SHA512: 813c19a145f09e7da11963598b70dc438acba784eb722e509d6af59dc3af8f5da97628c454bed2b03cc919689603e070796de2db8d784d9162ae34e7b85a77d9
The code in `sha_x86_shani.cpp` uses the `_mm_blend_epi16` function from
the SSE4.1 instruction set. However, it is possible that SHA-NI is
enabled even when SSE4.1 is disabled.
This changes avoid compilation errors in such a condition.
This change uses the `_mm_blend_epi16` SSE4.1 function used in our code
and fixes false-positive cases, for example, when CXXFLAGS="-mno-sse4.1"
provided.
e67ab174c9 test: fix flaky wallet_send functional test (Max Edwards)
3c49e69670 test: fix weight estimates in functional tests (Max Edwards)
Pull request description:
Fixes: https://github.com/bitcoin/bitcoin/issues/25164
The wallet_send functional test has been flaky due to a slightly overestimated weight calculation. This PR makes the weight calculation more accurate, although occasionally, due to how ECDSA signatures can be different lengths it might slightly over estimate. The assertion in the test can handle this slight variation and so should continue passing.
Update:
Because the signature can be shorter that is used in the weight estimation or the final transaction the estimate could be both slightly smaller or slightly larger.
ACKs for top commit:
achow101:
ACK e67ab174c9
S3RK:
Code review ACK e67ab174c9
Tree-SHA512: 3bf73b355309dce860fa1520afb8461e94268e4bcf0e92a8273c279b41b058c44472cf59daafa15a515529b50bd665b5d498bbe4d934f2315dbe810a05bc73f9
A test suite should not leave any artifacts except for those explicitly
expected.
This change is easy to review with `git diff --ignore-all-space`
command.
If `setOrganizationName()` and `setApplicationName()` has not been
previously called, the `QSettings` object will not be able to read or
write any settings.
6e5eda83ad doc: remove rel note fragments (fanquake)
Pull request description:
These have been added to https://github.com/bitcoin-core/bitcoin-devwiki/wiki/27.0-Release-Notes-Draft, where they can be improved further.
ACKs for top commit:
stickies-v:
ACK 6e5eda83ad
Tree-SHA512: 66874fe9a64ac3a99a15a602ac68ae0a9e08f52a0fe732e48136b245c2127ed04e8217f86c44459696b03b01532a926ab8d41101c6e670902c1fc31e583b4dc9
6962c66b4a build, msvc: Do not compile redundant sources (Hennadii Stepanov)
Pull request description:
The `test\util\setup_common.cpp` and `wallet\test\util.cpp` sources are already compiled and included in the `libtest_util` library, which is linked to the `test_bitcoin-qt.exe` binary. This PR follows the same logic as `Makefile.qttest.include`.
Useful for comparing project files across the master branch and the developing [cmake-staging](https://github.com/hebasto/bitcoin/tree/cmake-staging) branch.
ACKs for top commit:
sipsorcery:
utACK 6962c66b4a.
Tree-SHA512: 5c40f52f3c7df3fff994fb136d4b2779ade3857fa14cf167d3f8600f28e821294e3779ebd4f4ab10ad57bdc8e952f99f6eb211e746a986ec24e26c7d1a74c04f
b7aa717cdd refactor: gui, simplify boost signals disconnection (furszy)
f3a612f901 gui: guard accessing a nullptr 'clientModel' (furszy)
Pull request description:
Fixing #800.
During shutdown, already queue events dispatched from the backend such
'numConnectionsChanged' and 0networkActiveChanged' could try to access
the clientModel object, which might not exist because we manually delete
it inside 'BitcoinApplication::requestShutdown()'.
This happen because boost does not clears the queued events when they arise
concurrently with the signal disconnection (see https://www.boost.org/doc/libs/1_55_0/doc/html/signals2/thread-safety.html).
From the docs:
1) "Note that since we unlock the connection's mutex before executing its associated slot, it is possible a slot will still be executing after it has been disconnected by a [connection::disconnect](https://www.boost.org/doc/libs/1_55_0/doc/html/boost/signals2/connection.html#idp89761576-bb)(), if the disconnect was called concurrently with signal invocation."
2) "The fact that concurrent signal invocations use the same combiner object means you need to insure any custom combiner you write is thread-safe"
So, we need to guard `clientModel` before accessing it at the handler side.
ACKs for top commit:
hebasto:
re-ACK b7aa717cdd
Tree-SHA512: f1a21d69248628f6a13556a9438c9e4ea9f0a3678aab09ddfe836e78e4eee405a6730d37d39f1445068ada3a110b655b619cf0e090fc2d0cdf99bed061364aeb
The `test\util\setup_common.cpp` and `wallet\test\util.cpp` sources are
already compiled and included in the `libtest_util` library, which is
linked to the `test_bitcoin-qt.exe` binary. This change follows the same
logic as `Makefile.qttest.include`.
86b7f28d6c serialization: use internal endian conversion functions (Cory Fields)
432b18ca8d serialization: detect byteswap builtins without autoconf tests (Cory Fields)
297367b3bb crypto: replace CountBits with std::bit_width (Cory Fields)
52f9bba889 crypto: replace non-standard CLZ builtins with c++20's bit_width (Cory Fields)
Pull request description:
This replaces #28674, #29036, and #29057. Now ready for testing and review.
Replaces platform-specific endian and byteswap functions. This is especially useful for kernel, as it means that our deep serialization code no longer requires bitcoin-config.h.
I apologize for the size of the last commit, but it's hard to avoid making those changes at once.
All platforms now use our internal functions rather than libc or platform-specific ones, with the exception of MSVC.
Sadly, benchmarking showed that not all compilers are capable of detecting and optimizing byteswap functions, so compiler builtins are instead used where possible. However, they're now detected via macros rather than autoconf checks.
This[ matches how libc++ implements std::byteswap for c++23](https://github.com/llvm/llvm-project/blob/main/libcxx/include/__bit/byteswap.h#L26).
I suggest we move/rename `compat/endian.h`, but I left that out of this PR to avoid bikeshedding.
#29057 pointed out some irregularities in benchmarks. After messing with various compilers and configs for a few weeks with these changes, I'm of the opinion that we can't win on every platform every time, so we should take the code that makes sense going forward. That said, if any real-world slowdowns are caused here, we should obviously investigate.
ACKs for top commit:
maflcko:
ACK 86b7f28d6c📘
fanquake:
ACK 86b7f28d6c - we can finish pruning out the __builtin_clz* checks/usage once the minisketch code has been updated. This is more good cleanup pre-CMake & for the kernal.
Tree-SHA512: 715a32ec190c70505ffbce70bfe81fc7b6aa33e376b60292e801f60cf17025aabfcab4e8c53ebb2e28ffc5cf4c20b74fe3dd8548371ad772085c13aec8b7970e
efb70cd645 doc: correct function name in AssumeUTXO design docs (jrakibi)
Pull request description:
Corrected the function name from `CompleteSnapshotValidation()` to `MaybeCompleteSnapshotValidation()` in the assumeutxo design documentation.
This change ensures that the documentation accurately reflects the actual function name used in the code
ACKs for top commit:
Empact:
ACK efb70cd645
Tree-SHA512: 68b9be3ba710d91a2a955189e227f86b46ccb6a2a13c345d46f276cec6ff12b77ebf9814c4bcb00db7c17e221510e4a2e71175c78a6faf0e0e3159c761bc9b94
Rather than asserting that the exact fees are the same, check the fee rate rounded to nearest interger. This will account for small differences in fees caused by variability in ECDSA signature lengths.
Updates the weight estimate to be more accurate by removing byte buffers and calculating the length of the count of scriptWitnesses rather than just using the count itself.
f8a06f7a02 doc: remove references to disable-asm option now that it's gone (Cory Fields)
376f0f6d07 build: remove confusing and inconsistent disable-asm option (Cory Fields)
Pull request description:
1. It didn't actually disable asm usage in our code. Regardless of the setting, asm is used in random.cpp and support/cleanse.cpp.
2. The value wasn't forwarded to libsecp as a user might have reasonably expected.
3. We now have the DISABLE_OPTIMIZED_SHA256 define which is what disable-asm actually did in practice.
If there is any desire, we can hook DISABLE_OPTIMIZED_SHA256 up to a new configure option that actually does what it says.
Additionally, this is one of the last (THE last?) remaining uses of autoconf defines in our crypto code. As such it seems like low-hanging fruit.
ACKs for top commit:
fanquake:
ACK f8a06f7a02
Tree-SHA512: 4a99c2130225acbe9dc7399ed572a04ca155cbfa3eef8178a632ba533017d264691e6482cceb1d8f9c5d768619d99a2466dea4b82b27b18b872bceae91b92fbb
a8c3454ba1 test: speedup bip324_cipher.py unit test (Sebastian Falbesoner)
Pull request description:
Executing the unit tests for the bip324_cipher.py module currently takes quite long (>60 seconds on my older notebook). Most time here is spent in empty plaintext/ciphertext encryption/decryption loops in `test_fschacha20poly1305aead`:
9eeee7caa3/test/functional/test_framework/crypto/bip324_cipher.py (L193-L194)9eeee7caa3/test/functional/test_framework/crypto/bip324_cipher.py (L198-L199)
Their sole purpose is increasing the FSChaCha20Poly1305 packet counter in order to trigger rekeying, i.e. the actual encryption/decryption is not relevant, as the result is thrown away. This commit speeds up the tests by supporting to pass "None" as plaintext/ciphertext, indicating to the routines that no actual encryption/decryption should be done.
The approach here is a bit hacky, a cleaner alternative would probably be to introduce a special `seek`/`skip_packets` method and not touch the encrypt/decrypt routines, but that seemed overkill to me only for speeding up a unit test. Open for suggestions.
master branch:
```
$ python3 -m unittest ./test/functional/test_framework/crypto/bip324_cipher.py
..
----------------------------------------------------------------------
Ran 2 tests in 64.658s
```
PR branch:
```
$ python3 -m unittest ./test/functional/test_framework/crypto/bip324_cipher.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.822s
```
ACKs for top commit:
delta1:
Concept ACK a8c3454
epiccurious:
Tested ACK a8c3454ba1.
achow101:
ACK a8c3454ba1
marcofleon:
ACK a8c3454ba1. The comments at the top of `bip324_cipher.py` specify that this should only be used for testing, so I think this optimization makes sense in that context.
cbergqvist:
ACK a8c3454!
stratospher:
ACK a8c3454. I think it's worth it because of the significant speedup in the unit test.
Tree-SHA512: 737dd805a850be6e035aa3c6d9e2c5b5b5e89ddc564f84a045c37e0238fef6419912de7c902139b64914abdd647c649fe02a694f1a5e1741d7d4459c041caccc
6ee3997d03 test: removes unnecessary check from validation_tests (Sergi Delgado Segura)
Pull request description:
An unnecessary check was added to the block mutation tests in #29412 where IsBlockMutated is returning true for the invalid reasons: we try to check mutation via transaction duplication, but the merkle root is not updated before the check, therefore the check fails because the provided root and the computed root differ, but not because the block contains the same transaction twice.
Notice that a proper check to test the duplication case is added a few lines later, so this check is just meaningless and can be removed. Check https://github.com/bitcoin/bitcoin/pull/29412#discussion_r1506490281 for context.
ACKs for top commit:
maflcko:
ACK 6ee3997d03
dergoegge:
utACK 6ee3997d03
BrandonOdiwuor:
utACK 6ee3997d03
Tree-SHA512: e4627668091dda5f589e4c15edac39dc84aabc9b34b8f7fadbf512beb7111d5477e1b69567a34b4a657e48ba66dfb864db5ff37c9bbe3ff24cd32931b2dd89e6
1. It didn't actually disable asm usage in our code. Regardless of the setting,
asm is used in random.cpp and support/cleanse.cpp.
2. The value wasn't forwarded to libsecp as a user might have reasonably
expected.
3. We now have the DISABLE_OPTIMIZED_SHA256 define which is what disable-asm
actually did in practice.
If there is any desire, we can hook DISABLE_OPTIMIZED_SHA256 up to a new
configure option that actually does what it says.
e073f1dfda test: make sure keypool sizes do not change on `getrawchangeaddress`/`getnewaddress` failures (UdjinM6)
367bb7a80c wallet: Avoid updating `ReserveDestination::nIndex` when `GetReservedDestination` fails (UdjinM6)
Pull request description:
I think the expected behaviour of `getrawchangeaddress` and `getnewaddress` RPCs is that their failure should not affect keypool in any way. At least that's how legacy wallets work, you can confirm this behaviour by running `wallet_keypool.py --legacy-wallet` on master with e073f1dfda applied on top. However running `wallet_keypool.py --descriptors` on the same commit results in the following failure:
```
File "/path/to/bitcoin/test/functional/test_framework/test_framework.py", line 131, in main
self.run_test()
File "/path/to/bitcoin/test/functional/wallet_keypool.py", line 114, in run_test
assert_equal(kp_size_before, kp_size_after)
File "/path/to/bitcoin/test/functional/test_framework/util.py", line 57, in assert_equal
raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
AssertionError: not([18, 24] == [19, 24])
```
This happens because we pass `nIndex` (which is a class member) into `GetReservedDestination` and since it's passed by reference we get an updated value back, so `nIndex` won't be equal `-1` anymore, no matter if the function failed or succeeded. This means that `ReturnDestination` (called by dtor of `ReserveDestination`) will try to return something we did not actually reserve.
The fix is to simply use a temporary variable instead of a class member and only update `nIndex` when `op_address` actually has value, basically do it the same way we do for other class members (`address` and `fInternal`) already.
ACKs for top commit:
achow101:
ACK e073f1dfda
josibake:
ACK e073f1dfda
Tree-SHA512: 1128288a60dd4d8f306ef6f7ac66cdfeae3c9cc35c66ecada2d78fa61ac759f2a757b70fc3976ba8b5081200942b58dfabc184c01ccf911af40ba8c145344651
0487f91a20 test: Fix intermittent failure in rpc_net.py --v2transport (stratospher)
Pull request description:
Fixes#29508.
Make sure that v2 handshake is complete before comparing getpeerinfo outputs so that `transport_protocol_type` isn't stuck at 'detecting'.
This is done by adding a wait_until statement till `transport_protocol_type = v2` so that bitcoind waits until the v2 handshake is complete. (on the python side, this is ensured by default since `wait_for_handshake = True` inside `add_p2p_connection()`)
ACKs for top commit:
Sjors:
ACK 0487f91a20
mzumsande:
Code Review ACK 0487f91a20
achow101:
ACK 0487f91a20
vasild:
ACK 0487f91a20
Tree-SHA512: 44dd646a61cd38da243f527df7321e22d1821c2b090be43673027746098caf450c6671708ed731ba257952df6b5886e64c9c2f9686a82f6ef0f25780b7a87d3d
This option beats the --v2transport option and is meant to be used in
test_runner.py.
It applies these to a few tests that are particulary interesting
in terms of the transport type.
This ensures that these tests arei always run with both v1 and v2, irrespective of
whether the global --v2transport test_runner option is set or not.
An unnecessary check was added to the block mutation tests
in #29412 where IsBlockMutated is returning true for the invalid
reasons: we try to check mutation via transaction duplication,
but the merkle root is not updated before the check, therefore
the check fails because the provided root and the computed root
differ, but not because the block contains the same transaction twice.
The check is meaningless so it can be removed.
In the functional tests, we often compare dicts with assert_equal, but the
output makes it very hard to tell exactly which entry in the dicts don't
match when there are a lot of entries and only minor differences. Change
the output to make it clearer.
Make sure that v2 handshake is complete before comparing getpeerinfo
outputs so that `transport_protocol_type` isn't stuck at 'detecting'.
- on the python side, this is ensured by default
`wait_for_handshake = True` inside `add_p2p_connection()`.
- on the c++ side, add a wait_until statement till
`transport_protocol_type = v2` so that v2 handshake is complete.
Co-Authored-By: Martin Zumsande <mzumsande@gmail.com>
d8087adc7e [test] IsBlockMutated unit tests (dergoegge)
1ed2c98297 Add transaction_identifier::size to allow Span conversion (dergoegge)
1ec6bbeb8d [validation] Cache merkle root and witness commitment checks (dergoegge)
5bf4f5ba32 [test] Add regression test for #27608 (dergoegge)
49257c0304 [net processing] Don't process mutated blocks (dergoegge)
2d8495e080 [validation] Merkle root malleation should be caught by IsBlockMutated (dergoegge)
66abce1d98 [validation] Introduce IsBlockMutated (dergoegge)
e7669e1343 [refactor] Cleanup merkle root checks (dergoegge)
95bddb930a [validation] Isolate merkle root checks (dergoegge)
Pull request description:
This PR proposes to check for mutated blocks early as a defense-in-depth mitigation against attacks leveraging mutated blocks.
We introduce `IsBlockMutated` which catches all known forms of block malleation and use it to do an early mutation check whenever we receive a `block` message.
We have observed attacks that abused mutated blocks in the past, which could have been prevented by simply not processing mutated blocks (e.g. https://github.com/bitcoin/bitcoin/pull/27608 for which a regression test is included in this PR).
ACKs for top commit:
achow101:
ACK d8087adc7e
maflcko:
ACK d8087adc7e🏄
fjahr:
Code review ACK d8087adc7e
sr-gi:
Code review ACK d8087adc7e
Tree-SHA512: 618ff4ea7f168e10f07504d3651290efbb1bb2ab3b838ffff3527c028caf6c52dedad18d04d3dbc627977479710930e200f2dfae18a08f627efe7e64a57e535f
1484998b6b ci: print python version on win64 native job (Max Edwards)
Pull request description:
Adds python version output to the Win64 Native CI job on Github Actions. Also clarifies that one of the versions already printed is the VCToolsVersion.
Before:

After:

Should the individual python test runners print the python version instead or also?
ACKs for top commit:
hebasto:
ACK 1484998b6b.
Tree-SHA512: 6d084ff4a667156fa8797450de83bbcf596ddd3b2fa8ec04c1ca9a532a6fec716817b66da34db4ea0184bd802ef613e2b8f6142be9a511c5397785cfbfede0c3
51bc1c7126 test: Remove Windows-specific code from `system_tests/run_command` (Hennadii Stepanov)
Pull request description:
The removed code has been dead since https://github.com/bitcoin/bitcoin/pull/28967.
Required as a precondition for replacing Boost.Process with [cpp-subprocess](https://github.com/bitcoin/bitcoin/pull/28981) to make diff for this code meaningful and reviewable.
The plan is to reintroduce Windows-specific code in this test simultaneously with enabling Windows support in cpp-subprocess.
ACKs for top commit:
Sjors:
utACK 51bc1c7126
theStack:
Code-review ACK 51bc1c7126
Tree-SHA512: 0e3875c4dc20564332555633daf2227223b10dc3d052557635eced2734575d1e0252fb19e46ea6e6c47a15c51c345f70b6d437e33435abcd0e4fcf29edb50887
ad7584d8b6 serialization: replace char-is-int8_t autoconf detection with c++20 concept (Cory Fields)
Pull request description:
Doesn't depend on #29263, but it's really only relevant after that one's merged.
This removes the only remaining autoconf macro in our serialization code (after #29263), so it can now be used trivially and safely out-of-tree.
~Our code does not currently contain any concepts, but couldn't find any discussion or docs about avoiding them. I guess we'll see if this blows up our c-i.~
Edit: Ignore this. ajtowns pointed out that we're already using a few concepts.
This was introduced in #13580. Please check my logic on this as I'm unable to test on a SmartOS system. Even better would be a confirmation from someone who can build there.
ACKs for top commit:
Empact:
Code review ACK ad7584d8b6
Tree-SHA512: 1faf65c900700efb1cf3092c607a2230321b393cb2f029fbfb94bc8e50df1dabd7a9e4b91e3b34f0d2f3471aaf18ee7e56d91869db5c5f4bae84da95443e1120
b052b2d1f2 build: remove -Wdocumentation conditional (fanquake)
Pull request description:
Now that `--enable-suppress-external-warnings` is on by default, we can drop it. CIs are all already building with this flag.
ACKs for top commit:
Empact:
Code review ACK b052b2d1f2
theuni:
utACK b052b2d1f2
Tree-SHA512: 8b55f366dfeece082090fb87de67d8811967f4c89987a346431b2deb73c3c94401b59ec98bb1cbf790e18894f3d4c4aebb57cbc5fbf931c1046bf40239bc7a58
During shutdown, already queue events dispatched from the backend such
'numConnectionsChanged' and 'networkActiveChanged' could try to access
the clientModel object, which might not exist because we manually delete
it inside 'BitcoinApplication::requestShutdown()'.
Turn the `std::vector` to `std::array` because it is cheaper and
allows us to have the number of the messages as a compile time
constant: `ALL_NET_MESSAGE_TYPES.size()` which can be used in
future code to build other `std::array`s with that size.
These replace our platform-specific mess in favor of c++20 endian detection
via std::endian and internal byteswap functions when necessary.
They no longer rely on autoconf detection.
Rather than a complicated set of tests to decide which bswap functions to
use, always prefer the compiler built-ins when available.
These builtins and fallbacks can all be removed once we're using c++23, which
adds std::byteswap.
This code has been dead since https://github.com/bitcoin/bitcoin/pull/28967.
Required as a precondition for replacing Boost.Process with
cpp-subprocess to make diff for this code meaningful and reviewable.
The plan is to reintroduce Windows-specific code in this test
simultaneously with enabling Windows support in cpp-subprocess.
We preemptively perform a block mutation check before further processing
a block message (similar to early sanity checks on other messsage
types). The main reasons for this change are as follows:
- `CBlock::GetHash()` is a foot-gun without a prior mutation check, as
the hash returned only commits to the header but not to the actual
transactions (`CBlock::vtx`) contained in the block.
- We have observed attacks that abused mutated blocks in the past, which
could have been prevented by simply not processing mutated blocks
(e.g. https://github.com/bitcoin/bitcoin/pull/27608).
fccfdb25b2 doc: Update OpenBSD build docs to 7.4 (Jesse Barton)
Pull request description:
Updated OpenBSD Build doc for 7.4 after testing all build options. No issues on my end.
Also added a note about referring to depends/README.md for detailed instructions on required dependencies.
This was added in reference to a conversation in #29443
ACKs for top commit:
fanquake:
ACK fccfdb25b2
theStack:
lgtm ACK fccfdb25b2
Tree-SHA512: be6d22b605140b37a71e11c5bbed54f60655832d78cd3cb221eddc77c7621a65c0d71baf436f90819be536d9b5dbf1a0b2c82b6b23d62356addc495403f2ba35
bf5662c678 test: enable v2 for python p2p depending on global --v2transport flag (Martin Zumsande)
6e9e39da43 test: Don't use v2transport when it's too slow. (Martin Zumsande)
87549c8f89 test: enable p2p_invalid_messages.py with v2transport (Martin Zumsande)
5fc9db504b test: enable p2p_sendtxrcncl.py with v2transport (Martin Zumsande)
Pull request description:
#24748 added v2 transport to the python `P2PConnection`, but so far each test that wants to make use of it needs to enable it on an individual basis.
This PR changes it so that if the test suite is run with `--v2transport` option, v2 is used in each test by default, not only for connections between two bitcoind instances as before, but also wherever `P2PConnection` is used. Individual tests can override this global option.
To do that, a few tests need to be adjusted.
In addition, I added a commit to always use v1 in a few select subtests that send a large number of large messages (e.g. large reorgs). These tests don't have a fundamental problem with v2 but become very slow due to the unoptimised python ChaCha20 implementation (~30 minutes on my computer, so probably not suitable to be run in the CI).
As a result, `python3 test_runner.py --v2transport` should succeed and use `v2` everywhere (unless v1 is chosen explicitly).
[Edit]: To make the "test each commit" CI pass, several test fixes were squashed into the last commit, which actually enables v2 p2p for `P2PConnection`. I have an unsquashed version at https://github.com/mzumsande/bitcoin/tree/202401_bip324_alltests_unsquashed, in case that helps with review.
ACKs for top commit:
fjahr:
tACK bf5662c678
vasild:
ACK bf5662c678
stratospher:
reACK bf5662c6.
theStack:
Tested ACK bf5662c678
Tree-SHA512: 4f5a08248ba8a755f7d0f48deb2b79bef03292345cacb7deef01be955481093800e4e56ff218ea56734eef5de1fb3ab0f04657447ea27d393bb536539d7b289d
fa3a4102ef fuzz: Set -rss_limit_mb=8000 for generate as well (MarcoFalke)
fa4e396e1d fuzz: Generate with random libFuzzer settings (MarcoFalke)
Pull request description:
Sometimes a libFuzzer setting like `-use_value_profile=1` helps [0], sometimes it hurts [1].
[0] https://github.com/bitcoin/bitcoin/pull/20789#issuecomment-752961937
[1] https://github.com/bitcoin/bitcoin/pull/27888#issuecomment-1645976254
By picking a random value, it is ensured that at least some of the runs will have the beneficial configuration set.
Also, set `-max_total_time` to prevent slow fuzz targets from getting a larger time share, or possibly peg to a single core for a long time and block the python script from exiting for a long time. This can be improved in the future. For example, the python script can exit after some time (https://github.com/bitcoin/bitcoin/pull/20752#discussion_r549248791). Alternatively, it can measure if coverage progress was made and run for less time if no progress has been made recently anyway, so that more time can be spent on targets that are new or still make progress.
ACKs for top commit:
murchandamus:
utACK fa3a4102ef
dergoegge:
utACK fa3a4102ef
brunoerg:
light ACK fa3a4102ef
Tree-SHA512: bfd04a76ca09aec612397bae5f3f263a608faa7087697169bd4c506c8195c4d2dd84ddc7fcd3ebbc75771eab618fad840af819114968ca3668fc730092376768
d2fe90571e test: Drop `x` modifier in `fsbridge::fopen` call for mingw builds (Hennadii Stepanov)
Pull request description:
The MinGW-w64 toolchain links executables to the old msvcrt C Runtime Library that does not support the `x` modifier for the [`_wfopen()`](https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/fopen-wfopen?view=msvc-170) function.
Fixes https://github.com/bitcoin/bitcoin/issues/29014.
ACKs for top commit:
maflcko:
ACK d2fe90571e
fanquake:
ACK d2fe90571e - the plan here should still be to migrate to the newer windows runtime.
Tree-SHA512: 0269b66531e58c093ecda3a3e355a20ee8274e165d7e010f8f125881b3c8d4cfe801abdca4605d81efd3b2dbe9a81896968971f6f53da7f6c6093b76b47c5bc9
Tested and used all build options on OpenBSD 7.4 with no issues.
Added a note about referring to depends/README.md for detailed instructions on required dependencies.
This was added in reference to a conversation in #29443
0fbf051fec depends: fix BDB compilation on OpenBSD (Sebastian Falbesoner)
Pull request description:
Compiling C++ code with `-D_XOPEN_SOURCE=600` causes problems on OpenBSD. If that define is set, the C++ standard header detection routine in BDB's configure script fails due to a missing type name for `locale_t` (see https://gist.github.com/theStack/b41884e31ebc5cdca3220bcaa674cb70 for the relevant config.log part).
This results in `HAVE_CXX_STDHEADERS` not being defined, which then it turn leads to the inclusion of `<iostream.h>` (rather than `<iostream>`), which doesn't exist, as described in #28963.
According to a mailing list post discussing a similar problem [1], "OpenBSD provides the POSIX APIs by default", so we don't need this define anyway and can remove it. This fixes the BDB build problem as described in issue #28963. See also f87e75ae71 for a similar fix for google's flatbuffer project.
Tested on OpenBSD 7.4 with clang 13.0.0. Fixes#28963.
[1] https://www.mail-archive.com/tech@openbsd.org/msg63386.html
ACKs for top commit:
fanquake:
ACK 0fbf051fec
Tree-SHA512: 02139e9081ed855e067bfba8c81b54c657417576e553cc1035a916ada9be049358f5e14d756d5f234c5226bd7e943f61c6ae8990c1b152f9125681b7b777c9b3
b03b20685a Fix CI-detected codespell warnings (Lőrinc)
Pull request description:
Split out the typo fixes encountered in https://github.com/bitcoin/bitcoin/pull/29458 to a separate PR.
ACKs for top commit:
maflcko:
ACK b03b20685a
Tree-SHA512: 99b6fac01ba2ae6e6de9c50d2b481387899844a4b3a77d544c7b8afe7cfd25251a982329688d4739cde8b98ad35afcfd49be7c7cc3dad9bdff1d5915861a206d
faa30a4c56 rpc: Do not wait for headers inside loadtxoutset (MarcoFalke)
Pull request description:
While the `loadtxoutset` default 10 minute timeout is convenient when it is sufficient, it may cause hassle where it is not. For example:
* When P2P connections are missing, it seems better to abort early than wait for the timeout.
* When the 10 minute timeout is not sufficient, the RPC will have to be called again, so a check or loop is needed outside the RPC either way. So might as well remove the loop inside the RPC.
ACKs for top commit:
fjahr:
ACK faa30a4c56
theStack:
Code-review ACK faa30a4c56
pablomartin4btc:
tACK faa30a4c56
TheCharlatan:
ACK faa30a4c56
Tree-SHA512: 9167c7d8b2889bb3fd369de4acd2cc4d24a2fe225018d82bd9568ecd737093f6e19be7cc62815b574137b61076a6f773c29bff75398991b5cd702423aab2322b
5f240ab2e8 test: Add option to skip unit tests for the test runner (Martin Zumsande)
Pull request description:
In the python `test_runner`, it's possible to disable specific functional tests (or just enable a few specific ones), but the unit tests for the python test framework cannot be skipped.
Add this option (`--skipunit` or `-u`), it would save some time for devs not interested in running those every time.
ACKs for top commit:
fjahr:
re-ACK 5f240ab2e8
tdb3:
Code review and tested ACK 5f240ab2e8
stratospher:
tested ACK 5f240ab.
Tree-SHA512: f7c9cfefc18a6510e24ca4601309b40fdf4180a4c5fe592be9cf7607be6541784b283c46c8d6e60740ff3eba83025dd5d0db36e55bf8bad1404b38120859e113
fa58ae74ea refactor: Add missing include for USE_BDB, USE_SQLITE to bench/wallet_ismine.cpp (MarcoFalke)
fa31908ea8 lint: Check for missing or redundant bitcoin-config.h includes (MarcoFalke)
fa63b0e351 lint: Make lint error easier to spot in output (MarcoFalke)
fa770fd368 doc: Add missing RUST_BACKTRACE=1 (MarcoFalke)
fa10051267 lint: Add get_subtrees() helper (MarcoFalke)
Pull request description:
Missing `bitcoin-config.h` includes are problematic, because the build could silently pass, but produce an unintended result. For example, a slower fallback algorithm could be picked, even though `bitcoin-config.h` indicates that a faster feature is available and should be used.
As the build succeeds silently, this problem is not possible to detect with iwyu.
Thus, fix this by using a linter based on grepping the source code.
ACKs for top commit:
theuni:
Weak ACK fa58ae74ea.
TheCharlatan:
ACK fa58ae74ea
hebasto:
ACK fa58ae74ea, tested on Ubuntu 23.10 -- it catches bugs properly. I didn't review rust code changes.
Tree-SHA512: cf4346f81ea5b8c215da6004cb2403d1aaf569589613c305d8ba00329b82b3841da94fe1a69815ce15f2edecbef9b031758ec9b6433564976190e3cf91ec8181
84388c942c ci: avoid running git diff after patching (Ryan Ofsky)
Pull request description:
Drop `git diff` command so it is easier to run CI locally if git checkout is a worktree. Currently it fails because the directory is not recognized as a git repository.
The `git diff` command was added recently in #28359 commit fa07ac48d8 and can be avoided just by teeing the patch to stdout
ACKs for top commit:
maflcko:
lgtm ACK 84388c942c
TheCharlatan:
ACK 84388c942c
Tree-SHA512: 089c8ff62f9c56a1df06686e72420a9a54a079d2ef9eaf7c9cfcd97cb5cce50c8c169890e599ef875aaf1ee426f590851b1f19d6c9e386671460ee6507d8d872
9dae3b970a [fuzz] Avoid partial negative result (Murch)
Pull request description:
May address the problem reported by maflcko in https://github.com/bitcoin/bitcoin/pull/27877#pullrequestreview-1890304914.
For some values, `MAX_MONEY - max_spendable - max_output_groups` could result in a partial negative value. By putting the addition of `group_pos.size()` first, all partial results in this line will be strictly positive.
I opened this as a draft, since I was unable to reproduce the issue, so I’m waiting for confirmation whether this in fact mitigates the problem.
ACKs for top commit:
maflcko:
ACK 9dae3b970a
sipa:
utACK 9dae3b970a
achow101:
ACK 9dae3b970a
brunoerg:
crACK 9dae3b970a
Tree-SHA512: 744b4706268d8dfd77538b99492ecf3cf77d229095f9bcd416a412131336830e2f134f2b2846c79abd3d193426f97c1f71eeaf68b16ab00e76318d57ee3673c7
Drop `git diff` command so it is easier to run CI locally if git checkout is a
worktree. Currently it fails because the directory is not recognized as a git
repository.
The `git diff` command was added recently in #28359 commit
fa07ac48d8 and can be avoided just by teeing the
patch to stdout
Adds test in mempool_accept to check if a negative maxfeerate is inputed
into check_mempool_result, asserts "Amount out of range" error message
and -3 error code
345169a752 test: assert rpc error for addnode v2transport not enabled (kevkevin)
Pull request description:
Added coverage for the `addnode` rpc when v2transport is not enabled,
but is set as true when calling `addnode` rpc.
I ran the following to check if this rpc error message
was covered in the functional tests.
`grep -nr "v2transport requested but not enabled" ./test/functional --binary-files=without-match`
Adds test coverage to this line.
https://github.com/bitcoin/bitcoin/blob/master/src/rpc/net.cpp#L339
ACKs for top commit:
maflcko:
lgtm ACK 345169a752
brunoerg:
utACK 345169a752
BrandonOdiwuor:
Code Review ACK 345169a752
theStack:
Code-review ACK 345169a752
Tree-SHA512: fb82409485efe25a1193b1dafca8ae694b397a301bb8bcb33c7572d21ff244ee45fbbd4364141e9421733873b343554a34614a59b1450ce0cac5c420203c3d35
540282905d docs: ci multi-arch requires qemu (Max Edwards)
Pull request description:
On a fresh Debian system qemu isn't installed and therefore the multi-architecture CI system doesn't run.
This documentation notes that qemu is required and how to install it.
ACKs for top commit:
maflcko:
lgtm ACK 540282905d
willcl-ark:
utACK 540282905d
Tree-SHA512: 903c12cf7b16f2146b99a952577c5550d60faf65f9e72b9f4d9479b52228118ab46349e5130de5281b39da05d3bc0b4ae8a8165601e62ce145647a98ef197131
Before this change it was possible but awkward to create ParamStream streams
with multiple parameter objects. After this change it is straightforward.
The change to support multiple parameters is implemented by letting
ParamsStream contain substream instances, instead of just references to
external substreams. So a side-effect of this change is that ParamStream can
now accept rvalue stream arguments and be easier to use in some other cases. A
test for rvalues is added in this commit, and some simplifications to non-test
code are made in the next commit.
Move parameter argument after stream argument so will be possible to accept
multiple variadic parameter arguments in the following commit.
Also reverse template parameter order for consistency.
Drop unnecessary ParamsStream references from CTransaction and
CMutableTransaction constructors. This just couples these classes unnecessarily
to the ParamsStream class, making the ParamsStream class harder to modify, and
making the transaction classes in some cases (depending on parameter order)
unable to work with stream classes that have multiple parameters set.
9d1dbbd4ce scripted-diff: Fix bitcoin_config_h includes (TheCharlatan)
Pull request description:
As mentioned in https://github.com/bitcoin/bitcoin/pull/26924#issuecomment-1403449932 and https://github.com/bitcoin/bitcoin/pull/29263#issuecomment-1922334399, it is currently not safe to remove `bitcoin-config.h` includes from headers because some unrelated file might be depending on it.
See also #26972 for discussion.
Solve this by including the file directly everywhere it's required, regardless of whether or not it's already included by another header.
There should be no functional change here, but it will allow us to safely remove includes from headers in the future.
~I'm afraid it's a bit tedious to reproduce these commits, but it's reasonably straightforward:~
Edit: See note below
```bash
# All commands executed from the src/ subdir.
# Collect all tokens from bitcoin-config.h.in
# Isolate the tokens and remove blank lines
# Replace newlines with | and remove the last trailing one
# Collect all files which use these tokens
# Filter out subprojects (proper forwarding can be verified from Makefiles)
# Filter out .rc files
# Save to a text file
git grep -E -l `grep undef config/bitcoin-config.h.in | cut -d" " -f2 | grep -v '^$' | tr '\n' '|' | sed 's/|$//'` | grep -v -e "^leveldb/" -e "^secp256k1/" -e "^crc32c/" -e "^minisketch/" -e "^Makefile" -e "\.rc$" > files-with-config-include.txt
# Find all files from the above list which don't include bitcoin-config.h
git grep -L -E "config/bitcoin-config.h" -- `cat files-with-config-include.txt`
# Include them manually with the exception of some files in crypto:
# crypto/sha256_arm_shani.cpp crypto/sha256_avx2.cpp crypto/sha256_sse41.cpp crypto/sha256_x86_shani.cpp
# These are exceptions which don't use bitcoin-config.h, rather the Makefile.am adds these cppflags manually.
# Commit changes. This should match the first commit of this PR.
# Use the same search as above to find all files which DON'T use any config tokens
git grep -E -L `grep undef config/bitcoin-config.h.in | cut -d" " -f2 | grep -v '^$' | tr '\n' '|' | sed 's/|$//'` | grep -v -e "^leveldb/" -e "^secp256k1/" -e "^crc32c/" -e "^minisketch/" -e "^Makefile" -e "\.rc$" > files-without-config-include.txt
# Manually remove the includes and commit changes. This should match the second commit of this PR.
```
Edit: I'll keep this old description for posterity, but the manual approach has been replaced with a scripted diff from TheCharlatan
ACKs for top commit:
maflcko:
ACK 9d1dbbd4ce🚪
TheCharlatan:
ACK 9d1dbbd4ce
hebasto:
ACK 9d1dbbd4ce, I have reviewed the code and it looks OK.
fanquake:
ACK 9d1dbbd4ce
Tree-SHA512: f11ddc4ae6a887f96b954a6b77f310558ddb271088a3fda3edc833669c4251b7f392515224bbb8e5f67eb2c799b4ffed3b07d96454e82ec635c686d0df545872
fa91bf2559 ci: Skip git install if it is already installed (MarcoFalke)
c65fde4831 ci: vary /tmp/env (Sjors Provoost)
Pull request description:
* Currently, running separate CI tasks at the same time may intermittently fail, because they race to read/write `/tmp/env`. Fix this by adding `$CONTAINER_NAME` to the file name.
* Also, add `$USER`, while touching the line, to allow different users to run the same CI task at the same time.
* Also, skip the git install if there is no need.
Ref: https://github.com/bitcoin/bitcoin/pull/29274
ACKs for top commit:
Sjors:
ACK fa91bf2559
BrandonOdiwuor:
ACK fa91bf2559
hebasto:
ACK fa91bf2559.
Tree-SHA512: 9a8479255a2afb6618f9d0796488d9430ba95266b90ce39536a9817c1974ca4049beeaab5355a38b25171f76fc386dbec06b1919aaa079f08a5a0c0a146232c8
e041ed9b75 wallet: Retrieve ID from loaded DescSPKM directly (Ava Chow)
39640dd34e wallet: Use scriptPubKeyCache in GetSolvingProvider (Ava Chow)
b410f68791 wallet: Use scriptPubKey cache in GetScriptPubKeyMans (Ava Chow)
edf4e73a16 wallet: Use scriptPubKey cache in IsMine (Ava Chow)
37232332bd wallet: Cache scriptPubKeys for all DescriptorSPKMs (Ava Chow)
99a0cddbc0 wallet: Introduce a callback called after TopUp completes (Ava Chow)
b276825932 bench: Add a benchmark for ismine (Ava Chow)
Pull request description:
Wallets that have a ton of non-ranged descriptors (such as a migrated non-HD wallet) perform fairly poorly due to looping through all of the wallet's `ScriptPubKeyMan`s. This is done in various places, such as `IsMine`, and helper functions for fetching a `ScriptPubKeyMan` and a `SolvingProvider`. This also has a bit of a performance impact on standard descriptor wallets, although less noticeable due to the small number of SPKMs.
As these functions are based on doing `IsMine` for each `ScriptPubKeyMan`, we can improve this performance by caching `IsMine` scriptPubKeys for all descriptors and use that to determine which `ScriptPubKeyMan` to actually use for those things. This cache is used exclusively and we no longer iterate the SPKMs.
Also added a benchmark for `IsMine`.
ACKs for top commit:
ryanofsky:
Code review ACK e041ed9b75. Just suggested changes since last review
josibake:
ACK e041ed9b75
furszy:
Code review ACK e041ed9b
Tree-SHA512: 8e7081991a025e682e9dea838b4543b0d179832d1c47397fb9fe7a97fa01eb699c15a5d5a785634926844fc83a46e6ac07ef753119f39d84423220ef8a548894
dddd7be9bf doc: Clarify maxfeerate help (MarcoFalke)
fa2a4fdef7 rpc: Fixed signed integer overflow for large feerates (MarcoFalke)
fade94d11a rpc: Add ParseFeeRate helper (MarcoFalke)
fa0ff66109 rpc: Implement RPCHelpMan::ArgValue<> for UniValue (MarcoFalke)
Pull request description:
Passing large BTC/kvB feerates to RPCs is problematic, because:
* They are likely a typo. 1BTC/kvB (or larger) seems absurd.
* They may cause signed integer overflow.
* Anyone really wanting to pick such a large value can set `0` to disable the check.
Fix all issues by rejecting anything more than 1BTC/kvB during parsing.
ACKs for top commit:
brunoerg:
crACK dddd7be9bf
achow101:
ACK dddd7be9bf
vasild:
ACK dddd7be9bf
tdb3:
Code review ACK and basic test ACK for dddd7be9bf.
fjahr:
utACK dddd7be9bf
Tree-SHA512: 5dcce1f0abe059dc6b2ff56787e11081d73a45b4ddd6dcc2c1ea13709ebc13af5e7265e84fffb97ef32027b56b81955672a67ed7702e8fa30c2e849d67727bac
Compiling C++ code with `-D_XOPEN_SOURCE=600` causes problems on
OpenBSD. If that define is set, the C++ standard header detection
routine in BDB's configure script fails. This results in
`HAVE_CXX_STDHEADERS` not being defined, which then it turn leads to
the inclusion of `<iostream.h>` (rather than `<iostream>`), which
doesn't exist.
According to a mailing list post discussing a similar problem [1],
"OpenBSD provides the POSIX APIs by default", so we don't need this
define anyway and can remove it. This fixes the BDB build problem as
described in issue #28963.
Tested on OpenBSD 7.4 with clang 13.0.0.
[1] https://www.mail-archive.com/tech@openbsd.org/msg63386.html
During the initial sync, the Tab moves the focus to the widgets
of the main window, even when the ModalOverlay is visible. This
creates some weird rectangular *selections on the screen*.
This PR fixes this by keeping the focus on the "Hide" button while
the ModalOverlay is visible.
Fixes#783
Instead of iterating m_spk_managers a DescriptorSPKM has been loaded in
order to get it's ID to compare, have LoadDescriptorSPKM return a
reference to the loaded DescriptorSPKM so it can be queried directly.
Have CWallet maintain a cache of all known scriptPubKeys for its
DescriptorSPKMs in order to improve performance of the functions that
require searching for scriptPubKeys.
After TopUp completes, the wallet containing each SPKM will want to know
what new scriptPubKeys were generated. In order for all TopUp calls
(including ones internal the the SPKM), we use a callback function in
the WalletStorage interface.
By defining a virtual interface class for the scheduler client, users of
the kernel can now define their own event consuming infrastructure,
without having to spawn threads or rely on the scheduler design.
Removing CScheduler also allows removing the thread and
exception modules from the kernel library.
f1684bb88a rpc: mention that migratewallet can take a while (Andrew Chow)
9ecff997e1 rpc: Drop migratewallet experimental warning (Andrew Chow)
Pull request description:
The migration process itself hasn't fundamentally changed since it was added, so I think it's reasonable to say that it is no longer experimental.
ACKs for top commit:
maflcko:
lgtm ACK f1684bb88a
josibake:
ACK f1684bb88a
furszy:
ACK f1684bb88a
ryanofsky:
Code review ACK f1684bb88a
willcl-ark:
ACK f1684bb88a
Tree-SHA512: 99b176cddbf3878c76bd4c80c030106200bf03139785e26dbae3341e1a675b623a13cd6dc7a0bb78344335bf859ae7548d97b2b58eb650c6e7b305d7cdc86e40
This is done in preparation for the next two commits, where the
CMainSignals are de-globalized.
This avoids adding new constructor arguments to the ChainstateManager
and CTxMemPool classes over the next two commits.
This could also allow future tests that are only interested in the
internal behaviour of the classes to forgo instantiating the signals.
3d1bb1a122 qt: Update translation source file for v27.0 string freeze (Hennadii Stepanov)
Pull request description:
This PR updates the `src/qt/locale/bitcoin_en.xlf` translation source file according to the [Release schedule for 27.0](https://github.com/bitcoin/bitcoin/issues/29028).
Note for reviewers: it is expected to get a zero diff after running `make -C src translate` locally.
ACKs for top commit:
jarolrod:
ACK 3d1bb1a122
Tree-SHA512: 9b6e5aa3aaabb918d0a6418559bc3eb14297abc48b99e8c6e6de770aa1478b8b28881f8965fd15fe23cf4aa377b88ba903e978c8b75681c4f11e428ca1588b96
-BEGIN VERIFY SCRIPT-
regex_string='^(?!//).*(AC_APPLE_UNIVERSAL_BUILD|BOOST_PROCESS_USE_STD_FS|CHAR_EQUALS_INT8|CLIENT_VERSION_BUILD|CLIENT_VERSION_IS_RELEASE|CLIENT_VERSION_MAJOR|CLIENT_VERSION_MINOR|COPYRIGHT_HOLDERS|COPYRIGHT_HOLDERS_FINAL|COPYRIGHT_HOLDERS_SUBSTITUTION|COPYRIGHT_YEAR|ENABLE_ARM_SHANI|ENABLE_AVX2|ENABLE_EXTERNAL_SIGNER|ENABLE_SSE41|ENABLE_TRACING|ENABLE_WALLET|ENABLE_X86_SHANI|ENABLE_ZMQ|HAVE_BOOST|HAVE_BUILTIN_CLZL|HAVE_BUILTIN_CLZLL|HAVE_BYTESWAP_H|HAVE_CLMUL|HAVE_CONSENSUS_LIB|HAVE_CXX20|HAVE_DECL_BE16TOH|HAVE_DECL_BE32TOH|HAVE_DECL_BE64TOH|HAVE_DECL_BSWAP_16|HAVE_DECL_BSWAP_32|HAVE_DECL_BSWAP_64|HAVE_DECL_FORK|HAVE_DECL_FREEIFADDRS|HAVE_DECL_GETIFADDRS|HAVE_DECL_HTOBE16|HAVE_DECL_HTOBE32|HAVE_DECL_HTOBE64|HAVE_DECL_HTOLE16|HAVE_DECL_HTOLE32|HAVE_DECL_HTOLE64|HAVE_DECL_LE16TOH|HAVE_DECL_LE32TOH|HAVE_DECL_LE64TOH|HAVE_DECL_PIPE2|HAVE_DECL_SETSID|HAVE_DECL_STRERROR_R|HAVE_DEFAULT_VISIBILITY_ATTRIBUTE|HAVE_DLFCN_H|HAVE_DLLEXPORT_ATTRIBUTE|HAVE_ENDIAN_H|HAVE_EVHTTP_CONNECTION_GET_PEER_CONST_CHAR|HAVE_FDATASYNC|HAVE_GETENTROPY_RAND|HAVE_GETRANDOM|HAVE_GMTIME_R|HAVE_INTTYPES_H|HAVE_LIBADVAPI32|HAVE_LIBCOMCTL32|HAVE_LIBCOMDLG32|HAVE_LIBGDI32|HAVE_LIBIPHLPAPI|HAVE_LIBKERNEL32|HAVE_LIBOLE32|HAVE_LIBOLEAUT32|HAVE_LIBSHELL32|HAVE_LIBSHLWAPI|HAVE_LIBUSER32|HAVE_LIBUUID|HAVE_LIBWINMM|HAVE_LIBWS2_32|HAVE_MALLOC_INFO|HAVE_MALLOPT_ARENA_MAX|HAVE_MINIUPNPC_MINIUPNPC_H|HAVE_MINIUPNPC_UPNPCOMMANDS_H|HAVE_MINIUPNPC_UPNPERRORS_H|HAVE_NATPMP_H|HAVE_O_CLOEXEC|HAVE_POSIX_FALLOCATE|HAVE_PTHREAD|HAVE_PTHREAD_PRIO_INHERIT|HAVE_STDINT_H|HAVE_STDIO_H|HAVE_STDLIB_H|HAVE_STRERROR_R|HAVE_STRINGS_H|HAVE_STRING_H|HAVE_STRONG_GETAUXVAL|HAVE_SYSCTL|HAVE_SYSCTL_ARND|HAVE_SYSTEM|HAVE_SYS_ENDIAN_H|HAVE_SYS_PRCTL_H|HAVE_SYS_RESOURCES_H|HAVE_SYS_SELECT_H|HAVE_SYS_STAT_H|HAVE_SYS_SYSCTL_H|HAVE_SYS_TYPES_H|HAVE_SYS_VMMETER_H|HAVE_THREAD_LOCAL|HAVE_TIMINGSAFE_BCMP|HAVE_UNISTD_H|HAVE_VM_VM_PARAM_H|LT_OBJDIR|PACKAGE_BUGREPORT|PACKAGE_NAME|PACKAGE_STRING|PACKAGE_TARNAME|PACKAGE_URL|PACKAGE_VERSION|PTHREAD_CREATE_JOINABLE|QT_QPA_PLATFORM_ANDROID|QT_QPA_PLATFORM_COCOA|QT_QPA_PLATFORM_MINIMAL|QT_QPA_PLATFORM_WINDOWS|QT_QPA_PLATFORM_XCB|QT_STATICPLUGIN|STDC_HEADERS|STRERROR_R_CHAR_P|USE_ASM|USE_BDB|USE_DBUS|USE_NATPMP|USE_QRCODE|USE_SQLITE|USE_UPNP|_FILE_OFFSET_BITS|_LARGE_FILES)'
exclusion_files=":(exclude)src/minisketch :(exclude)src/crc32c :(exclude)src/secp256k1 :(exclude)src/crypto/sha256_arm_shani.cpp :(exclude)src/crypto/sha256_avx2.cpp :(exclude)src/crypto/sha256_sse41.cpp :(exclude)src/crypto/sha256_x86_shani.cpp"
git grep --perl-regexp --files-with-matches "$regex_string" -- '*.cpp' $exclusion_files | xargs git grep -L "bitcoin-config.h" | while read -r file; do line_number=$(awk -v my_file="$file" '/\/\/ file COPYING or https?:\/\/www.opensource.org\/licenses\/mit-license.php\./ {line = NR} /^\/\// && NR == line + 1 {while(getline && /^\/\//) line = NR} END {print line+1}' "$file"); sed -i "${line_number}i\\\\n\#if defined(HAVE_CONFIG_H)\\n#include <config/bitcoin-config.h>\\n\#endif" "$file"; done;
git grep --perl-regexp --files-with-matches "$regex_string" -- '*.h' $exclusion_files | xargs git grep -L "bitcoin-config.h" | while read -r file; do sed -i "/#define.*_H/a \\\\n\#if defined(HAVE_CONFIG_H)\\n#include <config/bitcoin-config.h>\\n\#endif" "$file"; done;
for file in $(git grep --files-with-matches 'bitcoin-config.h' -- '*.cpp' '*.h' $exclusion_files); do if ! grep -q --perl-regexp "$regex_string" $file; then sed -i '/HAVE_CONFIG_H/{N;N;N;d;}' $file; fi; done;
-END VERIFY SCRIPT-
The first command creates a regular expression for matching all bitcoin-config.h symbols in the following form: ^(?!//).*(AC_APPLE_UNIVERSAL_BUILD|BOOST_PROCESS_USE_STD_FS|...|_LARGE_FILES). It was generated with:
./autogen.sh && printf '^(?!//).*(%s)' $(awk '/^#undef/ {print $2}' src/config/bitcoin-config.h.in | paste -sd "|" -)
The second command holds a list of files and directories that should not be processed. These include subtree directories as well as some crypto files that already get their symbols through the makefile.
The third command checks for missing bitcoin-config headers in .cpp files and adds the header if it is missing.
The fourth command checks for missing bitcoin-config headers in .h files and adds the header if it is missing.
The fifth command checks for unneeded bitcoin-config headers in sources files and removes the header if it is unneeded.
77331aa2a1 wallet: simplify EraseRecords by using 'ErasePrefix' (furszy)
33757814ce wallet: bdb batch 'ErasePrefix', do not create txn internally (furszy)
cf4d72a75e wallet: db, introduce 'RunWithinTxn()' helper function (furszy)
Pull request description:
Seeks to optimize and simplify `WalletBatch::EraseRecords`. Currently, this process opens a cursor to iterate over the entire database, searching for records that match the type prefixes, to then call the `WalletBatch::Erase` function for each of the matching records.
This PR rewrites this 40-line manual process into a single line; instead of performing all of those actions manually, we can simply utilize the `ErasePrefix()` functionality. The result is 06216b344dea6ad6c385fda0b37808ff9ae5273b.
Moreover, it expands the test coverage for the `ErasePrefix` functionality and documents the db txn requirement for `BerkeleyBatch::ErasePrefix` .
ACKs for top commit:
achow101:
reACK 77331aa2a1
josibake:
code review ACK 77331aa2a1
Tree-SHA512: 9f78dda658677ff19b5979ba0efd11cf9fabf3d315feb79ed1160526f010fe843c41903fc18c0b092f78aa88bc874cf24edad8fc1ea6e96aabdc4fd1daf21ca5
864e2e9097 fuzz: increase length of string used for `NetWhitelist{bind}Permissions::TryParse` (brunoerg)
Pull request description:
The string `s` represents the value from `-whitelist`/`-whitebind` (e.g. "bloom,forcerelay,noban@1.2.3.4:32") and it is used in `NetWhitelistPermissions::TryParse` and `NetWhitebindPermissions::TryParse`. However, a max length of 32 is not enough to cover a lot of cases. Even disconsidering the permissions, 32 would not be enough to cover a lot of addresses. This PR fixes it.
ACKs for top commit:
maflcko:
lgtm ACK 864e2e9097
epiccurious:
utACK 864e2e9097.
vasild:
ACK 864e2e9097
Tree-SHA512: 2b89031b9f2ea92d636f05fd167b1e5ac726742a7e7c1af8ddaeaf90236e659731aaa6b7c23f65ec16ce52ac1b9e68e7b16e23c59e355312d057e001976d172a
8d20602e55 test, assumeutxo: Add test to ensure failure when mempool not empty (Hernan Marino)
Pull request description:
Add a test to ensure that loadtxoutset fails when the node's mempool is not empty, as suggested by maflcko here: https://github.com/bitcoin/bitcoin/pull/27596#discussion_r1344713537
ACKs for top commit:
maflcko:
re-ACK 8d20602e55
BrandonOdiwuor:
ACK 8d20602e55
Tree-SHA512: 97c9668c0f38897934bf0d326515d898d4e682ff219deba9d751b35125b5cf33d51c9df116a74117ecf0394f28995a3d0cae1266b1e5acb4365ff4f309ce3f6c
44d11532f8 test: fix intermittent failure in wallet_reorgrestore.py (Martin Zumsande)
Pull request description:
By adding a missing `sync_blocks` call.
There was a race at `node2` between connecting the block produced by `node0`, and using `-generate` to create new blocks itself. In the failed run, block generation started before connecting the block, resulting in a final block height that was smaller by 1 than expected.
See https://github.com/bitcoin/bitcoin/issues/29392#issuecomment-1939541603 for a more detailed analysis of the failed run.
Can be reproduced by adding a sleep to [this spot](6ff0aa089c/src/validation.cpp (L4217)) in `ChainstateManager::ProcessNewBlock()`:
```
if (util::ThreadGetInternalName() == "msghand") {
std::this_thread::sleep_for(0.2s);
}
```
which fails for me on master and succeeds with the fix.
Fixes#29392
ACKs for top commit:
maflcko:
lgtm ACK 44d11532f8
Tree-SHA512: c08699e5ae348d4c0626022b519449d052f511d3f44601bcd8dac836a130a3f67fca149532e1e3690367ebfdcbcdd32e527170d039209c1f599ce861136ae29f
...by adding a missing sync_blocks call.
There was a race at node2 between connecting the block
produced by node 0, and using -generate to create new blocks
itself. In the failed run, the latter happened first,
resulting in a final block height that was smaller by 1 than
expected.
Transactions are intended to be started on upper layers rather than
internally by the bdb batch object. This enables us to consolidate
different write operations within a procedure in the same db txn,
improving consistency due to the atomic property of the transaction,
as well as its performance due to the reduction of disk write
operations.
Important Note:
This approach also ensures that the BerkeleyBatch::ErasePrefix
function behaves exactly as the SQLiteBatch::ErasePrefix function,
which does not create a db txn internally.
Furthermore, since the `BerkeleyBatch::ErasePrefix' implementation
erases records one by one (by traversing the db), this change
ensures that the function is always called within an active txn
context. Without this measure, there's a potential risk to consistency;
certain records may be removed while others could persist due to an
internal failure during the procedure.
'RunWithinTxn()' provides a way to execute db operations within a
transactional context. It avoids writing repetitive boilerplate code for
starting and committing the database transaction.
9a3c5c8697 scripted-diff: rename ZapSelectTx to RemoveTxs (furszy)
83b762845f wallet: batch and simplify ZapSelectTx process (furszy)
595d50a103 wallet: migration, remove extra NotifyTransactionChanged call (furszy)
a2b071f992 wallet: ZapSelectTx, remove db rewrite code (furszy)
Pull request description:
Work decoupled from #28574. Brother of #28894.
Includes two different, yet interconnected, performance and code improvements to the zap wallet transactions process.
1) As the goal of the `ZapSelectTx` function is to erase tx records that match any of the inputted hashes. There is no need to traverse the whole database record by record. We could just check if the tx exist, and remove it directly by calling `EraseTx()`.
2) Instead of performing single write operations per removed tx record, this PR batches them all within a single atomic db txn.
Moreover, these changes will enable us to consolidate all individual write operations that take place during the wallet migration process into a single db txn in the future.
ACKs for top commit:
achow101:
ACK 9a3c5c8697
josibake:
ACK 9a3c5c8697
Tree-SHA512: fb2ecc48224c400ab3b1fbb32e174b5b13bf03794717727f80f01f55fb183883b067a68c0a127b2de8885564da15425d021a96541953bf38a72becc2e9929ccf
517c7f9cba gui: Check for private keys disabled before attempting unlock (Andrew Chow)
Pull request description:
Before trying to unlock a wallet, first check if it has private keys disabled. If so, there is no need to unlock.
Note that such wallets are not expected to occur in typical usage. However bugs in previous versions allowed such wallets to be created, and so we need to handle them.
Fixes#772
For some additional context, see #631
ACKs for top commit:
hebasto:
ACK 517c7f9cba, I have reviewed the code and it looks OK.
BrandonOdiwuor:
ACK 517c7f9cba
Tree-SHA512: c92aa34344d04667b70b059d2aa0a1da999cb7239cd1413f3009781aa82379f309ff9808d7dc91d385e2c8afe2abda3564568e2091ef833b1536ebfcf80f7c3c
This changes the default behavior, individual tests can overwrite this option.
As a result, it is possible to run the entire test suite with
--v2transport, and all connections to the python p2p will then use it.
Also adjust several tests that are already running with --v2transport in the
test runner (although they actually made v1 connection before this change).
This is done in the same commit so that there isn't an
intermediate commit in which the CI fails.
bee0ffbecf GUI/Intro: Never change the prune checkbox after the user has touched it (Luke Dashjr)
420a983e25 Bugfix: GUI/Intro: Disable GUI prune option if -prune is set, regardless of set value (Luke Dashjr)
Pull request description:
Re-PR from https://github.com/bitcoin/bitcoin/pull/18729
Now includes a bugfix too (`-prune=2+` disabled the checkbox, but `-prune=0/1` did not; this behaviour is necessary since `-prune` overrides GUI settings)
ACKs for top commit:
hebasto:
ACK bee0ffbecf, both commits are improvements of the current behaviour. Tested on Ubuntu 23.10.
Tree-SHA512: 8eb7d90af37deb30fe226179db3bc9df8ab59e4f3218c8e447ed31fc9ddc81ac1a1629da63347518587a56a4c8558b05cf7ec474024c5f5dfc6d49d6ff0eb0cc
fa0ceae970 test: Fix utxo set hash serialisation signedness (MarcoFalke)
Pull request description:
It is unsigned in Bitcoin Core, so the tests should match it:
5b8990a1f3/src/kernel/coinstats.cpp (L54)
Large positive values for the block height are too difficult to hit in tests, but it still seems fine to fix this.
The bug was introduced when the code was written in 6ccc8fc067.
(Lowercase `i` means signed, see https://docs.python.org/3/library/struct.html#format-characters)
ACKs for top commit:
epiccurious:
Tested ACK fa0ceae970.
fjahr:
utACK fa0ceae970
Tree-SHA512: ab4405c74fb191fff8520b456d3a800cd084d616bb9ddca27d56b8e5c8969bd537490f6e204c1870dbb09a3e130b03b22a27b6644252a024059c200bbd9004e7
b2e531e70a qt: update widgets availability on wallet selection (pablomartin4btc)
Pull request description:
This PR addresses an issue where, with no wallet selected, ticking on "Settings -> Mask values" checkbox twice enables the transaction tab when the checkbox is unticked.
<details>
<summary>Current behavior display on master</summary>

</details>
<details>
<summary>Correction display from this branch</summary>

</details>
Note for maintaners: this PR should be backported to both 25.x and 26.x.
---
Originally this PR was disabling the "Mask Values" checkbox when no wallet was selected but since a reviewer pointed out that a user might want to open a wallet already on "privacy mode" I rolled that change out.
<details>
<summary>Original correction display disabling "Mask Values" </summary>

</details>
ACKs for top commit:
alfonsoromanz:
Tested ACK b2e531e70a
hebasto:
ACK b2e531e70a, tested on Ubuntu 22.04.
Tree-SHA512: 6be77ab4d5ec86267a9b0a289a4d8600bb67d279f7e0be65e47b608ec392fe705cf026e32f3c082d2f27449b697d1d9e6a1d110035900d7a804ba823c9f5dfd4
ede5014c44 Modify command line help to show support for BIP21 URIs (Hernan Marino)
Pull request description:
While reviewing a different PR (see https://github.com/bitcoin-core/gui/pull/742 ) **hebasto** suggested that the help for bitcoin-qt should be updated to reflect the fact that bitcoin-qt supports an optional BIP21 URI parameter.
Since this reflects actual behaviour of bitcoin-qt and is independent of whether or not the other PR gets merged, I created this simple PR to fix the help message.
ACKs for top commit:
kristapsk:
utACK ede5014c44
pablomartin4btc:
lgtm, re ACK ede5014c44
hebasto:
ACK ede5014c44.
Tree-SHA512: c456297c486bc5cc65e0e092e7ba9d51b0bd7a584d4fabca7f7ca1f8e58cbcc66e96226539c689ed0f5e7f40da220bbc4ea30b90e31e1aeeb8867a385a90209c
The code in `logging.cpp` needs to:
* Get the category name given the flag (e.g. `BCLog::PRUNE` -> `"prune"`)
* Get the flag given the category name (e.g. `"prune"` -> `BCLog::PRUNE`)
* Get the list of category names sorted in alphabetical order
Achieve this by using the proper std containers. The result is
* less code (this diff is +62 / -129)
* faster code (to linear search and no copy+sort)
* more maintainable code (the categories are no longer duplicated in
`LogCategories[]` and `LogCategoryToStr()`)
This behavior is preserved:
`BCLog::NONE` -> `""` (lookup by `LogCategoryToStr()`)
`""` -> `BCLog::ALL` (lookup by `GetLogCategory("")`)
29029df5c7 [doc] v3 signaling in mempool-replacements.md (glozow)
e643ea795e [fuzz] v3 transactions and sigop-adjusted vsize (glozow)
1fd16b5c62 [functional test] v3 transaction submission (glozow)
27c8786ba9 test framework: Add and use option for tx-version in MiniWallet methods (MarcoFalke)
9a1fea55b2 [policy/validation] allow v3 transactions with certain restrictions (glozow)
eb8d5a2e7d [policy] add v3 policy rules (glozow)
9a29d470fb [rpc] return full string for package_msg and package-error (glozow)
158623b8e0 [refactor] change Workspace::m_conflicts and adjacent funcs/structs to use Txid (glozow)
Pull request description:
See #27463 for overall package relay tracking.
Delving Bitcoin discussion thread: https://delvingbitcoin.org/t/v3-transaction-policy-for-anti-pinning/340
Delving Bitcoin discussion for LN usage: https://delvingbitcoin.org/t/lightning-transactions-with-v3-and-ephemeral-anchors/418
Rationale:
- There are various pinning problems with RBF and our general ancestor/descendant limits. These policies help mitigate many pinning attacks and make package RBF feasible (see #28984 which implements package RBF on top of this). I would focus the most here on Rule 3 pinning. [1][2]
- Switching to a cluster-based mempool (see #27677 and #28676) requires the removal of CPFP carve out, which applications depend on. V3 + package RBF + ephemeral anchors + 1-parent-1-child package relay provides an intermediate solution.
V3 policy is for "Priority Transactions." [3][4] It allows users to opt in to more restrictive topological limits for shared transactions, in exchange for the more robust fee-bumping abilities that offers. Even though we don't have cluster limits, we are able to treat these transactions as having as having a maximum cluster size of 2.
Immediate benefits:
- You can presign a transaction with 0 fees (not just 1sat/vB!) and add a fee-bump later.
- Rule 3 pinning is reduced by a significant amount, since the attacker can only attach a maximum of 1000vB to your shared transaction.
This also enables some other cool things (again see #27463 for overall roadmap):
- Ephemeral Anchors
- Package RBF for these 1-parent-1-child packages. That means e.g. a commitment tx + child can replace another commitment tx using the child's fees.
- We can transition to a "single anchor" universe without worrying about package limit pinning. So current users of CPFP carve out would have something else to use.
- We can switch to a cluster-based mempool [5] (#27677#28676), which removes CPFP carve out [6].
[1]: Original mailing list post and discussion about RBF pinning problems https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff, https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-January/019817.html
[2]: A FAQ is "we need this for cluster mempool, but is this still necessary afterwards?" There are some pinning issues that are fixed here and not fully fixed in cluster mempool, so we will still want this or something similar afterward.
[3]: Mailing list post for v3 https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020937.html
[4]: Original PR #25038 also contains a lot of the discussion
[5]: https://delvingbitcoin.org/t/an-overview-of-the-cluster-mempool-proposal/393/7
[6]: https://delvingbitcoin.org/t/an-overview-of-the-cluster-mempool-proposal/393#the-cpfp-carveout-rule-can-no-longer-be-supported-12
ACKs for top commit:
sdaftuar:
ACK 29029df5c7
achow101:
ACK 29029df5c7
instagibbs:
ACK 29029df5c7 modulo that
Tree-SHA512: 9664b078890cfdca2a146439f8835c9d9ab483f43b30af8c7cd6962f09aa557fb1ce7689d5e130a2ec142235dbc8f21213881baa75241c5881660f9008d68450
13161ecf03 opt: Skip over barren combinations of tiny UTXOs (Murch)
b7672c7cdd opt: Skip checking max_weight separately (Murch)
1edd2baa37 opt: Cut if last addition was minimal weight (Murch)
5248e2a60d opt: Skip heavier UTXOs with same effective value (Murch)
9124c73742 opt: Tiebreak UTXOs by weight for CoinGrinder (Murch)
451be19dc1 opt: Skip evaluation of equivalent input sets (Murch)
407b1e3432 opt: Track remaining effective_value in lookahead (Murch)
5f84f3cc04 opt: Skip branches with worse weight (Murch)
d68bc74fb2 fuzz: Test optimality of CoinGrinder (Murch)
67df6c629a fuzz: Add CoinGrinder fuzz target (Murch)
1502231229 coinselection: Track whether CG completed (Murch)
7488acc646 test: Add coin_grinder_tests (Murch)
6cc9a46cd0 coinselection: Add CoinGrinder algorithm (Murch)
89d0956643 opt: Tie-break UTXO sort by waste for BnB (Murch)
aaee65823c doc: Document max_weight on BnB (Murch)
Pull request description:
***Please refer to the [topic on Delving Bitcoin](https://delvingbitcoin.org/t/gutterguard-and-coingrinder-simulation-results/279) discussing Gutter Guard/Coingrinder simulation results.***
Adds a coin selection algorithm that minimizes the weight of the input set while creating change.
Motivations
---
- At high feerates, using unnecessary inputs can significantly increase the fees
- Users are upset when fees are relatively large compared to the amount sent
- Some users struggle to maintain a sufficient count of UTXOs in their wallet
Approach
---
So far, Bitcoin Core has used a balanced approach to coin selection, where it will generate multiple input set candidates using various coin selection algorithms and pick the least wasteful among their results, but not explicitly minimize the input set weight. Under some circumstances, we _do_ want to minimize the weight of the input set. Sometimes changeless solutions require many or heavy inputs, and there is not always a changeless solution for Branch and Bound to find in the first place. This can cause expensive transactions unnecessarily. Given a wallet with sufficient funds, `CoinGrinder` will pick the minimal-waste input set for a transaction with a change output. The current implementation only runs `CoinGrinder` at feerates over 3×long-term-feerate-estimate (by default 30 ṩ/vB), which may be a decent compromise between our goal to reduce costs for the users, but still permit transactions at lower feerates to naturally reduce the wallet’s UTXO pool to curb bloat.
Trade-offs
---
Simulations for my thesis on coin selection ([see Section 6.3.2.1 [PDF]](https://murch.one/erhardt2016coinselection.pdf)) suggest that minimizing the input set for all transactions tends to grind a wallet’s UTXO pool to dust (pun intended): an approach selecting inputs per coin-age-priority (in effect similar to “largest first selection”) on average produced a UTXO pool with 15× the UTXO count as Bitcoin Core’s Knapsack-based Coin Selection then (in 2016). Therefore, I do not recommend running `CoinGrinder` under all circumstances, but only at extreme feerates or when we have another good reason to minimize the input set for other reasons. In the long-term, we should introduce additional metrics to score different input set candidates, e.g. on basis of their privacy and wallet health impact, to pick from all our coin selection results, but until then, we may want to limit use of `CoinGrinder` in other ways.
ACKs for top commit:
achow101:
ACK 13161ecf03
sr-gi:
ACK [13161ec](13161ecf03)
sipa:
ACK 13161ecf03
Tree-SHA512: 895b08b2ebfd0b71127949b7dba27146a6d10700bf8590402b14f261e7b937f4e2e1b24ca46de440c35f19349043ed2eba4159dc2aa3edae57721384186dae40
The goal of the function is to erase the wallet transactions that
match the inputted hashes. There is no need to traverse the database,
reading record by record, to then perform single entry removals for
each of them.
To ensure consistency and improve performance, this change-set removes
all tx records within a single atomic db batch operation, as well as
it cleans up code, improves error handling and simplifies the
transactions removal process entirely.
This optimizes the removal of watch-only transactions during the wallet
migration process and the 'removeprunedfunds' RPC command.
This deduplicates code and leads to more consistent and detailed error
messages. Affected are legacy import RPCs (`importpubkey`,
`importmulti`) and other ones where solving data can be provided
(`fundrawtransaction`, `walletcreatefundedpsbt`, `send`, `sendall`).
In the helper `HexToPubKey`, check for three different causes of legacy
public key parsing errors (in this order):
- pubkey is not a hex string
- pubkey doesn't have a valid length (33 or 65 bytes) [NEW]
- pubkey is cryptographically invalid, i.e. not on curve
(`IsFullyValid` check)
and throw a specific error message for each one. Note that the error
code is identical for all of them (-5), so this doesn't break RPC API
compatibility.
The helper is currently used for the RPCs `createmultisig` and
`addmultisigaddress`. The length checks can be removed from the
call-sites and error message checks in the functional tests are adapted.
Given a lot of small amount UTXOs it is possible that the lookahead
indicates sufficient funds, but any combination of them would push us
beyond the current best_weight.
We can estimate a lower bound for the minimal necessary weight to reach
target from the maximal amount and minimal weight in the tail of the
UTXO pool: if adding a number of hypothetical UTXOs of this maximum
amount and minimum weight would not be able to beat `best_weight`, we
can SHIFT to the omission branch, and CUT if the last selected UTXO is
not heavier than the minimum weight of the remainder.
In situations where we have UTXO groups of various weight, we can CUT
rather than SHIFT when we exceeded the max_weight or the best
selection’s weight while the last step was equal to the minimum weight
in the lookahead.
When two successive UTXOs differ in weight but match in effective value,
we can skip the second if the first is not selected, because all input
sets we can generate by swapping out a lighter UTXOs with a heavier UTXO
of matching effective value would be strictly worse.
When two successive UTXOs match in effective value and weight, we can
skip the second if the prior is not selected: adding it would create an
equivalent input set to a previously evaluated.
E.g. if we have three UTXOs with effective values {5, 3, 3} of the same
weight each, we want to evaluate
{5, _, _}, {5, 3, _}, {5, 3, 3}, {_, 3, _}, {_, 3, 3},
but skip {5, _, 3}, and {_, _, 3}, because the first 3 is not selected,
and we therefore do not need to evaluate the second 3 at the same
position in the input set.
If we reach the end of the branch, we must SHIFT the previously selected
UTXO group instead.
Introduces a dedicated data structure to track the total
effective_value available in the remaining UTXOs at each index of the
UTXO pool. In contrast to the approach in BnB, this allows us to
immediately jump to a lower index instead of visiting every UTXO to add
back their eff_value to the lookahead.
CoinGrinder may not be able to exhaustively search all potentially
interesting combinations for large UTXO pools, so we keep track of
whether the search was terminated by the iteration limit.
b58f009d95 test: check that mempool msgs lead to disconnect if uploadtarget is reached (Sebastian Falbesoner)
dd5cf38818 test: check for specific disconnect reasons in feature_maxuploadtarget.py (Sebastian Falbesoner)
73d7372115 test: verify `-maxuploadtarget` limit state via `getnettotals` RPC result (Sebastian Falbesoner)
Pull request description:
This PR improves existing and adds new test coverage for the `-maxuploadtarget` mechanism (feature_maxuploadtarget.py) in the following ways, one commit each:
* verify the uploadtarget state via the `getnettotals` RPC (`uploadtarget` result field):
160d23677a/src/rpc/net.cpp (L581-L582)
Note that reaching the total limit (`target_reached` == True) always implies that the historical blocks serving limits is also reached (`serve_historical_blocks` == False), i.e. it's impossible that both flags are set to True.
* check for peer's specific disconnect reason (in this case, `"historical block serving limit reached, disconnect peer"`):
160d23677a/src/net_processing.cpp (L2272-L2280)
* add a test for a peer disconnect if the uploadtarget is reached and a `mempool` message is received (if bloom filters are enabled):
160d23677a/src/net_processing.cpp (L4755-L4763)
Note that another reason for disconnect after receiving a MEMPOOL msg of a peer is if bloom filters are disabled on the node. This case is already covered in the functional test `p2p_nobloomfilter_messages.py`.
ACKs for top commit:
maflcko:
lgtm ACK b58f009d95
achow101:
ACK b58f009d95
sr-gi:
tACK [b58f009](b58f009d95)
Tree-SHA512: 7439134129695c9c3a7ddc5e39f2ed700f91e7c91f0b7a9e0a783f275c6aa2f9918529cbfd38bb37f9139184e05e0f0354ef3c3df56da310177ec1d6b48b43d0
5ca9b24da1 test: Add makefile target for running unit tests (TheCharlatan)
Pull request description:
`make check` runs a bunch of other subtree tests that exercise code that is hardly ever changed and have a comparatively long runtime. There seems to be no target for running just the unit tests, so add one.
Alternatively the secp256k1 tests could be removed from the `check-local` target, reducing its runtime. This was rejected before though in https://github.com/bitcoin/bitcoin/pull/20264.
ACKs for top commit:
delta1:
utACK 5ca9b24da1
edilmedeiros:
Tested ACK 5ca9b24da1
achow101:
ACK 5ca9b24da1
ryanofsky:
Tested ACK 5ca9b24da1.
Tree-SHA512: 470969d44585d7cc33ad038a16e791db9e2be8469f52ddf122c46f20776fad34e6a48f988861a132c42540158fed05f3cf66fcc3bea05708253daaa35af54339
cc87ee4c39 test: fix intermittent failure in rpc_setban.py --v2transport (Martin Zumsande)
Pull request description:
This test failed for me on master locally:
The reason is that when initiating a v2 connection and being immediately disconnected, a node cannot know if the disconnect happens because the peer only supports v1, or because it has banned you, so it schedules to reconnect with v1. If the test doesn't wait for that, the reconnect can happen at a bad time, resulting in failure in a later `connect_nodes` call.
Also add the test with `--v2transport` to the test runner because banning with v2 seems like a useful thing to have test coverage for.
ACKs for top commit:
delta1:
tested ACK cc87ee4c39
epiccurious:
Concept ACK cc87ee4c39.
achow101:
ACK cc87ee4c39
stratospher:
tested ACK cc87ee4. nice find!
Tree-SHA512: ae234d9b771d9c9c11501ddd93c99cf93257c999de3da62280d4d51806cd246b289c10a5f41fa7d5651b2fb4fdaee753f5b2d6939a99f89d71aa012af4a4d231
Ensure we are checking sigop-adjusted virtual size by creating setups
and packages where sigop cost is larger than bip141 vsize.
Co-authored-by: Gregory Sanders <gsanders87@gmail.com>
e064487ca2 addrman, refactor: improve stochastic test in `AddSingle` (brunoerg)
Pull request description:
This PR changes this algorithm to be O(1) instead of O(n). Also, in the current implementation, if `pinfo->nRefCount` is 0, we created an unnecessary variable (`nFactor`), this changes it. the change is relatively simple and does not cause conflicts.
ACKs for top commit:
achow101:
ACK e064487ca2
amitiuttarwar:
ACK e064487ca2
stratospher:
ACK e064487ca2. simple use of << instead of a loop, didn't observe any behaviour difference before and after.
Tree-SHA512: ff0a65155e47f65d2ce3cb5a3fd7a86efef1861181143df13a9d8e59cb16aee9be2f8801457bba8478b17fac47b015bff5cc656f6fac2ccc071ee7178a38d291
fab41697a5 Allow int8_t optimized vector serialization (MarcoFalke)
facaa14785 Faster std::byte (pre)vector (un)serialize (MarcoFalke)
Pull request description:
Currently, large vectors of `std::byte` are (un)serialized byte-by-byte, which is slow. Fix this, by enabling the already existing optimization for them.
On my system this gives a 10x speedup for `./src/bench/bench_bitcoin --filter=PrevectorDeserializeTrivial`, when `std::byte` are used:
```diff
diff --git a/src/bench/prevector.cpp b/src/bench/prevector.cpp
index 2524e215e4..76b16bc34e 100644
--- a/src/bench/prevector.cpp
+++ b/src/bench/prevector.cpp
@@ -17,7 +17,7 @@ struct nontrivial_t {
static_assert(!std::is_trivially_default_constructible<nontrivial_t>::value,
"expected nontrivial_t to not be trivially constructible");
-typedef unsigned char trivial_t;
+typedef std::byte trivial_t;
static_assert(std::is_trivially_default_constructible<trivial_t>::value,
"expected trivial_t to be trivially constructible");
```
However, the optimization does not cover `signed char`. Fix that as well.
ACKs for top commit:
sipa:
utACK fab41697a5
achow101:
ACK fab41697a5
TheCharlatan:
ACK fab41697a5
Tree-SHA512: a3e20f375fd1d0e0dedb827a8ce528de1173ea69660c8c891ad1343a86b422072f6505096fca0d3f8af4442fbe1378a02e32d5974919d4e88ff06934d0258cba
86960cdb7f wallet: migration, batch addressbook records removal (furszy)
342c45f80e wallet: addressbook migration, batch db writes (furszy)
595bbe6e81 refactor: wallet, simplify addressbook migration (furszy)
d0943315b1 refactor: SetAddressBookWithDB, minimize number of map lookups (furszy)
bba4f8dcb5 refactor: SetAddrBookWithDB, signal only if write succeeded (furszy)
97b0753923 wallet: clean redundancies in DelAddressBook (furszy)
Pull request description:
Commits decoupled from #28574, focused on the address book cloning process
Includes:
1) DB batch operations and flow simplification for the address book migration process.
2) Code improvements to `CWallet::DelAddressBook` and `Wallet::SetAddrBookWithDB` methods.
These changes will let us consolidate all individual write operations that take place during the wallet migration process into a single db txn in the future.
ACKs for top commit:
achow101:
ACK 86960cdb7f
josibake:
reACK 86960cdb7f
Tree-SHA512: 10c941df3cd84fd8662b9c9ca6a1ed2c7402d38c677d2fc66b8b6c9edc6d73e827a5821487bbcacb5569d502934fa548fd10699e2ec45185f869e43174d8b2a1
cfcb9b1ecf test: wallet, coverage for concurrent db transactions (furszy)
548ecd1155 tests: Test for concurrent writes with db tx (Ava Chow)
395bcd2454 sqlite: Ensure that only one SQLiteBatch is writing to db at a time (Ava Chow)
Pull request description:
The way that we have configured SQLite to run means that only one database transaction can be open at a time. Typically, each individual read and write operation will be its own transaction that is opened and committed automatically by SQLite. However, sometimes we want these operations to be batched into a multi-statement transaction, so `SQLiteBatch::TxnBegin`, `SQLiteBatch::TxnCommit`, and `SQLiteBatch::TxnAbort` are used to manage the transaction of the database.
However, once a db transaction is begun with one `SQLiteBatch`, any operations performed by another `SQLiteBatch` will also occur within the same transaction. Furthermore, those other `SQLiteBatch`s will not be expecting a transaction to be active, and will abort it once the `SQLiteBatch` is destructed. This is problematic as it will prevent some data from being written, and also cause the `SQLiteBatch` that opened the transaction in the first place to be in an unexpected state and throw an error.
To avoid this situation, we need to prevent the multiple batches from writing at the same time. To do so, I've implemented added a `CSemaphore` within `SQLiteDatabase` which will be used by any `SQLiteBatch` trying to do a write operation. `wait()` is called by `TxnBegin`, and at the beginning of `WriteKey`, `EraseKey`, and `ErasePrefix`. `post()` is called in `TxnCommit`, `TxnAbort` and at the end of `WriteKey`, `EraseKey`, and `ErasePrefix`. To avoid deadlocking on ` TxnBegin()` followed by a `WriteKey()`, `SQLiteBatch will now also track whether a transaction is in progress so that it knows whether to use the semaphore.
This issue is not a problem for BDB wallets since BDB uses WAL and provides transaction objects that must be used if an operation is to occur within a transaction. Specifically, we either pass a transaction pointer, or a nullptr, to all BDB operations, and this allows for concurrent transactions so it doesn't have this problem.
Fixes#29110
ACKs for top commit:
josibake:
ACK cfcb9b1ecf
furszy:
ACK cfcb9b1ecf
ryanofsky:
Code review ACK cfcb9b1ecf. This looks great and I think it is ready for merge. Just holding off because josibake seemed ready to review https://github.com/bitcoin/bitcoin/pull/29112#issuecomment-1930372190 and might have more feedback.
Tree-SHA512: 2dd5a8e76df52451a40e0b8a87c7139d68a0d8e1bf2ebc79168cc313e192dab87cfa4270ff17fea4f7b370060d3bc9b5d294d50f7e07994d9b5a69b40397c927
Instead of doing one db transaction per removed record,
we now batch all removals in a single db transaction.
Speeding up the process and preventing the wallet from entering
an inconsistent state when any of the intermediate writes fail.
1) Encode destination only once (instead of three).
2) Fail if the entry's linked data cannot be removed.
3) Don't remove entry from memory if db write fail.
4) Notify GUI only if removal succeeded
a17fd33edd GUI: OptionsDialog: Replace verbose two-option font selector with simple combobox with Custom... choice (Luke Dashjr)
98e9ac5199 GUI: Use FontChoice type in OptionsModel settings abstraction (Luke Dashjr)
3a6757eed9 GUI: Load custom FontForMoney from QSettings (Luke Dashjr)
49eb97eff9 GUI: Add possibility for an explicit QFont for FontForMoney in OptionsModel (Luke Dashjr)
f2dfde80b8 GUI: Move "embedded font or not" decision into new OptionsModel::getFontForMoney method (Luke Dashjr)
Pull request description:
This replaces the overly-verbose radio-button font setting (which only allows embedded or autodetected from system) with a simple combobox providing both existing options as well as a custom option to allow the user to select any font of their choice/style.
ACKs for top commit:
pablomartin4btc:
tested ACK a17fd33edd
hebasto:
ACK a17fd33edd, I have reviewed the code and tested it on Ubuntu 22.04. This is a UX improvement. https://github.com/bitcoin-core/gui/pull/497#issuecomment-1341222673 might be addressed in a follow-up.
Tree-SHA512: 2f0a8bc1242a374c4b7dc6e34014400428b6d36063fa0b01c9f62a8bd6078adfbbca93d95c87e4ccb580d982fe10173e1d9a28bcec586591dd3f966c7b90fc5d
facafa90f7 test: Fix CPartialMerkleTree.nTransactions signedness (MarcoFalke)
Pull request description:
It is unsigned in Bitcoin Core, so the tests should match it:
aa9231fafe/src/merkleblock.h (L59)
Large positive values, or "negative" values, are rejected anyway, but it still seems fine to fix this.
The bug was introduced when the code was written in d280617bf5.
(Lowercase `i` means signed, see https://docs.python.org/3/library/struct.html#format-characters)
ACKs for top commit:
theStack:
LGTM ACK facafa90f7
Empact:
ACK facafa90f7
Tree-SHA512: 35ac11bb5382dffe132bfae6097efc343ef6c06b1b4b1545130ca27b228ca6894679004862fee921b095172abaddbef5972c24d9bc195ce970f35643bd4a0f09
b14298c5bc fuzz: remove unused `args` and `context` from `FuzzedWallet` (brunoerg)
Pull request description:
`ArgsManager args` and `WalletContext context` were previously used to create the wallet into `FuzzedWallet`. After fa15861763, they are not used anymore. This PR removes them.
ACKs for top commit:
maflcko:
lgtm ACK b14298c5bc
epiccurious:
utACK b14298c5bc
ryanofsky:
Code review ACK b14298c5bc
Tree-SHA512: 164e6a66ba05e11176a0cf68db6257f0ac07459cf7aa01ec4302b303c156c205a68128373a0b8daba0a6dfbff990af7fa14465a6341a296312fb20ea778c7a8c
Sending multiple large messages is rather slow with the non-optimized python
implementation of ChaCha20.
Apart from the slowness, these tests would also run successfully with v2.
By adding to the test framework a wait until the v2 handshake
is completed, so that p2p_sendtxrcncl.py (which doesn't need
to be changed itself) doesnt't send out any other messages before that.
e2ad343f69 wallet: remove unused `SignatureData` instances in spkm's `FillPSBT` methods (Sebastian Falbesoner)
Pull request description:
These are filled with signature data from a PSBT input, but not used anywhere after, hence they can be removed. Note that the same code is in the `SignPSBTInput` function where the `sigdata` result is indeed used.
ACKs for top commit:
achow101:
ACK e2ad343f69
brunoerg:
crACK e2ad343f69
Tree-SHA512: f0cabcc000bcea6bc7d7ec9d3be2e2a8accbdbffbe35252250ea2305b65a5813fde2b8096fbdd2c7cccdf417ea285183dc325fc2d210d88bce62978ce642930e
2bb25ce502 wallet: remove unused 'accept_no_keys' arg from decryption process (furszy)
Pull request description:
Found it while reviewing other PR. Couldn't contain myself from cleaning it up.
The wallet decryption process (`CheckDecryptionKey()` and `Unlock()`)
contains an arg 'accept_no_keys,' introduced in #13926, that has
never been used.
Additionally, this also removes the unimplemented `SplitWalletPath`
function.
ACKs for top commit:
delta1:
ACK 2bb25ce502
epiccurious:
utACK 2bb25ce502.
achow101:
ACK 2bb25ce502
theStack:
Code-review ACK 2bb25ce502
Tree-SHA512: e0537c994be19ca0032551d8a64cf1755c8997e04d21ee0522b31de26ad90b9eb02a8b415448257b60bced484b9d2a23b37586e12dc5ff6e35bdd8ff2165c6bf
There are occasions where a multi-statement tx is begun in one batch,
and a second batch is created which does a normal write (without a
multi-statement tx). These should not conflict with each other and all
of the data should end up being written to disk.
A SQLiteBatch need to wait for any other batch to finish writing before
it can begin writing, otherwise db txn state may be incorrectly
modified. To enforce this, each SQLiteDatabase has a semaphore which
acts as a lock and is acquired by a batch when it begins a write, erase,
or a transaction, and is released by it when it is done.
To avoid deadlocking on itself for writing during a transaction,
SQLiteBatch also keeps track of whether it has begun a transaction.
e7fd70f4b6 [test] make v2transport arg in addconnection mandatory and few cleanups (stratospher)
Pull request description:
- make `v2transport` argument in `addconnection` regression-testing only RPC mandatory. https://github.com/bitcoin/bitcoin/pull/24748#discussion_r1470738750
- previously it was an optional arg with default `false` value.
- only place this RPC is used is in the [functional tests](11b436a66a/test/functional/test_framework/test_node.py (L742)) where we always pass the appropriate `v2transport` option to the RPC anyways. (and that too just for python dummy peer(`P2PInterface`) and bitcoind(`TestNode`) interactions)
- rename `v2_handshake()` to `_on_data_v2_handshake()` https://github.com/bitcoin/bitcoin/pull/24748#discussion_r1466958424
- more compact return statement in `wait_for_reconnect()` https://github.com/bitcoin/bitcoin/pull/24748#discussion_r1466979708
- assertion to check that empty version packets are received from `TestNode`.
ACKs for top commit:
glozow:
ACK e7fd70f4b6
theStack:
Code-review ACK e7fd70f4b6
mzumsande:
Code Review ACK e7fd70f4b6
Tree-SHA512: e66e29baccd91e1e4398b91f7d45c5fc7c2841d77d8a6178734586017bf2be63496721649da91848dec71da605ee31664352407d5bb896e624cc693767c61a1f
c340503b67 test: p2p: adhere to typical VERSION message protocol flow (Sebastian Falbesoner)
7ddfc28309 test: p2p: process post-v2-handshake data immediately (Sebastian Falbesoner)
b198b9c2ce test: p2p: introduce helper for sending prepared VERSION message (Sebastian Falbesoner)
Pull request description:
This PR addresses a quirk in the test framework's p2p implementation regarding the version handshake protocol:
Currently, the VERSION message is sent immediately after an inbound connection (i.e. TestNode outbound connection) is made. This doesn't follow the usual protocol flow where the initiator sends a version first, the responder processes that and only then responds with its own version message. Change that accordingly by only sending immediate VERSION message for outbound connections (or after v2 handshake for v2 connections, respectively), and sending out VERSION message as response for incoming VERSION messages (i.e. in the function `on_version`) for inbound connections.
I first stumbled upon this issue through reading comment https://mirror.b10c.me/bitcoin-bitcoin/24748/#discussion_r1465420112 (see last paragraph) and recently again in the course of working on a v2-followup for #29279, where this causes issues for TestNode outbound connections that disconnect *before* sending out their own version message.
Note that these changes lead to slightly more code in some functional tests that override the `on_version` method, as the version reply has to be sent explicitly now, but I think is less confusing and reflects better what is actually happening.
ACKs for top commit:
epiccurious:
utACK c340503b67
stratospher:
tested ACK c340503b67. very useful to have since we'd want real node behaviour!
mzumsande:
ACK c340503b67
sr-gi:
tACK c340503b67
Tree-SHA512: 63eac287d3e1c87a01852bfd9f0530363354bbb642280298673b9c8817056356373adf348955c4e92af95c7c6efa8cc515cee2892e9f077bfbe1bce8e97ad082
Executing the unit tests for the bip324_cipher.py module currently
takes quite long (>60 seconds on my notebook). Most time here is spent
in empty plaintext/ciphertext encryption/decryption loops:
....
for _ in range(msg_idx):
enc_aead.encrypt(b"", b"")
...
for _ in range(msg_idx):
enc_aead.decrypt(b"", bytes(16))
...
Their sole purpose is increasing the FSChaCha20Poly1305 packet
counters in order to trigger rekeying, i.e. the actual
encryption/decryption is not relevant, as the result is thrown away.
This commit speeds up the tests by supporting to pass "None" as
plaintext/ciphertext, indicating to the routines that no actual
encryption/decryption should be done.
master branch:
$ python3 -m unittest ./test/functional/test_framework/crypto/bip324_cipher.py
..
----------------------------------------------------------------------
Ran 2 tests in 64.658s
PR branch:
$ python3 -m unittest ./test/functional/test_framework/crypto/bip324_cipher.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.822s
Note that another reason for disconnect after receiving a MEMPOOL msg of a peer
is if bloom filters are disabled on the node. This case is covered in the
functional test `p2p_nobloomfilter_messages.py`.
This ensures that the disconnect happens for the expected reason and
also makes it easier to navigate between implementation and test code,
i.e. both the questions "do we have test coverage for this disconnect?"
(from an implementation reader's perspective) and "where is the code
handling this disconnect?" (from a test reader's perspective) can be
answered simply by grep-ping the corresponding debug message.
c819a83b4d Don't use scientific notation in log messages (Kristaps Kaupe)
Pull request description:
Don't see any benefits here, only harder to read for most of the users.
Before:
```
2024-01-16T13:11:36Z Dumped mempool: 8.165e-06s to copy, 0.00224268s to dump
```
After:
```
2024-01-16T13:11:36Z Dumped mempool: 0.000s to copy, 0.002s to dump
```
ACKs for top commit:
kristapsk:
> > > > lgtm ACK [c819a83](c819a83b4d). can you update the PR description?
glozow:
lgtm ACK c819a83b4d. can you update the PR description?
Tree-SHA512: 0972e0a05934e1b014fdeca0c235065aa017ba9abf74b3018f514e4d8022ef02b7f042a07d3675144b51449492468aea6b5b0183233ad7f1bab887d18e3d06af
fa5cd66f0a test: Assumeutxo with more than just coinbase transactions (MarcoFalke)
Pull request description:
Currently the AU tests only check that loading a txout set with only coinbase outputs works.
Fix that by adding other transactions.
ACKs for top commit:
jamesob:
ACK fa5cd66f0a
glozow:
concept ACK fa5cd66f0a
Tree-SHA512: e090c41f73490ad72e36c478405bfd0716d46fbf5a131415095999da6503094a86689a179a84addae3562b760df64cdb67488f81692178c8ca8bf771b1e931ff
make check runs a bunch of other subtree tests that exercise code that
is hardly ever changed and have a comparatively long runtime. There
seems to be no target for running just the unit tests, so add one.
The wallet decryption process (CheckDecryptionKey() and Unlock())
contains an arg 'accept_no_keys,' introduced in #13926, that has
never been used.
Additionally, this also removes the unimplemented SplitWalletPath
function.
4da76ca247 test: Test migration of tx with both spendable and watchonly (Ava Chow)
c62a8d03a8 wallet: Keep txs that belong to both watchonly and migrated wallets (Ava Chow)
71cb28ea8c test: Make sure that migration test does not rescan on reloading (Ava Chow)
78ba0e6748 wallet: Reload the wallet if migration exited early (Ava Chow)
9332c7edda wallet: Write bestblock to watchonly and solvable wallets (Ava Chow)
Pull request description:
A transaction does not necessarily have to belong to either the migrated wallet (with the private keys) and the watchonly wallet (with watchonly things), it could have multiple outputs with each isminetype. So we should be putting such transactions in one or the other wallet, but rather putting it in both.
I've added a test for this behavior, however the test also revealed a few other issues. Notably, it revealed that `migratewallet` would have the watchonly wallet rescan from genesis when it is reloaded at the end of migration. This could be a cause for migration appearing to be very slow. This is resolved by first writing best block records to the watchonly and solvable wallets, as well as updating the test to make sure that rescans don't happen.
The change to avoid rescans also found an issue where some of our early exits would result in unloading the wallet even though nothing happened. So there is also a commit to reload the wallet for such early exits.
ACKs for top commit:
ryanofsky:
Code review ACK 4da76ca247. This looks great. The code is actually cleaner than before, two bugs are fixed, and the test checking for rescanning is pretty clever and broadens test coverage.
furszy:
Code review ACK 4da76ca2
Tree-SHA512: 5fc210cff16ca6720d7b2d0616d7e3f295c974147854abc704cf99a3bfaad17572ada084859e7a1b1ca94da647ad130303219678f429b7995f85e040236db35c
3904123da9 tests: Test that descriptors flag is set for migrated blank wallets (Ava Chow)
072d506240 wallet: Make sure that the descriptors flag is set for blank wallets (Ava Chow)
Pull request description:
While rebasing #28710 after #28976 was merged, I realized that although blank wallets were being moved to sqlite, `WALLET_FLAG_DESCRIPTORS` was not being set so those blank wallets would still continue to be treated as legacy wallets.
To fix that, just set the descriptor flags for blank wallets. Also added a test to catch this.
ACKs for top commit:
epiccurious:
Tested ACK 3904123da9.
delta1:
tested ACK 3904123da9
ryanofsky:
Code review ACK 3904123da9
murchandamus:
code review ACK 3904123da9
Tree-SHA512: 9f6fe9c1899ca387ab909b1bb6956cd6bc5acbf941686ddc6c061f9b1ceec2cc9d009ff472486fc86e963f6068f0e2f1ae96282e7c630193797a9734c4f424ab
When initiating a v2 connection and being immediately disconnected,
a node cannot know if the disconnect happens because the peer only
supports v1, or because it has banned you, so it schedules to reconnect with v1.
If the test doesn't wait for that, the reconnect can happen at a bad time,
resulting in failure in a later connect_nodes call.
Also add the test with --v2transport to the test runner.
Even when the node believes it completed IBD, need to avoid
requesting historical blocks from network-limited peers.
Otherwise, the limited peer will disconnect right away.
Requesting historical blocks from network limited peers is a
direct disconnection cause.
The node must only request the blocks who know for sure the
limited peer can provide.
It is possible for a transaction that has an output that belongs to the
mgirated wallet, and another output that belongs to the watchonly
wallet. Such transaction should appear in both wallets during migration.
We want to make sure that all of the transactions are being copied to
the watchonly and solvable wallets as expected. The automatic rescanning
behavior can cause us to pass a test by finding the transaction
on loading rather than having it be copied as expected.
Migration will unload loaded wallets prior to beginning. It will then
perform some checks which may exit early. Such unloaded wallets should
be reloaded prior to exiting.
When migrating, we should also be writing the bestblock record to the
watchonly and solvable wallets to avoid rescanning on the reload as that
can be slow.
25dc87e6f8 libconsensus: deprecate (Cory Fields)
Pull request description:
This library has existed for nearly 10 years with very little known uptake or impact. It has become a maintenance burden. In several cases it dictates our code/library structure (for example necessitating LIBBITCOIN_CRYPTO_BASE), as well as build-system procedures (building multiple copies of object files especially for the lib).
Several discussions have arisen wrt migrating it to CMake and it has become difficult to justify adding more complexity for a library that is virtually unused anyway.
See for example the discussions:
https://github.com/hebasto/bitcoin/pull/41https://github.com/bitcoin/bitcoin/pull/29123
And here: https://github.com/bitcoin/bitcoin/pull/29180
Where it is pointed out that the libbitcoinconsensus functions are slower than those the internal bitcoind equivalents due to the missing sha2 implementations.
Instead, we (fanquake, hebasto, TheCharlatan, and I) propose simply not migrating it to CMake and letting it end with v27. Any remaining use-cases could be handled in the future by libbitcoinkernel.
If there are any users currently using libbitcoinconsensus, please chime in with your use-case!
Edit: Corrected final release to be v27.
ACKs for top commit:
TheCharlatan:
ACK 25dc87e6f8
fanquake:
ACK 25dc87e6f8 - this library has very little, if any impactful real world usage. It has been entirely broken (on various platforms) for long periods of its existence, where nobody even noticed. Pruning this out to save porting, and starting anew with the kernel, is the sane thing to do.
Tree-SHA512: baff2b3c4f76f520c96021035f751fdcb51bedf00e767660249e92a7bc7c5c176786bcf2c4cfe2d2351c200f932b39eb886bcfb22fbec824a41617590d6a1638
fad74bbbd0 refactor: Mark prevector iterator with std::contiguous_iterator_tag (MarcoFalke)
fab8a01048 refactor: Fix binary operator+ for prevector iterators (MarcoFalke)
fa44a60b2b refactor: Fix constness for prevector iterators (MarcoFalke)
facaa66b49 refactor: Add missing default constructor to prevector iterators (MarcoFalke)
Pull request description:
Currently prevector iterators have many issues:
* Forward iterators (and stronger) must be default constructible (https://eel.is/c++draft/forward.iterators#1.2). Otherwise, some functions can not be instantiated, like `std::minmax_element`.
* Various `const` issues with random access iterators. For example, a `const iterator` is different from a `const_iterator`, because the first one holds a mutable reference and must also return it without `const`. Also, `operator+` must be callable regardless of the iterator object's `const`-ness.
* When adding an offset to random access iterators, both `x+n` and `n+x` must be specified, see https://eel.is/c++draft/random.access.iterators#tab:randomaccessiterator
Fix all issues.
Also, upgrade the `std::random_access_iterator_tag` (C++17) to `std::contiguous_iterator_tag` (C++20)
ACKs for top commit:
TheCharlatan:
ACK fad74bbbd0
stickies-v:
ACK fad74bbbd0
willcl-ark:
ACK fad74bbbd0
Tree-SHA512: b1ca778a31602af94b323b8feaf993833ec78be09f1d438a68335485a4ba97f52125fdd977ffb9541b89f8d45be0105076aa07b5726936133519aae832556e0b
The test framework's p2p implementation currently sends out it's VERSION
message immediately after an inbound connection (i.e. TestNode outbound
connection) is made. This doesn't follow the usual protocol flow where
the initiator sends a version first, and the responders processes that
and only then responds with its own version message. Change that
accordingly by only sending immediate VERSION message for outbound
connections (or after v2 handshake for v2 connections, respectively),
and sending out VERSION messages as response for incoming VERSION
messages (i.e. in the function `on_version`) for inbound connections.
Note that some of the overruled `on_version` methods in functional tests
needed to be changed to send the version explicitly.
In the course of executing the asyncio data reception callback during a
v2 handshake, it's possible that the receive buffer already contains
data for after the handshake (usually a VERSION message for inbound
connections).
If we don't process that data immediately, we would do so after the next
message is received, but with the adapted protocol flow introduced in
the next commit, there is no next message, as the TestNode wouldn't
continue until we send back our own version in `on_version`. Fix this by
calling `self._on_data` immediately if there's data left in the receive
buffer after a completed v2 handshake.
This deduplicates code for sending out the VERSION message
(if available and not sent yet), currently used at three
different places:
1) in the `connection_made` asyncio callback
(for v1 connections that are not v2 reconnects)
2) at the end of `v2_handshake`, if the v2 handshake succeeded
3) in the `on_version` callback, if a reconnection with v1 happens
b851c5385d fuzz: extend ConsumeNetAddr() to return I2P and CJDNS addresses (Vasil Dimov)
Pull request description:
In the process of doing so, refactor `ConsumeNetAddr()` to generate the addresses from IPv4, IPv6, Tor, I2P and CJDNS networks in the same way - by preparing some random stream and deserializing from it. Similar code was already found in `RandAddr()`.
ACKs for top commit:
achow101:
ACK b851c5385d
mzumsande:
ACK b851c5385d
brunoerg:
utACK b851c5385d
Tree-SHA512: 9905acff0e996f30ddac0c14e5ee9e1db926c7751472c06d6441111304242b563f7c942b162b209d80e8fb65a97249792eef9ae0a96100419565bf7f59f59676
9642aefb81 test: fix intermittent failure in p2p_v2_earlykeyresponse (Martin Zumsande)
Pull request description:
The test fails intermittently, see https://cirrus-ci.com/task/6403578080788480?logs=ci#L3521 and https://github.com/bitcoin/bitcoin/pull/24748#issuecomment-1916996716.
I think it's because of a race between the python NetworkThread and the actual
test, which will both call `initiate_v2_handshake`. I could reproduce it by adding a sleep into `initiate_v2_handshake` after the line `self.sent_garbage = random.randbytes(garbage_len)`.
Fix this by waiting for the first `initiate_v2_handshake` to have finished before calling it a second time.
ACKs for top commit:
stratospher:
tested ACK 9642aef.
achow101:
ACK 9642aefb81
theStack:
Tested ACK 9642aefb81
Tree-SHA512: f728bbceaf816ddefeee4957494ccb608ad4fc912cb5cbf5f2acf09836df969c4e8fa2bb441aadb94fa39b3ffbb005d4132e7b6a5a98d80811810d8bd1d624e3
987a1b51ee init: settings, do not load auto-generated warning msg (furszy)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/pull/29144#issuecomment-1907071391.
The settings warning message is meant to be used only to discourage users from
modifying the file manually. Therefore, there is no need to keep it in memory.
ACKs for top commit:
achow101:
ACK 987a1b51ee
ryanofsky:
Code review ACK 987a1b51ee. Seems like a clean, simple fix
Tree-SHA512: 3f2bdcf4b4a9cadb396dcff9b43155211eeed018527a07356970a341d139ad18edbd1a4d369377c8907b8ec1f19ee2ab8aacf85a887379e6d57a8a6db2403d51
c11c404281 tests: Test migration of blank wallets (Andrew Chow)
563b2a60d6 wallet: Better error message when missing LegacySPKM during migration (Andrew Chow)
b1d2c771d4 wallet: Check for descriptors flag before migration (Andrew Chow)
8c127ff1ed wallet: Skip key and script migration for blank wallets (Andrew Chow)
Pull request description:
Blank wallets (wallets without any keys are scripts) are being detected as already being descriptor wallets even though they are not. This is because the check for whether a wallet is already a descriptor wallet uses the presence of a `LegacyScriptPubKeyMan` which is only setup when keys or scripts are found. This PR resolves this issue by checking for the descriptor wallet flag instead and subsequently skipping the keys and scripts part of migration for blank wallets.
Fixes the issue mentioned in https://github.com/bitcoin/bitcoin/pull/28868#issuecomment-1809641110
ACKs for top commit:
furszy:
reACK c11c404281. CI failure is unrelated.
ryanofsky:
Code review ACK c11c404281
Tree-SHA512: 2466fdf1542eb8489c841253191f85dc88365493f0bb3395b67dee3e43709a9993c68b9d7623657b54b779adbe68fc81962d60efef4802c5d461f154167af7f4
ff9039f6ea Remove GetAdjustedTime (dergoegge)
Pull request description:
This picks up parts of #25908.
The use of adjusted time is removed from validation code while the warning to users if their clock is out of sync with the rest of the network remains.
ACKs for top commit:
naumenkogs:
ACK ff9039f6ea
achow101:
ACK ff9039f6ea
maflcko:
lgtm ACK ff9039f6ea🤽
stickies-v:
ACK ff9039f6ea
Tree-SHA512: d1f6b9445c236915503fd2ea828f0d3b92285a5dbc677b168453276115e349972edbad37194d8becd9136d8e7219b576af64ec51c72bdb1923e57e405c0483fc
0bef1042ce net: enable v2transport by default (Pieter Wuille)
Pull request description:
This enables BIP324's v2 transport by default (see #27634):
* Inbound connections will auto-sense whether v1 or v2 is in use.
* Automatic outbound connections will use v2 if `NODE_P2P_V2` was set in addr gossip, but retry with v1 if met with immediate failure.
* Manual outbound connections will default to v2, but retry with v1 if met with immediate failure.
It remains possible to run with `-v2transport=0` to disable all of these, and make all outbound and inbound connections v1. It also remains possible to specify the `v2transport` argument to the `addnode` RPC as `false`, to disable attempting a v2 connection for that particular added node.
ACKs for top commit:
stratospher:
ACK 0bef104.
josibake:
reACK 0bef1042ce
achow101:
ACK 0bef1042ce
naumenkogs:
ACK 0bef1042ce
theStack:
ACK 0bef1042ce
willcl-ark:
crACK 0bef1042ce
BrandonOdiwuor:
utACK 0bef1042ce
pablomartin4btc:
re ACK 0bef1042ce
kristapsk:
utACK 0bef1042ce
Tree-SHA512: 3f17a91e318b9304c40c74a7a5b231149f664ae684d13e9739a05be6c05ba9720f3c3c62da6a73ace0ae8ce733f1c8410b211f9fa15694e6a8d28999ab9882d8
b298242c8d test: sqlite, add coverage for dangling to-be-reverted db txns (furszy)
fc0e747192 sqlite: guard against dangling to-be-reverted db transactions (furszy)
472d2ca981 sqlite: introduce HasActiveTxn method (furszy)
dca874e838 sqlite: add ability to interrupt statements (furszy)
fdf9f66909 test: wallet db, exercise deadlock after write failure (furszy)
Pull request description:
Discovered while was reviewing #29112, specifically https://github.com/bitcoin/bitcoin/pull/29112#pullrequestreview-1821862931.
If the db handler that initiated the database transaction is destroyed,
the ongoing transaction cannot be left dangling when the db txn fails
to abort. It must be forcefully reverted; otherwise, any subsequent
db handler executing a write operation will dump the dangling,
to-be-reverted transaction data to disk.
This not only breaks the isolation property but also results in the
improper storage of incomplete information on disk, impacting
the wallet consistency.
This PR fixes the issue by resetting the db connection, automatically
rolling back the transaction (per https://www.sqlite.org/c3ref/close.html)
when the handler object is being destroyed and the txn abortion failed.
Testing Notes
Can verify the failure by reverting the fix e5217fea and running the test.
It will fail without e5217fea and pass with it.
ACKs for top commit:
achow101:
ACK b298242c8d
ryanofsky:
Code review ACK b298242c8d. Just fix for exec result codes and comment update since last review.
Tree-SHA512: 44ba0323ab21440e79e9d7791bc1c56a8873c8bd3e8f6a85641b91576e1293011fa8032d8ae5b0580f3fb7a949356f7b9676693d7ceffa617aaad9f6569993eb
`TestNode::add_outbound_p2p_connection()` is the only place where
addconnection test-only RPC is used. here, we always pass the
appropriate v2transport option to addconnection RPC.
currently the v2transport option for addconnection RPC is optional.
so simply make the v2transport option mandatory instead.
27f260aa6e net: remove now unused global 'g_initial_block_download_completed' (furszy)
aff7d92b15 test: add coverage for peerman adaptive connections service flags (furszy)
6ed53602ac net: peer manager, dynamically adjust desirable services flag (furszy)
9f36e591c5 net: move state dependent peer services flags (furszy)
f9ac96b8d6 net: decouple state independent service flags from desirable ones (furszy)
97df4e3887 net: store best block tip time inside PeerManager (furszy)
Pull request description:
Derived from #28120 discussion.
By relocating the peer desirable services flags into the peer manager, we
allow the connections acceptance process to handle post-IBD potential
stalling scenarios.
The peer manager will be able to dynamically adjust the services flags
based on the node's proximity to the tip (back and forth). Allowing the node
to recover from the following post-IBD scenario:
Suppose the node has successfully synced the chain, but later experienced
dropped connections and remained inactive for a duration longer than the limited
peers threshold (the timeframe within which limited peers can provide blocks). In
such cases, upon reconnecting to the network, the node might only establish
connections with limited peers, filling up all available outbound slots. Resulting
in an inability to synchronize the chain (because limited peers will not provide
blocks older than the `NODE_NETWORK_LIMITED_MIN_BLOCKS` threshold).
ACKs for top commit:
achow101:
ACK 27f260aa6e
vasild:
ACK 27f260aa6e
naumenkogs:
ACK 27f260aa6e
mzumsande:
Light Code Review ACK 27f260aa6e
andrewtoth:
ACK 27f260aa6e
Tree-SHA512: 07befb9bcd0b60a4e7c45e4429c02e7b6c66244f0910f4b2ad97c9b98258b6f46c914660a717b5ed4ef4814d0dbfae6e18e6559fe9bec7d0fbc2034109200953
26ad2aeb29 test: fix wallet_import_rescan unrounded minimum amount (stickies-v)
Pull request description:
Addresses https://github.com/bitcoin/bitcoin/pull/29283#discussion_r1468842089.
Fixes a `JSONRPCException: Invalid amount (-3)` exception by ensuring the amount sent to `sendtoaddress` is rounded to 8 decimals.
See https://cirrus-ci.com/task/5562947183837184?logs=ci#L2559
Note: since `round` can also round down, `min_amount` is not _exactly_ guaranteed, but this is not a problem for the current usage. I've added a docstring to highlight this.
ACKs for top commit:
sr-gi:
ACK [26ad2ae](26ad2aeb29)
Tree-SHA512: 82ce16447f30535f17fa73336f7e4f74639e33215a228294b9b8005b8050a760b90a3726de279cce98c7e439f09104172b74072be3a300dbd461bf0c3f54b954
This library has existed for nearly 10 years with very little known uptake or
impact. It has become a maintenance burden. In several cases it dictates our
code/library structure (for example necessitating LIBBITCOIN_CRYPTO_BASE), as
well as build-system procedures (building multiple copies of object files
especially for the lib).
Several discussions have arisen wrt migrating it to CMake and it has become
difficult to justify adding more complexity for a library that is virtually
unused anyway.
See for example the discussions:
https://github.com/hebasto/bitcoin/pull/41https://github.com/bitcoin/bitcoin/pull/29123
Instead, we (fanquake, hebasto, TheCharlatan, and I) propose simply not
migrating it to CMake and letting it end with v27. Any remaining use-cases
could be handled in the future by libbitcoinkernel.
If the handler that initiated the database transaction is destroyed,
the ongoing transaction cannot be left dangling when the db txn fails
to abort. It must be forcefully reversed; otherwise, any subsequent
db handler executing a write operation will dump the dangling,
to-be-reverted transaction data to disk.
This not only breaks the database isolation property but also results
in the improper storage of incomplete information on disk, impacting
the wallet consistency.
By encapsulating sqlite3_exec into its own standalone method
and introducing the 'SQliteExecHandler' class, we enable the
ability to test db statements execution failures within the
unit test framework.
This is used in the following-up commit to exercise a deadlock
and improve our wallet db error handling code.
Moreover, the future encapsulation of other sqlite functions
within this class will contribute to minimize the impact of
any future API changes.
31cce4a1bd doc: update `BroadcastTransaction` comment (ismaelsadeeq)
Pull request description:
`BroadcastTransaction` is also called by `submitpackage` RPC.
All transactions that are accepted into the mempool post package processing are broadcasted to peers individually here
ea4ddd8652/src/rpc/mempool.cpp (L926)
It's not maintainable to list all the callers of a function.
ACKs for top commit:
stickies-v:
ACK 31cce4a1bd
kristapsk:
ACK 31cce4a1bd
naumenkogs:
ACK 31cce4a1bd
Tree-SHA512: 8aea92c53c1911a0ac36fe9e3a24d37d83e7d9b40a16f0832bfa7a719328697621e3f94a5dc80d1840e7ae705e0c3aab7a3df7064986e1e53a4a4114adf078a8
9819db4cca validation: move nChainTx assert down in CheckBlockIndex (Martin Zumsande)
033477dba6 doc: fix checkblockindex comments (Martin Zumsande)
Pull request description:
The two assumptions there were described as test-only, which has led to confusion whether they should exist.
However, they are necessary in general, as the changed comment explains - without them, the check would fail everywhere where it is enabled.
The second commit moves this assert down to the other checks.
Closes#29261
ACKs for top commit:
maflcko:
ACK 9819db4cca 🌦
naumenkogs:
ACK 9819db4cca
ryanofsky:
Code review ACK 9819db4cca. Thanks for figuring this issue out and fixing it. Would suggest changing pr name from "improve comments" to "fix misleading comments" since previous comments were wrong about the reasons the conditions are needed.
Tree-SHA512: 3f77791253eb0c97f8153dd8ae1c567f43f6387ea7a53efea94817463c672a4e11d548aa7eff62235346ff0713ff4d6fe08f9ec50d0c30a1e6b6d27b9918b419
55556a64a8 test: Remove struct import from messages.py (MarcoFalke)
fa3fa86dda scripted-diff: test: Use int from_bytes and to_bytes over struct packing (MarcoFalke)
fafc0d68ee test: Use int from_bytes and to_bytes over struct packing (MarcoFalke)
fa3886b7c6 test: Treat msg_version.relay as unsigned (MarcoFalke)
Pull request description:
`struct` has many issues in messages.py:
* For unpacking, it requires to specify the length a second time, even when it is already clear from the `f.read(num_bytes)` context.
* For unpacking, it is designed to support a long format string and returning a tuple of many values. However, except for 3 instances in `messages.py`, usually only a single value is unpacked and all those cases require an `[0]` access.
* For packing and unpacking of a single value, the format string consists of characters that may be confusing and may need to be looked up in the documentation, as opposed to using easy to understand self-documenting code.
I presume the above issues lead to accidentally treat `msg_version.relay` as a "signed bool", which is fine, but confusing.
Fix all issues by using the built-in `int` helpers `to_bytes` and `from_bytes` via a scripted diff.
Review notes:
* `struct.unpack` throws an error if the number of bytes passed is incorrect. `int.from_bytes` doesn't know about "missing" bytes and treats an empty byte array as `int(0)`. "Extraneous" bytes should never happen, because all `read` calls are limited in this file. If it is important to keep this error behavior, a helper `int_from_stream(stream, num_bytes, bytes, byteorder, *, **kwargs)` can be added, which checks the number of bytes read from the stream.
* For `struct.pack` and `int.to_bytes` the error behavior is the same, although the error messages are not identical.
ACKs for top commit:
stickies-v:
ACK 55556a64a8
theStack:
re-ACK 55556a64a8
Tree-SHA512: 1cef8cdfd763fb424ed4b8be850a834b83fd0ef47fbea626a29784eb4f4832d44e42c4fe05b298b6070a933ef278b0222289a9955a97c86707e091e20bbb247a
bc9283c441 [test] Add functional test to test early key response behaviour in BIP 324 (stratospher)
ffe6a56d75 [test] Check whether v2 TestNode performs downgrading (stratospher)
ba737358a3 [test] Add functional tests to test v2 P2P behaviour (stratospher)
4115cf9956 [test] Ignore BIP324 decoy messages (stratospher)
8c054aa04d [test] Allow inbound and outbound connections supporting v2 P2P protocol (stratospher)
382894c3ac [test] Reconnect using v1 P2P when v2 P2P terminates due to magic byte mismatch (stratospher)
a94e350ac0 [test] Build v2 P2P messages (stratospher)
bb7bffed79 [test] Use lock for sending P2P messages in test framework (stratospher)
5b91fb14ab [test] Read v2 P2P messages (stratospher)
05bddb20f5 [test] Perform initial v2 handshake (stratospher)
a049d1bd08 [test] Introduce EncryptedP2PState object in P2PConnection (stratospher)
b89fa59e71 [test] Construct class to handle v2 P2P protocol functions (stratospher)
8d6c848a48 [test] Move MAGIC_BYTES to messages.py (stratospher)
595ad4b168 [test/crypto] Add ECDH (stratospher)
4487b80517 [rpc/net] Allow v2 p2p support in addconnection (stratospher)
Pull request description:
This PR introduces support for v2 P2P encryption(BIP 324) in the existing functional test framework and adds functional tests for the same.
### commits overview
1. introduces a new class `EncryptedP2PState` to store the keys, functions for performing the initial v2 handshake and encryption/decryption.
3. this class is used by `P2PConnection` in inbound/outbound connections to perform the initial v2 handshake before the v1 version handshake. Only after the initial v2 handshake is performed do application layer P2P messages(version, verack etc..) get exchanged. (in a v2 connection)
- `v2_state` is the object of class `EncryptedP2PState` in `P2PConnection` used to store its keys, session-id etc.
- a node [advertising](https://github.com/stratospher/blogosphere/blob/main/integration_test_bip324.md#advertising-to-support-v2-p2p) support for v2 P2P is different from a node actually [supporting v2 P2P](https://github.com/stratospher/blogosphere/blob/main/integration_test_bip324.md#supporting-v2-p2p) (differ when false advertisement of services occur)
- introduce a boolean variable `supports_v2_p2p` in `P2PConnection` to denote if it supports v2 P2P.
- introduce a boolean variable `advertises_v2_p2p` to denote whether `P2PConnection` which mimics peer behaviour advertises V2 P2P support. Default option is `False`.
- In the test framework, you can create Inbound and Outbound connections to `TestNode`
1. During **Inbound Connections**, `P2PConnection` is the initiator [`TestNode` <--------- `P2PConnection`]
- Case 1:
- if the `TestNode` advertises/signals v2 P2P support (means `self.nodes[i]` set up with `"-v2transport=1"`), different behaviour will be exhibited based on whether:
1. `P2PConnection` supports v2 P2P
2. `P2PConnection` does not support v2 P2P
- In a real world scenario, the initiator node would intrinsically know if they support v2 P2P based on whatever code they choose to run. However, in the test scenario where we mimic peer behaviour, we have no way of knowing if `P2PConnection` should support v2 P2P or not. So `supports_v2_p2p` boolean variable is used as an option to enable support for v2 P2P in `P2PConnection`.
- Since the `TestNode` advertises v2 P2P support (using "-v2transport=1"), our initiator `P2PConnection` would send:
1. (if the `P2PConnection` supports v2 P2P) ellswift + garbage bytes to initiate the connection
2. (if the `P2PConnection` does not support v2 P2P) version message to initiate the connection
- Case 2:
- if the `TestNode` doesn't signal v2 P2P support; `P2PConnection` being the initiator would send version message to initiate a connection.
2. During **Outbound Connections** [TestNode --------> P2PConnection]
- initiator `TestNode` would send:
- (if the `P2PConnection` advertises v2 P2P) ellswift + garbage bytes to initiate the connection
- (if the `P2PConnection` advertises v2 P2P) version message to initiate the connection
- Suppose `P2PConnection` advertises v2 P2P support when it actually doesn't support v2 P2P (false advertisement scenario)
- `TestNode` sends ellswift + garbage bytes
- `P2PConnection` receives but can't process it and disconnects.
- `TestNode` then tries using v1 P2P and sends version message
- `P2PConnection` receives/processes this successfully and they communicate on v1 P2P
4. the encrypted P2P messages follow a different format - 3 byte length + 1-13 byte message_type + payload + 16 byte MAC
5. includes support for testing decoy messages and v2 connection downgrade(using false advertisement - when a v2 node makes an outbound connection to a node which doesn't support v2 but is advertised as v2 by some malicious
intermediary)
### run the tests
* functional test - `test/functional/p2p_v2_encrypted.py` `test/functional/p2p_v2_earlykeyresponse.py`
I'm also super grateful to @ dhruv for his really valuable feedback on this branch.
Also written a more elaborate explanation here - https://github.com/stratospher/blogosphere/blob/main/integration_test_bip324.md
ACKs for top commit:
naumenkogs:
ACK bc9283c441
mzumsande:
Code Review ACK bc9283c441
theStack:
Code-review ACK bc9283c441
glozow:
ACK bc9283c441
Tree-SHA512: 9b54ed27e925e1775e0e0d35e959cdbf2a9a1aab7bcf5d027e66f8b59780bdd0458a7a4311ddc7dd67657a4a2a2cd5034ead75524420d58a83f642a8304c9811
8531e1e731 ci: Use DEBUG=1 in depends for MSAN jobs (fanquake)
800ddef6b9 ci: use LLVM 17.0.6 in MSAN jobs (fanquake)
Pull request description:
Switch to using LLVM 17.0.6 and `DEBUG=1` in MSAN CI jobs.
ACKs for top commit:
maflcko:
lgtm ACK 8531e1e731
Tree-SHA512: 819889762aeb78f95c4f955978890c6d98884bed0c7ff97ec072f4c7c1119ee3e3268ccab795bb1c801d36a206e16c6c1195e7a2bc7af94b580d17e49c632161
fab97d81ce fuzz: Print coverage summary after run_once (MarcoFalke)
Pull request description:
This can be used to quickly check the coverage effects of a code change or qa-assets change.
ACKs for top commit:
dergoegge:
ACK fab97d81ce
Tree-SHA512: 0ac913c14698f39e76e0e7bf124f182220031796d6443edb34c6e4615e128157cf746da661b216c4640a41964e977249712445ca9c005b1b4a3737adabdb4a7d
5b9d5bf866 depends: remove (darwin) libtool now that it's no longer used (Cory Fields)
3ef6563495 depends: use ar rather than libtool for miniupnpc/libnatpmp (Cory Fields)
Pull request description:
An alternative to https://github.com/bitcoin/bitcoin/pull/29232
Rather than switching to the CMake builds which [proved problematic](https://github.com/bitcoin/bitcoin/pull/29232#issuecomment-1898513919), do the quick and dirty thing of just patching out libtool. Doesn't seem to introduce any new issues.
This should buy us time to upstream the necessary CMake fixes.
ACKs for top commit:
TheCharlatan:
ACK 5b9d5bf866
fanquake:
ACK 5b9d5bf866
Tree-SHA512: c75c4bcc9332d8c1fc3395e2b5fc7265849186afc7005700f662ab291e6ea1f111025fad733d0b0b39d35029d1b757d3f1937d63aad3c0c3b88d0f8ac902ee18
The C++ code treats bool as uint8_t, so the python tests should as well.
This also allows to simplify the code, because converting an empty byte
array to int gives int(0).
>>> int.from_bytes(b'')
0
bbf218d061 crypto: remove sha256_sse4 from the base crypto helper lib (Cory Fields)
4dbd0475d8 crypto: remove use of BUILD_BITCOIN_INTERNAL macro in sha256 (Cory Fields)
Pull request description:
Replace it with a more explicit `DISABLE_OPTIMIZED_SHA256` and clean up some.
The macro was originally used by libbitcoinconsensus which opts out of optimized sha256 for the sake of simplicity.
Also remove the `BUILD_BITCOIN_INTERNAL` define from libbitcoinkernel for now as it does not export an api. When it does we can pick a less confusing define to control its exports.
Removing the define should have the effect of enabling sha256 optimizations for the kernel.
ACKs for top commit:
TheCharlatan:
Re-ACK bbf218d061
hebasto:
re-ACK bbf218d061
Tree-SHA512: 7c17592bb2d3e671779f96903cb36887c5785408213bffbda1ae37b66e6bcfaffaefd0c1bf2d1a407060cd377e3d4881cde3a73c429a1aacb677f370314a066a
3bfc5bd36e test: ensure output is large enough to pay for its fees (stickies-v)
Pull request description:
Fixes a (rare) intermittency issue in wallet_import_rescan.py
Since we [use](03752444cd/test/functional/wallet_import_rescan.py (L296)) `subtract_fee_from_outputs=[0]` in the `send` command, the output amount must at least be as large as the fee we're paying.
Example in CI: https://api.cirrus-ci.com/v1/task/6107972259020800/logs/ci.log
```
2024-01-18T22:16:12.383000Z TestFramework (INFO): Test that the mempool is rescanned as well if the rescan parameter is set to true
2024-01-18T22:16:20.187000Z TestFramework (ERROR): JSONRPC error
Traceback (most recent call last):
File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 131, in main
self.run_test()
File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/wallet_import_rescan.py", line 292, in run_test
child = self.nodes[1].send(
File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/coverage.py", line 50, in __call__
return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/authproxy.py", line 129, in __call__
raise JSONRPCException(response['error'], status)
test_framework.authproxy.JSONRPCException: The transaction amount is too small to pay the fee (-4)
```
Can be reproduced locally by forcing usage of the lowest possible value produced by `get_rand_amount()` ([thanks furszy](https://github.com/bitcoin/bitcoin/pull/29283#pullrequestreview-1832956095)):
<details>
<summary>git diff on 5f3a0574c4</summary>
```diff
diff --git a/test/functional/wallet_import_rescan.py b/test/functional/wallet_import_rescan.py
index 7f01d23941..925849d5c0 100755
--- a/test/functional/wallet_import_rescan.py
+++ b/test/functional/wallet_import_rescan.py
@@ -270,7 +270,7 @@ class ImportRescanTest(BitcoinTestFramework):
address_type=variant.address_type.value,
))
variant.key = self.nodes[1].dumpprivkey(variant.address["address"])
- variant.initial_amount = get_rand_amount() * 2
+ variant.initial_amount = Decimal(str(round(AMOUNT_DUST, 8))) * 2
variant.initial_txid = self.nodes[0].sendtoaddress(variant.address["address"], variant.initial_amount)
variant.confirmation_height = 0
variant.timestamp = timestamp
```
</details>
ACKs for top commit:
achow101:
ACK 3bfc5bd36e
glozow:
utACK 3bfc5bd36e, didn't experience this issue but in theory a minimum of `AMOUNT_DUST` could be too low to pay the fees
furszy:
utACK 3bfc5bd36
Tree-SHA512: 821ab94a510772e90528b2cef368bbf70309d8fd1dcda53dce75dd1bf91622358e80fea4d9fc68249b9d598892306c66f6c843b4a6855a9f9a9175f7b41109c6
8023640a71 qt: Avoid non-self-contained Windows header (Hennadii Stepanov)
Pull request description:
Using the `windows.h` header guarantees correctness regardless of the content of other headers.
For more details, please refer to https://stackoverflow.com/questions/4845198/fatal-error-no-target-architecture-in-visual-studio
Fixes the MSVC build when using the upcoming CMake-based build system and Qt packages installed via the vcpkg package manager.
Related to https://github.com/hebasto/bitcoin/pull/77.
ACKs for top commit:
theuni:
ACK 8023640a71. It's not completely clear to me why this currently works, but I don't think it's worth wasting more time on. `windows.h` seems more correct regardless.
Tree-SHA512: 1c03f909943111fb2663f86d33ec9a947bc5903819e5bd94f436f6b0782d9f5c5d80d9cd3490674ecd8921b2981c509e97e41580bccc436f8b5c7db84b4e493c
cf937b2068 fuzz: also set MSAN_SYMBOLIZER_PATH (fanquake)
Pull request description:
Should resolve: https://github.com/bitcoin-core/qa-assets/issues/167.
ACKs for top commit:
dergoegge:
utACK cf937b2068
Tree-SHA512: a7670b5054c2c9ec830db2a4dd4d78d8a0ee7d793a80d32942d78b5e459015344040fa9ce9d73f4f23cd920d5ca2e65c110e201723e4935de8f57fda0b6d5ce7
This commit update CheckFeeRate's incrementalRelayFee to use relayIncrementalFee
not max of (walletIncrementalRelayfee and relayIncrementalFee).
The restriction is not needed since user provided the fee rate.
fa3373d3ad refactor: Compile unreachable code (MarcoFalke)
Pull request description:
When unreachable code isn't compiled, compile failures are not detected.
Fix this by leaving it unreachable, but compiling it.
Fixes https://github.com/bitcoin/bitcoin/pull/28999#discussion_r1465010916
ACKs for top commit:
achow101:
ACK fa3373d3ad
ryanofsky:
Code review ACK fa3373d3ad. This looks good, and should prevent code in the else blocks from accidentally breaking.
Tree-SHA512: 3a3764915dfc935bf5d7a48f1ca151dcbac340c1cbdce8236b24ae9b4f04d6ee9771ed058ca60bcbca6e19d13671de3517f828a8f7ab6444c7cc4e3538d1ba4e
d298ff8b62 During IBD, prune as much as possible until we get close to where we will eventually keep blocks (Luke Dashjr)
Pull request description:
This should reduce pruning flushes even more, speeding up IBD with pruning on systems that have a sufficient dbcache.
Assumes 1 MB per block between tip and best header chain. Simply adds this to the buffer pruning is trying to leave available, which results in pruning almost everything up until we get close to where we need to be keeping blocks.
ACKs for top commit:
andrewtoth:
ACK d298ff8b62
fjahr:
utACK d298ff8b62
achow101:
ACK d298ff8b62
Tree-SHA512: 2a482376bfb177e2ba7c2f0bb0b58b02efdb38b34755a18d1fc3e869df5959c85b6f1009e1386fa8b89c4f90d520383e36bd3e21dec221042315134efb1a455b
5fb8f0f80f depends: Do not override CFLAGS when building SQLite with DEBUG=1 (Hennadii Stepanov)
2b0dd88f1c depends: Ensure definitions are passed when building SQLite with DEBUG=1 (Hennadii Stepanov)
Pull request description:
The `--enable-debug` configure option for the SQLite package does two things:
```autoconf
#-----------------------------------------------------------------------
# --enable-debug
#
AC_ARG_ENABLE(debug, [AS_HELP_STRING(
[--enable-debug], [build with debugging features enabled [default=no]])],
[], [])
AC_MSG_CHECKING([Build type])
if test x"$enable_debug" = "xyes"; then
BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_DEBUG -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE"
CFLAGS="-g -O0"
AC_MSG_RESULT([debug])
else
AC_MSG_RESULT([release])
fi
#-----------------------------------------------------------------------
```
It adds three preprocessor definitions and overrides `CFLAGS` with `"-g -O0"`. The latter breaks the user's ability to provide sanitizer and LTO flags.
This PR might be especially useful for OSS-Fuzz where `DEBUG=1` has been used since https://github.com/google/oss-fuzz/pull/10503.
Also it makes a workaround for building SQLite for 32-bit unneeded. For details, please refer to https://github.com/hebasto/oss-fuzz/tree/240120-sqlite.
Changes in https://github.com/bitcoin/bitcoin/pull/29282 might not be strictly required now. However, I consider them an improvement.
ACKs for top commit:
fanquake:
ACK 5fb8f0f80f - downstream is also green, so i'll fixup the PR there.
Tree-SHA512: 8593d8a0237ebb270d5da763fb65ed642ab8ed0d44e57704a34154621f49e3d5c58b462cc0070251fa1ba556c58a3c7d3620530d6839dc6dc9e0887010330eca
The `--enable-debug` configure option for the SQLite package does two
things. It adds three preprocessor definitions and overrides CFLAGS with
"-g -O0". The latter breaks the user's ability to provide sanitizer and
LTO flags.
00c1e2aa44 build: fix optimisation flags used for --coverage (fanquake)
1dc2c9b385 ci: cleanup C*FLAG usage in Valgrind jobs (fanquake)
6cc2a38c13 build: add sanitizer flags to configure output (fanquake)
08cd5aca18 build: always set -g -O2 in CORE_CXXFLAGS (fanquake)
Pull request description:
Rather than trying to sporadically rely on / override Autoconf default behaviour. Just always override (if unset), and always set the flags we want (which are the same as the Autoconf defaults).
Removes the need for duplicate code to clear (if not overridden) `CXXFLAGS`.
Fixes cases of "missing" `-O2`. i.e this PR when running a Valgrind CI job with changes here:
```bash
CXXFLAGS = -g -O2 -fdebug-prefix-map=$(abs_top_srcdir)=. -Wstack-protector -fstack-protector-all -mbranch-protection=bti -Werror -fsanitize=fuzzer -gdwarf-4
```
Fixes configure output to reflect actual compilation flag ordering, so it's useful.
Note that if we do still end up with a duplicate "-g -O2" when compiling, that has no effect, and I don't really thinks it's something worth trying to optimize.
ACKs for top commit:
TheCharlatan:
lgtm ACK 00c1e2aa44
hebasto:
ACK 00c1e2aa44, I have reviewed the code and it looks OK. Also tested `ci/test/00_setup_env_native_valgrind.sh`.
theuni:
ACK 00c1e2aa44
Tree-SHA512: cf6c7acf813ba10b198561e83eb72e9b2532a39cb1767c452d031e82921dcd42a47b129735b24c4e36131fd0c8fe7457f7cae870c1e011cdfdd430bdc4d4912b
- A node initiates a v2 connection by sending 64 bytes ellswift
- In BIP 324 "The responder waits until one byte is received which does not match the
V1_PREFIX (16 bytes consisting of the network magic followed by "version\x00\x00\x00\x00\x00".)"
- It's possible that the 64 bytes ellswift sent by an initiator starts with a prefix of V1_PREFIX
- Example form of 64 bytes ellswift could be:
4 bytes network magic + 60 bytes which aren't prefixed with remaining V1_PREFIX
- We test this behaviour:
- when responder receives 4 byte network magic -> no response received by initiator
- when first mismatch happens -> response received by initiator
- Add an optional `supports_v2_p2p` parameter to specify if the inbound
and outbound connections support v2 P2P protocol.
- In the `addconnection_callback` which gets called when creating
outbound connections, call the `addconnection` RPC with v2 P2P protocol
support enabled.
- When a v2 TestNode makes an outbound connection to a P2PInterface node
which doesn't support v2 but is advertised as v2 by some malicious
intermediary, the TestNode sends 64 bytes ellswift. The v1 node doesn't
understand this and disconnects. Then the v2 TestNode reconnects by
sending a v1/version message.
Messages are built, encrypted and sent over the socket in v2
connections. If a race condition happens between python's main
thread and p2p thread with both of them trying to send a message,
it's possible that the messages get encrypted with wrong keystream.
Messages are built and sent over the socket in v1 connections.
So there's no problem if messages are sent in the wrong order.
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
Co-authored-by: theStack <sebastian.falbesoner@gmail.com>
Instantiate this object when the connection supports v2 P2P transport
protocol.
- When a P2PConnection is opened, perform initiate_v2_handshake() if the
connection is an initiator. application layer messages are only sent after
the initial v2 handshake is over (for both initiator and responder).
ff54314d4a wallet: clarify replaced_by_txid and replaces_txid in help output (marco)
Pull request description:
Resolves issue #27781
ACKs for top commit:
achow101:
ACK ff54314d4a
ryanofsky:
Code review ACK ff54314d4a. Seems like a helpful clarification
Tree-SHA512: b13a0e24505dfaee083467ac6f357b96460b5d1841dc29c4df4a503c290d379cef3d50fcc76f33bbc95741f484dd9d2461b0c2e8bdebf57a8a72edfbeece2a79
9d09c873a5 fuzz: Exit and log stderr for parse_test_list errors (dergoegge)
Pull request description:
We should log all errors that occur when attempting to print the harness list in the fuzz test runner.
ACKs for top commit:
maflcko:
lgtm ACK 9d09c873a5
Tree-SHA512: 50471b732c8cbe287dacba14487e7c8a5826f146432d93aa3bb55d063a8ba158d01641d6cb1360241dd4cd54ef5e045b0412f9cc34d06c181134921d1f1ceced
The class `EncryptedP2PState` stores the 4 32-byte keys, session id,
garbage terminators, whether it's an initiator/responder, whether the
initial handshake has been completed etc.. It also contains functions
to perform the v2 handshake and to encrypt/decrypt p2p v2 messages.
- In an inbound connection to TestNode, P2PConnection is the initiator
and `initiate_v2_handshake()`, `complete_handshake()`, `authenticate_handshake()`
are called on it. [ TestNode <----------------- P2PConnection ]
- In an outbound connection from TestNode, P2PConnection is the responder
and `respond_v2_handshake()`, `complete_handshake()`, `authenticate_handshake()`
are called on it. [ TestNode -----------------> P2PConnection ]
The settings warning message is meant to be used only to discourage
users from modifying the file manually. Therefore, there is no need
to keep it in memory.
These exceptions are not related to situations specific to tests,
but are required in general:
Without the first check CheckBlockindex could fail for blocks where we
only know the header.
Without the second, it could fail when blocks are received out of order.
18ad1b9142 refactor: pass CRecipient to FundTransaction (josibake)
5ad19668db refactor: simplify `CreateRecipients` (josibake)
47353a608d refactor: remove out param from `ParseRecipients` (josibake)
f7384b921c refactor: move parsing to new function (josibake)
6f569ac903 refactor: move normalization to new function (josibake)
435fe5cd96 test: add tests for fundrawtx and sendmany rpcs (josibake)
Pull request description:
## Motivation
The primary motivation for this PR is to enable `FundTransaction` to take a vector of `CRecipient` objects to allow passing BIP352 silent payment addresses to RPCs that use `FundTransaction` (e.g. `send`, `walletcreatefundedpsbt`). To do that, SFFO logic needs to be moved out of `FundTransaction` so the `CRecipient` objects with the correct SFFO information can be created and then passed to `FundTransaction`.
As a secondary motivation, this PR moves the SFFO stuff closer to the caller, making the code cleaner and easier to understand. This is done by having a single function which parses RPC inputs for SFFO and consistently using the `set<int>` method for communicating SFFO.
I'm also not convinced we need to pass a full `CMutableTx` object to `FundTransaction`, but I'm leaving that for a follow-up PR/discussion, as its not a blocker for silent payments.
ACKs for top commit:
S3RK:
reACK 18ad1b9142
josibake:
> According to my `range-diff` nothing changed. reACK [18ad1b9](18ad1b9142)
achow101:
ACK 18ad1b9142
Tree-SHA512: d61f017cf7d98489ef216475b68693fd77e7b53a26a6477dcd73e7e5ceff5036b2d21476e377839e710bb73644759d42c4f9f4b14ed96b3e56ed87b07aa6d1a7
6acec6b9ff multiprocess: Add type conversion code for UniValue types (Ryan Ofsky)
0cc74fce72 multiprocess: Add type conversion code for serializable types (Ryan Ofsky)
4aaee23921 test: add ipc test to test multiprocess type conversion code (Ryan Ofsky)
Pull request description:
Add type conversion hooks to allow `UniValue` objects, and objects that have `CDataStream` `Serialize` and `Unserialize` methods to be used as arguments and return values in Cap'nProto interface methods. Also add unit test to verify the hooks are working and data can be round-tripped correctly.
The non-test code in this PR was previously part of #10102 and has been split off for easier review, but the test code is new.
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
achow101:
ACK 6acec6b9ff
dergoegge:
reACK 6acec6b9ff
Tree-SHA512: 5d2cbc5215d488b876d34420adf91205dabf09b736183dcc85aa86255e3804c2bac5bab6792dacd585ef99a1d92cf29c8afb3eb65e4d953abc7ffe41994340c6
e9014042a6 settings: add auto-generated warning msg for editing the file manually (furszy)
966f5de99a init: improve corrupted/empty settings file error msg (furszy)
Pull request description:
Small and simple issue reported [here](https://community.umbrel.com/t/bitcoin-docker-container-keeps-restarting/2144).
Improving a confusing situation reported by users who did not understand why a
settings parsing error occurred when the file was empty and did not know how to solve it.
Empty setting file could be due (1) corruption or (2) an user manually cleaning up the file content.
In both scenarios, the 'Unable to parse settings file' error does not help the user move forward.
ACKs for top commit:
achow101:
ACK e9014042a6
hebasto:
re-ACK e9014042a6.
ryanofsky:
Code review ACK e9014042a6. Just whitespace formatting changes and shortening a test string literal since last review
shaavan:
Code review ACK e9014042a6
Tree-SHA512: 2910654c6b9e9112de391eedb8e46980280f822fa3059724dd278db7436804dd27fae628d2003f2c6ac1599b07ac5c589af016be693486e949f558515e662bec
32a9f13cb8 wallet: avoid returning a reference to vMasterKey after releasing the mutex that guards it (Vasil Dimov)
Pull request description:
`CWallet::GetEncryptionKey()` would return a reference to the internal
`CWallet::vMasterKey`, guarded by `CWallet::cs_wallet`, which is unsafe.
Returning a copy would be a shorter solution, but could have security
implications of the master key remaining somewhere in the memory even
after `CWallet::Lock()` (the current calls to
`CWallet::GetEncryptionKey()` are safe, but that is not future proof).
So, instead of `EncryptSecret(m_storage.GetEncryptionKey(), ...)`
change the `GetEncryptionKey()` method to provide the encryption
key to a given callback:
`m_storage.WithEncryptionKey([](const CKeyingMaterial& k) { EncryptSecret(k, ...); })`
This silences the following (clang 18):
```
wallet/wallet.cpp:3520:12: error: returning variable 'vMasterKey' by reference requires holding mutex 'cs_wallet' [-Werror,-Wthread-safety-reference-return]
3520 | return vMasterKey;
| ^
```
---
_Previously this PR modified both ArgsManager and wallet code. But the ArgsManager commit 856c88776f was merged in https://github.com/bitcoin/bitcoin/pull/29040 so now this only affects wallet code. The previous PR description was:_
Avoid this unsafe pattern from `ArgsManager` and `CWallet`:
```cpp
class A
{
Mutex mutex;
Foo member GUARDED_BY(mutex);
const Foo& Get()
{
LOCK(mutex);
return member;
} // callers of `Get()` will have access to `member` without owning the mutex.
```
ACKs for top commit:
achow101:
ACK 32a9f13cb8
ryanofsky:
Code review ACK 32a9f13cb8. This seems like a potentially real race condition, and the fix here is pretty simple.
furszy:
ACK 32a9f13c
Tree-SHA512: 133da84691642afc1a73cf14ad004a7266cb4be1a6a3ec634d131dca5dbcdef52522c1d5eb04f5b6c4e06e1fc3e6ac57315f8fe1e207b464ca025c2b4edefdc1
d55fdb1a49 Move TRACEx parameters to seperate lines (Richard Myers)
2d58629ee6 wallet: fix coin selection tracing to return -1 when no change pos (Richard Myers)
Pull request description:
This is a bugfix for from when [optional was introduced](758501b713) for `change_pos` in the wallet. When optional `change_pos` is unset, we should return -1 and not 0.
I added two new checks to the `test/functional/interface_usdt_coinselection.py` which adds coverage for the situations when `normal_create_tx_internal` and `aps_create_tx_internal` events occur with no change.
You can reproduce this bug using the coin-selection-simulation scripts as described in [issue #16](https://github.com/achow101/coin-selection-simulation/issues/16). You can also run the `interface_usdt_coinselection.py` test without the changes to `wallet/spend.cpp`.
ACKs for top commit:
0xB10C:
ACK d55fdb1a49
achow101:
ACK d55fdb1a49
murchandamus:
ACK d55fdb1a49
Tree-SHA512: 6efac3b756bdf51debbcb759dc3c4b7a4304626bc047b70025cec02f3a04937ace7712e9558ac71e560fd136005a98c518ac5bb4b90c3282d776beccd0de9749
97181decf5 Add test for negative transaction version w/ CSV to tx_valid.json (Chris Stewart)
Pull request description:
This PR adds a static test vector corresponding to the bug found in various implementations of the bitcoin protocol discovered by dergoegge
For more information see:
https://delvingbitcoin.org/t/disclosure-btcd-consensus-bugs-due-to-usage-of-signed-transaction-version/455
ACKs for top commit:
darosior:
ACK 97181decf5
dergoegge:
ACK 97181decf5
Tree-SHA512: 92bbcd3cd10a569757b4de91e1b2bcfebc2b75ddb0160be36d8e512a6fa4623cced1aba93bd1cc044962cd2b10e1d184ef109ccdfe3cfcf85cf4b9585d80d115
This avoids circular dependency happening when importing MAGIC_BYTES.
Before,
p2p.py <--import for EncryptedP2PState-- v2_p2p.py
| ^
| |
└---------import for MAGIC_BYTES----------┘
Now, MAGIC_BYTES are kept separately in messages.py
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
This test-only RPC is required when a TestNode initiates
an outbound v2 p2p connection. Add a new arg `v2transport`
so that the node can attempt v2 connections.
Introduces functionality to detect when limited peers connections
are desirable or not. Ensuring that the new connections desirable
services flags stay relevant throughout the software's lifecycle.
(Unlike the previous approach, where once the validation IBD flag
was set, the desirable services flags remained constant forever).
This will let us recover from stalling scenarios where the node had
successfully synced, but subsequently dropped connections and remained
inactive for a duration longer than the limited peers threshold (the
timeframe within which limited peers can provide blocks). Then, upon
reconnection to the network, the node may end up only establishing
connections with limited peers, leading to an inability to synchronize
the chain.
This also fixes a possible limited peers threshold violation during IBD,
when the user configures `-maxtipage` further than the BIP159's limits.
This rule violation could lead to sync delays and, in the worst-case
scenario, trigger the same post-IBD stalling scenario (mentioned above)
but during IBD.
In the process of doing so, refactor `ConsumeNetAddr()` to generate the
addresses from IPv4, IPv6, Tor, I2P and CJDNS networks in the same way -
by preparing some random stream and deserializing from it. Similar code
was already found in `RandAddr()`.
Hopefully, refraining users from modifying the file unless they are
certain about the potential consequences.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
The preceding "Unable to parse settings file" message lacked
the necessary detail and guidance for users on what steps to
take next in order to resolve the startup error.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
282b12ddb0 refactor: remove CTxMemPool::queryHashes() (stickies-v)
Pull request description:
`CTxMemPool::queryHashes()` is only used in `MempoolToJSON()`, where it can just as easily be replaced with the more general `CTxMemPool::entryAll()`. No behaviour change, just cleans up the code.
ACKs for top commit:
dergoegge:
Code review ACK 282b12ddb0
TheCharlatan:
ACK 282b12ddb0
glozow:
ACK 282b12ddb0. Looks like there's no conflicts.
Tree-SHA512: 16160dec8e1f2457fa0f62dc96d2d2efd92c4bab810ecdb0e08918b8e85a667702c8e41421eeb4ea6abe92a5956a2a39a7a6368514973b78be0d22de2ad299b2
Fixes a (rare) intermittency issue in wallet_import_rescan.
Since we use `subtract_fee_from_outputs=[0]` in the `send` command,
the output amount must at least be as large as the fee we're paying.
Instead turning tx.vout into a vector of `CRecipient`, make `FundTransaction`
take a `CRecipient` vector directly. This allows us to remove SFFO logic from
the wrapper RPC `FundTransaction` since the `CRecipient` objects have already
been created with the correct SFFO values. This also allows us to remove
SFFO from both `FundTransaction` function signatures.
This sets us up in a future PR to be able to use these RPCs with BIP352
static payment codes.
Move validation logic out of `CreateRecipients` and instead take the
already validated outputs from `ParseOutputs` as an input.
Move SFFO parsing out of `CreateRecipients` into a new function,
`InterpretSubtractFeeFromOutputsInstructions`. This takes the SFFO instructions
from `sendmany` and `sendtoaddress` and turns them into a set of integers.
In a later commit, we will also move the SFFO parsing logic from
`FundTransaction` into this function.
Worth noting: a user can pass duplicate addresses and addresses that dont exist
in the transaction outputs as SFFO args to `sendmany` and `sendtoaddress`
without triggering a warning. This behavior is preserved in to keep this commit
strictly a refactor.
Move the parsing and validation out of `AddOutputs` into its own function,
`ParseOutputs`. This allows us to re-use this logic in `ParseRecipients` in a
later commit, where the code is currently duplicated.
The new `ParseOutputs` function returns a CTxDestination,CAmount tuples.
This allows the caller to then translate the validated outputs into
either CRecipients or CTxOuts.
Move the univalue formatting logic out of AddOutputs and into its own function,
`NormalizeOutputs`. This allows us to re-use this logic in later commits.
If the serialized transaction passed to `fundrawtransaction` contains
duplicates, they will be deserialized and added to the transaction. Add
a test to ensure this behavior is not changed during the refactor.
A user can pass any number of duplicated and unrelated addresses as an
SFFO argument to `sendmany` and the RPC will not throw an error (note,
all the rest of the RPCs which take SFFO as an argument will error if
the user passes duplicates or specifies outputs not present in the
transaction). Add a test to ensure this behavior is not changed during
the refactor.
6ec2813cd8 depends: add NM output to gen_id (fanquake)
Pull request description:
`NM` is part of the current toolset, and can be set by the user. Include it in `gen_id`.
ACKs for top commit:
TheCharlatan:
Re-ACK 6ec2813cd8
Tree-SHA512: 2ada61e03783f9eb441f285ef5da50557ad729cb52ce2d2c4b2c38103dab29920a26262d4545fd2ac7fbf1cedc4902cd2359833544fbc0debf829c12a63e9769
5555d8db33 test: Use blocks_path where possible (MarcoFalke)
fa9108941f rpc: Fix race in loadtxoutset (MarcoFalke)
Pull request description:
The tip may have advanced, also if it did not, there is no reason to
have two variables point to the same block.
Fixes https://github.com/bitcoin/bitcoin/pull/27596#discussion_r1344694600
ACKs for top commit:
achow101:
ACK 5555d8db33
pablomartin4btc:
ACK 5555d8db33
BrandonOdiwuor:
Code Review ACK 5555d8db33
Tree-SHA512: 23a82924a915b61bb1adab8ad20ec8914139c8ee647817af34ca27ee310a2e45833d8b285503e0feebe63e4667193d6d98cfcbbc1509bf40712225e04dd19e8b
fa2b95cf3f test: Remove all-lint.py script (MarcoFalke)
fadb06c361 doc: move-only lint docs to one place (MarcoFalke)
Pull request description:
Seems confusing to have a test runner that calls another runner (`all-lint.py`), which calls a subset of the lint tests.
Fix that by just calling this subset of lint tests in the test runner directly, and remove the now unused `all-lint.py`.
To run all lint checks locally, refer to the documentation: https://github.com/bitcoin/bitcoin/blob/master/test/lint/README.md#running-locally
ACKs for top commit:
kevkevinpal:
ACK [fa2b95c](fa2b95cf3f)
achow101:
ACK fa2b95cf3f
TheCharlatan:
ACK fa2b95cf3f
pablomartin4btc:
tACK fa2b95cf3f
brunoerg:
utACK fa2b95cf3f
Tree-SHA512: 43fac9acb4e9a6744d695dd49c7202e19ab4bf480f4cccff768647d0157a065f40e6ad70b9f6a65ba42048cc5fa9834365aa8e7aa0ed64c09e0cd4eb8dccb831
`CWallet::GetEncryptionKey()` would return a reference to the internal
`CWallet::vMasterKey`, guarded by `CWallet::cs_wallet`, which is unsafe.
Returning a copy would be a shorter solution, but could have security
implications of the master key remaining somewhere in the memory even
after `CWallet::Lock()` (the current calls to
`CWallet::GetEncryptionKey()` are safe, but that is not future proof).
So, instead of `EncryptSecret(m_storage.GetEncryptionKey(), ...)`
change the `GetEncryptionKey()` method to provide the encryption
key to a given callback:
`m_storage.WithEncryptionKey([](const CKeyingMaterial& k) { EncryptSecret(k, ...); })`
This silences the following (clang 18):
```
wallet/wallet.cpp:3520:12: error: returning variable 'vMasterKey' by reference requires holding mutex 'cs_wallet' [-Werror,-Wthread-safety-reference-return]
3520 | return vMasterKey;
| ^
```
6044628543 crypto, hash: replace custom rotl32 with std::rotl (Fabian Jahr)
Pull request description:
While exploring some C++20 changes and checking against our code I found this potential improvement:
1. We can replace our custom implementation of `rotl32` in crypto/chacha20 with `std::rotl` from the [new `bit` header](https://en.cppreference.com/w/cpp/header/bit).
ACKs for top commit:
fanquake:
ACK 6044628543
Tree-SHA512: db55b366f20fca2ef62e5f10a838f8a709d531678c35c1dba20898754029c788a2fd47995208ed6d187cf814109a7ca397bc2c301504500aee79da04c95d6895
52149b7a2c contrib: Fix clang-format-diff.py lint errors (TheCharlatan)
008e81e025 contrib: Latest clang-format-diff.py script (TheCharlatan)
Pull request description:
This was taken from 900bb318b5/clang/tools/clang-format/clang-format-diff.py and is useful for systems where clang tools are shipped with a version suffix.
ACKs for top commit:
maflcko:
lgtm ACK 52149b7a2c🌱
Tree-SHA512: cad720b283650e36c5b3ec597278112899ae6413a42c79b6296c58242000f32ae6ab7ed732a3a0f30f1f2586d7e3c0d1ef856a4821e28272bae6f428ed9497e9
fa96d93711 refactor: Allow std::span construction from CKey (MarcoFalke)
999962d68d Add missing XOnlyPubKey::data() to get mutable data (MarcoFalke)
Pull request description:
Is is possible to construct a `Span` from a reference to a `CKey`. However, the same is not possible with `std::span`.
Fix that.
ACKs for top commit:
shaavan:
ReACK fa96d93711
willcl-ark:
ACK fa96d93711
Tree-SHA512: 44fccdce5f32bc16b44f3b1bd32e86d9eabfd09bca6abe79f2d6db0cb0b5e4aaeaff710f023cb21ccde9315d2007d55f1b43f29416e81bceeeabe3948f673d3a
cbc9bf11fe build: move -mlinker-version to *FLAGS (fanquake)
42b2283765 depends: deduplicate use of mmacosx-version-min in macOS build (fanquake)
Pull request description:
Move some C/CXX FLAGS out of C/CXX. The remaining flags are host/SDK related, and will need some more thought.
This is more correct in any case, and simplifies future changes.
Related to #21778.
ACKs for top commit:
theuni:
utACK cbc9bf11fe
TheCharlatan:
ACK cbc9bf11fe
Tree-SHA512: 373216c1de32375faddc161ecc09b14fed0e0994cbd5ed62c862c38a5aea80c7f1740f32f36b8a60ac1caf749309750d22164c50b89916f41838d6453296ac4a
We assume to be using python3, so don't check for it. This removes a
type error on the line `from io import BytesIO as StringIO`.
Specify the encoding as "utf8" when opening a file.
3ba815b42d Make v2transport default for addnode RPC when enabled (Pieter Wuille)
Pull request description:
Since #29058, several types of manually configured connections will attempt v2 connections when `-v2transport` is enabled, except for the `addnode` RPC, as that one has an explicit argument to enable or disable.
Make the default for that RPC match the `-v2transport` setting so the behavior matches that of other manual connections from a user perspective.
ACKs for top commit:
achow101:
ACK 3ba815b42d
kristapsk:
ACK 3ba815b42d
theStack:
Code-review ACK 3ba815b42d
Tree-SHA512: 31ef48cf1e533abb17866020378c004df929e626074dc98b3229fb60a66de58435e95c8fda8d1b463e1208aa39d1f42d239818e7e58595a3738089920598befc
cdc6ac4126 snapshots: don't core dump when running -checkblockindex after `loadtxoutset` (Mark Friedenbach)
Pull request description:
Transaction counts aren't known for block history loaded from a snapshot. If you start with `-checkblockindex` after loading a snapshot, the bitcoin daemon will core dump. The test suite does not check for this because all the snapshots have no non-coinbase transactions (all blocks prior to the snapshot are assumed to have `nTx = 1`).
Recommend for backport to 26.x
ACKs for top commit:
fjahr:
utACK cdc6ac4126
achow101:
ACK cdc6ac4126
pablomartin4btc:
tACK cdc6ac4126
Tree-SHA512: f7488a85cc29056e2ac443ce8f34aea4dfde6ba246efce82235d6a4dca2dca4344f07b93c93424b4addcb83e4cb2ae49a3ebb37d89840d42d2aeea35904cab04
74ebd4d135 doc, test: Test and explain service flag handling (Martin Zumsande)
Pull request description:
Service flags received from the peer-to-peer network are handled differently, depending on how we receive them.
If received directly from an outbound peer the flags belong to, they replace existing flags.
If received via gossip relay (so that anyone could send them), new flags are added, but existing ones but cannot be overwritten.
Document that and add test coverage for it.
ACKs for top commit:
achow101:
ACK 74ebd4d135
furszy:
ACK 74ebd4d135
brunoerg:
utACK 74ebd4d135
Tree-SHA512: 604adc3304b8e3cb1a10dfd017025c10b029bebd3ef533f96bcb5856fee5d4396a9aed4949908b8e7ef267ad21320d1814dd80f88426330c5c9c2c529c497591
df30247705 [test] import descriptor wallet with reorged parent + IsFromMe child in mempool (glozow)
c3d02be536 [test] rescan legacy wallet with reorged parent + IsFromMe child in mempool (Gloria Zhao)
Pull request description:
Originally motivated by #29019, which reverts back to having `requestMempoolTransactions` emit `transactionAddedToMempool` in `mapTx` default order instead of `GetSortedDepthAndScore` order.
It's important that these notifications happen in topological order, otherwise the wallet rescan may miss transactions that belong to it. Notably, checking whether a transaction `IsFromMe` requires knowing its inputs, which may be from a mempool parent.
When using `mapTx` order, a parent may come later than its child if it was added from a block disconnected in a reorg.
This PR adds a test for this case.
ACKs for top commit:
achow101:
ACK df30247705
furszy:
Code review ACK df30247705, nits can be disregarded.
Tree-SHA512: 2f1d9ef92313228adbbef94e634e5f7a9ec6e6a2c88e16aa343bdc95ffc9b9f9c82a569b412c9a3841db9d789e52f9283e8b9385731668d59355903e26e58a5d
5335e454c0 contrib: add macho branch protection check (fanquake)
Pull request description:
Followup to https://github.com/bitcoin/bitcoin/pull/28459. Add a sanity check that `bti` instructions are present in the arm macho binary, similar to our x86_64 check for control flow.
Could do something similar for aarch64 linux in future, and maybe could use https://github.com/lief-project/LIEF/issues/975.
ACKs for top commit:
TheCharlatan:
ACK 5335e454c0
Tree-SHA512: 6cc8721209fe07fe07f0524ef6f114004e2b98844f73d31ff16547f7055c7cb4a5609480058c45ede21b457b2dea5357f1475eaa5063ea1f9772aa260f49039b
This was take from
900bb318b5/clang/tools/clang-format/clang-format-diff.py
Updating it introduces some new options. For example specifying the
clang-format binary, which is useful for systems where clang tools are
shipped with a version suffix.
It's preferable to use type-safe transaction identifiers to avoid
confusing txid and wtxid. The next commit will add a reference to this
set; we use this opportunity to change it to Txid ahead of time instead
of adding new uses of uint256.
ec779a2b8e doc: add unconditional info loglevel following merge of PR 28318 (Jon Atack)
Pull request description:
Commit ab34dc6012 of #28318 was an incomplete version of [`118c756` (#25203)](118c7567f6) from the `Severity-based logging` parent PR.
Add the missing text to update the `-loglevel` help doc.
While here, make the help text a little easier to understand.
Can be tested by running:
```
./src/bitcoind -regtest -help-debug | grep -A12 loglevel=
```
before
```
-loglevel=<level>|<category>:<level>
Set the global or per-category severity level for logging categories
enabled with the -debug configuration option or the logging RPC:
info, debug, trace (default=debug); warning and error levels are
always logged.
```
after
```
-loglevel=<level>|<category>:<level>
Set the global or per-category severity level for logging categories
enabled with the -debug configuration option or the logging RPC.
Possible values are info, debug, trace (default=debug). The
following levels are always logged: error, warning, info.
```
ACKs for top commit:
stickies-v:
ACK ec779a2b8e
Tree-SHA512: 0c375e30a5a4c168ca7d97720e8c287f598216767afedae329824e09a480830faf8537b792c5c4bb647c68681c287fe3005c62093708ce85624e9a71c8245e42
This avoids cases of missing -O2, when *FLAGS has been overriden.
Removes the need for duplicate code to clear autoconf defaults.
Also, move CORE_CXXFLAGS before DEBUG_CXXFLAGS, so that -O2 is always
overriden if debugging etc.
2d1b1c7dae build: remove --enable-lto (fanquake)
Pull request description:
This has outlived its usefulness, doesn't gel well with newer compilers & `-flto` related options, i.e thin vs full, or `=auto`, and having `-flto` as the only option means that sometimes this just needs to be worked around, i.e in oss-fuzz:
https://github.com/google/oss-fuzz/blob/master/projects/bitcoin-core/build.sh.
While it was convenient when `-flto` was newer, support for `-flto` is now in all compilers we use, and there's also no-longer any real need for us to treat `-flto` different to any other optimization option.
Remove it, to remove build complexity, and so there's no need to port a similar option to CMake.
Note that the LTO option remains in depends, because we still a way to build packages that have LTO specific patches/options.
ACKs for top commit:
TheCharlatan:
ACK 2d1b1c7dae
hebasto:
ACK 2d1b1c7dae.
Tree-SHA512: 91812de7da35346f51850714a188fcffbac478bc8b348bf756c2555fcbde86ba622ac2fb77d294dea0378c741d3656f06121ef3a795aeed63fd170fc31bfa5af
eb78ea4eeb [log] mempool loading (glozow)
Pull request description:
Motivated by #29193. Currently, we only log something (non-debug) when we fail to load the file and at the end of importing all the transactions. That means it's hard to tell what's happening if it's taking a long time to load.
This PR adds a maximum of 10 new unconditional log lines:
- When we start to load transactions.
- Our progress percentage when it advances by at least 10% from the last time we logged. Percentage is based on the number of transactions.
If there are lots of transactions in the mempool, the logs will look like this:
```
2024-01-11T11:36:30.410726Z Loading 401 mempool transactions from disk...
2024-01-11T11:36:30.423374Z Progress loading mempool transactions from disk: 10% (tried 41, 360 remaining)
2024-01-11T11:36:30.435539Z Progress loading mempool transactions from disk: 20% (tried 81, 320 remaining)
2024-01-11T11:36:30.447874Z Progress loading mempool transactions from disk: 30% (tried 121, 280 remaining)
2024-01-11T11:36:30.460474Z Progress loading mempool transactions from disk: 40% (tried 161, 240 remaining)
2024-01-11T11:36:30.473731Z Progress loading mempool transactions from disk: 50% (tried 201, 200 remaining)
2024-01-11T11:36:30.487806Z Progress loading mempool transactions from disk: 60% (tried 241, 160 remaining)
2024-01-11T11:36:30.501739Z Progress loading mempool transactions from disk: 70% (tried 281, 120 remaining)
2024-01-11T11:36:30.516334Z Progress loading mempool transactions from disk: 80% (tried 321, 80 remaining)
2024-01-11T11:36:30.531309Z Progress loading mempool transactions from disk: 90% (tried 361, 40 remaining)
2024-01-11T11:36:30.549019Z Imported mempool transactions from disk: 401 succeeded, 0 failed, 0 expired, 0 already there, 400 waiting for initial broadcast
```
If there are 0 or 1 transactions, progress logs aren't printed.
ACKs for top commit:
kevkevinpal:
Concept ACK [eb78ea4](eb78ea4eeb)
ismaelsadeeq:
ACK eb78ea4eeb
dergoegge:
Code review ACK eb78ea4eeb
theStack:
re-ACK eb78ea4eeb
mzumsande:
tested ACK eb78ea4eeb
Tree-SHA512: ae4420986dc7bd5cb675a7ebc76b24c8ee60007f0296ed37e272f1c3415764d44963bea84c51948da319a65661dca8a95eac2a59bf7e745519b6fcafa09812cf
No behavior change. Just an intermediate refactoring.
By relocating the peer desirable services flags into the peer
manager, we allow the connections acceptance process to handle
post-IBD potential stalling scenarios.
In the follow-up commit(s), the desirable service flags will be
dynamically adjusted to detect post-IBD stalling scenarios (such
as a +48-hour inactive node that must prefer full node connections
instead of limited peer connections because they cannot provide
historical blocks). Additionally, this encapsulation enable us
to customize the connections decision-making process based on
new user's configurations in the future.
And implement 'ApproximateBestBlockDepth()' to estimate
the distance, in blocks, between the best-known block
and the network chain tip. Utilizing the best-block time
and the chainparams blocks spacing to approximate it.
4756114e50 [depends] Allow PATH with spaces in directory names. (Mark Friedenbach)
Pull request description:
The goal of this PR is to help close https://github.com/bitcoin/bitcoin/pull/28733. I reverted the change on `depends/config.guess` based on the feedback provided in the previous PR. I've also incorporated the test mentioned by maflcko
ACKs for top commit:
maflcko:
lgtm ACK 4756114e50
hebasto:
ACK 4756114e50, successfully built depends on Ubuntu 22.04.
TheCharlatan:
ACK 4756114e50
Tree-SHA512: ee257f6efd235839156bc236384f08d77b91debc3c257168368a71e70742639f28a3289572b8693609c1109062dc9968e461103d1f4f5679906506e94b54e649
c003562120 doc: Add missing backtick in developer notes logging section (Fabian Jahr)
Pull request description:
Newly added logging section from https://github.com/bitcoin/bitcoin/pull/28318 is missing a single backtick. Also fixes some minor punctuation errors in that section.
ACKs for top commit:
jonatack:
ACK c003562120
alfonsoromanz:
ACK c003562120
Tree-SHA512: 2f75f9472d212ce7c7ebf3e7404f86b3bd8c695f63e2a7447d7a55bb54dcdb5e3bfd15e8eac5b92efbdcf1216c4e8d699cae0250d021f72b9d1c32a7db91989d
ea2551e55d wallet: Reset chain notifications handler if AttachChain fails (Ava Chow)
Pull request description:
AttachChain will create the chain notifications handler which contains a reference to the wallet's shared_ptr. If AttachChain fails, the wallet needs to be unloaded, and this is expected to happen with its custom deleter ReleaseWallet. However, if the chain notifications handler is still set, then the shared_ptr is still referenced by something, so the wallet is never actually released.
This behavior can also be verified by looking at the debug.log file. When the wallet is released, the line "Releasing wallet" should appear in the debug.log file. However the failing test does not contain that line, indicating that the problem is that the `CWallet` object is not being destroyed. After this PR, that log line now appears, and the test also passes.
Fixes#29234
ACKs for top commit:
jamesob:
ACK ea2551e55d
murchandamus:
ACK ea2551e55d
TheCharlatan:
ACK ea2551e55d
furszy:
Code review ACK ea2551e5
BrandonOdiwuor:
Code Review ACK ea2551e55d
Tree-SHA512: 73d676068c699303d9bcf70c9288ecb05f5f2e400ff3b7201367658d39d2fab63def97ab5ce4a742a6f2ca5e325f598fdbb6ce8f157a0423c07abc9a19bd5c81
AttachChain will create the chain notifications handler which contains a
reference to the wallet's shared_ptr. If AttachChain fails, the wallet
needs to be unloaded, and this is expected to happen with its custom
deleter ReleaseWallet. However, if the chain notifications handler is
still set, then the shared_ptr is still referenced by something, so the
wallet is never actually released.
Test that wallet rescans process transactions topologically, even if a
parent's entry into the mempool is later than that of its child.
This behavior is important because IsFromMe requires the ability to look
up a transaction's inputs.
Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
0d627c4ca8 doc: refer to "Node relay options" in policy/README (djschnei21)
Pull request description:
Fixed up #29095, to refer to `-help`, rather than listing every option.
ACKs for top commit:
stickies-v:
ACK 0d627c4ca8
glozow:
lgtm ACK 0d627c4ca8
Tree-SHA512: 37d36ffa48297371eb0032ed48dce28802f862f6c18bdb50207555a228ce252e51a93a6fdef86b3e596d486c5107594d64db89f077b77fc885fe84cecb1dadc3
0eebd6fe7d test: Assert that a new tx with a delta of 0 is never added (kevkevin)
cfdbcd19b3 rpc: exposing modified_fee in getprioritisedtransactions (kevkevin)
252a86729a rpc: renaming txid -> transactionid (kevkevin)
2fca6c2dd0 rpc: changed prioritisation-map -> "" (kevkevin)
3a118e19e1 test: Directly constructing 2 entry map for getprioritisedtransactions (kevkevin)
Pull request description:
In this PR I am addressing some comments in https://github.com/bitcoin/bitcoin/pull/27501 as a followup.
- changed `prioritisation-map` in the `RPCResult` to `""`
- Directly constructing 2 entry map for getprioritisedtransactions in functional tests
- renamed `txid` to `transactionid` in `RPCResult` to be more consistent with naming elsewhere
- exposed the `modified_fee` field instead of having it be a useless arg
- Created a new test that asserts when `prioritisedtransaction` is called with a fee_delta of 0 it is not added to mempool
ACKs for top commit:
glozow:
reACK 0eebd6fe7d, only change is the doc suggestion
Tree-SHA512: e99056e37a8b1cfc511d87c83edba7c928b50d2cd6c2fd7c038976779850677ad37fddeb2b983e8bc007ca8567eb21ebb78d7eae9b773657c2b297299993ec05
aaaace2fd1 fuzz: Assume presence of __builtin_*_overflow, without checks (MarcoFalke)
fa223ba5eb Revert "build: Fix undefined reference to __mulodi4" (MarcoFalke)
fa7c751bd9 build: Bump clang minimum supported version to 14 (MarcoFalke)
Pull request description:
Most supported operating systems ship with clang-14 (or later), so bump the minimum to that and allow new code to drop workarounds for previous clang bugs.
For reference:
* https://packages.debian.org/bookworm/clang (`clang-14`)
* https://packages.ubuntu.com/jammy/clang (`clang-14`)
* CentOS-like 8/9 Stream: All Clang versions from 15 to 17
* FreeBSD 12/13: All Clang versions from 15 to 16
* OpenSuse Tumbleweed ships with https://software.opensuse.org/package/clang (`clang17`); No idea about OpenSuse Leap
On operating systems where the clang version is not shipped by default, the user would have to use GCC, or install clang in a different way. For example:
* https://packages.debian.org/bullseye/g++ (g++-10)
* https://packages.ubuntu.com/focal/g++-10
* https://apt.llvm.org/, or nix, or guix, or compile clang from source, ...
ACKs for top commit:
fanquake:
ACK aaaace2fd1
Tree-SHA512: 81d066b14cc568d27312f1cc814b09540b038a10a0a8e9d71fc9745b024fb6c32a959af673e6819b817ea7cef98da4abfa63dff16cffb7821b40083016b0291f
fa0c594b33 ci: Rename tasks (previous releases, macOS cross) (MarcoFalke)
Pull request description:
The previous releases task no longer uses the qt5 dev package, but the depends package, so fix that in the name.
Also, remove a detail from the macOS cross task name, because anyone can look it up in the source, if they really want to. Otherwise, it may go out of date in the name.
Also, rename the two tasks' config file to reflect the same.
ACKs for top commit:
fanquake:
ACK fa0c594b33
Tree-SHA512: e6f1d04128d35462c49367c98a0227988695d75add88c569804551e3fd30c22292d22b88fa19c54f02fab0c9784c77a078447de0280553a3fc7162dcf992d7ae
Test that wallet rescans process transactions topologically, even if a
parent's entry into the mempool is later than that of its child.
This behavior is important because IsFromMe requires the ability to look
up a transaction's inputs.
Previously we would check that there is no LegacySPKM in order to
determine whether a wallet is already a descriptor wallet and doesn't
need to be migrated. However blank legacy wallets will also not have a
LegacySPKM, so we need to be checking for the descriptors flag instead.
5fa74609b8 Fix -netinfo backward compat with getpeerinfo pre-v26 (Jon Atack)
Pull request description:
Commit fb5bfed26a in #29058 will cause `-netinfo` to break when calling it on a node that is running pre-v26 bitcoind, as `getpeerinfo` doesn't yet return a "transport_protocol_type" field.
Fix this by adding an `IsNull()` check, as already done for other recent getpeerinfo fields, and also in the same commit:
a) avoid checking for the full string "detecting", and instead do the cheaper check for the most frequent case of the string starting with "v"
b) drop displaying the "v" prefix in all the rows, as it doesn't add useful information, and instead use "v" for the column header
c) display nothing when a value isn't determined yet, like for the -netinfo mping and ping columns (as `*` already has a separate meaning in this dashboard, and `?` might look like there is a bug)
ACKs for top commit:
mzumsande:
Code Review ACK 5fa74609b8
achow101:
ACK 5fa74609b8
kristapsk:
ACK 5fa74609b8
Tree-SHA512: 4afc513dc669b95037180008eb4c57fc0a0d742c02f24b236562d6b8daad5c120eb1ce0d90e51696e0f9b8361a72fc930c0b64f04902cf96fb48c8e042e58624
878d914777 doc: test: mention OS detection preferences in style guideline (Sebastian Falbesoner)
4c65ac96f8 test: detect OS consistently using `platform.system()` (Sebastian Falbesoner)
37324ae3df test: use `skip_if_platform_not_linux` helper where possible (Sebastian Falbesoner)
Pull request description:
There are at least three ways to detect the operating system in Python3:
- `os.name` (https://docs.python.org/3.9/library/os.html#os.name)
- `sys.platform` (https://docs.python.org/3.9/library/sys.html#sys.platform)
- `platform.system()` (https://docs.python.org/3.9/library/platform.html#platform.system)
We are currently using all of them in functional tests (both in individual tests and shared test framework code), which seems a bit messy. This PR consolidates into using `platform.system()`, as it appears to be one most consistent and easy to read (see also [IRC discussion](https://bitcoin-irc.chaincode.com/bitcoin-core-dev/2023-12-08#989301;) and table below). `sys.platform` is inconsistent as it has the major version number encoded for BSD systems, which doesn't make much sense for e.g. OpenBSD, where there is no concept of major versions, but instead the version is simply increased by 0.1 on each release.
Note that `os.name` is still useful to detect whether we are running a POSIX system (see `BitcoinTestFramework.skip_if_platform_not_posix`), so for this use-case it is kept as only exception. The following table shows values for common operating systems, found via
```
$ python3 -c "import os; import sys; import platform; print(os.name, sys.platform, platform.system())"
```
| OS | os.name | sys.platform | platform.system() |
|--------------|---------|--------------|--------------------|
| Linux 6.2.0 | posix | linux | Linux |
| MacOS* | posix | darwin | Darwin |
| OpenBSD 7.4 | posix | openbsd7 | OpenBSD |
| Windows* | nt | win32 | Windows |
\* = I neither have a MacOS nor a Windows machine available, so I extracted the values from documentation and our current code. Also I'm relying on CI for testing the relevant code-paths. Having reviewers to this this locally would be very appreciated, if this gets Concept ACKed.
ACKs for top commit:
kevkevinpal:
ACK [878d914](878d914777)
achow101:
ACK 878d914777
hebasto:
ACK 878d914777, I have reviewed the code and it looks OK.
pablomartin4btc:
tACK 878d914777
Tree-SHA512: 24513d493e47f572028c843260b81c47c2c29bfb701991050255c9f9529cd19065ecbc7b3b6e15619da7f3f608b4825c345ce6fee30d8fd1eaadbd08cff400fc
1f8450f066 doc: upgrade Bitcoin Core license to 2024 (22388o⚡️)
Pull request description:
See https://github.com/bitcoin/bitcoin/pull/26748.
Cherry-picked these commits from 22388o and then squashed them.
ACKs for top commit:
fanquake:
ACK 1f8450f066
Tree-SHA512: 6d12f24a6c7cd421f7d975d7e445de4583144a4d0902d4e68e7648395074ca804e3ee585b2d3f307d193690e2ed2f9fbd8e7938f8fb8af263888b8701993782a
ff3f51b402 depends: Include `config.guess` and `config.sub` into `meta_depends` (Hennadii Stepanov)
Pull request description:
ACKs for top commit:
theuni:
ACK ff3f51b402.
Tree-SHA512: e8575473d3fca2293181131c76bd6d43017fe753d2e670c53227a646b64b069dc542a0fc50a77b43e74bc6a0c0159ffa2fb1c3ff3aef9625684e0f78c16ad960
997b9a73e5 test: add assumeutxo wallet test (Sjors Provoost)
Pull request description:
Extracted from #28616, this adds a (very) basic wallet test for assume utxo. It checks some circumstances where a backup can and can't be loaded.
ACKs for top commit:
maflcko:
lgtm ACK 997b9a73e5
achow101:
ACK 997b9a73e5
theStack:
Code-review ACK 997b9a73e5
Tree-SHA512: 69474e56c6a46bb4f30fc54f8e5844766ac2a5f8226bb0b168d11ae1e3d4eae58570c1f1b4cc2b2f6f51b5d0e055bbe2bbd11684265215e01d4eb81ab4b7b0bb
f3ca6db8d3 ci: move CMake into base packages (fanquake)
Pull request description:
This is already used in multiple CIs, and will soon become a requirement for most CIs, i.e when we migrate depends packages to use CMake, for example: https://github.com/bitcoin/bitcoin/pull/21778#issuecomment-1885576324.
Some of the CIs in 21778 are failing because CMake isn't available, so just break this out and make CMake globally available.
ACKs for top commit:
maflcko:
lgtm ACK f3ca6db8d3
Tree-SHA512: b3daa82c1ead29600168b3f064dc65c8e632baa6e7efb5f2e87ba1e9130a48c31becdf0a89e6ede479c5b08dd97eb4c9cfd4cc10207235c601dd089d5b808b59
This commit makes a minimal change to the ParamsStream class to let it retrieve
multiple parameters. Followup commits after this commit clean up code using
ParamsStream and make it easier to set multiple parameters.
Currently it is only possible to attach one serialization parameter to a stream
at a time. For example, it is not possible to set a parameter controlling the
transaction format and a parameter controlling the address format at the same
time because one parameter will override the other.
This limitation is inconvenient for multiprocess code since it is not possible
to create just one type of stream and serialize any object to it. Instead it is
necessary to create different streams for different object types, which
requires extra boilerplate and makes using the new parameter fields a lot more
awkward than the older version and type fields.
Fix this problem by allowing an unlimited number of serialization stream
parameters to be set, and allowing them to be requested by type. Later
parameters will still override earlier parameters, but only if they have the
same type.
This change requires replacing the stream.GetParams() method with a
stream.GetParams<T>() method in order for serialization code to retrieve the
desired parameters. This change is more verbose, but probably a good thing for
readability because previously it could be difficult to know what type the
GetParams() method would return, and now it is more obvious.
This is already used in multiple CIs, and will soon become a requirement
for most CIs, i.e when we migrate depends packages to use CMake, for
example:
https://github.com/bitcoin/bitcoin/pull/21778#issuecomment-1885576324.
Some of the CIs in 21778 are failing because CMake isn't available, so
just break this out and make CMake globally available.
4fdd836db9 Use hardened runtime on macOS release builds. (Mark Friedenbach)
Pull request description:
The Apple notary service requires submitted app bundles to be configured to use the hardened runtime libraries. This is configured at signing time, and supported by the signapple tool Bitcoin Core uses for reproduceable signed binaries. We simply need to pass "--hardened-runtime" when the signature is created. Once attached to the bundle, the resulting codesigned binary can be successfully submitted to the Apple binary notarization service by any Apple Developer.
This partially resolves#15774. The release maintainer, or any authorized Apple Developer, will need to run `xcrun notarytool` to prevent gatekeeper warnings on macOS. Using `xcrun staple` to generate a binary that doesn't call home on first launch would be bonus, but at least this would massively improve the user experience.
ACKs for top commit:
fanquake:
ACK 4fdd836db9 - we can move ahead with this, and figure out notarisation / stapling as a followup.
Tree-SHA512: 7b8ba50030fb230d44bd63d12ed082537e8eaaa61396114c5df715f8dd6772fd8d84b00dc819f88d9a463996c2170a84981fce1bde7f7999b4bdb914fbcdfdac
016cc807f7 test: wallet migration, add coverage for tx extra data (furszy)
Pull request description:
Quick follow-up to #28610, coming from https://github.com/bitcoin/bitcoin/pull/28610#pullrequestreview-1802823938.
Verifying that the 'replaced_by_txid' and 'replaces_txid' tx data is preserved after migration,
as well as the extra tx comments.
ACKs for top commit:
jamesob:
Nice, ACK 016cc807f7
achow101:
ACK 016cc807f7
pablomartin4btc:
ACK 016cc807f7
BrandonOdiwuor:
lgtm ACK 016cc807f7
Tree-SHA512: 697cabece730cbe5c5947bf98455e80a8877c0352fbe2a66362ce5ea530b67882b0bec561a67d48fee200cdad717cd62c57fd809e2a94ff83c3fad30021e1d9e
e60fc7d5d3 logging: Replace uses of LogPrintfCategory (Anthony Towns)
f7ce5ac08c logging: add LogError, LogWarning, LogInfo, LogDebug, LogTrace (Anthony Towns)
fbd7642c8e logging: add -loglevelalways=1 option (Anthony Towns)
782bb6a056 logging: treat BCLog::ALL like BCLog::NONE (Anthony Towns)
667ce3e329 logging: Drop BCLog::Level::None (Anthony Towns)
ab34dc6012 logging: Log Info messages unconditionally (Anthony Towns)
dfe98b6874 logging: make [cat:debug] and [info] implicit (Anthony Towns)
c5c76dc615 logging: refactor: pull prefix code out (Anthony Towns)
Pull request description:
Replace `LogPrint*` functions with severity based logging functions:
* `LogInfo(...)`, `LogWarning(...)`, `LogError(...)` for unconditional (uncategorised) logging (replaces `LogPrintf`)
* `LogDebug(CATEGORY, ...)` and `LogTrace(CATEGORY, ...)` for conditional logging (replaces `LogPrint`)
* `LogPrintLevel(CATEGORY, LEVEL, ...)` for when the level isn't known in advance, or a category needs to be added for an info/warning/error log message (mostly unchanged, but rarely needed)
Logs look roughly as they do now with `LogInfo` not having an `[info]` prefix, and `LogDebug` having a `[cat]` prefix, rather than a `[cat:debug]` prefix. This removes `BCLog::Level::None` entirely -- for `LogFlags::NONE` just use `Level::Info`, for any actual category, use `Level::Debug`.
Adds docs to developer-notes about when to use which level.
Adds `-loglevelalways=1` option so that you get `[net:debug]`, `[all:info]`, `[all:warning]` etc, which might be helpful for automated parsing, or just if you like everything to be consistent. Defaults to off to reduce noise in the default config, and to avoid unnecessary changes on upgrades.
Changes the behaviour of `LogPrintLevel(CATEGORY, BCLog::Level::Info, ...)` to be logged unconditionally, rather than only being an additional optional logging level in addition to trace and debug. Does not change the behaviour of `LogPrintLevel(NONE, Debug, ...)` and `LogPrintLevel(NONE, Trace, ...)` being no-ops.
ACKs for top commit:
maflcko:
re-ACK e60fc7d5d3🌚
achow101:
ACK e60fc7d5d3
stickies-v:
ACK e60fc7d5d3
jamesob:
ACK e60fc7d5d3 ([`jamesob/ackr/28318.1.ajtowns.logging_simplify_api_for`](https://github.com/jamesob/bitcoin/tree/ackr/28318.1.ajtowns.logging_simplify_api_for))
Tree-SHA512: e7a4588779b148242495b7b6f64198a00c314cd57100affab11c43e9d39c9bbf85118ee2002792087fdcffdea08c84576e20844b3079f27083e26ddd7ca15d7f
931575418e test: assumeutxo: spend coin from snapshot chainstate after loading (Sebastian Falbesoner)
Pull request description:
This PR extends the AssumeUTXO functional test by submitting a spending transaction for an UTXO that is only available in a the snapshot chainstate (after loading via `loadtxoutset`), i.e. it hasn't been seen in a block before. With that we can verify that snapshot coins are visible to the mempool.
Note that we unfortunately can't use MiniWallet here, as the only available UTXO to spend from the snapshot chainstate is at height 200, where a P2PKH created from the test framework's deterministic private key is used (see `TestNode.generate(...)` and the `PRIV_KEYS` array). Coinbase outputs with smaller heights (<= 199) would be part of the pre-generated chain and hence not qualify for the "UTXO is only in snapshot chainstate and has never been seen in a block" scenario, coinbase outputs with larger heights (>= 201) can't be spent due to immaturity, as the snapshot chainstate block height is 299.
One could of course mine a different chain with outputs that MiniWallet supports (e.g. taproot anyone-can-spend), but this would change the hardcoded AssumeUTXO hash, colliding with other PRs like #28838, so I wanted to avoid that.
ACKs for top commit:
maflcko:
lgtm ACK 931575418e
jamesob:
ACK 931575418e
Tree-SHA512: 0665868e1e91fe74f408d0a239cc264bbbc11a6b55bcc0e86cc8b4b2ec1f44977884b817dbe9065a7c768332cab464636656858bc8b9c8e7d7810498e0a17d78
`CPubKey::VerifyPubKey` uses rng internally which leads to instability
in the fuzz test.
We fix this by avoiding `VerifyPubKey` in the test and verifying the
decoded public key with a fuzzer chosen message instead.
CLI -netinfo will currently break when calling it on a node that is running
pre-v26 bitcoind, as `getpeerinfo` doesn't yet return a transport_protocol_type
field.
Fix this by adding an `IsNull()` check as already done for other fields, and also:
- avoid checking for the full string "detecting", and instead do the cheaper
check for the most frequent case of the string starting with "v"
- drop displaying the "v" prefix in all the rows, as it doesn't add useful
information, and instead use "v" for the column header
- display nothing during peer setup, like for the -netinfo mping and ping columns
fb5bfed26a cli: add transport protcol column to -netinfo (Martin Zumsande)
9eed22e870 net: attempt v2 transport for addrfetch connections if we support it (Martin Zumsande)
770c0311ef net: attempt v2 transport for manual connections if we support it (Martin Zumsande)
Pull request description:
Some preparations before enabling `-v2transport` as the default:
* Use v2 for `-connect`, `-addnode` config arg and `-seednode` if `-v2transport` is enabled.
Our peer may or may not support v2, but I don't think an extra option is necessary for any of these (we have that for the `addnode` rpc), because we have the reconnection mechanism that will try again with `v1` if our peer doesn't support `v2`.
* Add a column for the transport protocol to `-netinfo`. I added it next to the `net` column because I thought it looked nice there, but if people prefer it somewhere else I'm happy to move it.

ACKs for top commit:
sipa:
utACK fb5bfed26a
achow101:
ACK fb5bfed26a
stratospher:
tested ACK fb5bfed. addrfetch + manual connections aren't frequent and it would be useful to have this for transition to v2 one day.
theStack:
ACK fb5bfed26a
kristapsk:
ACK fb5bfed26a
Tree-SHA512: c4575ad11b99613870b342acae369fa08f877ac79e6e04eb62e94ad7a92d528e289183c0963c78aa779ba11cb91e2a6fad7c8b0d813126c46c3e5b54bd962c26
9d728916b2 net: create I2P sessions with both ECIES-X25519 and ElGamal encryption (Jon Atack)
Pull request description:
A Bitcoin Core node may only connect to a peer destination via I2P if both sides have sessions with the same encryption type. Encryption type is a property of the session, not the destination. Sessions may support multiple encryption types.
As Bitcoin Core is not currently setting the encryption type when creating I2P sessions, it uses the older default, ElGamal (type 0).
This pull updates our I2P session creation to use both ECIES-X25519 and ElGamal (types 4 and 0, respectively). This allows to connect to I2P peers of either type, and the newer, faster ECIES-X25519 will be preferred.
See also:
- discussion around https://github.com/qbittorrent/qBittorrent/issues/19625#issuecomment-1879582395
- recently updated "Signature and Encryption Types" in https://geti2p.net/en/docs/api/samv3
Thank you and credit to zzzi2p for reporting and to vort for the patch.
Closes https://github.com/bitcoin/bitcoin/issues/29197.
ACKs for top commit:
zzzi2p:
ACK 9d728916b2
recursive-rat4:
ACK 9d728916b2
kristapsk:
cr utACK 9d728916b2
brunoerg:
crACK 9d728916b2
shaavan:
crACK 9d728916b2
Tree-SHA512: 0912fc01af9706914a7854f7479b9d82fc86c9530466cad8674e30f7eb4894d90d514efbc1aee8b7ea690faa6ff4a23b62cf5de8737cffdbc463300082c9b917
080763a058 build: Drop `ALLOW_HOST_PACKAGES` support in depends (Hennadii Stepanov)
Pull request description:
The `ALLOW_HOST_PACKAGES` variable was introduced in bitcoin#10508 "to speed up build and avoid timeout".
It is no longer the case for our CI infrastructure, which uses self- hosted persistent workers and depends caching.
In the current circumstances, it does not seem worth porting this feature to the upcoming [CMake-based](https://github.com/bitcoin/bitcoin/issues/28607) build system.
ACKs for top commit:
fanquake:
ACK 080763a058 - I can't imagine this option got any use outside our CI. It's also mostly just at odds with the idea of a self-contained dependency builder.
TheCharlatan:
ACK 080763a058
Tree-SHA512: 36f52690be913479c5d12be36760b8de1a6e891fe7c2cf98a7b8d6561006a6b18631e431351d79e97edb9409f9902d032aedf7b963aa7615e54b59fc2a58f7d6
e5b9ee0221 fuzz: set `nMaxOutboundLimit` in connman target (brunoerg)
Pull request description:
Setting `nMaxOutboundLimit` (`-maxuploadtarget`) will make fuzz to reach more coverage in connman target. This value is used in `GetMaxOutboundTimeLeftInCycle`, `OutboundTargetReached` and `GetOutboundTargetBytesLeft`.
ACKs for top commit:
dergoegge:
utACK e5b9ee0221
jonatack:
ACK e5b9ee0221
Tree-SHA512: d19c83602b0a487e6da0e3be539aa2abc95b8bbf36cf9a3e391a4af53b959f68ca38548a96d27d56742e3b772f648da04e2bf8973dfc0ab1cdabf4f2e8d44de6
The nodes are restarted with an empty addrman and populated
with addresses from different networks using a helper function.
We can safely add multiple addresses to addrman tables without
worrying about unpredictable collisions since bucket:position
is fixed in a deterministic addrman.
Currently in tests where we are interested in contents of addrman,
addresses which were added to the node's addrman in previous tests
leak into the current test. example: addresses added in addpeeraddress
test leak into getaddrmaninfo and getrawaddrman tests.
It is cleaner to design the tests to be modular and without such
leaks so that we don't need to deal with context from previous tests
this test inserts 1 address into the new table and 1 address into
the tried table so that no collisions can happen in either table
if a second address is added. this setup does not need to be
maintained anymore since we can use a deterministic addrman and
safely add many addresses in both tables without collisions. Remove
comment explaining why previous setup needed to be maintained.
Supposing there are 2 different addresses to be placed in an addrman
table. During every test run, a different [bucket,position] would be
calculated for each address. These calculated [bucket,position] could
end up being the same for the 2 different addresses in some test runs
and result in collisions in the addrman. We wouldn't be able to
predict when the collisions are going to happen because we can't
predict the nKey value which is chosen at random. This can cause
flaky tests.
Improve this by allowing deterministic addrman creation in the
functional tests. This creates an addrman with fixed `nKey` = 1 and
we can know the [bucket,position] collisions beforehand, safely add
more addresses in an addrman table and write more extensive tests.
-addrmantest is only used in `p2p_node_network_limited.py` test to
test if the node self-advertises a hard-coded local address
(which wouldn't be advertised in the tests because it's unroutable
without the test-only code path) to check pruning-related services
are correct in that addr.
Remove -addrmantest because the self advertisement happens because
of hard coded test path logic, and expected services are nominal
due to how easily the test-only code could diverge from mainnet
logic. It's also being used only in 1 test.
some of the existing command line args are to be only used in
functional tests. ex: addrmantest, fastprune etc.. make a separate
category -test=<option> for these so that code is cleaner and
user's debug-help output is straightforward.
The `ALLOW_HOST_PACKAGES` variable was introduced in bitcoin#10508 "to
speed up build and avoid timeout".
It is no longer the case for our CI infrastructure, which uses self-
hosted persistent workers and depends caching.
In the current circumstances, it does not seem worth porting this
feature to the upcoming CMake-based build system.
406b71abcb wallet: Migrate entire address book entries (Andrew Chow)
Pull request description:
Not all of the data in an address book entry was being copied to the watchonly and solvables wallets. This includes information such as whether the address was previously spent, and any receive requests that may exist. A test has been added to check that the previously spent information is copied, although it passes without the changes in this PR since this information is also regenerated when a transaction is loaded/added into a wallet.
ACKs for top commit:
ryanofsky:
Code review ACK 406b71abcb. Just suggested change since last review
furszy:
Code review ACK 406b71ab
Tree-SHA512: 13de42b16a1d8524fe0555764744139566b2e7d29741ceffc1158a905dd537136b762330568b3b5cac28cbee1bfd363a20de97d0a6c5296738cb3aa99133945b
5779010ed7 RPC/Blockchain: scanblocks: Accept named param for filter_false_positives (Luke Dashjr)
Pull request description:
Possibly due to a silent cross-merge, `scanblocks` was left out of 96233146dd
ACKs for top commit:
stickies-v:
ACK 5779010ed7
theStack:
ACK 5779010ed7
Tree-SHA512: bade107c7cb5fdd1265224c263a1e1edfc8bc0698b3abfac8d65c49a270181f0311713f7243813de17932a7a7ca65a36850e527ab0b433cf64c32191d3adde70
A Bitcoin Core node may only connect to a peer destination via I2P if both sides
have sessions with the same encryption type. The encryption type is a property
of the session, not the destination. Sessions may support multiple encryption
types.
As Bitcoin Core is not currently setting the I2P encryption type when creating
sessions, it is using the older default, ElGamal (type 0).
This pull updates Bitcoin Core to use both ECIES-X25519 and ElGamal (types 4 and
0, respectively). This allows to connect to I2P peers with either type, and the
newer, faster ECIES-X25519 will be preferred.
See also the recently updated section "Signature and Encryption Types" in
https://geti2p.net/en/docs/api/samv3
Thanks and credit to zzzi2p (https://github.com/zzzi2p) for reporting.
Closes https://github.com/bitcoin/bitcoin/issues/29197.
d83bea42d1 wallettool: Don't create CWallet when dumping DB (Andrew Chow)
40c80e36b1 wallettool: Don't unilaterally reset wallet_instance if loading error (Ava Chow)
Pull request description:
https://github.com/bitcoin/bitcoin/issues/29109#issuecomment-1863449058 reports that a wallet with noncritical errors cannot be dumped with `bitcoin-wallet dump`. This was caused by an erroneous reset of the wallet pointer when the loading the wallet returns something other than `LOAD_OK`. Not all errors are errors that require aborting, so unilaterally resetting the pointer at that time is incorrect. The first commit resolves this issue.
Furthermore, if a wallet has loading errors, that should not prevent the wallet tool from dumping the wallet. The wallet application logic should not get in the way of performing such a low level database operation, especially when it's primary usage is for debugging potentially corrupted wallets. The 2nd commit is taken from #28710 and changes the `dump` to stop at making a `WalletDatabase` rather than making a `CWallet` only to retrieve the underlying `WalletDatabase`.
ACKs for top commit:
furszy:
Code review ACK d83bea42d1
BrandonOdiwuor:
Code Review ACK d83bea42d1
Tree-SHA512: 425d712dfff1002bd81272aca0bae1016f9126a3c89506f8cb7cf0a0ec9f33d0c03b8d03896394f3a45c2998e59047e19218dfd08dc8a5f40e8625134e886b0f
fa87f8feb7 doc: Clarify C++20 comments (MarcoFalke)
Pull request description:
Turns out "class template argument deduction for aggregates" is one of the few things implemented only in recent compilers, see https://en.cppreference.com/w/cpp/compiler_support/20
So clarify the comments.
ACKs for top commit:
hebasto:
ACK fa87f8feb7, I verified the code with clang-{16,17}.
Tree-SHA512: f6d20f946cb6f8e34db224e074ed8f9dfa598377c066d1b58a8feb9e64d007444f1e2c0399e91a3e282fd5d59f90e0d7df90aa3956824d96bc78070ee12f603c
This has outlived its usefulness, doesn't gel well with
newer compilers & `-flto` related options, i.e thin vs full, or `=auto`,
and having `-flto` as the only option means that sometimes this just
needs to be worked around, i.e in oss-fuzz:
https://github.com/google/oss-fuzz/blob/master/projects/bitcoin-core/build.sh.
While it was convenient when `-flto` was newer, support for `-flto` is now
in all compilers we use, and there's also no-longer any real need
for us to treat `-flto` different to any other optimization option.
Remove it, to remove build complexity, and so there's no need
to port a similar option to CMake.
Note that the LTO option remains in depends, because we still a way to
build packages that have LTO specific patches/options.
If we decide to merge this, I'll follow up downstream in oss-fuzz first,
to make sure we don't break the build.
Replace it with a more explicit DISABLE_OPTIMIZED_SHA256 and clean up some.
The macro was originally used by libbitcoinconsensus which opts out of
optimized sha256 for the sake of simplicity.
Also remove the BUILD_BITCOIN_INTERNAL define from libbitcoinkernel for now
as it does not export an api. When it does we can pick a less confusing define
to control its exports.
Removing the define should have the effect of enabling sha256 optimizations
for the kernel.
f8ca1357c8 build: Fix check whether `-latomic` needed (Hennadii Stepanov)
Pull request description:
Clang >=15 still might need linking against `libatomic`.
We use `std::atomic<std::chrono::seconds>::compare_exchange_strong` in `net_processing.cpp`.
Addresses the https://github.com/bitcoin/bitcoin/pull/29165#discussion_r1440293694.
ACKs for top commit:
maflcko:
lgtm ACK f8ca1357c8
fanquake:
ACK f8ca1357c8
Tree-SHA512: ba8b6a88fd3471a206d068e8a000a053c99cb46d26bd04624418ddb066b3b9664a569ec8a1569af67c96b3e27f13dccbd5e24f985290ac072b6d74c92524e35d
fa46cc22bc Remove deprecated -rpcserialversion (MarcoFalke)
Pull request description:
The flag is problematic for many reasons:
* It is deprecated
* It is a global flag, requiring a restart to change, as opposed to a flag that can be set on each RPC invocation
* It may be hidden in config files by accident, hard to debug, causing LND crashes and bugs, see https://github.com/bitcoin/bitcoin/issues/28730#issuecomment-1780940868
* It makes performance improvements harder to implement: https://github.com/bitcoin/bitcoin/pull/17529#issuecomment-556082818
Fix all issues by removing it.
If there is a use-case, likely a per-RPC flag can be added, if needed.
ACKs for top commit:
ajtowns:
crACK fa46cc22bc
TheCharlatan:
lgtm ACK fa46cc22bc
Tree-SHA512: 96ba1c60356ce93954fe5c2a59045771c6d1516ad0d9dc436ef1800a1f1b0153f0d5fb78ca99d53ad54ba25fbce36962bdf1d4325aceedfc8154a61347a6a915
a44808fb43 fuzz: rule-out too deep derivation paths in descriptor parsing targets (Antoine Poinsot)
Pull request description:
This fixes the `mocked_descriptor_parse` timeout reported in #28812 and direct the targets more toward what they are intended to fuzz: the descriptor syntax.
ACKs for top commit:
sipa:
utACK a44808fb43
achow101:
ACK a44808fb43
dergoegge:
ACK a44808fb43 - Not running into timeouts anymore
TheCharlatan:
ACK a44808fb43
Tree-SHA512: a5dd1dbe9adf8f088bdc435addab88b56f435e6d7d2065bd6d5c6d80a32e3f1f97d3d2323131ab233618cd6dcc477c458abe3c4c865ab569449b8bc176231e93
29fde0223a Squashed 'src/secp256k1/' changes from 199d27cea3..efe85c70a2 (fanquake)
Pull request description:
This includes changes from the 0.4.1 release: https://github.com/bitcoin-core/secp256k1/releases/tag/v0.4.1.
> The point multiplication algorithm used for ECDH operations (module ecdh) was replaced with a slightly faster one.
> Optional handwritten x86_64 assembly for field operations was removed because modern C compilers are able to output more efficient assembly. This change results in a significant speedup of some library functions when handwritten x86_64 assembly is enabled (--with-asm=x86_64 in GNU Autotools, -DSECP256K1_ASM=x86_64 in CMake), which is the default on x86_64. Benchmarks with GCC 10.5.0 show a 10% speedup for secp256k1_ecdsa_verify and secp256k1_schnorrsig_verify.
ACKs for top commit:
hebasto:
re-ACK e2cdeb5925
jonasnick:
reACK e2cdeb5925
Tree-SHA512: eaa82721b63e84b9d8dae82956d5e75dbcee50c58c9049b7901055d79aef938bd268e18ce4ff85feb73aae7ee1cf58018b93067692f8f69f80216d336bd6f10a
faebf1df2a wallet: Fix use-after-free in WalletBatch::EraseRecords (MarcoFalke)
Pull request description:
Creating a copy of the pointer to the underlying data of the stream is not enough to copy the data.
Currently this happens to work sometimes, because the stream may not immediately free unused memory. However, there is no guarantee by the stream interface to always behave this way. Also, if `vector::clear` is called on the underlying memory, any pointers to it are invalid.
Fix this, by creating a full copy of all bytes.
ACKs for top commit:
achow101:
ACK faebf1df2a
Tree-SHA512: 79ede9bc16cf257609545597bc6d9623ceead4531780ea6037cc5684aa3a7c7d80601354d315358defe47193f978a8ce40c5dc4637e32936c76157679b549ac5
b1318dcc56 test: change `m_submitted_in_package` input to fuzz data provider boolean (ismaelsadeeq)
5615e16b70 tx fees: update `m_from_disconnected_block` to `m_mempool_limit_bypassed` (ismaelsadeeq)
fcd4296648 doc: fix typo and update incorrect comment (ismaelsadeeq)
562664d263 test: wait for fee estimator to catch up before estimating fees (ismaelsadeeq)
Pull request description:
This is a simple PR that does two things
1. Fixes#29000 by waiting for the fee estimator to catch up after `removeForBlock` calls before calling `estimateFee` in the `BlockPolicyEstimates` unit test.
2. Addressed some outstanding review comments from #28368
- Updated `NewMempoolTransactionInfo::m_from_disconnected_block` to `NewMempoolTransactionInfo::m_mempool_limit_bypassed` which now correctly indicates what the boolean does.
- Changed input of `processTransaction`'s tx_info `m_submitted_in_package` input from false to fuzz data provider boolean.
- Fixed some typos, and update incorrect comment
ACKs for top commit:
martinus:
re-ACK b1318dcc56
glozow:
utACK b1318dcc56
Tree-SHA512: 45268729bc044da4748fe004524e0df696d2ec92c5bd053db9aad6e15675f3838429b2a7b9061a6b694be4dc319d1782a876b44df506ddd439d62ad07252d0e1
e03d6f7ed5 fuzz: set `m_fallback_fee`/`m_fee_mode` in `wallet_fees` target (brunoerg)
Pull request description:
`m_fallback_fee` and `m_fee_mode` are used in `GetMinimumFeeRate` but we're not setting any value for them in `wallet_fees` target. That's the reason fuzzing is never reaching the following code:

This PR fixes it.
ACKs for top commit:
maflcko:
review ACK e03d6f7ed5
achow101:
ACK e03d6f7ed5
murchandamus:
ACK e03d6f7ed5
Tree-SHA512: 5d364f5351d65762a3ddf88e3abb7bda401b7e4955285e083031d216fb50082b1ea98e2c065aff75a5a8a3d1bc4c2e5e3ca9f9478d902ee8f8d4347b6cbe53af
fa1d49542e refactor: share and use `GenerateRandomKey` helper (Sebastian Falbesoner)
Pull request description:
Making the `GeneratingRandomKey` helper (recently introduced in PR #28433, commit b6934fd03f) available to other modules via key.{h.cpp} allows us to create random private keys directly at CKey instantiation, in contrast to the currently needed two-step process of creating an (invalid) CKey instance first and then having to call `MakeNewKey(...)`.
This is mostly used in unit tests and a few instances in the wallet.
ACKs for top commit:
Sjors:
re-ACK fa1d49542e
achow101:
ACK fa1d49542e
sipa:
utACK fa1d49542e
kristapsk:
cr utACK fa1d49542e
stratospher:
ACK fa1d495.
Tree-SHA512: 6fec73f33efe5bd77ca7d3c2fc06725d96f789f229294c39377e682ff222cfc7990b77c92e0bfd4cb6cf891d007ab1f86d395907511f06e87044bae37652a2fd
In reality some mempool transaction might be submitted in a package,
so change m_submitted_in_package to fuzz data provider boolean just like
m_has_no_mempool_parents.
The boolean indicates whether the transaction was added without enforcing mempool
fee limits. m_mempool_limit_bypassed is the correct variable name.
Also changes NewMempoolTransactionInfo booleans descriptions to the format that
is consistent with the codebase.
This affects manual connections made either with -connect, or with
-addnode provided as a bitcoind config arg (the addnode RPC has an
extra option for v2).
We don't necessarily know if our peer supports v2, but will reconnect
with v1 if they don't. In order to do that, improve the reconnection
behavior such that we will reconnect after a sleep of 500ms
(which usually should be enough for our peer to send us their
version message).
Making the `GenerateRandomKey` helper available to other modules via
key.{h.cpp} allows us to create random private keys directly at
instantiation of CKey, in contrast to the two-step process of creating
the instance and then having to call `MakeNewKey(...)`.
b335710782 depends: patch around non-determinism in qt (fanquake)
e8ecec4575 build: rename native_clang to native_llvm (fanquake)
b0c290340c Revert "build: Patch Qt to handle minimum macOS version properly" (fanquake)
558250dec1 guix: use clang-toolchain-17 for macOS build (fanquake)
5ddd7c65b4 build: Bump `native_clang` up to 17.0.6 (Hennadii Stepanov)
Pull request description:
This is the last step before #21778. We need LLVM 17.x so that lld has `-fixup_chains`.
ACKs for top commit:
theuni:
ACK b335710782.
TheCharlatan:
ACK b335710782
Tree-SHA512: 8142956196a481178f360258c2e4d924178d552966b713323f29f2deba7e5ec73a3da1c9d79d97c9e7f6aa18ed7429cd6660826aa633e6dde1ac56000b9ad57f
fae526345d Allow std::byte C-style array serialization (MarcoFalke)
fa898e6836 refactor: Print verbose serialize compiler error messages (MarcoFalke)
Pull request description:
Currently, trying to serialize an object that can't be serialized will fail with a short error message. For example, the diff and the error message:
```diff
diff --git a/src/test/serialize_tests.cpp b/src/test/serialize_tests.cpp
index d75eb499b4..773f49845b 100644
--- a/src/test/serialize_tests.cpp
+++ b/src/test/serialize_tests.cpp
@@ -62,6 +62,8 @@ public:
BOOST_AUTO_TEST_CASE(sizes)
{
+ int b[4];
+ DataStream{} << b << Span{b};
BOOST_CHECK_EQUAL(sizeof(unsigned char), GetSerializeSize((unsigned char)0));
BOOST_CHECK_EQUAL(sizeof(int8_t), GetSerializeSize(int8_t(0)));
BOOST_CHECK_EQUAL(sizeof(uint8_t), GetSerializeSize(uint8_t(0)));
```
```
./serialize.h:765:6: error: member reference base type 'const int[4]' is not a structure or union
765 | a.Serialize(os);
| ~^~~~~~~~~~
```
```
./serialize.h:277:109: error: no matching function for call to 'UCharCast'
277 | template <typename Stream, typename B> void Serialize(Stream& s, Span<B> span) { (void)/* force byte-type */UCharCast(span.data()); s.write(AsBytes(span)); }
| ^~~~~~~~~
```
This is fine. However, it would be more helpful for developers and more accurate by the compiler to explain why each function is not selected.
Fix this by using C++20 concepts where appropriate.
ACKs for top commit:
ajtowns:
reACK fae526345d
achow101:
ACK fae526345d
TheCharlatan:
Re-ACK fae526345d
Tree-SHA512: e03a684ccfcc5fbcad7f8a4899945a05989b555175fdcaebdb113aff46b52b4ee7b467192748edf99c5c348a620f8e52ab98bed3f3fca88280a64dbca458fe8a
e1281f1bbd wallet: fix key parsing check for miniscript expressions in `ParseScript` (brunoerg)
Pull request description:
In `ParseScript`, when processing miniscript expressions, the way we check for key parsing error is wrong, the actual code is unreachable because we're checking it into `if (node)` (successful parsing) statement.
ACKs for top commit:
sipa:
utACK e1281f1bbd
RandyMcMillan:
utACK e1281f1bbd
achow101:
ACK e1281f1bbd
Tree-SHA512: c4b3765d32673928a1f6d84ecbaa311870da9a9625753ed15ea57c802a9f16ddafa48c1dc66c0e4be284c5862e7821ed94135498ed9b9f3d7342a080035da289
cd810075ed fuzz: coinselection, improve `min_viable_change`/`change_output_size` (brunoerg)
Pull request description:
Instead of "randomly" fuzzing `min_viable_change` and `change_output_size`, and since they're correlated, this PR changes the approach to fuzz them according to the logic in `CreateTransactionInternal`.
ACKs for top commit:
murchandamus:
ACK cd810075ed
achow101:
ACK cd810075ed
furszy:
Code ACK cd810075ed
Tree-SHA512: 4539b469f00cdf666078d80c07ed062726f804e390400348148cd3092db9cdc178c6d00ead39aef19acf97badfb6576ce23546d8967387e81c5398d52d7f4404
The Apple notary service requires submitted app bundles to be configured to use the hardened runtime libraries. This is configured at signing time, and supported by the signapple tool Bitcoin Core uses for reproduceable signed binaries. We simply need to pass "--hardened-runtime" when the signature is created. Once attached to the bundle, the resulting codesigned binary can be successfully submitted to the Apple binary notarization service by any Apple Developer.
19bb65bf25 [doc]: add doxygen return comment for CheckPackageLimits (ismaelsadeeq)
Pull request description:
This PR adds a doxygen comment on `CheckPackageLimits` describing what the method returns.
Fixes https://github.com/bitcoin/bitcoin/pull/28863#discussion_r1429805433
ACKs for top commit:
Sjors:
utACK 19bb65bf25
Zero-1729:
utACK 19bb65bf25
Tree-SHA512: ccf1cc00a44d3fff60f28ad6766019a9f61b349729eab3cb02bc76b13c2e55441348a1602d806e60e4b2eabeb1f5d1ddacddf86c0bcdb78b078bb3a863b650c2
These provide simple and clear ways to write the most common logging
operations:
LogInfo("msg");
LogDebug(BCLog::LogFlags::NET, "msg");
LogError("msg");
LogWarning("msg");
LogTrace(BCLog::LogFlags::NET, "msg");
For cases where the level cannot be hardcoded, LogPrintLevel(category,
level, ...) remains available.
It's not necessary to set up an entire CWallet just so we can get access
to the WalletDatabase and read the records. Instead we can go one level
lower and make just a WalletDatabase.
When there is a wallet loading error, it could be a noncritical one so
it is not necessary to make wallet_instance a nullptr. The wallet can
still go on with normal operation in that case, as we do for loading in
bitcoind and bitcoin-qt.
8dec9c560b wallet, mempool: propagete `checkChainLimits` error message to wallet (ismaelsadeeq)
Pull request description:
* Requested in [#28391 comment](https://github.com/bitcoin/bitcoin/pull/28391#discussion_r1382997719)
* The error message is static when a new transaction is created and package limit is reached.
`Transaction has too long of a mempool chain`
While the [`CTxMempool::CheckPackageLimits`](5800c558eb/src/txmempool.cpp (L199)) provide explicit information about the error message.
* This PR updates [`CTxMempool::CheckPackageLimits`](5800c558eb/src/txmempool.cpp (L199)) return type to `util::Result<void>`, `CheckPackageLimits` now returns void when package limit is not hit, and returns the error string whenever package limit is hit instead of using out parameter `errString`.
* The PR updates [`checkChainLimits`](5800c558eb/src/node/interfaces.cpp (L703)) return type to `util::Result<void>`.
* Now the wallet `CreateTransactionInternal` will have access to the package limit error string whenever its hit.
* Also Updated functional test to reflect the error message from `CTxMempool::CheckPackageLimits` output.
ACKs for top commit:
glozow:
utACK 8dec9c560b
Sjors:
utACK 8dec9c560b
TheCharlatan:
Re-ACK 8dec9c560b
Tree-SHA512: ddeac18aeba6f8e3be0e3fe76bf3db655352e3b415169f1f83ea1e8976a2f3e3de021c8da6880eb8382ab52d545e418e3f4d57adcc68ecb4f390339710ee6f30
b2fc7a2eda [fuzz] Improve fuzzing stability for minisketch harness (dergoegge)
Pull request description:
The `minisketch` harness has low stability due to:
* Rng internal to minisketch
* Benchmarkning for the best minisketch impl
Fix this by seeding the rng and letting the fuzzer choose the impl.
Also see #29018.
ACKs for top commit:
maflcko:
review ACK b2fc7a2eda
Tree-SHA512: 3d81414299c6803c34e928a53bcf843722fa8c38e1d3676cde7fa80923f9058b1ad4b9a2941f718303a6641b17eeb28b4a22eda09678102e9fb7c4e31d06f8f2
ff896d2581 contrib: drop GCC MAX_VERSION to 4.3.0 in symbol-check (fanquake)
Pull request description:
Reflect the actual symbols used, i.e:
```bash
bitcoind: symbol __bswapsi2 from unsupported version GCC_4.3.0(7)
```
ACKs for top commit:
TheCharlatan:
ACK ff896d2581
Tree-SHA512: b38ff8f4dd78d2d1c9063c53544dc4f240c3043f142e1581f7ba42f088a509293f6f17cc402c60ac82bff3b36668866b87e0e9e4d10d929484bb4c7a3e654f25
Update CheckPackageLimits to use util::Result to pass the error message
instead of out parameter.
Also update test to reflect the error message from `CTxMempool`
`CheckPackageLimits` output.
7b45744df3 tests: ensure functional tests set permitbaremultisig=1 when needed (Anthony Towns)
7dfabdcf86 tests: test both settings for permitbaremultisig in p2sh tests (Anthony Towns)
Pull request description:
Update unit and functional tests so that they continue to work if the default for `-permitbaremultisig` is changed.
ACKs for top commit:
maflcko:
lgtm ACK 7b45744df3
instagibbs:
crACK 7b45744df3
ajtowns:
> crACK [7b45744](7b45744df3)
achow101:
ACK 7b45744df3
glozow:
ACK 7b45744df3, changed default locally and all tests passed
Tree-SHA512: f89f9e2bb11f07662cfd57390196df9e531064e1bd662e1db7dcfc97694394ae5e8014e9d209b9405aa09195bf46fc331b7fba10378065cdb270cbd0669ae904
This option tells the logging system to always include a "[cat:level]"
prefix, so [net] becomes [net:debug], LogInfo/LogPrint statements will have
an [all:info] prefix, and LogWarning and LogError logs will become
[all:warning] and [all:error]. This may be easier for automated parsing
of logs, particularly if additional prefixes such as thread or source
location are enabled.
Previously Info-level logging when a category was specified (via
LogPrintLevel) would only print the corresponding log message if
`-debug=category` were specified, while Info-level logging without a
category would always be printed. Make this more consistent by having
Info messages always be logged, whether they include a category or not.
6666713041 refactor: Rename fs::path::u8string() to fs::path::utf8string() (MarcoFalke)
856c88776f ArgsManager: return path by value from GetBlocksDirPath() (Vasil Dimov)
fa3d9304e8 refactor: Remove pre-C++20 fs code (MarcoFalke)
fa00098e1a Add tests for C++20 std::u8string (MarcoFalke)
fa2bac08c2 refactor: Avoid copy/move in fs.h (MarcoFalke)
faea30227b refactor: Use C++20 std::chrono::days (MarcoFalke)
Pull request description:
This:
* Removes dead code.
* Avoids unused copies in some places.
* Adds copies in other places for safety.
ACKs for top commit:
achow101:
ACK 6666713041
ryanofsky:
Code review ACK 6666713041. Just documentation change since last review.
stickies-v:
re-ACK 6666713041
Tree-SHA512: 6176e44f30b310d51632ec2d3827c3819905d0ddc6a4b57acfcb6cfa1f9735176da75ee8ed4a4abd1296cb0b83bee9374cc6f91ffac87c19b63c435eeadf3f46
1ce45baed7 rpc: getwalletinfo, return wallet 'birthtime' (furszy)
83c66444d0 test: coverage for wallet birth time interaction with -reindex (furszy)
6f497377aa wallet: fix legacy spkm default birth time (furszy)
75fbf444c1 wallet: birth time update during tx scanning (furszy)
b4306e3c8d refactor: rename FirstKeyTimeChanged to MaybeUpdateBirthTime (furszy)
Pull request description:
Fixing #28897.
As the user may have imported a descriptor with a timestamp newer
than the actual birth time of the first key (by setting 'timestamp=now'),
the wallet needs to update the birth time when it detects a transaction
older than the oldest descriptor timestamp.
Testing Notes:
Can cherry-pick the test commit on top of master. It will fail there.
ACKs for top commit:
Sjors:
re-utACK 1ce45baed7
achow101:
ACK 1ce45baed7
Tree-SHA512: 10c2382f87356ae9ea3fcb637d7edc5ed0e51e13cc2729c314c9ffb57c684b9ac3c4b757b85810c0a674020b7287c43d3be8273bcf75e2aff0cc1c037f1159f9
6db04be102 Get rid of shutdown.cpp/shutdown.h, use SignalInterrupt directly (Ryan Ofsky)
213542b625 refactor: Add InitContext function to initialize NodeContext with global pointers (Ryan Ofsky)
feeb7b816a refactor: Remove calls to StartShutdown from KernelNotifications (Ryan Ofsky)
6824eecaf1 refactor: Remove call to StartShutdown from stop RPC (Ryan Ofsky)
1d92d89edb util: Get rid of uncaught exceptions thrown by SignalInterrupt class (Ryan Ofsky)
ba93966368 refactor: Remove call to ShutdownRequested from IndexWaitSynced (Ryan Ofsky)
42e5829d97 refactor: Remove call to ShutdownRequested from HTTPRequest (Ryan Ofsky)
73133c36aa refactor: Add NodeContext::shutdown member (Ryan Ofsky)
f4a8bd6e2f refactor: Remove call to StartShutdown from qt (Ryan Ofsky)
f0c73c1336 refactor: Remove call to ShutdownRequested from rpc/mining (Ryan Ofsky)
263b23f008 refactor: Remove call to ShutdownRequested from chainstate init (Ryan Ofsky)
Pull request description:
This change drops `shutdown.h` and `shutdown.cpp` files, replacing them with a `NodeContext::shutdown` member which is used to trigger shutdowns directly. This gets rid of an unnecessary layer of indirection, and allows getting rid of the `kernel::g_context` global.
Additionally, this PR tries to improve error handling of `SignalInterrupt` code by marking relevant methods `[[nodiscard]]` to avoid the possibility of uncaught exceptions mentioned https://github.com/bitcoin/bitcoin/pull/27861#discussion_r1255496707.
Behavior is changing In a few cases which are noted in individual commit messages. Particularly: GUI code more consistently interrupts RPCs when it is shutting down, shutdown state no longer persists between unit tests, the stop RPC now returns an RPC error if requesting shutdown fails instead of aborting, and other failed shutdown calls now log errors instead of aborting.
This PR is a net reduction in lines of code, but in some cases the explicit error handling and lack of global shutdown functions do make it more verbose. The verbosity can be seen as good thing if it discourages more code from directly triggering shutdowns, and instead encourages code to return errors or send notifications that could be translated into shutdowns. Probably a number of existing shutdown calls could just be replaced by better error handling.
ACKs for top commit:
achow101:
ACK 6db04be102
TheCharlatan:
Re-ACK 6db04be102
maflcko:
ACK 6db04be102👗
stickies-v:
re-ACK 6db04be102
Tree-SHA512: 7a34cb69085f37e813c43bdaded1a0cbf6c53bd95fdde96f0cb45346127fc934604c43bccd3328231ca2f1faf712a7418d047ceabd22ef2dca3c32ebb659e634
98afe78661 doc: Update bitcoin-tx replaceable documentation (Kashif Smith)
94feaf2b66 tests: Add unit tests for bitcoin-tx replaceable command (Kashif Smith)
c2b836b119 bitcoin-tx: Make replaceable value optional (Kashif Smith)
Pull request description:
This fixes#28638. The issue was originally raised by dooglus, who also suggested the patch found in this code. Additionally, test coverage has been added and documentation has been updated.
ACKs for top commit:
achow101:
ACK 98afe78661
pinheadmz:
ACK 98afe78661
hernanmarino:
Tested ACK 98afe78661
instagibbs:
untested ACK 98afe78661
Tree-SHA512: ea1384aba7b0014c8cbeb7280d66b1e617d406fb02471dff33873057132b80518c94c7caa4b0426c26d17ce8aa393107de319dde781ace8df72f0314c8c75159
43c3246af7 ci: Set `HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK` to avoid failures (Hennadii Stepanov)
Pull request description:
Homebrew attempts to check for outdated dependents or those with broken linkage. Such behavior might lead to failures when Homebrew updates them on old macOS images. For example, https://github.com/bitcoin/bitcoin/actions/runs/7199058794/job/19609891263 using the macOS image version `20231025.2`.
This PR prevents such behavior.
ACKs for top commit:
maflcko:
lgtm ACK 43c3246af7
ismaelsadeeq:
re ACK 43c3246af7
Tree-SHA512: cbe3cef5adf3f00eb618ba17aad3dc76c0c5d11142122a26b93619ae47dc50771e9e095caa898213325ed6ff41c07119429c0a9094bb98ead5601855d07bb2ea
Homebrew attempts to check for outdated dependents or those with broken
linkage. Such behavior might lead to failures when Homebrew updates them
on old macOS images.
This change prevents such behavior.
37c75c5820 test: wallet, fix change position out of range error (furszy)
Pull request description:
Fixes#29061. Only the benchmark is affected.
Since #25273, the behavior of 'inserting change at a random position'
is instructed by passing ´std::nullopt´ instead of -1.
Also, added missing documentation about the meaning of
'change_pos=std::nullopt' inside 'CWallet::CreateTransaction()'
ACKs for top commit:
achow101:
ACK 37c75c5820
kevkevinpal:
ACK [37c75c5](37c75c5820)
BrandonOdiwuor:
ACK 37c75c5820
Tree-SHA512: d9a8d8533540455716a5090fcf407573cad9f0d0018a05f903f89e51620302f9b256318db6f7338b85c047f7fab372d724e916b1721d7ed302dbf3d845b08734
49a90915aa build: Bump minimum required Boost to 1.73.0 to support C++20 (Hennadii Stepanov)
Pull request description:
Boost versions <1.73 have C++20-specific bugs that were fixed in the following commits:
- 15fcf21356
- 495c095dc0
I tested [`libboost1.71-dev`](https://packages.ubuntu.com/focal/libboost1.71-dev) in Ubuntu 20.04 and Boost 1.71, 1.72, 1.73 in our depends build system.
Closes https://github.com/bitcoin/bitcoin/issues/29063.
ACKs for top commit:
fanquake:
ACK 49a90915aa
Tree-SHA512: b8ebc08af85abfa3fda70961bd1136ee9e5149dd76a3f901e43acba624d231971873cba5cbf30837f9e5ab58790b8330f241a76cb76d8cf5dce5ad0cca33fba8
308aec3e56 build: disable external-signer for Windows (fanquake)
35537318a1 ci: remove --enable-external-signer from win64 job (fanquake)
Pull request description:
It's come to light that Boost ASIO (a Boost Process sub dep) has in some
instances, been quietly initialising our network stack on Windows (see
PR https://github.com/bitcoin/bitcoin/pull/28486 and discussion in https://github.com/bitcoin/bitcoin/issues/28940).
This has been shielding a bug in our own code, but the larger issue
is that Boost Process/ASIO is running code before main, and doing things
like setting up networking. This undermines our own assumptions about
how our binary works, happens before we run any sanity checks,
and before we call our own code to setup networking. Note that ASIO also
calls WSAStartup with version `2.0`, whereas we call with `2.2`.
It's also not clear why a feature like external signer would have a
dependency that would be doing anything network/socket related,
given it only exists to spawn a local process.
See also the discussion in https://github.com/bitcoin/bitcoin/issues/24907. Note that the maintaince of Boost Process in general,
has not really improved. For example, rather than fixing bugs like https://github.com/boostorg/process/issues/111,
i.e, https://github.com/boostorg/process/pull/317, the maintainer chooses to just wrap exception causing overflows
in try-catch blocks: 0c42a58eac. These changes get merged in large,
unreviewed PRs, i.e https://github.com/boostorg/process/pull/319.
This PR disables external-signer on Windows for now. If, in future, someone
changes how Boost Process works, or replaces it entirely with some
properly reviewed and maintained code, we could reenable this feature on
Windows.
ACKs for top commit:
hebasto:
re-ACK 308aec3e56.
TheCharlatan:
ACK 308aec3e56
Tree-SHA512: 7405f7fc9833eeaacd6836c4e5b1c1a7845a40c1fdd55c1060152f8d8189e4777464fde650e11eb1539556a75dddf49667105987078b1457493ee772945da66e
fa0534d7e4 test: Actually fail when a python unit test fails (MarcoFalke)
Pull request description:
Currently python unit test failures are ignored.
Fix this.
ACKs for top commit:
theStack:
ACK fa0534d7e4
BrandonOdiwuor:
ACK fa0534d7e4
Tree-SHA512: c136be4c8d861d966f380e04d5d14b711b90c4011101302dae1332496e493207c5c673927586ed35b02b61a0b050bf45053a31e6ff766ec52f1d054caf0985e2
bde8d63b17 depends: build libmultiprocess with position independant code (fanquake)
506634d79d depends: always install libmultiprocess to /lib (fanquake)
beb3096263 depends: always install capnp to /lib (fanquake)
Pull request description:
Change to always install libmultiprocess into `lib/`. On some systems (my Fedora aarch64 box), libmultiprocess was being installed into `lib64/`, and then configure would fail to pick it up, because we only add `lib/` to pkgconfig/ldflags out of depends. Rather than adding lib64 to those, I opted for installing libmultiprocess into lib, with every other dependency we build.
This was broken in our build after https://github.com/chaincodelabs/libmultiprocess/pull/79 upstream.
ACKs for top commit:
ryanofsky:
Code review ACK bde8d63b17. Only changes since last review were reverting the native_capnp change as suggested, and changing the order of the first two commits.
Tree-SHA512: ddd547e4ac224f2f199c569efd91104db7f2c243b124f9535aa0d9377315775ac566d699101580ce45ddd6676ad3e0c8cbe256334eeed9548205c2fa04d02102
fa3da629a1 Remove DirIsWritable, GetUniquePath (MarcoFalke)
fad3a9793b Return LockResult::ErrorWrite in LockDirectory (MarcoFalke)
fa0afe7408 refactor: Return enum in LockDirectory (MarcoFalke)
Pull request description:
`GetUniquePath` is only used in tests and in `DirIsWritable`. The check by `DirIsWritable` is redundant with the check done in `LockDirectory`.
Fix the redundancy by removing everything, except `LockDirectory`.
ACKs for top commit:
TheCharlatan:
Re-ACK fa3da629a1
hebasto:
ACK fa3da629a1, I have reviewed the code and it looks OK.
Tree-SHA512: e95f18cd586de7582e9c08ac7ddb860bfcfcbc8963804f45c5784c5e4c0598dc59ae7e45dd4daf30a5020dbf8433f5db2ad06e46a8676371982003790043c6c9
Since #25273, the behavior of 'inserting change at a random
position' is instructed by passing std::nullopt instead of -1.
Also, added missing documentation about the meaning of
'change_pos=std::nullopt' inside 'CWallet::CreateTransaction()'
576bee88fd fuzz: disable BnB when SFFO is enabled (furszy)
05e5ff194c test: add coverage for BnB-SFFO restriction (furszy)
0c5755761c wallet: create tx, log resulting coin selection info (furszy)
5cea25ba79 wallet: skip BnB when SFFO is active (Murch)
Pull request description:
Solves #28918. Coming from https://github.com/bitcoin/bitcoin/issues/28918#issuecomment-1838626406 discussion.
The intention is to decouple only the bugfix relevant commits from #28985, allowing them to be included in the 26.x release. This way, we can avoid disabling the coin selection fuzzing test for an entire release.
Note:
Have introduced few changes to the bug fix commit so that the unit tests pass without the additional burden introduced in #28985.
ACKs for top commit:
josibake:
ACK 576bee88fd
murchandamus:
ACK 576bee88fd
achow101:
ACK 576bee88fd
Tree-SHA512: f5d90eb3f3f524265afe4719495c9bf30f98b9af26cf039f7df5a7db977abae72caa7a3478cdd0ab10cd143bc1662e8fc5286b5bc10fc10f0dd582a45b45c31a
On some systems, libmultiprocess would be installed into `lib64`, I
assume due to the use of GNUInstallDirs, however all other libs we build
in depends, go into lib/. Rather than adding lib64/ to the pkg-config
and link flags, I opted for always installing into lib/.
This was changed in
https://github.com/chaincodelabs/libmultiprocess/pull/79 upstream.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
On some systems, capnp would be installed into `lib64`, I
assume due to the use of GNUInstallDirs, however all other libs we build
in depends, go into lib/. Rather than adding lib64/ to the pkg-config
and link flags, I opted for always installing into lib/.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
7b22cd80e0 Revert "ci: Only run functional tests on windows in master" (Hennadii Stepanov)
Pull request description:
This PR reverts commit aba4a5887b from https://github.com/bitcoin/bitcoin/pull/28567.
The Windows-specific code received [quality](https://github.com/bitcoin/bitcoin/pull/28486) and [performance](https://github.com/bitcoin/bitcoin/pull/29045) improvements recently. So there are no reasons to skip functional tests in PRs anymore.
In my own repo, I've run the GHA Windows job more than 100 times with no failure.
ACKs for top commit:
maflcko:
lgtm ACK 7b22cd80e0
TheCharlatan:
ACK 7b22cd80e0
Tree-SHA512: 1e8687e8efe12db506e7cd2d5df9e48b5acb98a339f84684dd0fd67280e22227e2a5a206f1108b09e49038fab7a3ca2ffbd985677f00048c0962b39b2b9a2ba5
bd7f5d33e3 wallet: Assert that the wallet is not initialized in LoadWallet (Andrew Chow)
fb0b6ca4e5 tests, bench: Remove incorrect LoadWallet() calls (Andrew Chow)
Pull request description:
`CWallet::LoadWallet()` expects to be called after a `CWallet` is constructed, but before any of its member functions called. Doing so invalidates pointers which causes issues with some PRs and branches that I am working on. This was being used incorrectly in a few tests and benchmarks, resulting in segfaults.
As a precaution for this kind of issue in the future, I've also added a few asserts to `LoadWallet()` so that developers will notice when it is used incorrectly.
As similar issue was fixed in #27666
ACKs for top commit:
S3RK:
ACK bd7f5d33e3
furszy:
ACK bd7f5d33
Tree-SHA512: 7664f12b8452994e7fc4d7d4f77697fb5f75edb0dba95ba99a4a23ec03d5b8e0ecbdcb7635547a0e8b4f89f708f98dcb5d039df0559e24b1ae411ed630e16e14
Verify the transaction creation process does not produce
a BnB solution when SFFO is enabled.
This is currently problematic because it could require a
change output. And BnB is specialized on changeless solutions.
Co-authored-by: Andrew Chow <achow101@gmail.com>
Co-authored-by: Murch <murch@murch.one>
LoadWallet() must only be called immediately after a CWallet is
constructed, or not at all. Doing so after any other CWallet member
functions have been called may cause pointers and other objects
setup by other those functions to become invalidated.
Since these tests and benchmarks are using completely new wallets with
mock databases, it's not necessary to call LoadWallet() anyways, so
these can be dropped.
`ArgsManager::m_cached_blocks_path` is protected by
`ArgsManager::cs_args` and returning a reference to it after releasing
the mutex is unsafe.
To resolve this, return a copy of the path. This has some performance
penalty which is presumably ok, given that paths are a few 100s bytes
at most and `GetBlocksDirPath()` is not called often.
This silences the following (clang 18):
```
common/args.cpp:288:31: error: returning variable 'm_cached_blocks_path' by reference requires holding mutex 'cs_args' [-Werror,-Wthread-safety-reference-return]
288 | if (!path.empty()) return path;
| ^
```
Do the same with
`ArgsManager::GetDataDir()`,
`ArgsManager::GetDataDirBase()` and
`ArgsManager::GetDataDirNet()`.
Treating std::string as UTF-8 is deprecated in std::filesystem::path
since C++20.
However, it makes this codebase easier to read and maintain to retain
the ability for std::string to hold UTF-8.
fa8adbe7c1 build: Enable -Wunreachable-code (MarcoFalke)
Pull request description:
It seems a bit confusing to write code after a `return`. This can even lead to bugs, or incorrect code, such as https://github.com/bitcoin/bitcoin/pull/28830/files#r1415372320 . (Edit: The linked instance is not found by clang's `-Wunreachable-code`).
Fix all issues by enabling `-Wunreachable-code`.
This flag also enables `-Wunreachable-code-loop-increment`, according to https://clang.llvm.org/docs/DiagnosticsReference.html#wunreachable-code, so remove that.
ACKs for top commit:
ajtowns:
> ACK [fa8adbe](fa8adbe7c1)
stickies-v:
ACK fa8adbe7c1
jonatack:
ACK fa8adbe7c1 tested with arm64 clang 17.0.6
Tree-SHA512: 12a2f74b69ae002e62ae08038f7458837090a12051a4c154d05ae4bb26fb19fc1fa76c63aedf2b3fbb36f048c593ca3b8c0efe03fe93cf07a0fd114fc84ce1e7
0295b44c25 wallet: return CreatedTransactionResult from FundTransaction (Andrew Chow)
758501b713 wallet: use optional for change position as an optional in CreateTransaction (Andrew Chow)
2d39db7aa1 wallet: Explicitly preserve scriptSig and scriptWitness in CreateTransaction (Andrew Chow)
14e50746f6 wallet: Explicitly preserve transaction version in CreateTransaction (Andrew Chow)
0fefcbb776 wallet: Explicitly preserve transaction locktime in CreateTransaction (Andrew Chow)
4d335bb1e0 wallet: Set preset input sequence through coin control (Andrew Chow)
596642c5a9 wallet: Replace SelectExternal with SetTxOut (Andrew Chow)
5321786b9d coincontrol: Replace HasInputWeight with returning optional from Get (Andrew Chow)
e1abfb5b20 wallet: Introduce and use PreselectedInput class in CCoinControl (Andrew Chow)
Pull request description:
Currently `FundTransaction` handles transaction locktime and preset input data by extracting the selected inputs and change output from `CreateTransaction`'s results. This means that `CreateTransaction` is actually unaware of any user desired locktime or sequence numbers. This can have an effect on whether and how anti-fee-sniping works.
This PR makes `CreateTransaction` aware of the locktime and preset input data by providing them to `CCoinControl`. `CreateTransasction` will then set the sequences, scriptSigs, scriptWItnesses, and locktime as appropriate if they are specified. This allows `FundTransaction` to actually use `CreateTransaction`'s result directly instead of having to extract the parts of it that it wants.
Additionally `FundTransaction` will return a `CreateTransactionResult` as `CreateTransaction` does instead of having several output parameters. Lastly, instead of using `-1` as a magic number for the change output position, the change position is changed to be an optional with no value set indicating no desired change output position (when provided as an input parameter) or no change output present (in the result).
ACKs for top commit:
josibake:
ACK 0295b44c25
S3RK:
Code review ACK 0295b44c25
Tree-SHA512: 016be4d41cbf97e1938506e70959bb5335b87006162a1c1c62fa0adb637cbe7aefb76d342b8efad5f37dc693f270c8d0a0839e239fd1ac32c6941a8172f1a710
9f265d8825 fuzz: Detect deadlocks in process_message (dergoegge)
fae1e7e012 fuzz: p2p: Detect peer deadlocks (MarcoFalke)
Pull request description:
It may be possible that a peer connection will deadlock, due to software bugs such as https://github.com/bitcoin/bitcoin/pull/18808.
Fix this by detecting them in the fuzz target.
Can be tested by introducing a bug such as:
```diff
diff --git a/src/net_processing.cpp b/src/net_processing.cpp
index 1067341495..97495a13df 100644
--- a/src/net_processing.cpp
+++ b/src/net_processing.cpp
@@ -2436,3 +2436,3 @@ void PeerManagerImpl::ProcessGetData(CNode& pfrom, Peer& peer, const std::atomic
if (it != peer.m_getdata_requests.end() && !pfrom.fPauseSend) {
- const CInv &inv = *it++;
+ const CInv& inv = *it;
if (inv.IsGenBlkMsg()) {
```
Using a fuzz input such as:
```
$ base64 ./timeout-ada0fecaba2b8c46c6e970cf637d9625b01bf7e5
kNptdNbW1tbWYghvXIpwb25vPQAA////////cwAjLv8AXAB2ZXJhY2sAQW5v/62tra3Pz///////
//////////////////////9c8GZpbHRlcmxvYWQAAAEAAwAAAABVYwC2XABmaWx0ZXJhZGQAAAAX
Fxdn/////2V0F861tcqvEmAAACEAAABjYXB0dXJldmUAAH4AgAA1PNfX11x0Z2V0ZGF0YQBDACOw
AQMAAAAGIm5GERoLWcqvEmBD61u/KMNPOl4zKh/HKLK3PPGIkQ9eE/////////8AAAAAAAAAAFtb
WyjDTzpeMSofx7K3PNfX11x0Z2V0ZGF0YQBDACMwAQMAAAAGIm5GERoLWcqvEmBD61u/KMNPOl4z
Kh/Hsrc88YiRD2/Nzc3Nzc3Nzc3NTc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3N
zWWj1NTUudTU1NTU1P///0j+P/9cdHR4AAAAAAAAy/4AAHR4AAAAAAAAP8v+AAD/+P//////////
AX55bJl8HWnz/////wAgXGF0YVPxY2RkAAAA
```
And running the fuzz target:
```
$ FUZZ=process_messages ./src/test/fuzz/fuzz -runs=1 -timeout=18 ./timeout-ada0fecaba2b8c46c6e970cf637d9625b01bf7e5
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3436516708
INFO: Loaded 1 modules (390807 inline 8-bit counters): 390807 [0x55d0d6221e80, 0x55d0d6281517),
INFO: Loaded 1 PC tables (390807 PCs): 390807 [0x55d0d6281518,0x55d0d6877e88),
./src/test/fuzz/fuzz: Running 1 inputs 1 time(s) each.
Running: ./timeout-ada0fecaba2b8c46c6e970cf637d9625b01bf7e5
ALARM: working on the last Unit for 19 seconds
and the timeout value is 18 (use -timeout=N to change)
==375014== ERROR: libFuzzer: timeout after 19 seconds
```
ACKs for top commit:
naumenkogs:
ACK 9f265d8825
dergoegge:
ACK 9f265d8825
brunoerg:
ACK 9f265d8825
Tree-SHA512: da83ff90962bb679aae00e8e9dba639c180b7aaba544e0c4d0978d36e28a9ff1cd7a2e13009d8ab407ef57767656aca1ebc767a7d2f1bc880284f8f57c197a50
15f5a0d0c8 fuzz: Improve fuzzing stability for txorphan harness (dergoegge)
Pull request description:
The `txorphan` harness has low stability as eviction of orphan txs is entirely random at the moment.
Fix this by passing the rng to `LimitOrphans`, which can be deterministic in tests.
Also see #29018.
ACKs for top commit:
maflcko:
lgtm ACK 15f5a0d0c8
brunoerg:
utACK 15f5a0d0c8
Tree-SHA512: 854ec34b3a0f16f26db6dc419096c6e7a380e8400119534aa278d6b1d54c253b572aa2fad13c383c796c431d8ff4263956e6f60326e99f8bf6abd16d9a280e97
1a5dae630d msvc: Define the same `QT_...` macros as in Autotools builds (Hennadii Stepanov)
Pull request description:
There are no reasons to have such a diversion.
Also it fixes https://github.com/bitcoin/bitcoin/pull/28960#issuecomment-1847971114.
ACKs for top commit:
sipsorcery:
tACK 1a5dae630d.
TheCharlatan:
ACK 1a5dae630d
Tree-SHA512: 75be5eabb8fec974b8d77a023c72323015a3d95fbc13b7fd85e5f25c250ae67850ddf0bcaef143828d75fe35a49e7c9b1966976b74f3ce7d14465174e6585ceb
ea00f982d2 test: fix intermittent error in rpc_net.py (#29030) (Sebastian Falbesoner)
Pull request description:
Asserting for the debug log message "Added connection peer=" is insufficient for ensuring that this new connection will show up in a following getpeerinfo() call, as the debug message is written in the CNode ctor, which means it hasn't necessarily been added to CConnman.m_nodes at this point.
Solve this by using the recently introduced `wait_for_new_peer` helper (see #29006, commit 00e0658e77), which is more robust.
Fixes#29030.
ACKs for top commit:
maflcko:
lgtm ACK ea00f982d2
Tree-SHA512: dda307949a466fb3b24408a8c213d307e0af2155f2e8b4e52c836a22397f9d218bf9d8c54ca55bae62a96d7566f27167db9311dd8801785c327234783af5ed00
fd0bde2793 test: fix `addnode` functional test failure on OpenBSD (Sebastian Falbesoner)
Pull request description:
This is the functional test counterpart of PR #28891 / commit 007d6f0e85 (unfortunately, I missed it back then and only ran the unit tests -- sorry for the noise).
master branch on OpenBSD 7.4:
```
$ ./test/functional/rpc_net.py
2023-12-08T17:29:05.057000Z TestFramework (INFO): PRNG seed is: 6024296850131317403
2023-12-08T17:29:05.058000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_au3zchif
2023-12-08T17:29:05.618000Z TestFramework (INFO): Test getconnectioncount
2023-12-08T17:29:05.618000Z TestFramework (INFO): Test getpeerinfo
2023-12-08T17:29:06.643000Z TestFramework (INFO): Check getpeerinfo output before a version message was sent
2023-12-08T17:29:06.709000Z TestFramework (INFO): Test getnettotals
2023-12-08T17:29:06.773000Z TestFramework (INFO): Test getnetworkinfo
2023-12-08T17:29:06.978000Z TestFramework (INFO): Test addnode and getaddednodeinfo
2023-12-08T17:29:06.980000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/home/thestack/bitcoin/test/functional/test_framework/test_framework.py", line 131, in main
self.run_test()
File "/home/thestack/bitcoin/./test/functional/rpc_net.py", line 65, in run_test
self.test_addnode_getaddednodeinfo()
File "/home/thestack/bitcoin/./test/functional/rpc_net.py", line 224, in test_addnode_getaddednodeinfo
assert_raises_rpc_error(-23, "Node already added", self.nodes[0].addnode, node=ip_port2, command='add')
File "/home/thestack/bitcoin/test/functional/test_framework/util.py", line 131, in assert_raises_rpc_error
assert try_rpc(code, message, fun, *args, **kwds), "No exception raised"
AssertionError: No exception raised
```
On the PR branch, the same call succeeds.
ACKs for top commit:
kevkevinpal:
ACK [fd0bde2](fd0bde2793)
Tree-SHA512: ae20816fa4025c212e115ebd267b5e5784bfcdf0051219eb686faaade47ec4f91a3947af6d24258b159290000d2dcc3f6e65e788b83b8a9297282945dbdafbfb
6e0f1d2abb msvc: Optimize "Release" builds (Hennadii Stepanov)
Pull request description:
It is awkward not using optimization.
In addition to the obvious benefits for Windows users, this PR reduces the duration of functional tests by an hour.
Picked from https://github.com/bitcoin/bitcoin/pull/24773.
ACKs for top commit:
sipsorcery:
tACK 6e0f1d2abb.
Tree-SHA512: 5aa7fd38cb1a81d58ea3206756a8099891866c82a747d3b8079cab0b2afa1f40ba53adff2f32eb233efcd1227babee80ab175e35a83678fafa8a4f63c356e5ca
d08e820abf Add a note to msvc readme re building Qt for Bitcoin Core. (Aaron Clauson)
Pull request description:
Updated the msvc readme with a note about avoiding path too long errors when building Qt with Bitcoin Core.
Would have saved me half an hour if I'd remembered this from the last time I did the build.
ACKs for top commit:
hebasto:
ACK d08e820abf.
TheCharlatan:
ACK d08e820abf
Tree-SHA512: f51017b15383dbcd39ad1e5e978bb255b9205dc23d72b5e3530c6aefcbbc2dc4ec3a85e5fc8c0019c8511173c298f80b837cb35f268deac424d19365b25fb335
There exist many usages of `fuzzed_data_provider` where it is evaluated directly in the function call.
Unfortunately, the order of evaluation of function arguments is unspecified. This means it can differ
between compilers/version/optimization levels etc. But when the evaluation order changes, the same
fuzzing input will produce different output, which is bad for coverage/reproducibility.
This PR fixes all these cases where by moving multiple calls to `fuzzed_data_provider` out of the
function arguments.
Asserting for the debug log message "Added connection peer=" is
insufficient for ensuring that this new connection will show up in a
following getpeerinfo() call, as the debug message is written in the
CNode ctor, which means it hasn't necessarily been added to
CConnman.m_nodes at this point.
Solve this by using the recently introduced `wait_for_new_peer`
helper, which is more robust.
Fixes#29030.
Instead of using the output parameters, return CreatedTransactionResult
from FundTransaction in the same way that CreateTransaction does.
Additionally, instead of modifying the original CMutableTransaction, the
result from CreateTransactionInternal is used.
When creating a transaction with preset inputs, also preserve the
scriptSig and scriptWitness for those preset inputs if they are provided
(e.g. in fundrawtransaction).
Instead of having a separate CCoinControl::SelectExternal function, we
can use the normal CCoinControl::Select function and explicitly use
PreselectedInput::SetTxOut in the caller. The semantics of what an
external input is remains.
Instead of having different maps for selected inputs, external inputs,
and input weight in CCoinControl, have a class PreselectedInput which
tracks stores that information for each input.
Rather than re-implementing these checks, we can use this test
framework's helper (introduced in commit
c934087b62, PR #24358) called in a test's
`skip_test_if_missing_module` method instead.
fa6e50d6c7 fuzz: Use C++20 starts_with in rpc.cpp (MarcoFalke)
faa48388bc Revert "tracepoints: Disables `-Wgnu-zero-variadic-macro-arguments` to compile without warnings" (MarcoFalke)
fae3b77a87 refactor: Drop unused _Pragma to ignore -Wgnu-zero-variadic-macro-arguments (MarcoFalke)
fa02fc0a86 refactor: modernize-use-default-member-init for bit-fields (C++20) (MarcoFalke)
fa67f096bd build: Require C++20 compiler (MarcoFalke)
Pull request description:
C++20 allows to write safer code, because it allows to enforce more stuff at compile time (`constinit`, `conteval`, `constexpr`, `std::span`, ...).
Also, it allows to write less verbose and easier to understand code (C++ 20 Concepts).
See https://github.com/bitcoin/bitcoin/issues/23363 and https://en.cppreference.com/w/cpp/compiler_support#cpp20
With g++-10 (https://github.com/bitcoin/bitcoin/pull/28348) and clang-13 (https://github.com/bitcoin/bitcoin/pull/28210), there is broad support for almost all features of C++20.
It should be fine to require a C++20 compiler for Bitcoin Core 27.0 in 2024 (next year), not the soon upcoming 26.0 next month.
This pull request includes three small cleanups to make use of C++20 features. If any issues are detected before or after merge, this should be easy to revert. If no issues arise, it should be fine to make use of more involved C++20 features later on.
ACKs for top commit:
fanquake:
ACK fa6e50d6c7
Tree-SHA512: 244d79bfb0b750a4bdd713f40573b9ca33816fb84b6c84a58f027b9d7d4bb0cc4f18642959e4cf3d094808a69e5b8a327ca8521d7c0c08af27dacb5da3e78e71
00e0658e77 test: fix v2 transport intermittent test failure (#29002) (Sebastian Falbesoner)
Pull request description:
This PR improves the following fragile construct for detection of a new connection to the node under test in `p2p_v2_transport.py`:
6d5790956f/test/functional/p2p_v2_transport.py (L154-L156)
Only relying on the number of peers for that suffers from race conditions, as unrelated previous peers could disconnect at anytime in-between. In the test run in #29002, the following happens:
- `getpeerinfo()` is called the first time -> assigned to `num_peers`
- **previous peer disconnects**, the node's peer count is now `num_peers - 1` (in most test runs, this happens before the first getpeerinfo call)
- new peer connects, the node's peer count is now `num_peers`
- the condition that the node's peer count is `num_peers + 1` is never true, test fails
Use the more robust approach of watching for an increased highest peer id instead (again using the `getpeerinfo` RPC call), with a newly introduced context manager method `TestNode.wait_for_new_peer()`. Note that for the opposite case of a disconnect, no new method is introduced; this is currently used only once in the test and is also simpler.
Still happy to take suggestions for alternative solutions.
Fixes#29002.
ACKs for top commit:
kevkevinpal:
Concept ACK [00e0658](00e0658e77)
maflcko:
Ok, lgtm ACK 00e0658e77
stratospher:
ACK 00e0658.
Tree-SHA512: 0118b87f54ea5e6e080ff44f29d6af6674c757a588534b3add040da435f4359e71bf85bc0a5eb7170f99cc9956e1a03c35cce653d642d31eed41bbed1f94f44f
97c0dfa894 test: Extends MEMPOOL msg functional test (Sergi Delgado Segura)
Pull request description:
Currently, p2p_filter.py::test_msg_mempool is not testing much. This extends the tests so the interaction between sending `MEMPOOL` messages with a filter that does not include all transactions in the mempool reacts, plus how it interacts with `INV` messages, especially after the changes introduced by #27675
ACKs for top commit:
instagibbs:
ACK 97c0dfa894
theStack:
re-ACK 97c0dfa894
Tree-SHA512: 746fdc867630f40509e6341f484a238dd855ae6d1be5eca121974491e4ca272dee88af4b90dda55ea9a5a19cbff198fa91ffa0c5bf1ddf0232b2c1b215b05b9a
f053024273 wallet: batch external signer descriptor import (Sjors Provoost)
1f65241b73 wallet: descriptors setup, batch db operations (furszy)
3eb769f150 wallet: batch legacy spkm TopUp (furszy)
075aa44ceb wallet: batch descriptor spkm TopUp (furszy)
bb4554c81e bench: add benchmark for wallet creation procedure (furszy)
Pull request description:
Work decoupled from #28574.
Instead of performing multiple single write operations per spkm
setup call, this PR batches them all within a single atomic db txn.
Speeding up the process and preventing the wallet from entering
an inconsistent state if any of the intermediate transactions fail
(which shouldn't happen but.. if it does, it is better to not store
any spkm rather than storing them partially).
To compare the changes, added benchmark in the first commit.
ACKs for top commit:
Sjors:
re-utACK f053024273
achow101:
ACK f053024273
BrandonOdiwuor:
ACK f053024273
theStack:
Code-review ACK f053024273
Tree-SHA512: aead8548473e17d4d53e8e7039bbaf5e8bf2fe83f33b33f81cdedefe8a31b7003ceb6d5379b1bad1ca2692e909492009a21284ec8338eede078df3d19046ab5a
fa88953d6f doc: Add link to needs-release-notes label (MarcoFalke)
Pull request description:
This makes it easier to spot and not forget. C.f. https://github.com/bitcoin/bitcoin/pull/28597#issuecomment-1845299642
ACKs for top commit:
kristapsk:
ACK fa88953d6f
TheCharlatan:
ACK fa88953d6f
Tree-SHA512: 28336cde36d62622d1b6627497291cbd4644bf1e4e6f17dc9cde39f254e7094dd02484da754e45558e59facb20941dd0c049ce7b33dcc62bfec6c26c16516cdf
ca5937553b doc: Missing additions to 26.0 release notes (fanquake)
7d4e47d184 doc: add historical release notes for 26.0 (fanquake)
8df4aaabbe doc: add minimum required Linux Kernel to release-notes (fanquake)
Pull request description:
Bins are now up, used for GH release etc.
ACKs for top commit:
willcl-ark:
ACK ca5937553b
achow101:
ACK ca5937553b
Tree-SHA512: 1fefd857092412231215dc72b5d79b2a7828a8c74aa6cb19a7dbc3c3b77feace3ce7fa89d517b4ce25ea41ed84e7ca4ba840d0923b97bf8f6b40b27ad96affa9
fa63f16018 test: Add uint256 string parse tests (MarcoFalke)
facf629ce8 refactor: Remove unused and fragile string interface from arith_uint256 (MarcoFalke)
Pull request description:
The string interface (`base_uint(const std::string&)`, as well as `base_uint::SetHex`) is problematic for many reasons:
* It is unused (except in test-only code).
* It is redundant with the `uint256` string interface: `std::string -> uint256 -> UintToArith256`.
* It is brittle, because it inherits the brittle `uint256` string interface, which is brittle due to the use of `c_str()` (embedded null will be treated as end-of string), etc ...
Instead of fixing the interface, remove it since it is unused and redundant with `UintToArith256`.
ACKs for top commit:
ajtowns:
ACK fa63f16018
TheCharlatan:
ACK fa63f16018
Tree-SHA512: a95d5b938ffd0473361336bbf6be093d01265a626c50be1345ce2c5e582c0f3f73eb11af5fd1884019f59d7ba27e670ecffdb41d2c624ffb9aa63bd52b780e62
The Transaction View should be only enabled when a wallet is selected.
Therefore it has been added a condition for a selected wallet on
enableHistoryAction() since its availability also depends on the mask
value checkbox.
All functions assume that the pointer is never null, so pass by
reference, to avoid accidental segfaults at runtime, or at least make
them more obvious.
Also, remove unused c-style casts in touched lines.
Also, add CHECK_NONFATAL checks, to turn segfault crashes into an
recoverable runtime error with debug information.
8ea45e626e build: use macOS 14 SDK (Xcode 15.0) (fanquake)
51c97ffb69 build: patch boost process for macOS 14 SDK (fanquake)
423949a13b depends: add -platform_version to macOS build flags (fanquake)
Pull request description:
This fixes: https://github.com/bitcoin/bitcoin/pull/28349#issuecomment-1748515277 (cross-compiling with C++20 for macOS). See https://developer.apple.com/xcode/cpp/#c++20 for C++20 support in Apples libc++, some features landed with Xcode 14.3, although many more landed with Xcode 15.0.
ACKs for top commit:
hebasto:
ACK 8ea45e626e.
TheCharlatan:
ACK 8ea45e626e
Tree-SHA512: 274ce2c9b9f8e4d755c07b8d0d4897a7f92708ac64e6afb7a3f75bdb485e863fc7f40badf3a88b129ce36f6cca72f768dc2ed7fba2bdf0bb6da2bf0c8fedee10
Currently, p2p_filter.py::test_msg_mempool is not testing much.
This extends the tests so the interaction between sending MEMPOOL messages with
a filter that does not include all transactions in the mempool reacts, plus how
it interacts with INV messages
fad1903b8a fuzz: Avoid timeout in bitdeque (MarcoFalke)
Pull request description:
Avoid timeouts such as https://github.com/bitcoin/bitcoin/issues/28812#issuecomment-1842914664
This is done by:
* Limiting the maximum number of iterations if the maximum size of the container is "large" (see the magic numbers in the code).
* Check the equality only once. This should be fine, because if a crash were to happen in the equality check, but the crash doesn't happen if further iterations were run, the fuzz engine should eventually find the crash by truncating the fuzz input.
ACKs for top commit:
sipa:
utACK fad1903b8a
dergoegge:
utACK fad1903b8a
brunoerg:
crACK fad1903b8a
Tree-SHA512: d3d83acb3e736b8fcaf5d17ce225ac82a9f9a2efea048512d2fed594ba6c76c25bae72eb0fab3276d4db37baec0752e5367cecfb18161301b921fed09693045e
3ea54e5db7 net: Add continuous ASMap health check logging (Fabian Jahr)
28d7e55dff test: Add tests for unfiltered GetAddr usage (Fabian Jahr)
b8843d37ae fuzz: Let fuzzers use filter options in GetAddr/GetAddresses (Fabian Jahr)
e16f420547 net: Optionally include terrible addresses in GetAddr results (Fabian Jahr)
Pull request description:
There are certain statistics we can collect by running all our known clearnet addresses against the ASMap file. This could show issues with a maliciously manipulated file or with an old file that has decayed with time.
This is just a proof of concept for now. My idea currently is to run the analysis once per day and print the results to logs if an ASMap file is used.
ACKs for top commit:
achow101:
ACK 3ea54e5db7
mzumsande:
ACK 3ea54e5db7
brunoerg:
crACK 3ea54e5db7
Tree-SHA512: 777acbfac43cc43ce4a0a3612434e4ddbc65f59ae8ffc9e24f21de09011bccb297f0599cbaa82bcf40ef68e5af582c4e98556379db7ceff7d9f97574a1cf8e09
ca09415e63 rpc, doc: encryptwallet, mention HD seed rotation and new backup (furszy)
Pull request description:
Small and simple PR, updating the `encryptwallet` help message.
Better to notify users about the HD seed rotation and the new
backup requirement before executing the encryption process.
Ensuring they are prepared to update previous backups and
securely safeguard the updated wallet file.
ACKs for top commit:
S3RK:
ACK ca09415e63
achow101:
ACK ca09415e63
Tree-SHA512: f0ee65f5cea66450566e3a85e066d4c06b3293dd0e0b2ed5fafdb7fb11da0a2cd94407299a3c57a0706c2ed782f8eabb73443e85d8099a62a3fb10a02636ab46
55e3dc3e03 test: Fix test by checking the actual exception instance (Hennadii Stepanov)
Pull request description:
The `system_tests/run_command` test is broken because it passes even with the diff as follows:
```diff
--- a/src/test/system_tests.cpp
+++ b/src/test/system_tests.cpp
@@ -90,7 +90,7 @@ BOOST_AUTO_TEST_CASE(run_command)
});
}
{
- BOOST_REQUIRE_THROW(RunCommandParseJSON("echo \"{\""), std::runtime_error); // Unable to parse JSON
+ BOOST_REQUIRE_THROW(RunCommandParseJSON("invalid_command \"{\""), std::runtime_error); // Unable to parse JSON
}
// Test std::in, except for Windows
#ifndef WIN32
```
The reason of such fragility is that the [`BOOST_REQUIRE_THROW`](https://www.boost.org/doc/libs/1_83_0/libs/test/doc/html/boost_test/utf_reference/testing_tool_ref/assertion_boost_level_throw.html) macro passes even if the command raises an exception in the underlying subprocess implementation, which might have a type derived from `std::runtime_error`.
ACKs for top commit:
maflcko:
lgtm ACK 55e3dc3e03
achow101:
ACK 55e3dc3e03
furszy:
Non-Windows code ACK 55e3dc3e
pablomartin4btc:
ACK 55e3dc3e03
Tree-SHA512: 32f49421bdcc94744c81e82dc10cfa02e3f8ed111974edf1c2a47bdaeb56d7baec1bede67301cc89464fba613029ecb131dedc6bc5948777ab52f0f12df8bfe9
9075a44646 test: add regression test for the getrawtransaction segfault (Martin Zumsande)
494a926d05 rpc: fix getrawtransaction segfault (Martin Zumsande)
Pull request description:
The crash, reported in #28986, happens when calling `getrawtransaction` for any mempool transaction with `verbosity=2`, while pruning, because the rpc calls `IsBlockPruned(const CBlockIndex* pblockindex)`, which dereferences `pblockindex` without a check.
For ease of backporting this PR fixes it just locally in `rpc/rawtransaction.cpp` by moving the check for`!blockindex` up so that `IsBlockPruned()` will not be called with a `nullptr`. We might also want to change `IsBlockPruned()` so it doesn't crash when called with a `nullptr`, but I didn't do that here.
Fixes#28986
ACKs for top commit:
maflcko:
lgtm test-was-added ACK 9075a44646
theStack:
Tested ACK 9075a44646
Tree-SHA512: 0f7ed52579487196c206e16b45582b64e4b02ecf2a2eb0a31d2f3b52415bc9c64278cb94259314ef14ab7fb393c6195f79b3027d6de471d67614e51474498b11
fad2392c58 ci: Use Ubuntu 24.04 Noble for asan (MarcoFalke)
fa83b65ef8 ci: Use Ubuntu 24.04 Noble for tsan,tidy,fuzz (MarcoFalke)
Pull request description:
23.10 will be EOL mid next year, so a bump is needed before then for the `master` branch (and possibly the `26.x` branch).
Doing the bump now is fine, because the clang version is pinned to 17 inside the CI tasks. So a default clang version change in the system image should not affect the tasks. Once clang-18 is available and the default in April next year (https://discourse.ubuntu.com/t/noble-numbat-release-schedule/35649#planned-and-potentially-disruptive-archive-wide-activities-2), the pinned version could be bumped (for CI tasks that require a pin, like tidy), or the pin can be removed (for CI tasks that usually do not require a pin, like fuzz or the sanitizers).
ACKs for top commit:
fanquake:
ACK fad2392c58
Tree-SHA512: c40aede4e2281a5d539d5f65d2c08a57bf92e4a00b4f45a4260b57b7443a63d1a0603115da4a3bbd100ac5f6ade3f2eda0916e4b565573741162a76294ec0ac5
Only relying on the number of peers for detecting a new connection
suffers from race conditions, as unrelated previous peers could
disconnect at anytime in-between. Use the more robust approach of
watching for an increased highest peer id instead (again using the
`getpeerinfo` RPC call), with a newly introduced context manager
method `TestNode.wait_for_new_peer()`.
Fixes#29009.
Verifying the wallet updates the birth time accordingly when it
detects a transaction with a time older than the oldest descriptor
timestamp.
This could happen when the user blindly imports a descriptor with
'timestamp=now'.
To avoid scanning blocks, as assumed by a wallet with no
generated keys or imported scripts, the default value for
the birth time needs to be set to the maximum int64_t value.
Once the first key is generated or the first script is imported,
the legacy SPKM will update the birth time automatically.
Better to notify users about the HD seed rotation and the new
backup requirement before executing the encryption process.
Ensuring they are prepared to update previous backups and
securely safeguard the updated wallet file.
Co-authored-by: jonatack <jon@atack.com>
11d797e3a0 depends: Build `native_capnp` package with CMake (Hennadii Stepanov)
90389c95e9 depends: Build `capnp` package with CMake (Hennadii Stepanov)
Pull request description:
The first commit fixes two bugs when cross-compiling the `capnp` package on the master branch @ 160d23677a:
- for `x86_64-w64-mingw32` (see https://github.com/bitcoin/bitcoin/pull/28735#issuecomment-1790406668):
```
libtool: link: x86_64-w64-mingw32-g++-posix -shared -nostdlib /usr/lib/gcc/x86_64-w64-mingw32/12-posix/../../../../x86_64-w64-mingw32/lib/dllcrt2.o /usr/lib/gcc/x86_64-w64-mingw32/12-posix/crtbegin.o src/kj/.libs/cidr.o src/kj/.libs/common.o src/kj/.libs/units.o src/kj/.libs/memory.o src/kj/.libs/refcount.o src/kj/.libs/array.o src/kj/.libs/list.o src/kj/.libs/string.o src/kj/.libs/string-tree.o src/kj/.libs/source-location.o src/kj/.libs/hash.o src/kj/.libs/table.o src/kj/.libs/encoding.o src/kj/.libs/exception.o src/kj/.libs/debug.o src/kj/.libs/arena.o src/kj/.libs/io.o src/kj/.libs/mutex.o src/kj/.libs/thread.o src/kj/.libs/time.o src/kj/.libs/filesystem.o src/kj/.libs/filesystem-disk-unix.o src/kj/.libs/filesystem-disk-win32.o src/kj/.libs/test-helpers.o src/kj/.libs/main.o src/kj/parse/.libs/char.o -L/home/hebasto/git/bitcoin/depends/x86_64-w64-mingw32/lib -L/usr/lib/gcc/x86_64-w64-mingw32/12-posix -L/usr/lib/gcc/x86_64-w64-mingw32/12-posix/../../../../x86_64-w64-mingw32/lib -lstdc++ -lmingw32 -lgcc_s -lgcc -lmoldname -lmingwex -lmsvcrt -lkernel32 -lpthread -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 -lgcc_s -lgcc -lmoldname -lmingwex -lmsvcrt -lkernel32 /usr/lib/gcc/x86_64-w64-mingw32/12-posix/crtend.o -mthreads -O2 -mthreads -mthreads -o .libs/libkj-1-0-1.dll -Wl,--enable-auto-image-base -Xlinker --out-implib -Xlinker .libs/libkj.dll.a
/usr/bin/x86_64-w64-mingw32-ld: src/kj/.libs/cidr.o:cidr.c++:(.text+0x1dc): undefined reference to `__imp_inet_ntop'
/usr/bin/x86_64-w64-mingw32-ld: src/kj/.libs/cidr.o:cidr.c++:(.text+0x44b): undefined reference to `__imp_inet_pton'
collect2: error: ld returned 1 exit status
```
- for `arm64-apple-darwin`:
```
checking build system type... x86_64-pc-linux-gnu
checking host system type... Invalid configuration `arm64-apple-darwin': machine `arm64-apple' not recognized
configure: error: /bin/bash build-aux/config.sub arm64-apple-darwin failed
```
The second commit applies the same changes for the `native_capnp` package for [consistency](https://github.com/bitcoin/bitcoin/pull/28856#issuecomment-1807936546).
ACKs for top commit:
ryanofsky:
Code review ACK 11d797e3a0. Since last review arm64-apple-darwin platform is now mentioned in the commit message, and the change to `depends/packages/libmultiprocess.mk` in d1604d4b1d1ee8df279a1776303e167cc3d06193 which was unrelated (but probably still a good optimization) was reverted.
Tree-SHA512: c636e53073ce6fcda9724723bc59f3990fa9629a3b2f73d93dbc102a5a1badfbe8f4c5fef841f03588ebcad5cd4883f3ce32b128afcd75f6bc21eb801796a586
38816ff64e fuzz: txorphan check wtxids using GenTxid::Wtxid not GenTxid::Txid (Greg Sanders)
Pull request description:
Fixes the bugs in the fuzz test with no more changes as an alternative to https://github.com/bitcoin/bitcoin/pull/28658
ACKs for top commit:
naumenkogs:
ACK 38816ff64e
dergoegge:
ACK 38816ff64e
Tree-SHA512: 5e46a83f2b2a2ac0672a63eb6200b019e01089ab1aa80c4ab869b6fcf27ccf2e84a064e96397f1a1869ccfa43b0c9638cbae681a27c4ca3c96ac71f41262601e
```bash
-platform_version platform min_version sdk_version
This is set to indicate the platform, oldest supported version
of that platform that output is to be used on, and the SDK that
the output was built against.
```
11b7269d83 script: Enhance validations in utxo_snapshot.sh (pablomartin4btc)
Pull request description:
This PR resolves#27841 and some more:
- Ensure that the snapshot height is higher than the pruned block height when the node is pruned (Suggested by @Sjors [here](https://github.com/bitcoin/bitcoin/pull/28553#issuecomment-1804941396)).
- Validate the correctness of the file path and check if the file already exists (@hazeycode's [#27845](https://github.com/bitcoin/bitcoin/pull/27845)).
- Make network activity disablement optional for the user (Suggested by @Sjors [here](https://github.com/bitcoin/bitcoin/pull/16899#discussion_r342735815) and [here](https://github.com/bitcoin/bitcoin/pull/16899#issuecomment-536520911)).
- Ensure the `reconsiderblock` command is triggered on exit (@hazeycode's same PR as above), even in the case of user interruption (Ctrl-C).
In order to perform some testing please follow the instructions in the description of previous @hazeycode's PR #27845.
ACKs for top commit:
Sjors:
tACK 11b7269d83
ryanofsky:
Code review ACK 11b7269d83
Tree-SHA512: 2b699894c6f732ad5104f5a2bcf5dc86ed31edcc9d664690cab55b94a8ab00e2ca5bde901ee1d63acddca7ea80ad1734d8cfe78f9c02f8470f264fe93a2af759
This change is mostly a refectoring that removes some code and gets rid of an
unnecessary layer of indirection after #27861
But it is not a pure refactoring since StartShutdown, AbortShutdown, and
WaitForShutdown functions used to abort on failure, and the replacement code
logs or returns errors instead.
8f6ab31863 init: don't delete PID file if it was not generated (willcl-ark)
Pull request description:
In a similar vein to #28784, if a second `bitcoind` is started using the same datadir it will fail to start up, but during shutdown remove the PID file from the first `bitcoind` instance.
ACKs for top commit:
achow101:
ACK 8f6ab31863
andrewtoth:
ACK 8f6ab31863
romanz:
ACK 8f6ab31863
Tree-SHA512: c9af703cbfa179d33ef9580a51e86c1b0acbd28daa18c8d2e5e5ff796ab4d3e2009a962a47e6046a0e5ece936f8a06ee8af5fdf8ff4ae1e52cbcdbec4b942271
Having InitContext() avoids the need to add duplicate code to src/init/*.cpp
files in the next commit. It also lets these files avoid referencing global
variables like gArgs.
There is no change in behavior in this commit.
Use SignalInterrupt object instead. There is a slight change in behavior here
because the previous StartShutdown code used to abort on failure and the
new code logs errors instead.
Use SignalInterrupt object instead. There is a slight change in behavior here
because the previous StartShutdown code used to abort on failure and the
new code returns an RPC error instead.
Replace exceptions thrown by signal and wait methods with [[nodiscard]] return
values.
This is mostly a refactoring, but there is a slight change of behavior if
AbortShutdown function fails. The original behavior which was unintentionally
changed in #27861 is restored, so it now triggers an assert failure again
instead of throwing an exception. (The AbortShutdown function is only ever
called in the the GUI version of Bitcoin Core when corruption is detected on
loading and the user tries to reindex.)
Problems with using exceptions were pointed out by MarcoFalke in
https://github.com/bitcoin/bitcoin/pull/27861#discussion_r1255496707.
Pass HTTP server an interrupt object instead of having it depend on shutdown.h
and global shutdown state.
There is no change in behavior in this commit.
Add NodeContext::shutdown variable and start using it to replace the
kernel::Context::interrupt variable. The latter can't easily be removed right
away but will be removed later in this PR.
Moving the interrupt object from the kernel context to the node context
increases flexibility of the kernel API so it is possible to use multiple
interrupt objects, or avoid creating one if one is not needed. It will also
allow getting rid of the kernel::g_context global later in this PR, replacing
it with a private SignalInterrupt instance in init.cpp
There is no change in behavior in this commit outside of unit tests. In unit
tests there should be no visible change either, but internally now each test
has its own interrupt variable so the variable will be automatically reset
between tests.
Use interfaces::Node object instead.
There is a minor change in behavior in this commit, because the new code calls
InterruptRPC() and StopRPC() when previous code did not do this. But this
should be a good thing since it makes sense to interrupt RPC when the system is
shutting down, and it is better for the GUI shut down in a consistent way
regardless of how the shutdown is triggered.
Previously, starting a second bitcoind using the same datadir would
correctly fail to init and shutdown. However during shutdown the PID
file belonging to the first instance would be erroneously removed by
the second process shutting down.
Fix this to only delete the PID file if we created it.
The BOOST_REQUIRE_THROW passes even if the command raises an exception
in the underlying subprocess implementation, which might have a type
derived from std::runtime_error.
05aca09381 build: Patch Qt to handle minimum macOS version properly (Hennadii Stepanov)
Pull request description:
This PR is:
- required to [switch](https://github.com/bitcoin/bitcoin/pull/28622) to macOS 14 SDK (Xcode 15).
- an alternative to https://github.com/bitcoin/bitcoin/pull/28732 and https://github.com/bitcoin/bitcoin/pull/28775.
Qt relies on the `__MAC_OS_X_VERSION_MIN_REQUIRED` macro, which is set in the `AvailabilityInternal.h` SDK header to
the value provided by the Clang driver from the `-mmacos-version-min` / `-mmacosx-version-min` option.
Xcode 12 SDK expects the OS-specific `__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__` macro:
```c++
#ifndef __MAC_OS_X_VERSION_MIN_REQUIRED
#ifdef __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__
/* compiler for Mac OS X sets __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ */
#define __MAC_OS_X_VERSION_MIN_REQUIRED __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__
#endif
#endif /* __MAC_OS_X_VERSION_MIN_REQUIRED*/
```
In the other hand, Xcode 15 SDK expects a general `__ENVIRONMENT_OS_VERSION_MIN_REQUIRED__` macro:
```c++
#ifndef __MAC_OS_X_VERSION_MIN_REQUIRED
#if defined(__has_builtin) && __has_builtin(__is_target_os)
#if __is_target_os(macos)
#define __MAC_OS_X_VERSION_MIN_REQUIRED __ENVIRONMENT_OS_VERSION_MIN_REQUIRED__
#define __MAC_OS_X_VERSION_MAX_ALLOWED __MAC_14_0
#endif
#elif __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__
#define __MAC_OS_X_VERSION_MIN_REQUIRED __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__
#define __MAC_OS_X_VERSION_MAX_ALLOWED __MAC_14_0
#endif /* __has_builtin(__is_target_os) && __is_target_os(macos) */
#endif /* __MAC_OS_X_VERSION_MIN_REQUIRED */
```
The latter macro is not provided by LLVM Clang until c8e2dd8c6f, which is available in Clang 17.
The suggested patch makes Qt "borrow" the `__MAC_OS_X_VERSION_MIN_REQUIRED` value from `MAC_OS_X_VERSION_MIN_REQUIRED`, which is set in the `AvailabilityMacros.h` SDK header.
ACKs for top commit:
maflcko:
lgtm ACK 05aca09381
Tree-SHA512: 8891aefde4b8a48885abf0648f4ec71a22f7fcfca1e17ebb8c70ce1ef44751ea5db6b8b652de6ee8a716ca5f96f720fef01600bc23986162d0146c946e2e8743
97d53e90ba ci: remove libz-dev from macOS build deps (fanquake)
Pull request description:
Remove no-longer used libz-dev (from DMG compression).
Followup to #28432.
Missed this in #28932.
ACKs for top commit:
TheCharlatan:
ACK 97d53e90ba
Tree-SHA512: ea4e46d83a11cfb0d77e7c5b370359864a026ca58e02de1a3d30458e4463dabd338db9546da63b782a7fa0ce199446c418fbfdfc400f3803b0ea9dd626aed42f
91504cbe0d rpc: `SyncWithValidationInterfaceQueue` on fee estimation RPC's (ismaelsadeeq)
714523918b tx fees, policy: CBlockPolicyEstimator update from `CValidationInterface` notifications (ismaelsadeeq)
dff5ad3b99 CValidationInterface: modify the parameter of `TransactionAddedToMempool` (ismaelsadeeq)
91532bd382 tx fees, policy: update `CBlockPolicyEstimator::processBlock` parameter (ismaelsadeeq)
bfcd401368 CValidationInterface, mempool: add new callback to `CValidationInterface` (ismaelsadeeq)
0889e07987 tx fees, policy: cast with static_cast instead of C-Style cast (ismaelsadeeq)
a0e3eb7549 tx fees, policy: bugfix: move `removeTx` into reason != `BLOCK` condition (ismaelsadeeq)
Pull request description:
This is an attempt to #11775
This Pr will enable fee estimator to listen to ValidationInterface notifications to process new transactions added and removed from the mempool.
This PR includes the following changes:
- Added a new callback to the Validation Interface `MempoolTransactionsRemovedForConnectedBlock`, which notifies listeners about the transactions that have been removed due to a new block being connected, along with the height at which the transactions were removed.
- Modified the `TransactionAddedToMempool` callback parameter to include additional information about the transaction needed for fee estimation.
- Updated `CBlockPolicyEstimator` to process transactions using` CTransactionRef` instead of `CTxMempoolEntry.`
- Implemented the `CValidationInterface` interface in `CBlockPolicyEstimater` and overridden the `TransactionAddedToMempool`, `TransactionRemovedFromMempool`, and `MempoolTransactionsRemovedForConnectedBlock` methods to receive updates from their notifications.
Prior to this PR, the fee estimator updates from the mempool, i.e whenever a new block is connected all transactions in the block that are in our mempool are going to be removed using the `removeForBlock` function in `txmempool.cpp`.
This removal triggered updates to the fee estimator. As a result, the fee estimator would block mempool's `cs` until it finished updating every time a new block was connected.
Instead of being blocked only on mempool tx removal, we were blocking on both tx removal and fee estimator updating.
If we want to further improve fee estimation, or add heavy-calulation steps to it, it is currently not viable as we would be slowing down block relay in the process
This PR is smaller in terms of the changes made compared to #11775, as it focuses solely on enabling fee estimator updates from the validationInterface/cscheduler thread notifications.
I have not split the validation interface because, as I understand it, the rationale behind the split in #11775 was to have `MempoolInterface` signals come from the mempool and `CValidationInterface` events come from validation. I believe this separation can be achieved in a separate refactoring PR when the need arises.
Also left out some commits from #11775
- Some refactoring which are no longer needed.
- Handle reorgs much better in fee estimator.
- Track witness hash malleation in fee estimator
I believe they are a separate change that can come in a follow-up after this.
ACKs for top commit:
achow101:
ACK 91504cbe0d
TheCharlatan:
Re-ACK 91504cbe0d
willcl-ark:
ACK 91504cbe0d
Tree-SHA512: 846dfb9da57a8a42458827b8975722d153907fe6302ad65748d74f311e1925557ad951c3d95fe71fb90ddcc8a3710c45abb343ab86b88780871cb9c38c72c7b1
7cb9367157 rpc: keep .cookie if it was not generated (Roman Zeyde)
Pull request description:
Otherwise, starting bitcoind twice may cause the `.cookie` file generated by the first instance to be deleted by the second instance shutdown (after failing to obtain a lock).
ACKs for top commit:
willcl-ark:
re-ACK 7cb9367157
achow101:
ACK 7cb9367157
kristapsk:
re-ACK 7cb9367157
stickies-v:
ACK 7cb9367157
Tree-SHA512: 0960dbc457975b0e0535f3d814824a879d7f85c9f1191537415b3fc253429a316a8e4badde56c8bc139778f132392983cec5fbe03891fb15ff61d3bc3f6e681b
f23ba24aa0 test_submitpackage: only make a chain of 3 txns (Greg Sanders)
e67a345162 doc: submitpackage vsize results are sigops-adjusted (Greg Sanders)
b67db52c39 RPC submitpackage: change return format to allow partial errors (Greg Sanders)
Pull request description:
This was prompted by errors being returned that didn't "make any sense" to me, because it would for example return a "fee too low" error, when the "real" error was the child had something invalid, which disallowed CPFP evaluation. Rather than make judgment calls on what error is important(which is currently just return the "first"!), we simply return all errors and let the callers determine what's best.
Added a top level `package_msg` for quick eye-balling of general success of the package.
This PR also fixes a couple bugs:
1) Currently we don't actually broadcast a transaction, even if it was entered into our mempool, if a subsequent transaction causes `PKG_TX` failure.
2) "other-wtxid" is uncovered by tests, but IIUC was previously required to return "fees" and "vsize" results, but did not. I just make those results optional.
ACKs for top commit:
Sjors:
Light re-utACK f23ba24aa0
achow101:
ACK f23ba24aa0
glozow:
utACK f23ba24aa0, thanks for taking the suggestions
Tree-SHA512: ebfd716a4fed9e8c2dea3d2181ba6a6171b06718d29ac2324c67b7a30b374d199f7e1739f91ab5d036be172d0479de9bc89c32263ee62143c0338b9b622d0cca
It's come to light that Boost ASIO (a Boost Process sub dep) has in some
instances, been queitly initialising our network stack on Windows (see
PR #28486 and discussion in #28940).
This has been shielding a bug in our own code, but the larger issue
is that Boost Process/ASIO is running code before main, and doing things
like setting up networking. This undermines our own assumptions about
how our binary works, happens before we get to run any sanity checks,
and also runs before we call our own code to setup networking.
It's also not clear why a feature like external signer would have a
dependency that would be doing anything network/socket related, given it
only exists to spawn a local process.
fa98a097a3 Rename version.h to node/protocol_version.h (MarcoFalke)
fa4fbd5816 Remove unused version.h include (MarcoFalke)
fa0ae22ff2 Remove unused SER_NETWORK, SER_DISK (MarcoFalke)
fae00fe9c2 Remove unused CDataStream (MarcoFalke)
fa7eb4f5c3 fuzz: Drop unused version from fuzz input format (MarcoFalke)
Pull request description:
Seems odd to have code that is completely dead.
Fix this by removing all of it.
ACKs for top commit:
sipa:
utACK fa98a097a3
ajtowns:
ACK fa98a097a3
ryanofsky:
Seems odd to not code review ACK fa98a097a3 (looks good)
Tree-SHA512: 9f1b9d9f92bda0512610bda6653e892756f637860362a9abfa439faab62de233cbad94b7df78ebacc160d9667aadfed4d9df08c0edefa618c040a049050fb913
e67634ef19 fuzz: BIP324: damage ciphertext/aad in full byte range (Sebastian Falbesoner)
Pull request description:
This PR is a tiny improvement for the `bip324_cipher_roundtrip` fuzz target: currently the damaging of input data for decryption (either ciphertext or aad) only ever happens in the lower nibble within the byte at the damage position, as the bit position for the `damage_val` byte was calculated with `damage_bit & 3` (corresponding to `% 4`) rather than `damage_bit & 7` (corresponding to the expected `% 8`).
Noticed while reviewing #28263 which uses similar constructs.
ACKs for top commit:
stratospher:
ACK e67634ef.
dergoegge:
utACK e67634ef19
Tree-SHA512: 1bab4df28708e079874feee939beef45eff235215375c339decc696f4c9aef04e4b417322b045491c8aec6e88ec8ec2db564e27ef1b0be352b6ff4ed38bad49a
fad82fea2b ci: Reduce use of bash -c (MarcoFalke)
fafcee4874 ci: Rename test script to 03_test_script.sh (MarcoFalke)
Pull request description:
It is confusing to treat commands as a single string. This change is
also required to support paths and strings with spaces in them in the
future.
ACKs for top commit:
RandyMcMillan:
utACK fad82fea2b
vasild:
ACK fad82fea2b
Tree-SHA512: fb79469d809400739e53da203842fda838f2ec9ab8dcd5e622ccd3db651d629161323bfcc04301562f13f5c407e8865036478a4ac7f6b5265dc4dda1a320c23d
66c4b58e51 guix: switch from guix environment to guix shell (fanquake)
Pull request description:
See https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-environment.html.
> Deprecation warning: The guix environment command is deprecated
in favor of guix shell, which performs similar functions but is more convenient to use. See Invoking guix shell.
> Being deprecated, guix environment is slated for eventual removal,
but the Guix project is committed to keeping it until May 1st, 2023. Please get in touch with us at guix-devel@gnu.org if you would like to discuss it.
See also https://guix.gnu.org/blog/2021/from-guix-environment-to-guix-shell/ for a blog post and additional details.
Guix `shell` was added to Guix ~1 year ago, in this commit, https://git.savannah.gnu.org/cgit/guix.git/commit/?id=80edb7df6586464aa40e84e103f0045452de95db, which isn't part of the 1.3.0 release binaries out of the box, but invoking a `guix pull`, and updating will make it available. i.e:
```bash
bash-5.1# guix --version
guix (GNU Guix) 1.3.0
Copyright (C) 2021 the Guix authors
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
bash-5.1# guix shell
guix: shell: command not found
Try 'guix --help' for more information.
bash-5.1# guix pull
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Authenticating channel 'guix', commits 9edb3f6 to 7a980bb (6,278 new commits)...
Building from this channel:
guix https://git.savannah.gnu.org/git/guix.git7a980bb
< snip >
building /gnu/store/2wwwsczxcw61m05p4mv0kf0advx4fqsb-inferior-script.scm.drv...
building package cache...
building profile with 1 package...
New in this revision:
6,866 new packages: a2jmidid, abjad,
bash-5.1# guix help shell
Usage: guix shell [OPTION] PACKAGES... [-- COMMAND...]
Build an environment that includes PACKAGES and execute COMMAND or an
interactive shell in that environment.
```
ACKs for top commit:
TheCharlatan:
ACK 66c4b58e51
Tree-SHA512: caa3fd2ca8d0f261c50ecdda3728a75389d24d89b51293dedc704ee77ab1342b2bb08ca8c871dcb4646229f056ec86cb15500934ded1b0c501a3ffc25aaa8ae6
Behavior prior to this commit allows some transactions to
enter into the local mempool but not be reported to the user
when encountering a PackageValidationResult::PCKG_TX result.
This is further compounded with the fact that any transactions
submitted to the mempool during this call would also not be
relayed to peers, resulting in unexpected behavior.
Fix this by, if encountering a package error, reporting all
wtxids, along with a new error field, and broadcasting every
transaction that was found in the mempool after submission.
Note that this also changes fees and vsize to optional,
which should also remove an issue with other-wtxid cases.
Also, add missing includes to scriptpubkeyman.
Also, export dependecies of the BasicTestingSetup from setup_common.h,
to avoid having to include them when setup_common.h is already included.
It is confusing to treat commands as a single string. This change is
also required to support paths and strings with spaces in them in the
future.
This requires replacing TEST_RUNNER_ENV with a global export, because it
no longer works. See:
```bash
$ export ENV="A=1" && $ENV ls
bash: A=1: command not found...
```
Or in the CI task:
+ DIR_UNIT_TEST_DATA=/ci_container_base/ci/scratch/qa-assets/unit_test_data/
+ LD_LIBRARY_PATH=/ci_container_base/depends/i686-pc-linux-gnu/lib
+ BITCOIND=bitcoin-node make -j10 check VERBOSE=1
/ci_container_base/ci/test/03_test_script.sh: line 166: BITCOIND=bitcoin-node: command not found
https://github.com/bitcoin/bitcoin/pull/28954/checks?check_run_id=19096858944https://cirrus-ci.com/task/6718317604372480
fa02c08c93 refactor: Use Txid in CMerkleBlock (MarcoFalke)
Pull request description:
This should also fix a gcc-13 compiler warning, see https://github.com/bitcoin/bitcoin/pull/28922#discussion_r1407856376
```
rpc/txoutproof.cpp: In lambda function:
rpc/txoutproof.cpp:72:33: error: possibly dangling reference to a temporary [-Werror=dangling-reference]
72 | const Coin& coin = AccessByTxid(active_chainstate.CoinsTip(), Txid::FromUint256(tx));
| ^~~~
rpc/txoutproof.cpp:72:52: note: the temporary was destroyed at the end of the full expression ‘AccessByTxid((*(const CCoinsViewCache*)(&(& active_chainstate)->Chainstate::CoinsTip())), transaction_identifier<false>::FromUint256((* & tx)))’
72 | const Coin& coin = AccessByTxid(active_chainstate.CoinsTip(), Txid::FromUint256(tx));
| ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: all warnings being treated as errors
ACKs for top commit:
TheCharlatan:
Re-ACK fa02c08c93
dergoegge:
reACK fa02c08c93
Tree-SHA512: 2e6837b9d0c90bd6e9d766330e7086d68c6ec80bb27fe2cfc4702b251b00d91a79f8bfbc76d998cbcd90bee5317402cf617f61099eee96d94e7ac8f37ba7a642
fa1a384706 Move compat.h include from system.h to system.cpp (MarcoFalke)
88887531b7 Move compat/assumptions.h include to one place that actually needs it (MarcoFalke)
77774110f4 Remove __cplusplus from compat/assumptions.h (MarcoFalke)
faa3d4f1d8 Remove duplicate NDEBUG check from compat/assumptions.h (MarcoFalke)
Pull request description:
Generally, compile-time checks should be close to the code that use them. Especially, since `compat/assumptions.h` is only included in one place, where iwyu suggests to remove it.
Fix all issues:
* The `NDEBUG` check is used in `util/check`, so it is redundant in `compat/assumptions.h`.
* The `__cplusplus` check is redundant with `doc/dependencies.md` (see commit message).
* Add missing `// IWYU pragma: keep` to avoid removing the include by accident.
ACKs for top commit:
achow101:
ACK fa1a384706
TheCharlatan:
re-ACK fa1a384706
theuni:
ACK fa1a384706
Tree-SHA512: f8b6db84be5d8844a2267345c0b1405fcbc39b8b5eeaa24db5b8412a74145fe44cf188b6b0c39cc2b062690ed37ca5b4662473484afe28dbec6469e79961389b
9ac114e5cd Throw error if invalid parameters passed to getnetworkhashps RPC endpoint (Jameson Lopp)
Pull request description:
When writing some scripts that iterated over many blocks to generate hashrate estimates I realized that my script was going out of range of the current chain tip height but was not encountering any errors.
I believe that passing an invalid block height to this function but receiving the hashrate estimate for the chain tip instead should be considered unexpected behavior.
ACKs for top commit:
Sjors:
re-utACK 9ac114e5cd
kevkevinpal:
reACK [9ac114e](9ac114e5cd)
achow101:
ACK 9ac114e5cd
Tree-SHA512: eefb465c2dd654fc48267f444e1809597ec5363cdd131ea9ec812458fed1e4bffbbbb0617d74687c9f7bb16274b598d8292f5eeb7953421e5d2a8dc2cc081f2b
705e3f1de0 refactor: Make CTxMemPoolEntry only explicitly copyable (TheCharlatan)
Pull request description:
This has the goal of prohibiting users from accidentally creating runtime failures, e.g. by interacting with iterator_to with a copied entry. This was brought up here: https://github.com/bitcoin/bitcoin/pull/28886#issuecomment-1814794954.
CTxMemPoolEntry is already implicitly not move-constructable. So be explicit about this and use a std::list to collect the values in the policy_estimator fuzz test instead of a std::vector.
ACKs for top commit:
maflcko:
ACK 705e3f1de0🌯
achow101:
ACK 705e3f1de0
ajtowns:
ACK 705e3f1de0
ismaelsadeeq:
ACK 705e3f1de0
Tree-SHA512: 62056905c679c919d00f9ae065ed66ac986e7e7062015aea542843d8deecda57104d7a68d002f7b20afa3164f8e9215d2d2d002c167224129540e3b1bd0712cc
35fb9930ad test: enable v2 transport for p2p_timeouts.py (Martin Zumsande)
2c1669c37a test: enable v2 transport for rpc_net.py (Sebastian Falbesoner)
cc961c2695 test: enable v2 transport for p2p_node_network_limited.py (Sebastian Falbesoner)
3598a1b5c9 test: enable --v2transport in combination with --usecli (Martin Zumsande)
68a9001751 test: persist -v2transport over restarts and respect -v2transport=0 (Martin Zumsande)
Pull request description:
This makes the functional test suite compatible with BIP324, so that
`python3 test_runner.py --v2transport`
should succeed (currently, 12 tests fail for me on master).
Includes two commits by TheStack I found in an old discussion https://github.com/bitcoin/bitcoin/pull/28331#discussion_r1326714164
Note that even though all tests should pass, the python `p2p.py` module will do v2 connections only after the merge of #24748, so that for now only connections between two full nodes will actually run v2.
Some of the fixed tests were added with `--v2transport` to the test runner. Though after #24748 we might also want to consider running the entire suite with `--v2transport` in some CI.
ACKs for top commit:
sipa:
utACK 35fb9930ad. Thanks for taking care of this.
achow101:
ACK 35fb9930ad
theStack:
ACK 35fb9930ad
stratospher:
ACK 35fb993.
Tree-SHA512: 80dc0bf211fa525ff1d092043aea9f222f14c02e5832a548fb8b83b9ede1fcee03c5e8ade0d05c331bdaa492af9c1cf3d0f0b15b846673c6eacea82dd4cefbc3
fae76a1f2a scripted-diff: Use DataStream in most places (MarcoFalke)
fac39b56b7 refactor: SpanReader without nVersion (MarcoFalke)
Pull request description:
The serialize version is unused, so remove it. This also allows to remove `GCS_SER_VERSION` and allows a scripted-diff to remove most of `CDataStream`.
ACKs for top commit:
ajtowns:
ACK fae76a1f2a
ryanofsky:
Code review ACK fae76a1f2a
Tree-SHA512: 3b487dba8ea380f1eacff9fdfb9197f025dbc30906813d3f4c3e6f1e9e4d9f2a169c6f163f51d135e18af538be78e2d2b13d694073ad25c5762980ae971a4c83
Allow any C++ object that has Serialize and Unserialize methods and can be
serialized to a bitcoin CDataStream to be converted to a capnproto Data field
and passed as arguments or return values to capnproto methods using the Data
type.
Extend IPC unit test to cover this and verify the serialization happens
correctly.
Add unit test to test IPC method calls and type conversion between bitcoin c++
types and capnproto messages.
Right now there are custom type hooks in bitcoin IPC code, so the test is
simple, but in upcoming commits, code will be added to convert bitcoin types to
capnproto messages, and the test will be expanded.
fa825975b5 fuzz: Avoid timeout in process_messages (MarcoFalke)
Pull request description:
Reduce the number of messages per fuzz input. There should be no reason to have more messages than that.
This should also avoid timeouts, such as https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64548. CC https://github.com/bitcoin/bitcoin/issues/28812
ACKs for top commit:
dergoegge:
utACK fa825975b5
Tree-SHA512: eeff732f7b0bd9a71f23aeecbf813d31fe34d355b906fd0384a43075cbc3cebc46a26df741b0f337208d8b33b3e28210c9b9437e2eed77844f03131bb8f5f2a1
fa79a881ce refactor: P2P transport without serialize version and type (MarcoFalke)
fa9b5f4fe3 refactor: NetMsg::Make() without nVersion (MarcoFalke)
66669da4a5 Remove unused Make() overload in netmessagemaker.h (MarcoFalke)
fa0ed07941 refactor: VectorWriter without nVersion (MarcoFalke)
Pull request description:
Now that the serialize framework ignores the serialize version and serialize type, everything related to it can be removed from the code.
This is the first step, removing dead code from the P2P stack. A different pull will remove it from the wallet and other parts.
ACKs for top commit:
ajtowns:
reACK fa79a881ce
Tree-SHA512: 785b413580d980f51f0d4f70ea5e0a99ce14cd12cb065393de2f5254891be94a14f4266110c8b87bd2dbc37467676655bce13bdb295ab139749fcd8b61bd5110
ecb46837e7 Change petertodd seeds to petertodd.net (Peter Todd)
Pull request description:
I changed my DNS seeds to .net from .org to avoid issues with DNS blacklisting, that falsely thinks my domain name is pointing to IP addresses with malware and similar things. Right now there are CNAME records, so the .org addresses still work. But eventually, if needed, I'll remove those CNAME's.
ACKs for top commit:
pablomartin4btc:
tACK ecb46837e7
fanquake:
ACK ecb46837e7 - tested that usable addresses are being returned.
Tree-SHA512: 285f7101198ea8e2e20900c17b38aa86db812308c6985d762e5fa8b6f1bc5b0d2d278da841fe2e10cf32e3fe18d4c984bc8cf195bd8d40c86b092b545c62acfa
70100f8584 Revert "ci: Avoid toolset ambiguity that MSVC can't handle" (Hennadii Stepanov)
1a889f7ea0 ci: Set MSVC toolset version explicitly (Hennadii Stepanov)
4335e55359 ci: Run vcpkg with path prefix (Hennadii Stepanov)
Pull request description:
This PR is an alternative to https://github.com/bitcoin/bitcoin/pull/28905 and reverts it.
To avoid toolset version incompatibilities, which result in errors like this:
```
LINK : fatal error C1900: Il mismatch between 'P1' version '20230904' and 'P2' version '20221215' [D:\a\bitcoin\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
LINK : fatal error LNK1257: code generation failed [D:\a\bitcoin\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
LINK : fatal error LNK1327: failure during running link.exe [D:\a\bitcoin\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
```
it is enough to set it explicitly in the vcpkg triplet file (see the second commit). The `VCToolsVersion` environment variable is set by the `ilammy/msvc-dev-cmd` action.
Please note that the https://github.com/bitcoin/bitcoin/pull/28905 is not [optimal](https://github.com/bitcoin/bitcoin/pull/28905#issuecomment-1822571419):
> I guess this is something we'll just have to maintain forever? That's a shame, because it also adds ~30% runtime to this CI job.
ACKs for top commit:
sipsorcery:
utACK 70100f8584.
pablomartin4btc:
ACK 70100f8584 since I've reviewed to be reverted #28905.
Tree-SHA512: 121a8e40c728060526f380b7946211b5d4eca8821bfe62e6451642ffdf95fe9ab7101e0cffa7f4a777bc9cf94278bb50c1b40b71768e1ac39801bb4831afeb90
Currently the damaging of input data for decryption (either ciphertext
or aad) only ever happens in the lower nibble within the byte at the
damage position, as the bit position for the `damage_val` byte was
calculated with `damage_bit & 3` (corresponding to `% 4`) rather than
`damage_bit & 7` (corresponding to the expected `% 8`).
faf1fb207f Fix IWYU for the script_flags fuzz target (MarcoFalke)
fa71285b73 fuzz: Limit fuzz buffer size in script_flags target (MarcoFalke)
fa6b87b9ee fuzz: CDataStream -> DataStream in script_flags (MarcoFalke)
Pull request description:
Most fuzz targets have an upper limit on the buffer size to avoid excessive runtime. Do the same for `script_flags` to avoid timeouts such as https://github.com/bitcoin/bitcoin/issues/28812#issuecomment-1824696971
Also, fix iwyu. Also, remove legacy `CDataStream`.
ACKs for top commit:
dergoegge:
ACK faf1fb207f
brunoerg:
utACK faf1fb207f
Tree-SHA512: 9301917b353f7409e448b6fd3635de19330856e0742431db5ef04e62873501b5b4cd6cb78ad81ada2747fa2bdae033115b5951d10489dd5d0d320426c8b96bee
I changed my DNS seeds to .net from .org to avoid issues with DNS blacklisting,
that falsely thinks my domain name is pointing to IP addresses with malware and
similar things. Right now there are CNAME records, so the .org addresses still
work. But eventually, if needed, I'll remove those CNAME's.
The GHA VS installation includes its own vcpkg package manager, which is
available since VS 17.6. This change avoids any ambiguity about which
copy of vcpkg we run.
9e58c5bcd9 Use Txid in COutpoint (dergoegge)
Pull request description:
This PR changes the type of the hash of a transaction outpoint from `uint256` to `Txid`.
ACKs for top commit:
Sjors:
ACK 9e58c5bcd9
stickies-v:
ACK 9e58c5bcd9. A sizeable diff, but very straightforward changes. Didn't see anything controversial. Left a few nits, but nothing blocking, only if you have to retouch.
TheCharlatan:
ACK 9e58c5bcd9
Tree-SHA512: 58f61ce1c58668f689513e62072a7775419c4d5af8f607669cd8cdc2e7be9645ba14af7f9e2d65da2670da3ec1ce7fc2a744037520caf799aba212fd1ac44b34
0ffcc5b680 ci: remove python3-setuptools from mac build deps (fanquake)
Pull request description:
Remove no-longer used python-setuptools.
Followup to #28432.
Related to #28845.
ACKs for top commit:
hebasto:
ACK 0ffcc5b680, I have reviewed the code and it looks OK.
TheCharlatan:
ACK 0ffcc5b680
Tree-SHA512: c3ac441c85f6f203414e5e2ad0c453ee35fa4765c72c5ff79699aa622c0db767651890ac0c8507f5ed18e18e3b0e23f2952f677476424cfda4df93647a367c26
47e5c9994c fuzz: add target for `DescriptorScriptPubKeyMan` (brunoerg)
641dddf018 fuzz: create ConsumeCoins (brunoerg)
2e1833ca13 fuzz: move `MockedDescriptorConverter` to `fuzz/util` (brunoerg)
Pull request description:
This PR adds fuzz target for `DescriptorScriptPubKeyMan`. Also, moves `MockedDescriptorConverter` to `fuzz/util/descriptor` to be used here and in `descriptor` target.
ACKs for top commit:
maflcko:
lgtm ACK 47e5c9994c🏓
dergoegge:
ACK 47e5c9994c
Tree-SHA512: 519acca6d7b7a3a0bfc031441b02d5980b12bfb97198bd1958a83cd815ceb9eb1499a48a3f0a7fe20e5d06d83b89335d987376fc0a014e2106b0bc0e9838dd02
As the user could have imported a descriptor with
a newer timestamp (by blindly setting 'timestamp=now'),
the wallet needs to update the birth time when it detects
a transaction older than the oldest descriptor timestamp.
In the following-up commit, the wallet birth time will also
be modified by the transactions scanning process. When a tx
older than all descriptor's timestamp is detected.
228d6a2969 build: Fix regression in "ARMv8 CRC32 intrinsics" test (Hennadii Stepanov)
Pull request description:
In the master branch, the `aarch64` binaries lack support for CRC32 intrinsics.
The `vmull_p64` is a part of the Crypto extensions from the ACLE. They are optional extensions, so they get enabled with a `+crypto` for architecture flags.
The regression was introduced in https://github.com/bitcoin/bitcoin/pull/26183 (v25.0).
The `./configure` script log excerpts:
- the master branch @ d752349029:
```
checking whether C++ compiler accepts -march=armv8-a+crc... yes
checking whether C++ compiler accepts -march=armv8-a+crypto... yes
checking for ARMv8 CRC32 intrinsics... no
checking for ARMv8 SHA-NI intrinsics... yes
```
- this PR:
```
checking whether C++ compiler accepts -march=armv8-a+crc+crypto... yes
checking whether C++ compiler accepts -march=armv8-a+crypto... yes
checking for ARMv8 CRC32 intrinsics... yes
checking for ARMv8 SHA-NI intrinsics... yes
```
Guix build:
```
x86_64
2afd81f540c6d3b36ff305e88bafe935e4272cd3efef3130aa69d49a0522541b guix-build-228d6a2969e4/output/aarch64-linux-gnu/SHA256SUMS.part
6c704d6d30d495adb3fb86befdb500eb389a02c1167163f14ab5c3c3e630e6b3 guix-build-228d6a2969e4/output/aarch64-linux-gnu/bitcoin-228d6a2969e4-aarch64-linux-gnu-debug.tar.gz
e4419963c9c0d99adc4e38538900b648f2c14f793b60c8ee2e6f5acc9d3fadd3 guix-build-228d6a2969e4/output/aarch64-linux-gnu/bitcoin-228d6a2969e4-aarch64-linux-gnu.tar.gz
7d11052b6bd28cdf26d5f2a4987f02d32c93a061907bcd048fb6d161a0466ca9 guix-build-228d6a2969e4/output/dist-archive/bitcoin-228d6a2969e4.tar.gz
```
ACKs for top commit:
TheCharlatan:
ACK 228d6a2969
Tree-SHA512: 4c27ca8acb953bf56e972d907a282ee19e3f30f7a4bf8a9822395fe0e28977cd6233e8b65b4a25cc1d3d5ff6a796d7af07653e18531c44ee3efaff1563d96d32
f95af98128 guix: default ssp for Windows GCC (fanquake)
95d55b96c2 guix: remove ssp workaround from Windows GCC (fanquake)
8f43302a0a build: remove explicit libssp linking from Windows build (fanquake)
Pull request description:
I was expecting this to fail to compile somewhere, maybe in the CI, but that doesn't seem to be the case?
Seems workable given the SSP related changes in the newer mingw-w64 headers (which are in Guix):
> Implement some of the stack protector functions/variables so -lssp is now optional when _FORTIFY_SOURCE or -fstack-protector-strong is used.
However I think this would still be broken in some older environments, so we might have to wait for a compiler bump, or similar. The optional -lssp also seems to work when using older headers, which doesn't make sense.
Would fix#28104.
ACKs for top commit:
hebasto:
ACK f95af98128, I've verified binaries from `bitcoin-f95af98128f1-win64.zip` on Windows 11 Pro 23H2.
TheCharlatan:
ACK f95af98128
Tree-SHA512: 71169ec513cfe692dfa7741d2bf37b45da05627c0af1cbd50cf8c3c04cc21c4bf88f3284532bddc1e3e648391ec78dbaca5170987a13c21ac204a7bcaf27f349
fa01f884d3 ci: Add missing COPY for ./test/lint/test_runner (MarcoFalke)
faff3e3b46 lint: Report all lint errors instead of early exit (MarcoFalke)
Pull request description:
`all-lint.py` currently collects all failures. However, the `06_script.sh` does not, since July this year (https://github.com/bitcoin/bitcoin/pull/28103#discussion_r1268115806).
Fix this by printing all failures before exiting.
Can be tested by modifying (for example) two subtrees in the same commit and then running the linters.
ACKs for top commit:
kevkevinpal:
ACK [fa01f88](fa01f884d3)
TheCharlatan:
lgtm ACK fa01f884d3
Tree-SHA512: c0f3110f2907d87e29c755e3b77a67dfae1f8a25833fe6ef8f2f2c58cfecf1aa46f1a20881576b62252b04930140a9e416c78b4edba0780d3c4fa7aaebabba81
21bfee0720 depends: bump libmultiprocess to fix capnproto deprecation warnings (Ryan Ofsky)
Pull request description:
This incorporates PR chaincodelabs/libmultiprocess#88 and reverts the NO_WERROR CI workaround added in #28735
Upstream diff: 61d5a0e661...414542f81e
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
maflcko:
lgtm ACK 21bfee0720
hebasto:
ACK 21bfee0720, I have reviewed the code and it looks OK. I've also skimmed through the related changes in the https://github.com/chaincodelabs/libmultiprocess repository.
Tree-SHA512: b5addb0deed694eeec62a0ae08b4715a811110201f39f3e6cadee8fc4e6231b0e66c844a98512072a1445bac122ab561dc1711e27fb4d7ac5c08ac46780a4acf
5e7cc4144b test: add unit test for CConnman::AddedNodesContain() (Jon Atack)
cc62716920 p2p: do not make automatic outbound connections to addnode peers (Jon Atack)
Pull request description:
to allocate our limited outbound slots correctly, and to ensure addnode
connections benefit from their intended protections.
Our addnode logic usually connects the addnode peers before the automatic
outbound logic does, but not always, as a connection race can occur. If an
addnode peer disconnects us and if it was the only one from its network, there
can be a race between reconnecting to it with the addnode thread, and it being
picked as automatic network-specific outbound peer. Or our internet connection
or router or the addnode peer could be temporarily offline, and then return
online during the automatic outbound thread. Or we could add a new manual peer
using the addnode RPC at that time.
The race can be more apparent when our node doesn't know many peers, or with
networks like cjdns that currently have few bitcoin peers.
When an addnode peer is connected as an automatic outbound peer and is the only
connection we have to a network, it can be protected by our new outbound
eviction logic and persist in the "wrong role".
Finally, there does not seem to be a reason to make block-relay or short-lived
feeler connections to addnode peers, as the addnode logic will ensure we connect
to them if they are up, within the addnode connection limit.
Fix these issues by checking if the address is an addnode peer in our automatic
outbound connection logic.
ACKs for top commit:
mzumsande:
Tested ACK 5e7cc4144b
brunoerg:
utACK 5e7cc4144b
vasild:
ACK 5e7cc4144b
guggero:
utACK 5e7cc4144b
Tree-SHA512: 2438c3ec92e98aebca2a0da960534e4655a9c6e1192a24a085fc01326d95cdb1b67d8c44e4ee706bc1d8af8564126d446a21b5579dcbec61bdea5fce2f0115ee
007d6f0e85 test: fix `AddNode` unit test failure on OpenBSD (Sebastian Falbesoner)
Pull request description:
On OpenBSD 7.4, the following check of the unit test `test_addnode_getaddednodeinfo_and_connection_detection` currently fails:
```
BOOST_CHECK(!connman->AddNode({/*m_added_node=*/"127.1", /*m_use_v2transport=*/true}));
```
The reason for that is that this OS seemingly doesn't support the IPv4 shorthand notation with omitted zero-bytes:
```
$ ping 127.1
ping: no address associated with name
```
As a simple fix, this PR skips the check for this with a pre-processor #if. On NetBSD and FreeBSD, `127.1` is resolved correctly to localhost and hence the test passes (thanks to vasild for verifying on the latter!).
ACKs for top commit:
vasild:
ACK 007d6f0e85
Tree-SHA512: 8ab8393c490e1ecc140e8ff74f6fa4d26d0dd77e6a77a241cd198314b8c5afee7422f95351ca05f4c1742433dab77016a8ccb8d28062f8edd4b703a918a2bbda
d5b4c0b69e pool: change memusage_test to use int64_t, add allocation check (Martin Leitner-Ankerl)
ce881bf9fc pool: make sure PoolAllocator uses the correct alignment (Martin Leitner-Ankerl)
Pull request description:
The class `CTxOut` has a member `CAmount` which is an int64_t, and on ARM 32bit int64_t are 8 byte aligned, which is larger than the pointer alignment of 4 bytes.
So for `CCoinsMap` to be able to use the pool, we need to use the alignment of the member instead of just `alignof(void*)`.
This fixes#28906 (first noted in https://github.com/bitcoin/bitcoin/issues/28718#issuecomment-1807197107) and #28440.
ACKs for top commit:
pinheadmz:
ACK d5b4c0b69e
hebasto:
re-ACK d5b4c0b69e, the only change since my recent [review](https://github.com/bitcoin/bitcoin/pull/28913#pullrequestreview-1739334189) is an updated test.
theStack:
Tested ACK d5b4c0b69e
Tree-SHA512: 4446793fad6d56f0fe22e09ac9ade051e86de11ac039cd61c0f6b7f79874242878a6a46a2c76ac3b8f1d53464872620d39139f54b1471daccad26d6bb1ae8ca1
`CBlockPolicyEstimator` will implement `CValidationInterface` and
subscribe to its notification to process transactions added and removed
from the mempool.
Re-delegate calculation of `validForFeeEstimation` from validation to fee estimator.
Also clean up the validForFeeEstimation arg thats no longer needed in `CTxMempool`.
Co-authored-by: Matt Corallo <git@bluematt.me>
This commit adds a new callback `MempoolTransactionsRemovedForBlock` which notify
its listeners of the transactions that are removed from the mempool because a new
block is connected, along with the block height the transactions were removed.
The transactions are in `RemovedMempoolTransactionInfo` format.
`CTransactionRef`, base fee, virtual size, and height which the transaction was added
to the mempool are all members of the struct called `RemovedMempoolTransactionInfo`.
A struct `NewMempoolTransactionInfo`, which has fields similar to `RemovedMempoolTransactionInfo`,
will be added in a later commit, create a struct `TransactionInfo` with all similar fields.
They can both have a member with type `TransactionInfo`.
If the removal reason of a transaction is BLOCK, then the `removeTx`
boolean argument should be true.
Before this PR, `CBlockPolicyEstimator` have to complete updating the fee stats
before the mempool clears that's why having removeTx call outside reason!= `BLOCK`
in `addUnchecked` was not a bug.
But in a case where the `CBlockPolicyEstimator` update is asynchronous, the mempool might
clear before we update the `CBlockPolicyEstimator` fee stats.
Transactions that are removed for `BLOCK` reasons will also be incorrectly removed from
`CBlockPolicyEstimator` stats as failures.
710da28c72 ci: Switch from `apt` to `apt-get` (Hennadii Stepanov)
a6cc059ea5 ci: Update apt cache (Hennadii Stepanov)
Pull request description:
This PR aims to fix the recent errors in the "test each commit" CI job.
ACKs for top commit:
maflcko:
lgtm ACK 710da28c72
ismaelsadeeq:
utACK 710da28c72
Tree-SHA512: b42340aea00e80f791000e19791629f27df2da98adefb839cb4389d81b5eee094089ea5092a2d7b56b3990683a72e4d2fa986fc86c823c7245649af37873b790
Instead of doing one db transaction per descriptor setup,
batch all descriptors' setup writes in a single db txn.
Speeding up the process and preventing the wallet from entering
an inconsistent state if any of the intermediate transactions
fail.
If alignment of the PoolAllocator would be insufficient, then the test would fail. This also catches the issue with ARM 32bit,
where int64_t is aligned to 8 bytes but void* is aligned to 4 bytes. The test adds a check to ensure the pool has allocated
a minimum number of chunks
The `vmull_p64` is a part of the Crypto extensions from the ACLE. They
are optional extensions, so they get enabled with a `+crypto` for
architecture flags.
The nVersion field is unused, so remove it.
This is also required for future commits.
Also, add PushMessage aliases in PeerManagerImpl to make calling code
less verbose.
Co-Authored-By: Anthony Towns <aj@erisian.com.au>
This changes the PoolAllocator to default the alignment to the given type. This makes the code simpler, and most importantly
fixes a bug on ARM 32bit that caused OOM: The class CTxOut has a member CAmount which is an int64_t and on ARM 32bit int64_t
are 8 byte aligned which is larger than the pointer alignment of 4 bytes. So for CCoinsMap to be able to use the pool, we
need to use the alignment of the member instead of just alignof(void*).
This has the goal of prohibiting users from accidentally creating
runtime failures, e.g. by interacting with iterator_to with a copied
entry.
CTxMemPoolEntry is already implicitly not move-constructable. So be
explicit about this and use a std::list to collect the values in the
policy_estimator fuzz test instead of a std::vector.
Co-authored-by: Anthony Towns <aj@erisian.com.au>
fa552e8a4e doc: Simplify guix install doc, after 1.4 release (MarcoFalke)
Pull request description:
Now that 1.4 is out (for a while), remove the recommendation to build a random commit.
ACKs for top commit:
fanquake:
ACK fa552e8a4e
hebasto:
ACK fa552e8a4e.
Tree-SHA512: f5642df201ff0e2af8a7ae9660a66920ddbb5f522b3e921f6f4aa7c411ced23afa91bdfe43b943ac012228eebbaad3396df505d00aa8f721a4358f03fda9d8e3
a478c817b2 test: replace `Callable`/`Iterable` with their `collections.abc` alternative (PEP 585) (stickies-v)
4b9afb18e6 scripted-diff: use PEP 585 built-in collection types for verify-binary script (Sebastian Falbesoner)
d516cf83ed test: use built-in collection types for type hints (Python 3.9 / PEP 585) (Sebastian Falbesoner)
Pull request description:
With Python 3.9 / [PEP 585](https://peps.python.org/pep-0585/), [type hinting has become a little less awkward](https://docs.python.org/3.9/whatsnew/3.9.html#type-hinting-generics-in-standard-collections), as for collection types one doesn't need to import the corresponding capitalized types (`Dict`, `List`, `Set`, `Tuple`, ...) anymore, but can use the built-in types directly (see https://peps.python.org/pep-0585/#implementation for the full list).
This PR applies the replacement for all Python scripts (i.e. in the contrib and test folders) for the basic types, i.e.:
- typing.Dict -> dict
- typing.List -> list
- typing.Set -> set
- typing.Tuple -> tuple
For an additional check, I ran mypy 1.6.1 on both master and the PR branch via
```
$ mypy --ignore-missing-imports --explicit-package-bases $(git ls-files "*.py")
```
and verified that the output is identical -- (from the 22 identified problems, most look like false-positives, it's probably worth it to go deeper here and address them in a follow-up though).
ACKs for top commit:
stickies-v:
ACK a478c817b2
fanquake:
ACK a478c817b2
Tree-SHA512: 6948c905f6abd644d84f09fcb3661d7edb2742e8f2b28560008697d251d77a61a1146ab4b070e65b0d27acede7a5256703da7bf6eb1c7c3a897755478c76c6e8
83986f464c Include version.h in fewer places (Anthony Towns)
c7b61fd61b Convert some CDataStream to DataStream (Anthony Towns)
1410d300df serialize: Drop useless version param from GetSerializeSize() (Anthony Towns)
bf574a7501 serialize: drop GetSerializeSizeMany (Anthony Towns)
efa9eb6d7c serialize: Drop nVersion from [C]SizeComputer (Anthony Towns)
Pull request description:
Drops the version field from `GetSerializeSize()`, simplifying the code in various places. Also drop `GetSerializeSizeMany()` (as just removing the version parameter could result in silent bugs) and remove unnecessary instances of `#include <version.h>`.
ACKs for top commit:
maflcko:
ACK 83986f464c📒
theuni:
ACK 83986f464c.
Tree-SHA512: 36617b6dfbb1b4b0afbf673e905525fc6d623d3f568d3f86e3b9d4f69820db97d099e83a88007bfff881f731ddca6755ebf1549e8d8a7762437dfadbf434c62e
faa25718b3 fuzz: AutoFile with XOR (MarcoFalke)
fab5cb9066 fuzz: Reduce LIMITED_WHILE limit for file fuzzing (MarcoFalke)
fa5388fad3 fuzz: Remove FuzzedAutoFileProvider (MarcoFalke)
Pull request description:
This should help to get fuzz coverage for https://maflcko.github.io/b-c-cov/fuzz.coverage/src/streams.cpp.gcov.html
Also, remove unused code and fix a timeout bug.
ACKs for top commit:
dergoegge:
ACK faa25718b3
Tree-SHA512: 56f1e6fd5cb2b66ffd9a7d9c09c9b8e396be3e7485feb03b35b6bd3c48e624fdaed50b472e4ffec21f09efb5e949d7ee32a13851849c9140b6b4cf25917dd7ac
to allocate our limited outbound slots correctly, and to ensure addnode
connections benefit from their intended protections.
Our addnode logic usually connects the addnode peers before the automatic
outbound logic does, but not always, as a connection race can occur. If an
addnode peer disconnects us and if it was the only one from its network, there
can be a race between reconnecting to it with the addnode thread, and it being
picked as automatic network-specific outbound peer. Or our internet connection
or router, or the addnode peer, could be temporarily offline, and then return
online during the automatic outbound thread. Or we could add a new manual peer
using the addnode RPC at that time.
The race can be more apparent when our node doesn't know many peers, or with
networks like cjdns that currently have few bitcoin peers.
When an addnode peer is connected as an automatic outbound peer and is the only
connection we have to a network, it can be protected by our new outbound
eviction logic and persist in the "wrong role".
Examples on mainnet using logging added in the same pull request:
2023-08-12T14:51:05.681743Z [opencon] [net.cpp:1949] [ThreadOpenConnections]
[net:debug] Not making automatic network-specific outbound-full-relay connection
to i2p peer selected for manual (addnode) connection: [geh...odq.b32.i2p]:0
2023-08-13T03:59:28.050853Z [opencon] [net.cpp:1949] [ThreadOpenConnections]
[net:debug] Not making automatic block-relay-only connection to onion peer
selected for manual (addnode) connection: kpg...aid.onion:8333
2023-08-13T16:21:26.979052Z [opencon] [net.cpp:1949] [ThreadOpenConnections]
[net:debug] Not making automatic network-specific outbound-full-relay connection
to cjdns peer selected for manual (addnode) connection: [fcc...8ce]:8333
2023-08-14T20:43:53.401271Z [opencon] [net.cpp:1949] [ThreadOpenConnections]
[net:debug] Not making automatic network-specific outbound-full-relay connection
to cjdns peer selected for manual (addnode) connection: [fc7...59e]:8333
2023-08-15T00:10:01.894147Z [opencon] [net.cpp:1949] [ThreadOpenConnections]
[net:debug] Not making automatic feeler connection to i2p peer selected for
manual (addnode) connection: geh...odq.b32.i2p:8333
Finally, there does not seem to be a reason to make block-relay or short-lived
feeler connections to addnode peers, as the addnode logic will ensure we connect
to them if they are up, within the addnode connection limit.
Fix these issues by checking if the address is an addnode peer in our automatic
outbound connection logic.
43de4d3630 doc: fix typos (Sjors Provoost)
Pull request description:
This PR fixes typos found by lint-spelling.py using codespell 2.2.6.
Our CI linter job uses codespell 2.2.5 and found fewer typos that I did locally. In any case it's happy now.
ACKs for top commit:
pablomartin4btc:
re ACK 43de4d3630
Tree-SHA512: c032fe86cb49c924a468385653b31f309a9db68c478d70335bba3e65a1ff3826abe80284fe00a090ab5a509e1edbf17e476f6922fb15d055e50f1103dad2ccb0
88e09ac2a1 tests: Fix LCOV_OPTS to be in the correct position (Andrew Chow)
Pull request description:
`lcov`'s `-a` option takes an argument. With `LCOV_OPTS` immediately after `-a`, the first additional argument becomes the argument to `-a` which is incorrect.
Also add `LCOV_OPTS` to more `lcov` calls.
ACKs for top commit:
fanquake:
ACK 88e09ac2a1
Tree-SHA512: 1ed657c96395bfe882041ded883cb5fa4d04d6ede91f66c319b5bbdd1f88468f8abb2a741dd7898904a78ed7e6c844316f7958ce9e4ccf2dbe666ebec308b7fb
3b19100303 depends: remove PYTHONPATH from config.site (fanquake)
Pull request description:
We no-longer need this, as we no-longer build python packages.
ACKs for top commit:
hebasto:
ACK 3b19100303, this PR effectively reverts no longer needed de619a37fd.
Tree-SHA512: 775354773f83fc98922f1d4ee84d8f1e866fb6fb2a59a3eaf06a7a5f0d846f7dc1b84862c58195dfb91ddfb02b2dc86bee78b51459f91c65a5b1464df9f3c53c
6a917918b7 fuzz: allow fake and duplicate inputs in tx_package_eval target (Greg Sanders)
a0626ccdad fuzz: allow reaching MempoolAcceptResult::ResultType::DIFFERENT_WITNESS in tx_package_eval target (Greg Sanders)
Pull request description:
Exercises `DIFFERENT_WITNESS` by using "blank" WSH() and allowing witness to determine wtxid, and attempts to make invalid/duplicate inputs.
ACKs for top commit:
dergoegge:
Coverage looks good to me ACK 6a917918b7
Tree-SHA512: db894f5f5b81c6b454874baf11f296462832285f41ccb09f23c0db92b9abc98f8ecacd72fc8f60dc92cb7947f543a2e55bed2fd210b0e8ca7c7d5389d90b14af
821a8a1125 doc: remove x86_64 build assumption from depends doc (fanquake)
Pull request description:
This dates from the introduction of depends, and has not been the case for some time now.
ACKs for top commit:
maflcko:
lgtm ACK 821a8a1125
hebasto:
ACK 821a8a1125.
theuni:
ACK 821a8a1125
Tree-SHA512: 640967a3e6dfab495fd733d3379aa916ac7f67e89a92ef6a94c3bea0494dc7921a9d7485e1b90a1beab00548b575cdab8fb08eb9267dcc5e890cc796ae1b6875
30bd4b1e4a doc: remove mention of missing bdb being a configure error (fanquake)
Pull request description:
This is no-longer the case, unless you're passing additional flags, which is not the case in this example.
ACKs for top commit:
maflcko:
lgtm ACK 30bd4b1e4a
TheCharlatan:
ACK 30bd4b1e4a
hebasto:
ACK 30bd4b1e4a.
Tree-SHA512: b3730546d7ff1f49854b88e710c72c4f6e4b6d238147599d4c4e4adeeb256424c2096635f6c51dcfe2e5a9c1155c1c9915fe03a09c5c38605bee2722756c8f6e
defdf67765 contrib: use a raw string for a regular expression literal that contains backslashes in signet/miner (muxator)
Pull request description:
Running `contrib/signet/miner` under python >= 3.12 causes a `SyntaxWarning`. The problem was already present in previous versions, but it only triggered a `DeprecationWarning`, which was not shown by default.
The change is useful for future-proofing the code base, since future python versions will start to exit with a runtime exception (see the reference given later).
Command to see the warning at runtime under python3.11 (`DeprecationWarning`, needs "-Walways"):
```
$ python3.11 -Walways ./contrib/signet/miner
<BASE>/contrib/signet/miner:33: DeprecationWarning: invalid escape sequence '\d'
RE_MULTIMINER = re.compile("^(\d+)(-(\d+))?/(\d+)$")
2023-11-15 16:02:49 ERROR Must specify command
```
Command to see the warning at runtime under python3.12 (`SyntaxWarning`, no modifiers needed):
```
$ python3.12 ./contrib/signet/miner
<BASE>/contrib/signet/miner:33: SyntaxWarning: invalid escape sequence '\d'
RE_MULTIMINER = re.compile("^(\d+)(-(\d+))?/(\d+)$")
2023-11-15 16:03:00 ERROR Must specify command
```
Reference (https://docs.python.org/3.8/library/re.html):
> Regular expressions use the backslash character ('\') [...]. This collides with Python’s usage of the same character for the same purpose in string literals; [...]
>
> Also, please note that any invalid escape sequences in Python’s usage of the backslash in string literals now generate a DeprecationWarning and in the future this will become a SyntaxError.
>
> The solution is to use Python’s raw string notation for regular expression patterns;
ACKs for top commit:
maflcko:
lgtm ACK defdf67765
ajtowns:
utACK defdf67765
Tree-SHA512: 81bd4892938e7d40a226ca20b5b61ff2470ad763743528da290271faefc535167b56f44665e2d03ed2607c4f7bc8a3200e7931f98fe28dbaf0d2a842c96549f5
Protocol version is no longer needed to work out the serialized size
of objects so drop that information from CSizeComputer and rename the
class to SizeComputer.
f718a74b12 guix: remove python-macholib (fanquake)
d3cbff16c2 guix: update signapple (fanquake)
Pull request description:
Update to the latest signapple, which includes https://github.com/achow101/signapple/pull/13.
Drop python-macholib and python-altgraph.
ACKs for top commit:
Sjors:
ACK f718a74b12
Tree-SHA512: 199b2108f2f063b6b0fb5354ac79a30b46e848c923ebe7d02f7d7d3f08749817a1f6b4c14d21658fd2f2d68f8be1698e1999edf7e2366b1cae3bf2709a665e30
a0c254c13a Drop CHashWriter (Anthony Towns)
c94f7e5b1c Drop OverrideStream (Anthony Towns)
6e9e4e6130 Use ParamsWrapper for witness serialization (Anthony Towns)
Pull request description:
Choose whether witness is included in transaction serialization via serialization parameter rather than the stream version. See #25284 and #19477 for previous context.
ACKs for top commit:
maflcko:
re-ACK a0c254c13a🐜
theuni:
ACK a0c254c13a
Tree-SHA512: 8fd5cadfd84c5128e36c34a51fb94fdccd956280e7f65b7d73c512d6a9cdb53cdd3649de99ffab5322bd34be26cb95ab4eb05932b3b9de9c11d85743f50dcb13
Running the miner under python >= 3.12 causes a SyntaxWarning. The problem was
already present in previous versions, but it only triggered a
DeprecationWarning, which was not shown by default.
The change is useful for future-proofing the code base, since future python
versions will start to exit with a runtime exception (see the reference given
later).
Command to see the warning at runtime under python3.11 (DeprecationWarning,
needs "-Walways"):
$ python3.11 -Walways ./contrib/signet/miner
<BASE>/contrib/signet/miner:33: DeprecationWarning: invalid escape sequence '\d'
RE_MULTIMINER = re.compile("^(\d+)(-(\d+))?/(\d+)$")
2023-11-15 16:02:49 ERROR Must specify command
Command to see the warning at runtime under python3.12 (SyntaxWarning, no
modifiers needed):
$ python3.12 ./contrib/signet/miner
<BASE>/contrib/signet/miner:33: SyntaxWarning: invalid escape sequence '\d'
RE_MULTIMINER = re.compile("^(\d+)(-(\d+))?/(\d+)$")
2023-11-15 16:03:00 ERROR Must specify command
Reference ( https://docs.python.org/3.8/library/re.html ):
Regular expressions use the backslash character ('\') [...]. This collides
with Python’s usage of the same character for the same purpose in string
literals; [...]
Also, please note that any invalid escape sequences in Python’s usage of the
backslash in string literals now generate a DeprecationWarning and in the
future this will become a SyntaxError.
The solution is to use Python’s raw string notation for regular expression
patterns;
fd30e9688e test: migrate to some per-symbol ubsan suppressions (fanquake)
Pull request description:
Now that the symbolizer should be hanging around (#28814), migrate some file-wide suppressions to be symbol specific. Should assist in catching new issues that may otherwise go unnoticed due to file-wide suppression.
Only tested (so far) on aarch64 using the native ASAN & FUZZ CI.
ACKs for top commit:
maflcko:
lgtm ACK fd30e9688e
dergoegge:
utACK fd30e9688e (if CI is green)
Tree-SHA512: fbc44464d22813969dd4d1cdeab00042fa45f0af9bf1aed4fd3b688dc7b3c377a7c0f5f0c0a37ba65b649cfb5c7ff8ab2774500fe182d702c4340ca19f08479f
d799ea26ed doc: rewrite explanation for -par= (fanquake)
Pull request description:
The negative bound for script threads comes from the machine which generates the man pages, so may only be correct for that machine. Any other placeholder value will also be wrong for some machines. Fix this be removing the value. This also fixes help2man incorrectly bolding the value, as if it were a paramater.
Closes#28850.
ACKs for top commit:
maflcko:
lgtm ACK d799ea26ed
theStack:
ACK d799ea26ed
Tree-SHA512: 2eec0086faf4cc64bbf46b22949662f84d8546d2322c3d507fc44a4e1f64d228a2901af4fa4535c0771e3e14600be8308fc5dbd407b66ae6ae4f8878d8372c0a
1e5b86171e test: Add test for array serialization (TheCharlatan)
d49d198840 refactor: Initialize magic bytes in constructor initializer (TheCharlatan)
Pull request description:
This is a followup-PR for #28423
* Initialize magic bytes in constructor
* Add a small unit test for serializing arrays.
ACKs for top commit:
sipa:
utACK 1e5b86171e
maflcko:
lgtm ACK 1e5b86171e
Tree-SHA512: 0f58d2332dc501ca9fd419f40ed4f977c83dce0169e9a0eee1ffc9f8daa2d2ef7e7df18205ba076f55d90ae6c4a20d2b51ab303150d38470a962bcc58a66f6e7
3c61c60b90 build: Add an old hack to remove bind_at_load from libtool. (Cory Fields)
45257601da build: remove -bind_at_load usage (fanquake)
Pull request description:
This is deprecated on macOS:
```bash
ld: warning: -bind_at_load is deprecated on macOS
```
and likely redundant anyways, given the behaviour of dyld3.
Unfortunately libtool is still injecting a `-bind_at_load`, because it's version check is broken:
```bash
# Don't allow lazy linking, it breaks C++ global constructors
# But is supposedly fixed on 10.4 or later (yay!).
if test CXX = "$tagname"; then
case ${MACOSX_DEPLOYMENT_TARGET-10.0} in
10.[0123])
func_append compile_command " $wl-bind_at_load"
func_append finalize_command " $wl-bind_at_load"
;;
esac
fi
```
so this adds another change to strip them out at the end of configure.
Note that anywhere the ld64 warnings are being emitted, we are already not adding this flag to our hardened ldflags, because of `-Wl,-fatal_warnings`.
ACKs for top commit:
theuni:
utACK 3c61c60b90.
hebasto:
ACK 3c61c60b90, tested on macOS Sonoma 14.1.1 (23B81, Apple M1) and Ubuntu 23.10 (cross-compiling for macOS). Also I've verified the actual diff in the `libtool` script.
Tree-SHA512: 98e6a095dc2d2409f8ec3b9d462e0db3643d7873d7903a12f8acd664829e7e84e797638556fa42ca8ebc1003f13a38fe9bb8a2a50cecfa991155da818574bf08
49a92579c7 build: latest config.sub in depends (fanquake)
ced0435a71 build: latest config.guess in depends (fanquake)
Pull request description:
Before we make any local modifications (i.e #28733) pull the latest files from upstream.
ACKs for top commit:
TheCharlatan:
ACK 49a92579c7
Tree-SHA512: fbbe0d6ef72a196a652467af0550b38da23b932fe68da4965a9b0dc4795db9c869969db98f660cd360f6af3a7659b46c25e3fd398e0ef127dae71726b9a915a6
- Ensure that the snapshot height is higher than the pruned block height when the node is pruned.
- Validate the correctness of the file path and check if the file already exists.
- Make network activity disablement optional for the user.
- Ensure the reconsiderblock command is triggered on exit, even in the case of user interruption (Ctrl-C).
Co-authored-by: Chris Heyes <22148308+hazeycode@users.noreply.github.com>
Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
fa6b053b5c mempool: persist with XOR (MarcoFalke)
Pull request description:
Currently the `mempool.dat` file stores data received from remote peers as-is. This may be problematic when a program other than Bitcoin Core tries to interpret them by accident. For example, an anti-virus program or other program may scan the file and move it into quarantine, or delete it, or corrupt it.
While the local wallet is expected to re-submit any pending transactions, unrelated transactions may be missing from the mempool after a restart. This may cause fee estimates to be off, or may cause block relay to be slower.
Fix this, similar to https://github.com/bitcoin/bitcoin/pull/6650, by rolling a random XOR pattern over the dat file when writing or reading it.
Obviously this can only protect against programs that accidentally and unintentionally are trying to mess with the dat file. Any program that intentionally wants to mess with the dat file can still trivially do so.
ACKs for top commit:
achow101:
re-ACK fa6b053b5c
glozow:
reACK fa6b053b5c
ismaelsadeeq:
ACK fa6b053b5c
Tree-SHA512: ded2ce3d81bc944b828263534e3178a1e45a914fe8e024f4a14c6561a73e301820944ecc75dd704b3d4221a7a3a5c0597ccab79546250c1197609ee981fe324e
bbbbdb0cd5 ci: Add filesystem lint check (MarcoFalke)
fada2f9110 refactor: Replace <filesystem> with <util/fs.h> (MarcoFalke)
Pull request description:
Using `std::filesystem` is problematic:
* There is a `fs` namespace wrapper for it. So having two ways to achieve the same is confusing.
* Not using the `fs` wrapper is dangerous and buggy, because it disables known bugs by deleting problematic functions.
Fix all issues by removing use of it and adding a linter to avoid using it again in the future.
ACKs for top commit:
TheCharlatan:
ACK bbbbdb0cd5
fanquake:
ACK bbbbdb0cd5🦀
Tree-SHA512: 0e2d49742b08eb2635e6fce41485277cb9c40fe20b81017c391d3472a43787db1278a236825714ca1e41c9d2f59913865cfb0c649e3c8ab1fb598c849f80c660
3b70f7b615 doc: fix broken doc/design/multiprocess.md links after #24352 (Ryan Ofsky)
6d43aad742 span: Make Span template deduction guides work in SFINAE context (Ryan Ofsky)
8062c3bdb9 util: Add ArgsManager SetConfigFilePath method (Ryan Ofsky)
441d00c60f interfaces: Rename CalculateBumpFees methods to be compatible with capn'proto (Ryan Ofsky)
156f49d682 interfaces: Change getUnspentOutput return type to avoid multiprocess segfault (Ryan Ofsky)
4978754c00 interfaces: Add schedulerMockForward method so mockscheduler RPC can work across processes (Ryan Ofsky)
924327eaf3 interfaces: Fix const virtual method that breaks multiprocess support (Ryan Ofsky)
82a379eca8 streams: Add SpanReader ignore method (Russell Yanofsky)
Pull request description:
This is a collection of small changes to interfaces and code which were needed as part of multiprocess PR #10102, but have been moved here to make that PR smaller.
All of these changes are refactoring changes which do not affect behavior of current code
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
achow101:
ACK 3b70f7b615
naumenkogs:
ACK 3b70f7b615
maflcko:
re-ACK 3b70f7b615🎆
Tree-SHA512: 2368772b887056ad8a9f84c299cfde76ba45943770e3b5353130580900afa9611302195b899ced7b6e303b11f053ff204cae7c28ff4e12c55562fcc81119ba4c
The negative bound for script threads comes from the machine which
generates the man pages, so may only be correct for that machine. Any
other placeholder value will also be wrong for some machines. Fix this
be removing the value. This also fixes help2man incorrectly bolding the
value, as if it were a paramater.
Closes#28850.
fca0a8938e ci: remove "--exclude banman" for fuzzing in mac (brunoerg)
f9b286353f fuzz: call lookup functions before calling `Ban` (brunoerg)
Pull request description:
Fixes#27924
To not have any discrepancy, it's required to call lookup functions before calling `Ban`. If we don't do it, the assertion `assert(banmap == banmap_read);` may fail because `BanMapFromJson` will call `LookupSubNet` and cause the discrepancy between the banned and the loaded one. It happens especially in MacOS (#27924).
Also, calling lookup functions before banning is what RPC `setban` does.
ACKs for top commit:
maflcko:
lgtm ACK fca0a8938e
dergoegge:
ACK fca0a8938e
Tree-SHA512: a3d635088a556df4507e65542157f10b41d4f87dce42927b58c3b812f262f4544b6b57f3384eef1097ffdd7c32b8dd1556aae201254960cbfbf48d45551200f7
4dd94ca18f [refactor] remove access to mapTx in validation_block_tests (TheCharlatan)
d0cd2e804e [refactor] rewrite BlockAssembler inBlock and failedTx as sets of txids (glozow)
55b0939cab scripted-diff: rename vTxHashes to txns_randomized (TheCharlatan)
a03aef9cec [refactor] rewrite vTxHashes as a vector of CTransactionRef (glozow)
938643c3b2 [refactor] remove access to mapTx in validation.cpp (glozow)
333367a940 [txmempool] make CTxMemPoolEntry::lockPoints mutable (glozow)
1bf4855016 [refactor] use CheckPackageLimits for checkChainLimits (glozow)
dbc5bdbf59 [refactor] remove access to mapTx.find in mempool_tests.cpp (glozow)
f80909e7a3 [refactor] remove access to mapTx in blockencodings_tests.cpp (glozow)
8892d6b744 [refactor] remove access to mapTx from rpc/mempool.cpp (glozow)
fad61aa561 [refactor] get wtxid from entry instead of vTxHashes (glozow)
9cd8cafb77 [refactor] use exists() instead of mapTx.find() (glozow)
14804699e5 [refactor] remove access to mapTx from policy/rbf.cpp (glozow)
1c6a73abbd [refactor] Add helper for retrieving mempool entry (TheCharlatan)
453b4813eb [refactor] Add helper for iterating through mempool entries (stickies-v)
Pull request description:
Motivation
* It seems preferable to use stdlib data structures instead of boost if they can achieve close to the same thing.
* Code external to mempool should ideally use its public helper methods instead of accessing `mapTx` or its iterators directly.
* Reduce the number of complex boost multi index type interactions
* Also see #28335 for further context/motivation. This PR together with #28385 simplifies that one.
Overview of things done in this PR:
* Make `vTxHashes` a vector of transaction references instead of a pair of transaction hash and iterator. The trade off here is that the data is retrieved on the fly with `GetEntry` instead of being cached in `vTxHashes`.
* Introduce `GetEntry` helper method to replace the more involved `GetIter` where applicable
* Replace `mapTx` access with `CTxMemPool` helper methods
* Simplify `checkChainLimits` call in `node/interfaces.cpp`
* Make `CTxMemPoolEntry`s `lockPoints`mutable such that they can be changed with a const iterator directly instead of going through `mapTx`
* Make `BlockAssembler`'s `inBlock` and `failedTx` sets of transaction hashes.
ACKs for top commit:
glozow:
reACK 4dd94ca
maflcko:
re-ACK 4dd94ca18f👝
stickies-v:
re-ACK 4dd94ca18f
Tree-SHA512: c4d043f2186e4fde337591883fac66cade3058173987b49502bd65cecf69207a3df1077f6626809652ab63230013167b7f39a2b39f1c5166959e5495df57065f
44445ae8f1 test: Avoid intermittent failures in feature_init (MarcoFalke)
Pull request description:
The code not only modifies block dat files, but also leveldb files, which may be of smaller size. Such corruption may not force leveldb to abort, according to the intermittent test failures.
Fix the intermittent test failures by reverting 5ab6419f38 .
ACKs for top commit:
kevkevinpal:
lgtm ACK [44445ae](44445ae8f1)
fjahr:
ACK 44445ae8f1
theStack:
ACK 44445ae8f1
Tree-SHA512: 8084e4aeb8a976c1706a1898d7854c55d0c4ec7b5a08f65f97ffc173c935f9b0e0c1caef7be1538a458e4c018f7bd1948173349ec76ca48bc4013a63f284bb0f
ebc7063c80 doc: update docs for CHECK_ATOMIC macro (fanquake)
Pull request description:
Clarify that supported versions of GCC are not affected, and that Clang
prior to version 15 still requires the explicit `-latomic` linking, when
compiling for 32-bit.
ACKs for top commit:
hebasto:
ACK ebc7063c80.
Tree-SHA512: 6044dc28547431cfde7e89b663b5f9a86a4cb801212a21c3dbb18a1c41a53640480c3e4e944050dc3ec4cded9bc4c1f8eae8dbb60596289fef49bb13a8b53b76
380e365563 guix: switch to 6.1 kernel headers over 5.15 (fanquake)
Pull request description:
6.1 is the current longterm release: https://kernel.org/.
Note that using an older version of the kernel headers inside Guix, is not a "hack" for compatibility, and is explicitly recommended against by glibc:
https://sourceware.org/glibc/wiki/FAQ#What_version_of_the_Linux_kernel_headers_should_be_used.3F.
so using the latest version of the longterm headers seems appropriate.
The last time we changed this was when we consolidated all builds to 5.15, in #25006.
ACKs for top commit:
TheCharlatan:
ACK 380e365563
Tree-SHA512: 78eb601e10261d99afd030dd7d039d962c106c48a57f16deb1c65b68fee4831e1070e4c35201f567fd24bbdab30a2b00804ddd118e1fee1dc8cdac7a3fb32ac5
22e38080ea test: fix node index bug when comparing peerinfo (Kashif Smith)
Pull request description:
fix node index bug when comparing peerinfo in test/functional/p2p_v2_transport.py
ACKs for top commit:
theStack:
ACK 22e38080ea
mzumsande:
ACK 22e38080ea, good find!
Tree-SHA512: 9ee336eea999c61fb9f8704cc6361cf289fd3a361ab636c97695121ca3bcb8b38fbbfb55484311c17faa76d02065d91d190c489e1f3defd628216bf80a93f1fe
Allows calling UpdateLockPoints() with a (const) txiter. Note that this
was already possible for caller using mapTx.modify(txiter). The point
here is to not be accessing mapTx when doing so.
The behavior is the same as CalculateMemPoolAncestors. The only
difference is the string returned, and the string is discarded anyway
since checkChainLimits only cares about pass/fail.
7de7685372 test, assumeutxo: Use assert_debug_log for error details (pablomartin4btc)
Pull request description:
This is a follow-up on the invalid hash dump fix#28698, [suggested](https://github.com/bitcoin/bitcoin/pull/28698#pullrequestreview-1698178157) by theStack and agreed by Sjors and ryanofsky.
ACKs for top commit:
Sjors:
ACK 7de7685372
maflcko:
lgtm ACK 7de7685372
Tree-SHA512: 036b3cef3084e3ead8923e8dcabe4fa7ebe97fb514d223aa38bc38df10337e3fe3113e42322178b58fb03fcd4511af4b5b56bceecbb7ded5b9758842c70db3f2
This is deprecated on macOS:
```bash
ld: warning: -bind_at_load is deprecated on macOS
```
and likely redundant anyways, given the behaviour of dyld3.
Unfortunately libtool is still injecting a `-bind_at_load`:
```bash
# Don't allow lazy linking, it breaks C++ global constructors
# But is supposedly fixed on 10.4 or later (yay!).
if test CXX = "$tagname"; then
case ${MACOSX_DEPLOYMENT_TARGET-10.0} in
10.[0123])
func_append compile_command " $wl-bind_at_load"
func_append finalize_command " $wl-bind_at_load"
;;
esac
fi
```
so this doesn't remove all the warnings, but removes us as a potential
source of them.
Note that anywhere the ld64 warnings are being emitted, we are already
not adding this flag to our hardened ldflags, because of `-Wl,-fatal_warnings`.
9f208c0171 ci: Switch IWYU to `clang_17` branch (Hennadii Stepanov)
Pull request description:
The IWYU version [0.21](https://github.com/include-what-you-use/include-what-you-use/releases/tag/0.21) has been tagged, and the `clang_17` branch is available now.
ACKs for top commit:
maflcko:
lgtm ACK 9f208c0171
Tree-SHA512: 8b8f8743d1c2719b6383b5a6a48356ac02a301d1ce9cee77f93cc04c12de22e9ac6b59e23550a589540e68292cfac0d85bacedc9ca26f6b589011d36ee1d38cf
5f0bf2ef69 ci: win64 task does use boost:process (fanquake)
Pull request description:
It passes `--enable-external-signer`.
ACKs for top commit:
maflcko:
lgtm ACK 5f0bf2ef69
Tree-SHA512: 789877aac0d36429f31256adc07812d1914a8a059a43ef22416be97270b083902c253ff0561b3de28e76db005387f14b2712bfcfb1334f69b293c39ce0e7467c
faa2ad88bc test: Add missing wait for version to be sent in add_outbound_p2p_connection (MarcoFalke)
Pull request description:
Can be tested with:
```diff
diff --git a/test/functional/test_framework/p2p.py b/test/functional/test_framework/p2p.py
index b1ed97b794..eb4f72c6b6 100755
--- a/test/functional/test_framework/p2p.py
+++ b/test/functional/test_framework/p2p.py
@@ -205,6 +205,7 @@ class P2PConnection(asyncio.Protocol):
assert not self._transport
logger.debug("Connected & Listening: %s:%d" % (self.dstaddr, self.dstport))
self._transport = transport
+ import time;time.sleep(.1);
if self.on_connection_send_msg:
self.send_message(self.on_connection_send_msg)
self.on_connection_send_msg = None # Never used again
```
Found and reported by mzumsande in https://github.com/bitcoin/bitcoin/pull/28782#pullrequestreview-1718560252
ACKs for top commit:
mzumsande:
ACK faa2ad88bc
Tree-SHA512: 863f06125dec40cccaa852d0d8ca2e2b9c0b74610205e9fd6c9c279bdf36801ff475e3d873fd1b18172eb8220e17b2caff60069ce63512e569934a43f27d03fd
- "transport_protocol_type" of inbound peer before version handshake
is "detecting" on p2p v2 nodes (as opposed to "v1" for p2p v1)
- size of a ping/pong message is 29 bytes (as opposed to 32 for p2p v1)
- for the sendmsgtopeer RPC sub-test, enforce p2p v1 connection to
have a peer id of zero
By renaming the "command" send_cli arg. The old name was unsuitable
because the "addnode" RPC has its own "command" arg, leading to
ambiguity when included in kwargs.
Can be tested with
"python3 wallet_multiwallet.py --usecli --v2transport"
which fails on master because of this (python throws a TypeError).
Before, a global -v2transport provided to the test would be dropped
when restarting the node within a test and specifying any extra_args.
Fix this by adding "v2transport=1" to args (not extra_args) based
on the global parameter, and deciding for each (re)start of the node
based on this default and test-specific extra_args
(which take precedence over args) whether v2 should be used.
8cbb619691 ci: remove note re M1 usage (fanquake)
Pull request description:
M1 is now available in GitHub CI, but we don't currently have a plan to use it, so remove the comment.
ACKs for top commit:
maflcko:
lgtm ACK 8cbb619691
achow101:
ACK 8cbb619691
hebasto:
ACK 8cbb619691.
Tree-SHA512: 13bbd4ad2358b0df6781031d6bdba456ffe706f30bf273a317ea8031f28276ef5821b5f767e4fb47d444b4e9ad7b8b7f67563927838552d275aca481d0e2fc2f
5039c346ca init: completely remove `-zapwallettxes` (remaining hidden option) (Sebastian Falbesoner)
Pull request description:
The `-zapwallettxes` functionality has been removed in v0.21.0 (see commit 3340dbadd3 / PR #19671), with the parameter being kept as hidden option, to inform users via an exit error that `abandontransaction` should be used instead.
As any guides that still suggest to use `-zapwallettxes` would refer to a Bitcoin Core version that is EOL since many years (i.e. <= v0.20.x), it is highly unlikely that the error caused by the option is still relevant for any user, hence it seems fine to remove it now.
ACKs for top commit:
achow101:
ACK 5039c346ca
BrandonOdiwuor:
ACK 5039c346ca
fanquake:
ACK 5039c346ca
Tree-SHA512: e3ccc6918e0f8fa68dbd1a7ec4999cc2a44e28038711919fcddaf0727648c73a9ba0fb77674317147592a113fad20755d4e727f48176bc17b048fbdebad2d6c9
fabb5046a7 fuzz: Avoid timeout and bloat in fuzz targets (MarcoFalke)
Pull request description:
If the fuzz input contains invalid data *in a loop*, abort early. This will teach the fuzz engine to look for useful data and avoids bloating the fuzz input folder with useless (repeated) data.
ACKs for top commit:
dergoegge:
utACK fabb5046a7
brunoerg:
crACK fabb5046a7
Tree-SHA512: 26da100d7558ae6fdd5292fb146d8858b2af8f78c546ca2509b9d27b33a33e9462ecb6035de142f9f36dd5de32f8cbad099d6c7a697902d23e1bb621cd27dc88
0420f99f42 Create net_peer_connection unit tests (Jon Atack)
4b834f6499 Allow unit tests to access additional CConnman members (Jon Atack)
34b9ef443b net/rpc: Makes CConnman::GetAddedNodeInfo able to return only non-connected address on request (Sergi Delgado Segura)
94e8882d82 rpc: Prevents adding the same ip more than once when formatted differently (Sergi Delgado Segura)
2574b7e177 net/rpc: Check all resolved addresses in ConnectNode rather than just one (Sergi Delgado Segura)
Pull request description:
## Rationale
Currently, `addnode` has a couple of corner cases that allow it to either connect to the same peer more than once, hence wasting outbound connection slots, or add redundant information to `m_added_nodes`, hence making Bitcoin iterate through useless data on a regular basis.
### Connecting to the same node more than once
In general, connecting to the same node more than once is something we should try to prevent. Currently, this is possible via `addnode` in two different ways:
1. Calling `addnode` more than once in a short time period, using two equivalent but distinct addresses
2. Calling `addnode add` using an IP, and `addnode onetry` after with an address that resolved to the same IP
For the former, the issue boils down to `CConnman::ThreadOpenAddedConnections` calling `CConnman::GetAddedNodeInfo` once, and iterating over the result to open connections (`CConman::OpenNetworkConnection`) on the same loop for all addresses.`CConnman::ConnectNode` only checks a single address, at random, when resolving from a hostname, and uses it to check whether we are already connected to it.
An example to test this would be calling:
```
bitcoin-cli addnode "127.0.0.1:port" add
bitcoin-cli addnode "localhost:port" add
```
And check how it allows us to perform both connections some times, and some times it fails.
The latter boils down to the same issue, but takes advantage of `onetry` bypassing the `CConnman::ThreadOpenAddedConnections` logic and calling `CConnman::OpenNetworkConnection` straightaway. A way to test this would be:
```
bitcoin-cli addnode "127.0.0.1:port" add
bitcoin-cli addnode "localhost:port" onetry
```
### Adding the same peer with two different, yet equivalent, addresses
The current implementation of `addnode` is pretty naive when checking what data is added to `m_added_nodes`. Given the collection stores strings, the checks at `CConnman::AddNode()` basically check wether the exact provided string is already in the collection. If so, the data is rejected, otherwise, it is accepted. However, ips can be formatted in several ways that would bypass those checks.
Two examples would be `127.0.0.1` being equal to `127.1` and `[::1]` being equal to `[0:0:0:0:0:0:0:1]`. Adding any pair of these will be allowed by the rpc command, and both will be reported as connected by `getaddednodeinfo`, given they map to the same `CService`.
This is less severe than the previous issue, since even tough both nodes are reported as connected by `getaddednodeinfo`, there is only a single connection to them (as properly reported by `getpeerinfo`). However, this adds redundant data to `m_added_nodes`, which is undesirable.
### Parametrize `CConnman::GetAddedNodeInfo`
Finally, this PR also parametrizes `CConnman::GetAddedNodeInfo` so it returns either all added nodes info, or only info about the nodes we are **not** connected to. This method is used both for `rpc`, in `getaddednodeinfo`, in which we are reporting all data to the user, so the former applies, and to check what nodes we are not connected to, in `CConnman::ThreadOpenAddedConnections`, in which we are currently returning more data than needed and then actively filtering using `CService.fConnected()`
ACKs for top commit:
jonatack:
re-ACK 0420f99f42
kashifs:
> > tACK [0420f9](0420f99f42)
sr-gi:
> > > tACK [0420f9](0420f99f42)
mzumsande:
Tested ACK 0420f99f42
Tree-SHA512: a3a10e748c12d98d439dfb193c75bc8d9486717cda5f41560f5c0ace1baef523d001d5e7eabac9fa466a9159a30bb925cc1327c2d6c4efb89dcaf54e176d1752
1147e00e59 [validation] change package-fee-too-low, return wtxid(s) and effective feerate (glozow)
10dd9f2441 [test] use CheckPackageMempoolAcceptResult in previous tests (glozow)
3979f1afcb [validation] add TxValidationResult::TX_RECONSIDERABLE, TX_UNKNOWN (glozow)
5c786a026a [refactor] use Wtxid for m_wtxids_fee_calculations (glozow)
Pull request description:
Split off from #26711 (suggested in https://github.com/bitcoin/bitcoin/pull/26711#issuecomment-1786392253). This is part of #27463.
- Add 2 new TxValidationResults
- `TX_RECONSIDERABLE` helps us encode transactions who have failed fee checks that can be bypassed using package validation. This is distinguished from `TX_MEMPOOL_POLICY` so that we re-validate a transaction if and only if it is eligible for package CPFP. In the future, we will have a separate cache for reconsiderable rejects so these transactions don't go in `m_recent_rejects`.
- `TX_UNKNOWN` helps us communicate that we aborted package validation and didn't finish looking at this transaction: it's not valid but it's also not invalid (i.e. don't cache it as a rejected tx)
- Return effective feerate and the wtxids of transactions used to calculate that effective feerate when the error is `TX_SINGLE_FAILURE`. Previously, we would only provide this information if the transaction passed. Now that we have package validation, it's much more helpful to the caller to know how the failing feerate was calculated. This can also be used to improve our submitpackage RPC result (which is currently a bit unhelpful when things fail).
- Use the newly added `CheckPackageMempoolAcceptResult` for existing package validation tests. This increases test coverage and helps test the changes made in this PR.
ACKs for top commit:
instagibbs:
reACK 1147e00e59
achow101:
ACK 1147e00e59
murchandamus:
reACK 1147e00e59
ismaelsadeeq:
ACK 1147e00e59
Tree-SHA512: ac1cd73c2b487a1b99d329875d39d8107c91345a5b0b241d54a6a4de67faf11be69a2721cc732c503024a9cca381dac33d61e187957279e3c82653bea118ba91
6559e4d27a tests: Increase wallet_miniscript.py rpc timeout to 90 seconds (Andrew Chow)
Pull request description:
The signing test for the large miniscript can sometimes take longer than the 30 second timeout, depending on the load on my system. Increasing it to 90 seconds seems to be good enough.
ACKs for top commit:
kevkevinpal:
but increasing seems fine ACK [6559e4d](6559e4d27a)
maflcko:
lgtm ACK 6559e4d27a
Tree-SHA512: 1b7bf94c77f85a0deddb1384aacbeb934205d0a630fecc8e75a4a98d1946d77d9bca36692fb6c1ab8e9276392f617281aafc4c685c248a8d3b0c77f896cda624
fa02598469 test: Add missing sync on send_version in peer_connect (MarcoFalke)
Pull request description:
Without the sync, the logic will be racy. For example, `p2p_sendtxrcncl.py` is failing locally (and on CI occasionally), because non-version messages will be sent before the version message:
```py
self.log.info('SENDTXRCNCL with version=0 triggers a disconnect')
sendtxrcncl_low_version = create_sendtxrcncl_msg()
sendtxrcncl_low_version.version = 0
peer = self.nodes[0].add_p2p_connection(PeerNoVerack(), send_version=True, wait_for_verack=False)
with self.nodes[0].assert_debug_log(["txreconciliation protocol violation"]):
peer.send_message(sendtxrcncl_low_version)
peer.wait_for_disconnect()
```
```
test 2023-11-02T08:15:19.620000Z TestFramework (INFO): SENDTXRCNCL with version=0 triggers a disconnect
test 2023-11-02T08:15:19.621000Z TestFramework.p2p (DEBUG): Connecting to Bitcoin Node: 127.0.0.1:11312
test 2023-11-02T08:15:19.624000Z TestFramework.p2p (DEBUG): Connected & Listening: 127.0.0.1:11312
test 2023-11-02T08:15:19.798000Z TestFramework.p2p (DEBUG): Send message to 127.0.0.1:11312: msg_sendtxrcncl(version=0, salt=2)
test 2023-11-02T08:15:19.799000Z TestFramework.p2p (DEBUG): Send message to 127.0.0.1:11312: msg_version(nVersion=70016 nServices=9 nTime=Thu Nov 2 08:15:19 2023 addrTo=CAddress(nServices=1 net=IPv4 addr=127.0.0.1 port=11312) addrFrom=CAddress(nServices=1 net=IPv4 addr=0.0.0.0 port=0) nNonce=0x369AC031CDA96022 strSubVer=/python-p2p-tester:0.0.3/ nStartingHeight=-1 relay=1)
node0 2023-11-02T08:15:19.804409Z [net] [net.cpp:3676] [CNode] [net] Added connection peer=0
node0 2023-11-02T08:15:19.805256Z [net] [net.cpp:1825] [CreateNodeFromAcceptedSocket] [net] connection from 127.0.0.1:55964 accepted
node0 2023-11-02T08:15:19.809861Z [msghand] [net_processing.cpp:3356] [ProcessMessage] [net] received: sendtxrcncl (12 bytes) peer=0
node0 2023-11-02T08:15:19.810297Z [msghand] [net_processing.cpp:3582] [ProcessMessage] [net] non-version message before version handshake. Message "sendtxrcncl" from peer=0
node0 2023-11-02T08:15:19.810928Z [msghand] [net_processing.cpp:3356] [ProcessMessage] [net] received: version (111 bytes) peer=0
...
test 2023-11-02T09:35:20.166000Z TestFramework.utils (ERROR): wait_until() failed. Predicate: ''''
def test_function():
if check_connected:
assert self.is_connected
return test_function_in()
'''
test 2023-11-02T09:35:20.187000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/ci_container_base/ci/scratch/build/bitcoin-s390x-linux-gnu/test/functional/test_framework/test_framework.py", line 132, in main
self.run_test()
File "/ci_container_base/ci/scratch/build/bitcoin-s390x-linux-gnu/test/functional/p2p_sendtxrcncl.py", line 188, in run_test
peer.wait_for_disconnect()
File "/ci_container_base/ci/scratch/build/bitcoin-s390x-linux-gnu/test/functional/test_framework/p2p.py", line 478, in wait_for_disconnect
self.wait_until(test_function, timeout=timeout, check_connected=False)
File "/ci_container_base/ci/scratch/build/bitcoin-s390x-linux-gnu/test/functional/test_framework/p2p.py", line 470, in wait_until
wait_until_helper_internal(test_function, timeout=timeout, lock=p2p_lock, timeout_factor=self.timeout_factor)
File "/ci_container_base/ci/scratch/build/bitcoin-s390x-linux-gnu/test/functional/test_framework/util.py", line 275, in wait_until_helper_internal
raise AssertionError("Predicate {} not true after {} seconds".format(predicate_source, timeout))
AssertionError: Predicate ''''
def test_function():
if check_connected:
assert self.is_connected
return test_function_in()
''' not true after 4800.0 seconds
ACKs for top commit:
mzumsande:
ACK fa02598469
Tree-SHA512: 78871f603d387e2df8c0acbdfa95441fa186f80e94593021bb219bbf1bc9dc7efc4e266bd254b5cc41114c38227ff3b7f6172335d9bb828427f0a2acffde752d
49d953281d fuzz: explicitly specify llvm-symbolizer path in runner (fanquake)
Pull request description:
It's not completely clear to me why this needs to be explicitly specified in some environments, and not in others, while at the same time that `llvm-symbolizer` is already in PATH, but this has fixed the 2 issues outlined in https://github.com/bitcoin/bitcoin/pull/28147.
Use `LLVM_SYMBOLIZER_PATH` as the env var, as that is somewhat also used inside LLVM, but not consistently, i.e it's checked for in the asan_symbolize script, but not in in the ubsan_symbolize script, or from in compiler-rt.
Alternative to #28804.
ACKs for top commit:
maflcko:
lgtm ACK 49d953281d
Tree-SHA512: c3d5bf1c3629793b342c70754a419b3c7a3cd39f800b9aa69ce3395cc2bf83b4d46f2b329974337b94b99573cd0b8600d3f147ed5c21387bf3812316570d1ee3
The signing test for the large miniscript can sometimes take longer than
the 30 second timeout, depending on the load on my system. Increasing it
to 90 seconds seems to be good enough.
df69b22f2e doc: improve documentation around connection limit maximums (Amiti Uttarwar)
adc171edf4 scripted-diff: Rename connection limit variables (Amiti Uttarwar)
e9fd9c0225 net: add m_max_inbound to connman (Amiti Uttarwar)
c25e0e0555 net, refactor: move calculations for connection type limits into connman (Amiti Uttarwar)
Pull request description:
This is joint work with amitiuttarwar.
This has the first few commits of #28463. It is not strictly a prerequisite for that, but has changes that in our opinion make sense on their own.
It improves the handling of maximum numbers for different connection types (that are set during init and don’t change after) by:
* moving all calculations into one place, `CConnMan::Init()`. Before, they were dispersed between `Init`, `CConnman::Init` and other parts of `CConnman`, resulting in some duplicated test code.
* removing the possibility of having a negative maximum of inbound connections, which is hard to argue about
* renaming of variables and doc improvements
ACKs for top commit:
amitiuttarwar:
co-author review ACK df69b22f2e
naumenkogs:
ACK df69b22f2e
achow101:
ACK df69b22f2e
Tree-SHA512: 913d56136bc1df739978de50db67302f88bac2a9d34748ae96763288d97093e998fc0f94f9b6eff12867712d7e86225af6128f4170bf2b5b8ab76f024870a22c
c1144f0076 tests: Reset node context members on ~BasicTestingSetup (TheCharlatan)
9759af17ff shutdown: Destroy kernel last (TheCharlatan)
Pull request description:
The destruction/resetting of node context members in the tests should roughly follow the behavior of the `Shutdown` function in `init.cpp`.
This was originally requested by MarcoFalke in this [comment](https://github.com/bitcoin/bitcoin/pull/25065#discussion_r890161249) in response to the [original pull request](https://github.com/bitcoin/bitcoin/pull/25065) introducing the `kernel::Context`.
ACKs for top commit:
maflcko:
ACK c1144f0076 🗣
achow101:
ACK c1144f0076
ryanofsky:
Code review ACK c1144f0076. No code changes since last review, just updated commits and descriptions
Tree-SHA512: 819bb85ff82a5c6c60e429674d5684f3692fe9062500d00a87b361cc59e6bda145be21b5a4466dee6791faed910cbde4d26baab325bf6daa1813af13a63588ff
aee5404e02 Add support for RNDR/RNDRRS for aarch64 on Linux (John Moffett)
Pull request description:
This checks whether the ARMv8.5-A optional TRNG extensions [RNDR](https://developer.arm.com/documentation/ddi0601/2022-12/AArch64-Registers/RNDR--Random-Number) and [RNDRRS](https://developer.arm.com/documentation/ddi0601/2022-12/AArch64-Registers/RNDRRS--Reseeded-Random-Number) are available and, if they are, uses them for random entropy purposes.
They are nearly functionally identical to the x86 RDRAND/RDSEED extensions and are used in a similar manner.
Currently, there [appears to be](https://marcin.juszkiewicz.com.pl/download/tables/arm-socs.html) only one actual hardware implementation -- the Amazon Graviton 3. (See the `rnd` column in the link.) However, future hardware implementations may become available.
It's not possible to directly query for the capability in userspace, but the Linux kernel [added support](1a50ec0b3b) for querying the extension via `getauxval` in version 5.6 (in 2020), so this is limited to Linux-only for now.
Reviewers may want to launch any of the `c7g` instances from AWS to test the Graviton 3 hardware. Alternatively, QEMU emulates these opcodes for `aarch64` with CPU setting `max`.
Output from Graviton 3 hardware:
```
ubuntu@ip:~/bitcoin$ src/bitcoind -regtest
2023-01-06T20:01:48Z Bitcoin Core version v24.99.0-3670266ce89a (release build)
2023-01-06T20:01:48Z Using the 'arm_shani(1way,2way)' SHA256 implementation
2023-01-06T20:01:48Z Using RNDR and RNDRRS as additional entropy sources
2023-01-06T20:01:48Z Default data directory /home/ubuntu/.bitcoin
```
Graviton 2 (doesn't support extensions):
```
ubuntu@ip:~/bitcoin$ src/bitcoind -regtest
2023-01-06T20:05:04Z Bitcoin Core version v24.99.0-3670266ce89a (release build)
2023-01-06T20:05:04Z Using the 'arm_shani(1way,2way)' SHA256 implementation
2023-01-06T20:05:04Z Default data directory /home/ubuntu/.bitcoin
```
This partially closes#26796. As noted in that issue, OpenSSL [added support](https://github.com/openssl/openssl/pull/15361) for these extensions a little over a year ago.
ACKs for top commit:
achow101:
ACK aee5404e02
laanwj:
Tested ACK aee5404e02
Tree-SHA512: 1c1eb345d6690f5307a87e9bac8f06a0d1fdc7ca35db38fa22192510a44289a03252e4677dc7cbf731a27e6e3a9a4e42b6eb4149fe063bc1c905eb2536cdb1d3
bbb68ffdbd refactor: drop protocol.h include header in rpc/util.h (Jon Atack)
1dd62c5295 refactor: move GetServicesNames from rpc/util.{h,cpp} to rpc/net.cpp (Jon Atack)
Pull request description:
Move `GetServicesNames()` from `rpc/util` to `rpc/net.cpp`, as it is only called from that compilation unit and there is no reason for other ones to need it.
Remove the `protocol.h` include in `rpc/util.h`, as it was only needed for `GetServicesNames()`, drop an unneeded forward declaration (the other IWYU suggestions would require more extensive changes in other files), and add 3 already-missing include headers in other translation units that are needed to compile without `protocol.h` in `rpc/util.h`, as `protocol.h` includes `netaddress.h`, which in turn includes `util/strencodings.h`.
ACKs for top commit:
kevkevinpal:
lgtm ACK [bbb68ff](bbb68ffdbd)
ns-xvrn:
ACK bbb68ff
achow101:
ACK bbb68ffdbd
Tree-SHA512: fcbe195874dd4aa9e86548685b6b28595a2c46f9869b79b6e2b3835f76b49cab4bef6a59c8ad6428063a41b7bb6f687229b06ea614fbd103e0531104af7de55d
af0fca530e netbase: use reliable send() during SOCKS5 handshake (Vasil Dimov)
1b19d1117c sock: change Sock::SendComplete() to take Span (Vasil Dimov)
Pull request description:
The `Socks5()` function which does the SOCKS5 handshake with the SOCKS5 proxy sends bytes to the socket without retrying partial writes.
`send(2)` may write only part of the provided data and return. In this case the caller is responsible for retrying the operation with the remaining data. Change `Socks5()` to do that. There is already a method `Sock::SendComplete()` which does exactly that, so use it in `Socks5()`.
A minor complication for this PR is that `Sock::SendComplete()` takes `std::string` argument whereas `Socks5()` has `std::vector<uint8_t>`. Thus the necessity for the first commit. It is possible to do also in other ways - convert the data in `Socks5()` to `std::string` or have just one `Sock::SendComplete()` that takes `void*` and change the callers to pass `str.data(), str.size()` or `vec.data(), vec.size()`.
This came up while testing https://github.com/bitcoin/bitcoin/pull/27375.
ACKs for top commit:
achow101:
ACK af0fca530e
jonatack:
ACK af0fca530e
pinheadmz:
ACK af0fca530e
Tree-SHA512: 1d4a53d0628f7607378038ac56dc3b8624ce9322b034c9547a0c3ce052eafb4b18213f258aa3b57bcb4d990a5e0548a37ec70af2bd55f6e8e6399936f1ce047a
It's not completely clear to me why this needs to be explicitly
specified in some environments, and not in others, while at the same time
that `llvm-symbolizer` is already in PATH, but this has fixed the 2 issues
outlined in #28147.
Use `LLVM_SYMBOLIZER_PATH` as the env var, as that is somewhat also used
inside LLVM, but not consistently, i.e it's checked for in the asan_symbolize
script, but not in in the ubsan_symbolize script, or from in compiler-rt.
f06016d77d wallet: Add asserts to detect unset transaction height values (Ryan Ofsky)
262a78b133 wallet, refactor: Add CWalletTx::updateState function (Ryan Ofsky)
Pull request description:
Originally, this PR fixed a wallet migration bug that could cause the watchonly wallet created by legacy wallet migration to have incorrect transaction height values. A different fix for the bug was implemented in #28609, but that PR did not add any test coverage that would have caught the bug, and didn't include other changes from this PR intended to prevent problems from invalid transaction heights.
This PR adds new asserts to catch invalid transaction heights, which would trigger test failures without bugfix in #28609. This PR also refactors code and adds comments to clarify assumptions and make it less likely a bug from invalid transaction height values would be introduced.
ACKs for top commit:
achow101:
ACK f06016d77d
Sjors:
utACK f06016d77d
furszy:
Code review ACK f06016d
Tree-SHA512: 82657c403724d60354f7676b53bcfcc95bdc5864e051a2eb8bfad09d8ad35615393b2d6b432b46f908def9be37bebded3a55ec9ae19e19371d35897fe842c92e
With subpackage evaluation and de-duplication, it's not always the
entire package that is used in CheckFeerate. To be more helpful to the
caller, specify which transactions were included in the evaluation and
what the feerate was.
Instead of PCKG_POLICY (which is supposed to be for package-wide
errors), use PCKG_TX.
fa7ba92630 fuzz: Avoid utxo_total_supply timeout (MarcoFalke)
Pull request description:
Looks like this still may take a long time to run large fuzz inputs. Thus, reduce it further, but still allow it to catch the regression, if re-introduced:
```diff
diff --git a/src/consensus/tx_check.cpp b/src/consensus/tx_check.cpp
index f949655909..4bdd15c5ee 100644
--- a/src/consensus/tx_check.cpp
+++ b/src/consensus/tx_check.cpp
@@ -40,7 +40,7 @@ bool CheckTransaction(const CTransaction& tx, TxValidationState& state)
std::set<COutPoint> vInOutPoints;
for (const auto& txin : tx.vin) {
if (!vInOutPoints.insert(txin.prevout).second)
- return state.Invalid(TxValidationResult::TX_CONSENSUS, "bad-txns-inputs-duplicate");
+ {}//return state.Invalid(TxValidationResult::TX_CONSENSUS, "bad-txns-inputs-duplicate");
}
if (tx.IsCoinBase())
```
This is the second take, see https://github.com/bitcoin/bitcoin/pull/27780. If in the future it still times out, I think the fuzz test can just be removed.
Example input:
```
JREROy5pcnAgQyw7IC4ODg4ODg4ODg4O0dEODg4ODg4ZDg4ODg4ODg4ODg7RDg4ODg4ODg4O0dEODg4ODg4ODg4ODg7R0Q4ODg4ODg4ODtHRDg4ODtHR0dEODg4O0dEODg7R0Q4ODg4ODg4ODtHRDg4ODg4ODg4ODg4O0dEODg4ODg4ODg7R0Q4ODg7R0Q4O0dEODg4ODg4ODg4ODg7R0Q4ODg4ODtHRDg4ODtHR
ACKs for top commit:
dergoegge:
ACK fa7ba92630
brunoerg:
utACK fa7ba92630
Tree-SHA512: 154a4895834babede6ce7b775562a7026637af1097e53e55676e92f6cf966ae0c092300ebf7e51a397eebd11f7b41d020586663e781f70d084efda1c0fe851b4
5e6bc6d830 test: remove custom rpc timeout for `wallet_miniscript.py`, reorder in test_runner (Sebastian Falbesoner)
f811a24421 wallet: cache descriptor ID to avoid repeated descriptor string creation (Sebastian Falbesoner)
Pull request description:
Right now a wallet descriptor is converted to its string representation (via `Descriptor::ToString`) repeatedly at different instances:
- on finding a `DescriptorScriptPubKeyMan` for a given descriptor (`CWallet::GetDescriptorScriptPubKeyMan`, e.g. used by the `importdescriptors` RPC); the string representation is created once for each spkm in the wallet and at each iteration again for the searched descriptor (`DescriptorScriptPubKeyMan::HasWalletDescriptor`)
- whenever `DescriptorScriptPubKeyMan::GetID()` is called, e.g. in `TopUp` or any instances where a descriptor is written to the DB to determine the database key, also at less obvious places like `FastWalletRescanFilter` etc.
As there is no good reason to calculate a fixed descriptor's string/ID more than once, add the ID as a field to `WalletDescriptor` and calculate it immediately at initialization (or deserialization). `HasWalletDescriptor` is changed to compare the spkm's and searched descriptor's ID instead of the string to take use of that.
This speeds up the functional test `wallet_miniscript.py` by a factor of 5-6x on my machine (3m30.95s on master vs. 0m38.02s on PR). The recently introduced "max-size TapMiniscript" test-case introduced a descriptor that takes 2-3 seconds to create a string representation, so the repeated calls to that were significantly hurting the performance.
Fixes https://github.com/bitcoin/bitcoin/issues/28800.
ACKs for top commit:
Sjors:
ACK 5e6bc6d830
S3RK:
Code Review ACK 5e6bc6d830
achow101:
ACK 5e6bc6d830
BrandonOdiwuor:
ACK 5e6bc6d830
Tree-SHA512: 98b43963a5dde6055bb26cecd3b878dadd837d6226af4c84142383310495da80b3c4bd552e73b9107f2f2ff1c11f5e18060c6fd3d9e44bbd5224114c4d245c1c
3c208cc05e Add offline signing tutorial (Brandon Odiwuor)
Pull request description:
This PR adds offline signing tutorial. Fixes https://github.com/bitcoin/bitcoin/issues/9492
Although there currently exists tutorials on external-signer and on multisig implemented on #24519 . The external-signer tutorial assumes a connected device and the multisig tutorial is only for multisig transactions and does not include using an offline wallet
- The tutorial uses signet(instead of regtest) to be as close as possible to mainnet
ACKs for top commit:
achow101:
ACK 3c208cc05e
willcl-ark:
ACK 3c208cc05e
pinheadmz:
ACK 3c208cc05e
Zero-1729:
ACK 3c208cc05e
Tree-SHA512: c1686043d9e9ed440e78d219a6b18d58d62efd05bdd535e74194d8cc2db0a91e94c6c619106453120a137e47220cf3ab27af3214e861f4e5cc419a73a8704dd6
With package validation rules, transactions that fail individually may
sometimes be eligible for reconsideration if submitted as part of a
(different) package. For now, that includes trasactions that failed for
being too low feerate. Add a new TxValidationResult type to distinguish
these failures from others. In the next commits, we will abort package
validation if a tx fails for any other reason. In the future, we will
also decide whether to cache failures in recent_rejects based on this
result (we won't want to reject a package containing a transaction that
was rejected previously for being low feerate).
Package validation also sometimes elects to skip some transactions when
it knows the package will not be submitted in order to quit sooner. Add
a result to specify this situation; we also don't want to cache these
as rejections.
Right now a wallet descriptor is converted to it's string representation
(via `Descriptor::ToString`) repeatedly at different instances:
- on finding a `DescriptorScriptPubKeyMan` for a given descriptor
(`CWallet::GetDescriptorScriptPubKeyMan`, e.g. used by the
`importdescriptors` RPC); the string representation is created once
for each spkm in the wallet and at each iteration again for
the searched descriptor (`DescriptorScriptPubKeyMan::HasWalletDescriptor`)
- whenever `DescriptorScriptPubKeyMan::GetID()` is called, e.g. in
`TopUp` or any instances where a descriptor is written to the DB
to determine the database key etc.
As there is no good reason to calculate a fixed descriptor's string/ID
more than once, add the ID as a field to `WalletDescriptor` and
calculate it immediately at initialization (or deserialization).
`HasWalletDescriptor` is changed to compare the spkm's and searched
descriptor's ID instead of the string to take use of that.
This speeds up the functional test `wallet_miniscript.py` by a factor of
5-6x on my machine (3m30.95s on master vs. 0m38.02s on PR). The recently
introduced "max-size TapMiniscript" test-case introduced a descriptor
that takes 2-3 seconds to create a string representation, so the
repeated calls to that were significantly hurting the performance.
3333f14efa depends: Bump to capnproto-c++-1.0.1 (MarcoFalke)
Pull request description:
Reasons:
* Debian is starting to ship this version in Trixie (https://packages.debian.org/trixie/capnproto), which will likely become the version shipped with Ubuntu 24.04 LTS. So testing with this version will help to find any issues before real users start to use those distro packages.
* The feature is currently experimental, so bumping the version shouldn't cause any production issues.
* With multiprocess begin a priority project for 27.0, it seems better to do build system changes/bumps early, rather than later, to allow for more time testing them.
ACKs for top commit:
TheCharlatan:
Re-ACK 3333f14efa
fanquake:
ACK 3333f14efa - the response from upstream is that [if we submit a PR, they can take a look](https://github.com/capnproto/capnproto/issues/1833#issuecomment-1792582206), so if anyone would like this to work for Windows, I'd suggest sending a patch.
ryanofsky:
Code review ACK 3333f14efa
Tree-SHA512: 7d53ad1536f042ab43dbc7847126b826e7fc76694f173c348b835fd1067b8f3dd682c5bcb4887f09ee85bab69130721cd7f8fb96b2e82053d4e28bd5c38bdc5f
5bd1b8d4f1 ci: Drop no longer needed "Fix Visual Studio installation" step (Hennadii Stepanov)
Pull request description:
The underlying issue has been [fixed](https://github.com/actions/runner-images/pull/8686) in the image version 20231029.
ACKs for top commit:
maflcko:
lgtm ACK 5bd1b8d4f1
Tree-SHA512: d0efef3086a147d863c9b5f45ba1142c6e7cc65e47d685b2094211e58036315fb7562253b7d7172b527fa1ded4b5a86634ba7c151e761ec20fe948145aff83fe
The `-zapwallettxes` functionality has been removed in v0.21.0
(see commit 3340dbadd3 / PR #19671),
with the parameter being kept as hidden option, to inform users via
an exit error that `abandontransaction` should be used instead.
As any guides that still suggest to use `-zapwallettxes` would refer to
a Bitcoin Core version that is EOL since many years (i.e. <= v0.20.x),
it is highly unlikely that the error caused by the option is still
relevant for any user, hence it seems fine to remove it now.
Otherwise, starting bitcoind twice may cause the `.cookie`
file generated by the first instance to be deleted by the
second instance shutdown (after failing to obtain a lock).
d9cc99d04e [test] MiniMiner::Linearize and manual construction (glozow)
dfd6a3788c [refactor] unify fee amounts in miniminer_tests (glozow)
f4b1b24a3b [MiniMiner] track inclusion order and add Linearize() function (glozow)
004075963f [test] add case for MiniMiner working with negative fee txns (glozow)
fe6332c0ba [MiniMiner] make target_feerate optional (glozow)
5a83f55c96 [MiniMiner] allow manual construction with non-mempool txns (glozow)
e3b2e630b2 [refactor] change MiniMinerMempoolEntry ctor to take values, update includes (glozow)
4aa98b79b2 [lint] update expected boost includes (glozow)
Pull request description:
This is part of #27463. It splits off the `MiniMiner`-specific changes from #26711 for ease of review, as suggested in https://github.com/bitcoin/bitcoin/pull/26711#issuecomment-1786392253.
- Allow using `MiniMiner` on transactions that aren't in the mempool.
- Make `target_feerate` param of `BuildMockTemplate` optional, meaning "don't stop building the template until all the transactions have been selected."
- Add clarification for how this is different from `target_feerate=0` (https://github.com/bitcoin/bitcoin/pull/26711#discussion_r1377019133)
- Track the order in which transactions are included in the template to get the "linearization order" of the transactions.
- Tests
Reviewers can take a look at #26711 to see how these functions are used to linearize the `AncestorPackage` there.
ACKs for top commit:
TheCharlatan:
ACK d9cc99d04e
kevkevinpal:
reACK [d9cc99d](d9cc99d04e)
achow101:
re-ACK d9cc99d04e
Tree-SHA512: 32b80064b6679536ac573d674825c5ca0cd6245e49c2fd5eaf260dc535335a57683c74ddd7ce1f249b5b12b2683de4362a7b0f1fc0814c3b3b9f14c682665583
b5a60abe87 MOVEONLY: CleanupTemporaryCoins into its own function (glozow)
10c0a8678c [test util] CreateValidTransaction multi-in/out, configurable feerate, signal BIP125 (glozow)
6ff647a7e0 scripted-diff: rename CheckPackage to IsWellFormedPackage (glozow)
da9aceba21 [refactor] move package checks into helper functions (glozow)
Pull request description:
This is part of #27463. It splits off the more trivial changes from #26711 for ease of review, as requested in https://github.com/bitcoin/bitcoin/pull/26711#issuecomment-1786392253.
- Split package sanitization in policy/packages.h into helper functions
- Add some tests for its quirks (https://github.com/bitcoin/bitcoin/pull/26711#discussion_r1340521597)
- Rename `CheckPackage` to `IsPackageWellFormed`
- Improve the `CreateValidTransaction` unit test utility to:
- Configure the target feerate and return the fee paid
- Signal BIP125 on transactions to enable RBF tests
- Allow the specification of multiple inputs and outputs
- Move `CleanupTemporaryCoins` into its own function to be reused later without duplication
ACKs for top commit:
dergoegge:
Code review ACK b5a60abe87
instagibbs:
ACK b5a60abe87
Tree-SHA512: 39d67a5f0041e381f0d0f802a98ccffbff11e44daa3a49611189d6306b03f18613d5ff16c618898d490c97a216753e99e0db231ff14d327f92c17ae4d269cfec
fcb3069fa3 Use CheckPackageMempoolAcceptResult for package evaluation fuzzing (Greg Sanders)
34088d6c9e [test util] CheckPackageMempoolAcceptResult for sanity-checking results (glozow)
651fa404e4 fuzz: tx_pool checks ATMP result invariants (Greg Sanders)
Pull request description:
Poached from https://github.com/bitcoin/bitcoin/pull/26711 since that PR is being split apart, and modified to match current behavior.
ACKs for top commit:
glozow:
reACK fcb3069fa3, only whitespace changes
dergoegge:
ACK fcb3069fa3
Tree-SHA512: abd687e526d8dfc8d65b3a873ece8ca35fdcbd6b0f7b93da6a723ef4e47cf85612de819e6f2b8631bdf897e1aba27cdd86f89b7bd85fc3356e74be275dcdf8cc
Sometimes we are just interested in the order in which transactions
would be included in a block (we want to "linearize" the transactions).
Track and store this information.
This doesn't change any of the bump fee calculations.
Add an option to keep building the template regardless of feerate. We
can't just use target_feerate=0 because it's possible for transactions
to have negative modified feerates.
No behavior change for users that pass in a target_feerate.
This is primarily intended for linearizing a package of transactions
prior to submitting them to mempool. Note that, if this ctor is used,
bump fees will not be calculated because we haven't instructed MiniMiner
which outpoints for which we want bump fees to be calculated.
No behavior change. All we are doing is copying out these values before
passing them into the ctor instead of within the ctor.
This makes it possible to use the MiniMiner algorithms to analyze
transactions that haven't been submitted to the mempool yet.
It also iwyu's the mini_miner includes.
bb91131d54 doc: remove out-of-date external link in src/util/strencodings.h (Jon Atack)
7d494a48dd refactor: use string_view to pass string literals to Parse{Hash,Hex} (Jon Atack)
Pull request description:
as `string_view` is optimized to be trivially copiable, whereas the current code creates a `std::string` copy at each call.
These utility methods are called by quite a few RPCs and tests, as well as by each other.
```
$ git grep "ParseHashV\|ParseHashO\|ParseHexV\|ParseHexO" | wc -l
61
```
Also remove an out-of-date external link.
ACKs for top commit:
jonatack:
Rebased per `git range-diff c9273f6 b94581a bb91131` for an include header from the merge of https://github.com/bitcoin/bitcoin/pull/28230. Should be trivial to re-ACK.
maflcko:
lgtm ACK bb91131d54
ns-xvrn:
ACK bb91131d54
achow101:
ACK bb91131d54
brunoerg:
crACK bb91131d54
Tree-SHA512: 9734fe022c9e43fd93c23a917770d332dbbd3132c80a234059714c32faa6469391e59349954749fc86c4ef0b18d5fd99bf8f4b7b82d9f799943799c1253272ae
376dc2cfb3 test: add coverage to rpc_blockchain.py (kevkevin)
Pull request description:
Included a test that checks the functionality of setting
the first param of getnetworkhashps to negative value returns
the average network hashes per second from the last difficulty change.
ACKs for top commit:
jlopp:
tACK 376dc2cfb3
achow101:
ACK 376dc2cfb3
ismaelsadeeq:
Tested ACK 376dc2cfb3
pablomartin4btc:
tACK 376dc2cfb3
Tree-SHA512: 02d52f622e9cb7a1240c5d124510dd75d03f696f119b2625b0befd60b004ec50ff1a2d5515e0e227601adeecd837e0778ed131ee2a8c5f75f1b824be711213a7
37d150d8c5 refactor: Add more negative `!m_banned_mutex` thread safety annotations (Hennadii Stepanov)
0fb2908708 refactor: replace RecursiveMutex m_banned_mutex with Mutex (w0xlt)
784c316f9c scripted-diff: rename m_cs_banned -> m_banned_mutex (w0xlt)
46709c5f27 refactor: Get rid of `BanMan::SetBannedSetDirty()` (Hennadii Stepanov)
d88c0d8440 refactor: Get rid of `BanMan::BannedSetIsDirty()` (Hennadii Stepanov)
Pull request description:
This PR is an alternative to bitcoin/bitcoin#24092. Last two commit have been cherry-picked from the latter.
ACKs for top commit:
maflcko:
ACK 37d150d8c5🎾
achow101:
ACK 37d150d8c5
theStack:
Code-review ACK 37d150d8c5
vasild:
ACK 37d150d8c5
Tree-SHA512: 5e9d40101a09af6e0645a6ede67432ea68631a1b960f9e6af0ad07415ca7718a30fcc1aad5182d1d5265dc54c26aba2008fc9973840255c09adbab8fedf10075
Clarify that supported versions of GCC are not affected, and that Clang
prior to version 15 still requires the explicit -latomic linking, when
compiling for 32-bit.
a5e39d325d Fee estimation: extend bucket ranges consistently (Anthony Towns)
Pull request description:
When calculating a median fee for a confirmation target at a particular threshold, we analyse buckets in ranges rather than individually in case some buckets have very little data. This patch ensures the breaks between ranges are independent of the the confirmation target.
Fixes#20725
ACKs for top commit:
ismaelsadeeq:
Code review ACK a5e39d325d
glozow:
btw what I meant by [this](https://github.com/bitcoin/bitcoin/pull/21161#pullrequestreview-1350258467) was ACK a5e39d325d
jonatack:
Initial ACK a5e39d325d
Tree-SHA512: 0edf4e56717c4ab8d4ab0bc0f1d7ab36a13b99de12f689e55c9142c6b81691367ffd8df2e8260c5e14335310b1a51770c6c22995db31109976239befcb558ef8
9b3da70bd0 [test] DisconnectedBlockTransactions::DynamicMemoryUsage (glozow)
b2d0447964 bugfix: correct DisconnectedBlockTransactions memory usage (stickies-v)
f4254e2098 assume duplicate transactions are not added to `iters_by_txid` (ismaelsadeeq)
29eb219c12 move only: move implementation code to disconnected_transactions.cpp (ismaelsadeeq)
81dfeddea7 refactor: update `MAX_DISCONNECTED_TX_POOL` from kb to bytes (ismaelsadeeq)
Pull request description:
This PR is a follow-up to fix review comments and a bugfix from #28385
The PR
- Updated `DisconnectedBlockTransactions`'s `MAX_DISCONNECTED_TX_POOL` from kb to bytes.
- Moved `DisconnectedBlockTransactions` implementation code to `kernel/disconnected_transactions.cpp`.
- `AddTransactionsFromBlock` now assume duplicate transactions are not passed by asserting after inserting each transaction to `iters_by_txid`.
- Included a Bug fix: In the current master we are underestimating the memory usage of `DisconnectedBlockTransactions`.
* When adding and subtracting `cachedInnerUsage` we call `RecursiveDynamicUsage` with `CTransaction` which invokes this [`RecursiveDynamicUsage(const CTransaction& tx)`](6e721c923c/src/core_memusage.h (L32)) version of `RecursiveDynamicUsage`, the output of that call only account for the memory usage of the inputs and outputs of the `CTransaction`, this omits the memory usage of the `CTransaction` object and the control block.
* This PR fixes this bug by calling `RecursiveDynamicUsage` with `CTransactionRef` when adding and subtracting `cachedInnerUsage` which invokes [`RecursiveDynamicUsage(const std::shared_ptr<X>& p)`](6e721c923c/src/core_memusage.h (L67)) version of `RecursiveDynamicUsage` the output of the calculation accounts for the` CTransaction` object, the control blocks, inputs and outputs memory usage.
* see [comment ](https://github.com/bitcoin/bitcoin/pull/28385#discussion_r1322948452)
- Added test for DisconnectedBlockTransactions memory limit.
ACKs for top commit:
stickies-v:
ACK 9b3da70bd0 - nice work!
BrandonOdiwuor:
re ACK 9b3da70bd0
glozow:
ACK 9b3da70bd0
Tree-SHA512: 69b9595d09f4d0209038f97081d790cea92ccf63efb94e9e372749979fcbe527f7f17a8e454720cedd12021be0c8e11cf99874625d3dafd9ec602b12dbeb4098
8047bb6fea build: Update `qt` package up to 5.15.11 (Hennadii Stepanov)
Pull request description:
In the light of https://github.com/bitcoin/bitcoin/pull/28622, we probably have to patch Qt. It seems reasonable to update it up to the latest available version before doing that.
ACKs for top commit:
TheCharlatan:
ACK 8047bb6fea
Tree-SHA512: b4d7df2ff059b8f58c3202d913237c0d39a962748658f1ce853884dca095fbda5f56d4d68f73a1bc8da2f295e96a20927306e148b41a9f4afc42c8edb11c3729
`lcov`'s `-a` option takes an argument. With `LCOV_OPTS` immediately
after `-a`, the first additional argument becomes the argument to `-a`
which is incorrect.
Also add `LCOV_OPTS` to more `lcov` calls.
Support the creation of a transaction with multiple specified inputs or
outputs. Also accept a target feerate and return the fee paid.
Also, signal BIP125 by default - a subsequent commit needs to RBF
something.
Co-authored-by: Andrew Chow <achow101@gmail.com>
This allows IsSorted() and IsConsistent() to be used by themselves.
IsSorted() with a precomputed set is used so that we don't create this
set multiple times.
9cfc1c9440 test: check that we don't send a getaddr msg to an inbound peer (Martin Zumsande)
88c33c6748 test: make python p2p not send getaddr messages when it's being connected to (Martin Zumsande)
Pull request description:
`bitcoind` nodes send `getaddr` messages only to outbound nodes (and ignore `getaddr` received by outgoing connections).
The python p2p node should mirror this behavior by not sending a `getaddr` message when it is not the initiator of the connection.
This is currently causing several unnecessary messages being sent and then ignored (`Ignoring "getaddr" from outbound-full-relay connection.`) in tests like `p2p_add_connections.py`.
ACKs for top commit:
pinheadmz:
concept ACK 9cfc1c9440
pablomartin4btc:
re ACK 9cfc1c9440
BrandonOdiwuor:
re ACK 9cfc1c9440
Tree-SHA512: 812bec5d8a4828b4384d4cdd4362d6eec09acb2363e888f2b3e3bf8b925e0e17f15e13dc297d6b616c68b93ace9ede7245b07b405d3f5f8eada98350f74230dc
02a4f1a385 addrman: log AS only when using asmap (brunoerg)
Pull request description:
This PR changes the log to just print the ASN when using asmap, same logic presented in other logs:
afa081a39b/src/net_processing.cpp (L3552-L3556)afa081a39b/src/net_processing.cpp (L3598-L3604)
ACKs for top commit:
naumenkogs:
ACK 02a4f1a385
mzumsande:
Code Review ACK 02a4f1a385
Tree-SHA512: adad5904ab163660d47554b32dc2dc3dfdff8dd64b94e5320ad11706381264d1e338654fa8239430eed4ccbebc8f6670698b4278895794055c37fc4bcefe71bc
b74e449ffa build: remove potential for duplciate natpmp linking (fanquake)
4e95096952 build: remove duplicate -lminiupnpc linking (fanquake)
Pull request description:
Having the link check in the header check loop means we get `-lminiupnpc -lminiupnpc -lminiupnpc` on the link line.
This is unnecessary, and results in warnings, i.e:
```bash
ld: warning: ignoring duplicate libraries: '-levent', '-lminiupnpc'
ld: warning: ignoring duplicate libraries: '-levent', '-lminiupnpc'
ld: warning: ignoring duplicate libraries: '-levent', '-lminiupnpc'
```
These warnings have been occurring since the new macOS linker released with Xcode 15, and also came up in https://github.com/hebasto/bitcoin/pull/34.
There are other duplicate lib issues, i.e with `-levent` + `-levent_pthreads -levent`, but those are less straight forward to solve, and won't be included here.
ACKs for top commit:
jonatack:
ACK b74e449ffa
hebasto:
ACK b74e449ffa, it fixes one issue mentioned in https://github.com/hebasto/bitcoin/pull/34#issuecomment-1782914787.
TheCharlatan:
ACK b74e449ffa
theuni:
ACK b74e449ffa
Tree-SHA512: 987a56ef17cbaf273cb672c41016f3f615b16889317325a9e88135d0c41f01af3840ad44a6f811a7df97f5873c9cd957e60aaa1b99bd408b17b4b1ffe2c68f36
e26e665f9f gui: fix crash on selecting "Mask values" in transaction view (Sebastian Falbesoner)
Pull request description:
This PR fixes a crash bug that can be caused with the following steps:
- change to the "Transactions" view
- right-click on an arbitrary transaction -> "Show transaction details"
- close the transaction detail window again
- select menu item "Settings" -> "Mask values"
The problem is that the list of opened dialogs, tracked in the member variable `m_opened_dialogs` (introduced in https://github.com/bitcoin-core/gui/pull/708, commit 4492de1be1), is only ever appended with newly opened transaction detail dialog pointers, but never removed. This leads to dangling pointers in the list, and if the "Mask values" menu item is selected, a crash is caused in the course of trying to close the opened transaction detail dialogs (see `closeOpenedDialogs()` method). Fix this by removing a pointer of the list if the corresponding widget is destroyed.
ACKs for top commit:
achow101:
ACK e26e665f9f
pablomartin4btc:
tACK e26e665f9f
furszy:
utACK e26e665f9
hebasto:
ACK e26e665f9f, tested on Ubuntu 22.04.
Tree-SHA512: 37885c22abae0ab065b4878bae46fd362f41b09609d081fd59e26bb05474f427b98771ee73f5480526afaef04e016c5ba62c956e0e85a57b6a0f44a905b68a83
for initial partial unit test coverage of these CConnman class methods:
- AddNode()
- ConnectNode()
- GetAddedNodeInfo()
- AlreadyConnectedToAddress()
- ThreadOpenAddedConnections()
and of the GetAddedNodeInfo() call in RPC addnode.
`send(2)` can be interrupted or for another reason it may not fully
complete sending all the bytes. We should be ready to retry the send
with the remaining bytes. This is what `Sock::SendComplete()` does,
thus use it in `Socks5()`.
Since `Sock::SendComplete()` takes a `CThreadInterrupt` argument,
change also the recv part of `Socks5()` to use `CThreadInterrupt`
instead of a boolean.
Easier reviewed with `git show -b` (ignore white-space changes).
79539fbfbf guix: update signapple (fanquake)
Pull request description:
Fixes#28449, and removes the need to boostrap Rust, by avoiding the `python-requests` dependency.
Comparing a `--no-substitutes` build of this PR, to master, signapple requires ~1350 _less_ packages to boostrap:
Master derivation - https://gist.github.com/fanquake/dbf69a62c9a78b7ae8c183a160e6d58d
PR derivation - https://gist.github.com/fanquake/0aa2d8eddaba861ba489ed3d936f727d
ACKs for top commit:
achow101:
ACK 79539fbfbf
Tree-SHA512: 341ddcae27e53c31d114465cb5173573dcc9e1c0874ee160715630f686da6f69255f6080ec0181ffeffc26efbdb545599d667784b1cd17dfa7e3da0998ec9bd6
f6f18eeaa8 guix: Zip needs to include all files with time as SOURCE_DATE_EPOCH (Andrew Chow)
Pull request description:
The zip for codesigned MacOS distribution needs to have all files included and have their timestamps set to the same value (`SOURCE_DATE_EPOCH`).
This uses the same pattern for zip as is done for the other zip files produced by guix.
ACKs for top commit:
hebasto:
ACK f6f18eeaa8.
TheCharlatan:
ACK f6f18eeaa8
Tree-SHA512: 569ff0d8bfe76b9b111a2454478523eeb514b44b691be8b57b61415db88356c683582550ea67ebd5fb392b4f486be170a925067b507979090535ca41cbc7351b
The zip for codesigned MacOS distribution needs to have all files have
the same timestamp. These files also need to be included in the zip as
zip is not automatically recursive. We use the same pattern for zip as
is done for the other zip files produced by guix.
Having the link check in the header check loop means we get `-lminiupnpc
-lminiupnpc -lminiupnpc` on the link line. This is unnecessary, and
results in warnings, i.e:
```bash
ld: warning: ignoring duplicate libraries: '-levent', '-lminiupnpc'
ld: warning: ignoring duplicate libraries: '-levent', '-lminiupnpc'
ld: warning: ignoring duplicate libraries: '-levent', '-lminiupnpc'
```
These warnings have been occurring since the new linker released with
Xcode 15, and also came up in https://github.com/hebasto/bitcoin/pull/34.
99990194ce Remove WithParams serialization helper (MarcoFalke)
ffffb4af83 scripted-diff: Use ser params operator (MarcoFalke)
fae9054793 test: Use SER_PARAMS_OPFUNC in serialize_tests.cpp (MarcoFalke)
Pull request description:
Every serialization parameter struct already has the `SER_PARAMS_OPFUNC`, except for one in the tests.
For consistency, and to remove verbose code, convert the test to `SER_PARAMS_OPFUNC`, and use it everywhere, then remove the `WithParams` helper.
ACKs for top commit:
ajtowns:
reACK 99990194ce
TheCharlatan:
Re-ACK 99990194ce
Tree-SHA512: be9cae4225a502486fe8d552aaf4b2cd2904a9f73cce9d931c6b7c757594ff1982fcc2c30d00d012cd12b0a9531fd609f8bcd7c94b811e965ac087eb8a3589d3
This commits fixes a crash bug that can be caused with the following steps:
- change to the "Transactions" view
- right-click on an arbitrary transaction -> "Show transaction details"
- close the transaction detail window again
- select "Settings" -> "Mask values"
The problem is that the list of opened dialogs, tracked in the member
variable `m_opened_dialogs`, is only ever appended with newly opened
transaction detail dialog pointers, but never removed. This leads to
dangling pointers in the list, and if the "Mask values" menu item is
selected, a crash is caused in the course of trying to close the opened
transaction detail dialogs (see `closeOpenedDialogs()` method). Fix this
by removing the pointer from the list if the corresponding widget is
destroyed.
fa56067a8f refactor: Fix bugprone-string-constructor warning (MarcoFalke)
Pull request description:
String literals in C++ have a trailing null character, so the current code is fine to rely on that implicitly. However,
* the sqlite documentation explicitly mentions the null character
* code readers may wonder if the code is intentional
* clang-tidy warns about the code via `bugprone-string-constructor`
Address the points by putting the null character into the code and enable the clang-tidy `bugprone-string-constructor` check.
ACKs for top commit:
stickies-v:
ACK fa56067a8f
Tree-SHA512: da519184d792a885a8151ffc44c8da5781f5aaae12ef768a187cc6d9e542ca8952aebc2ec6c1a05f673f29a86ef44902ee96e7b491af7b4705ad38e14624882e
that are otherwise private:
- CConnman::m_nodes
- CConnman::ConnectNodes()
- CConnman::AlreadyConnectedToAddress()
and update the #include headers per iwyu.
`CConnman::GetAddedNodeInfo` is used both to get a list of addresses to manually connect to
in `CConnman::ThreadOpenAddedConnections`, and to report about manually added connections in
`getaddednodeinfo`. In both cases, all addresses added to `m_added_nodes` are returned, however
the nodes we are already connected to are only relevant to the latter, in the former they are
actively discarded.
Parametrizes `CConnman::GetAddedNodeInfo` so we can ask for only addresses we are not connected to,
to avoid passing useless information around.
Currently it is possible to add the same node twice when formatting IPs in
different, yet equivalent, manner. This applies to both ipv4 and ipv6, e.g:
127.0.0.1 = 127.1 | [::1] = [0:0:0:0:0:0:0:1]
`addnode` will accept both and display both as connected (given they translate to
the same IP). This will not result in multiple connections to the same node, but
will report redundant info when querying `getaddednodeinfo` and populate `m_added_nodes`
with redundant data.
This can be avoided performing comparing the contents of `m_added_addr` and the address
to be added as `CServices` instead of as strings.
The current `addnode` rpc command has some edge cases in where it is possible to
connect to the same node twice by combining ip and address requests. This can happen under two situations:
The two commands are run one right after each other, in which case they will be processed
under the same loop in `CConnman::ThreadOpenAddedConnections` without refreshing `vInfo`, so both
will go trough. An example of this would be:
```
bitcoin-cli addnode "localhost:port" add
```
A node is added by IP using `addnode "add"` while the other is added by name using
`addnode "onetry"` with an address that resolves to multiple IPs. In this case, we currently
only check one of the resolved IPs (picked at random), instead of all the resolved ones, meaning
this will only probabilistically fail/succeed. An example of this would be:
```
bitcoin-cli addnode "127.0.0.1:port" add
[...]
bitcoin-cli addnode "localhost:port" onetry
```
Both cases can be fixed by iterating over all resolved addresses in `CConnman::ConnectNode` instead
of picking one at random
fa5423b5b5 refactor: Remove unused gcc-9 workaround in txrequest (MarcoFalke)
fa918d397d Always enable -Wsuggest-override (MarcoFalke)
faea58eee4 Bump g++ minimum supported version to 10 (MarcoFalke)
Pull request description:
All supported operating systems ship with g++ 10 (or later), so bumping the minimum should not cause any issues. The bump allows to drop some now-unused workarounds.
For reference:
* https://packages.debian.org/bullseye/g++ (`g++-10`)
* https://packages.ubuntu.com/focal/g++-10
* FreeBSD 12/13 ships with g++ 12
* CentOS-like 9 ships with g++ 11
* OpenSuse Tumbleweed ships with g++ 13 https://software.opensuse.org/package/gcc13-c++ (No idea about OpenSuse Leap)
This is for Bitcoin Core 27.0 in 2024 (next year), not the soon upcoming 26.0 next month.
ACKs for top commit:
fanquake:
ACK fa5423b5b5
Tree-SHA512: 6f0697ae4c0f578873591b7872bf158aba3af17f171c3556b593a70ec379bf94c7a9dd7697e8e79173edd4ac3c81a376e0cbbc0cfabde1a1cfe5f9b5eaea6831
faa769db5a Fix bugprone-lambda-function-name errors (MarcoFalke)
Pull request description:
Inside a lambda, `__func__` will evaluate to something like `"operator()"`. Fix this by either removing it, or by using the real name.
https://clang.llvm.org/extra/clang-tidy/checks/bugprone/lambda-function-name.html
ACKs for top commit:
TheCharlatan:
ACK faa769db5a
darosior:
utACK faa769db5a
Tree-SHA512: 0b562bd4ebd7f46ca3ebabeee67851ad30bd522fa57e5010e833b163664e51f5df645ff9ca35d22c3479fb27d9267d4e5d0d417d42729bf3ccf80d7944970e4e
811067ca1c test: add coverage for snapshot chainstate not matching AssumeUTXO parameters (pablomartin4btc)
4a5be10b92 assumeutxo, blockstorage: prevent core dump on invalid hash (pablomartin4btc)
Pull request description:
While reviewing #27596 (ran `loadtxoutset` in `mainnet` before `m_assumeutxo_data` is empty as [currently](434495a8c1/src/kernel/chainparams.cpp (L175-L177)) in master - back to 1b1d711), got a `core dumped`, so it seems there's a potential issue if new releases ever remove snapshot details or a semi-experienced user performs a `loadtxoutset` on a different "customised" binary version (not sure if this is a real use case).
```
2023-10-18T17:42:52Z [init] Using obfuscation key for /tmp/.test_utxo_2/blocks/index: 0000000000000000
node/blockstorage.cpp:390 LoadBlockIndex: Assertion `GetParams().AssumeutxoForBlockhash(*snapshot_blockhash)' failed.
Aborted (core dumped)
```
<details>
<summary>This is also happening before IBD is completed (<code>background validation</code> still being performed as it can be seen in rpc <code>getchainstates</code>)</summary>
```
/src/bitcoin-cli -datadir=${AU_DATADIR} getchainstates
{
"headers": 813097,
"chainstates": [
{
"blocks": 368249,
"bestblockhash": "00000000000000000b7a08224a1cb00d337100ba7a46c03d04b2c2d8964efc37",
"difficulty": 52278304845.59168,
"verificationprogress": 0.086288278873286,
"coins_db_cache_bytes": 7969177,
"coins_tip_cache_bytes": 14908338995,
"validated": true
},
{
"blocks": 813097,
"bestblockhash": "0000000000000000000270c9fdce7b17db64cca91f90106964b58e33a4d91089",
"difficulty": 61030681983175.59,
"verificationprogress": 0.999997140098457,
"coins_db_cache_bytes": 419430,
"coins_tip_cache_bytes": 784649420,
"snapshot_blockhash": "00000000000000000002a7c4c1e48d76c5a37902165a270156b7a8d72728a054",
"validated": false
}
]
}
```
</details>
<details>
<summary>Steps to reproduce the core dump error and its output:</summary>
1. Perform a `loadtxoutset` in `mainnet` on compiled `bitcoind` adding the block hash from Sjors's [commit](24deb2022b).
2. Once step 1 finishes, remove the added code from step 1 and compile again or just compile `master` without any changes on top.
3. Run `bitcoind`, soon it'll crash with:
```
2023-10-18T17:42:52Z [init] init message: Loading block index…
2023-10-18T17:42:52Z [init] Assuming ancestors of block 00000000000000000001a0a448d6cf2546b06801389cc030b2b18c6491266815 have valid signatures.
2023-10-18T17:42:52Z [init] Setting nMinimumChainWork=000000000000000000000000000000000000000052b2559353df4117b7348b64
2023-10-18T17:42:52Z [init] Prune configured to target 3000 MiB on disk for block and undo files.
2023-10-18T17:42:52Z [init] [snapshot] detected active snapshot chainstate (/tmp/.test_utxo_2/chainstate_snapshot) - loading
2023-10-18T17:42:52Z [init] [snapshot] switching active chainstate to Chainstate [snapshot] @ height -1 (null)
2023-10-18T17:42:52Z [init] Opening LevelDB in /tmp/.test_utxo_2/blocks/index
2023-10-18T17:42:52Z [init] Opened LevelDB successfully
2023-10-18T17:42:52Z [init] Using obfuscation key for /tmp/.test_utxo_2/blocks/index: 0000000000000000
node/blockstorage.cpp:390 LoadBlockIndex: Assertion `GetParams().AssumeutxoForBlockhash(*snapshot_blockhash)' failed.
Aborted (core dumped)
```
</details>
<details>
<summary>After original change, error message output:</summary>
```
2023-10-20T15:49:12Z [init] init message: Loading block index…
2023-10-20T15:49:12Z [init] Assuming ancestors of block 00000000000000000001a0a448d6cf2546b06801389cc030b2b18c6491266815 have valid signatures.
2023-10-20T15:49:12Z [init] Setting nMinimumChainWork=000000000000000000000000000000000000000052b2559353df4117b7348b64
2023-10-20T15:49:12Z [init] Prune configured to target 3000 MiB on disk for block and undo files.
2023-10-20T15:49:12Z [init] [snapshot] detected active snapshot chainstate (/tmp/.test_utxo_2/chainstate_snapshot) - loading
2023-10-20T15:49:12Z [init] [snapshot] switching active chainstate to Chainstate [snapshot] @ height -1 (null)
2023-10-20T15:49:12Z [init] Opening LevelDB in /tmp/.test_utxo_2/blocks/index
2023-10-20T15:49:12Z [init] Opened LevelDB successfully
2023-10-20T15:49:12Z [init] Using obfuscation key for /tmp/.test_utxo_2/blocks/index: 0000000000000000
2023-10-20T15:49:13Z [init] *** Assumeutxo data not found for the given blockhash '00000000000000000002a7c4c1e48d76c5a37902165a270156b7a8d72728a054'.
2023-10-20T15:49:13Z [init] Error: Assumeutxo data not found for the given blockhash '00000000000000000002a7c4c1e48d76c5a37902165a270156b7a8d72728a054'.
Error: Assumeutxo data not found for the given blockhash '00000000000000000002a7c4c1e48d76c5a37902165a270156b7a8d72728a054'.
2023-10-20T15:49:13Z [init] Shutdown requested. Exiting.
2023-10-20T15:49:13Z [init] Shutdown: In progress...
2023-10-20T15:49:13Z [scheduler] scheduler thread exit
2023-10-20T15:49:13Z [shutoff] Flushed fee estimates to fee_estimates.dat.
2023-10-20T15:49:13Z [shutoff] Shutdown: done
```
</details>
<details>
<summary>Alternative on error handling using <code>return error()</code> instead of <code>return FatalError()</code> used in this PR, which produces a different output and perhaps confusing:</summary>
```
2023-10-20T21:45:58Z [init] Using obfuscation key for /tmp/.test_utxo_2/blocks/index: 0000000000000000
2023-10-20T21:45:59Z [init] ERROR: Assumeutxo data not found for the given blockhash '00000000000000000002a7c4c1e48d76c5a37902165a270156b7a8d72728a054'.
2023-10-20T21:45:59Z [init] : Error loading block database.
Please restart with -reindex or -reindex-chainstate to recover.
: Error loading block database.
Please restart with -reindex or -reindex-chainstate to recover.
2023-10-20T21:45:59Z [init] Aborted block database rebuild. Exiting.
2023-10-20T21:45:59Z [init] Shutdown: In progress...
2023-10-20T21:45:59Z [scheduler] scheduler thread exit
2023-10-20T21:45:59Z [shutoff] Flushed fee estimates to fee_estimates.dat.
2023-10-20T21:45:59Z [shutoff] Shutdown: done
```
</details>
<details>
<summary>Current state (including ryanofsky <a href="https://github.com/bitcoin/bitcoin/pull/28698#discussion_r1368635965">suggestion</a>), after code change, error message output:</summary>
```
2023-10-25T02:29:57Z [init] Using obfuscation key for /home/pablo/.test_utxo_2/regtest/blocks/index: 0000000000000000
2023-10-25T02:29:57Z [init] *** Assumeutxo data not found for the given blockhash 'f09b5835f3f8b39481f2af3257bbc2e82845552d4d2d6d31cf520fc24263ed5b'.
2023-10-25T02:29:57Z [init] Error: A fatal internal error occurred, see debug.log for details
Error: A fatal internal error occurred, see debug.log for details
2023-10-25T02:29:57Z [init] Shutdown requested. Exiting.
2023-10-25T02:29:57Z [init] Shutdown: In progress...
2023-10-25T02:29:57Z [scheduler] scheduler thread exit
2023-10-25T02:29:57Z [shutoff] Flushed fee estimates to fee_estimates.dat.
2023-10-25T02:29:57Z [shutoff] Shutdown: done
```
</details>
ACKs for top commit:
naumenkogs:
ACK 811067ca1c
theStack:
ACK 811067ca1c
ryanofsky:
Code review ACK 811067ca1c.
Tree-SHA512: cfc137b0a4f638b99fd7dac2c35cc729ef71ae1166a2a8960a91055ec90841cb33aed589834012cfe0e157937e2a76a88d1020ea1df2bc98e1114eb1fc8eaae4
fe3ac3700d test: replace random_bytes with randbytes #28720 (ns-xvrn)
Pull request description:
With Python upgraded to 3.9 replaced the `random_bytes` function in util of functional tests and replaced it's usage with `random.randbytes`.
Closes#28720.
ACKs for top commit:
maflcko:
lgtm ACK fe3ac3700d
BrandonOdiwuor:
ACK fe3ac3700d
stickies-v:
ACK fe3ac3700d, thanks for picking this up
kristapsk:
utACK fe3ac3700d
Tree-SHA512: f65a75e73ebd840c2936eb133d42bccd552f25b717c8ca25c18d06e0593e12f292389cfcc0a0b0759004b67a46ea0c8ac237973ef90f246139778230be1e64e1
faec889f93 refactor: Add LIFETIMEBOUND to all (w)txid getters (MarcoFalke)
Pull request description:
Currently some getters return a reference, some don't. Fix this by returning a reference everywhere. Also, add `LIFETIMEBOUND` to all. Then, use the compiler warnings to create copies only where needed.
Also, fix iwyu includes while touching the includes.
ACKs for top commit:
dergoegge:
Code review ACK faec889f93
stickies-v:
ACK faec889f93
pablomartin4btc:
cr ACK faec889f93
Tree-SHA512: 0c2a151f39d0e007b4d33b0b85ad578cc220f3e9dd94890e812b3181c3901545b039325707731cc39a5e89557f59c1154c6320525f78f5de95f119a514d2d23f
91d0888921 sync: unpublish LocksHeld() which is used only in sync.cpp (Vasil Dimov)
3df37e0c78 doc: clarify that LOCK() does AssertLockNotHeld() internally (Vasil Dimov)
Pull request description:
Constructs like
```cpp
AssertLockNotHeld(m);
LOCK(m);
```
are equivalent to (almost, modulo some logging differences, see below)
```cpp
LOCK(m);
```
for non-recursive mutexes, so it is ok to omit `AssertLockNotHeld()` in such cases. Requests to do the former keep coming during review process. `developer-notes.md` explicitly states "Combine annotations in function declarations with run-time asserts in function definitions", but that seems to be too strong or unclear. `LOCK()` is also a run-time assert in this case.
Also remove `LocksHeld()` from the public interface in `sync.h` since it is only used in `sync.cpp`.
ACKs for top commit:
achow101:
ACK 91d0888921
hebasto:
ACK 91d0888921, I have reviewed the code and it looks OK.
Tree-SHA512: c4b7ef2c0bfeb28d1c4f55f497810f629873137e02f5a92137c02cb1ff603ac76473dcd2171e594491494a5cb87b8c0c803e06b86f190d4acb231791e28e802d
fb3e812277 p2p: return `CSubNet` in `LookupSubNet` (brunoerg)
Pull request description:
Analyzing the usage of `LookupSubNet`, noticed that most cases uses check if the subnet is valid by calling `subnet.IsValid()`, and the boolean returned by `LookupSubNet` hasn't been used so much, see:
29d540b7ad/src/httpserver.cpp (L172-L174)29d540b7ad/src/net_permissions.cpp (L114-L116)
It makes sense to return `CSubNet` instead of `bool`.
ACKs for top commit:
achow101:
ACK fb3e812277
vasild:
ACK fb3e812277
theStack:
Code-review ACK fb3e812277
stickies-v:
Concept ACK, but Approach ~0 (for now). Reviewed the code (fb3e812277) and it all looks good to me.
Tree-SHA512: ba50d6bd5d58dfdbe1ce1faebd80dd8cf8c92ac53ef33519860b83399afffab482d5658cb6921b849d7a3df6d5cea911412850e08f3f4e27f7af510fbde4b254
940a49978c Use type-safe txid types in orphanage (dergoegge)
ed70e65016 Introduce types for txids & wtxids (dergoegge)
cdb14d79e8 [net processing] Use HasWitness over comparing (w)txids (dergoegge)
Pull request description:
We currently have two different identifiers for transactions: `txid` (refering to the hash of a transaction without witness data) and `wtxid` (referring to the hash of a transaction including witness data). Both are typed as `uint256` which could lead to type-safety bugs in which one transaction identifier type is passed where the other would be expected.
This PR introduces explicit `Txid` and `Wtxid` types that (if used) would cause compilation errors for such type confusion bugs.
(Only the orphanage is converted to use these types in this PR)
ACKs for top commit:
achow101:
ACK 940a49978c
stickies-v:
ACK 940a49978c
hebasto:
ACK 940a49978c, I have reviewed the code and it looks OK.
instagibbs:
re-ACK 940a49978c
BrandonOdiwuor:
re-ACK 940a49978c
glozow:
reACK 940a49978c
Tree-SHA512: 55298d1c2bb82b7a6995e96e554571c22eaf4a89fb2a4d7a236d70e0f625e8cca62ff2490e1c179c47bd93153fe6527b56870198f026f5ee7753d64d7a424c92
1111475b41 bugfix: Mark CNoDestination and PubKeyDestination constructor explicit (MarcoFalke)
fa5ccc4137 iwyu: Export prevector.h from script.h (MarcoFalke)
Pull request description:
It seems confusing to allow any script, even one with a corresponding address, to silently convert to `CNoDestination`.
Make the converstion `explicit` in the code, and fix any bugs that were previously introduced.
In a follow-up, the class can be renamed, or the documentation can be updated to better reflect what the code does.
ACKs for top commit:
josibake:
ACK 1111475b41
achow101:
ACK 1111475b41
furszy:
Code review ACK 1111475
Tree-SHA512: d8b5f54d0cd8649a31e227ef164bb13e5b81ee9820f1976fd70c7a0de6841fba72d549c2f63e351c8cdda37dceb4763eca203e1c8ef385f46d9da6f1855c39ec
This should fix the bug reported in
https://github.com/bitcoin/bitcoin/pull/28246#discussion_r1371640502,
which caused the GUI to not detect the destination type of recipients,
thus picking the wrong change destination type.
Also, add missing lifetimebound attribute to a getter method.
50d1ac1207 test: remove unused `find_output` helper (Sebastian Falbesoner)
73a339abc3 test: refactor: support sending funds with outpoint result (Sebastian Falbesoner)
Pull request description:
In wallet-related functional tests we often want to send funds to an address and use the resulting (non-change) UTXO directly after as input for another transaction. Doing that is currently tedious, as it involves finding the index part of the outpoint manually by calling helpers like `find_vout_for_address` or `find_output` first. This results in two different txid/vout variables which then again have to be combined to a single dictionary `{"txid": ..., "vout": ...}` in order to be specified as input for RPCs like `createrawtransaction` or `createpsbt`. For example:
```
txid1 = node1.sendtoaddress(addr1, value1)
vout1 = find_vout_for_address(node1, txid1, addr1)
txid2 = node2.sendtoaddress(addr2, value2)
vout2 = find_vout_for_address(node2, txid2, addr2)
node.createrawtransaction([{'txid': txid1, 'vout': vout1}, {'txid': txid2, 'vout': vout2}], .....)
```
This PR introduces a helper `create_outpoints` to immediately return the outpoint as
UTXO dictionary in the common format, making the tests more readable and avoiding unnecessary duplication:
```
utxo1 = self.create_outpoints(node1, outputs=[{addr1: value1}])[0]
utxo2 = self.create_outpoints(node2, outputs=[{addr2: value2}])[0]
node.createrawtransaction([utxo1, utxo2], .....)
```
Tests are switched to work with UTXO-objects rather than two individual txid/vout variables accordingly.
The `find_output` helper is removed, as it seems generally a bad idea to search for an outpoint only based on the output value. If that's really ever needed in the future, it makes probably more sense to add it as an additional parameter to `find_vout_of_address`. Note that `find_output` supported specifying a block-hash for where to look for the transaction (being passed on to the `getrawtransaction` RPC). This seems to be unneeded, as txids are always unique and for the only test that used that parameter (rpc_psbt.py) there was no observed difference in run-time, so it was not reintroduced in the new helper.
There are still some `find_vout_of_address` calls remaining, used for detecting change outputs or for whenever the sending happens via `sendrawtransaction` instead, so this PR tackles not all, but the most common case.
ACKs for top commit:
achow101:
ACK 50d1ac1207
BrandonOdiwuor:
ACK 50d1ac1207
maflcko:
ACK 50d1ac1207 🖨
Tree-SHA512: af2bbf13a56cc840fefc1781390cf51625f1e41b3c030f07fc9abb1181b2d414ddbf795e887db029e119cbe45de14f7c987c0cba72ff0b8953080ee218a7915a
856325fac1 lint: Add `lint-qt-translation.py` (Hennadii Stepanov)
294a018bf5 qt: Avoid error prone leading spaces in translatable strings (Hennadii Stepanov)
d8298e7f06 qt, refactor: Drop superfluous type conversions (Hennadii Stepanov)
Pull request description:
While working on the GUI translation via Transifex web interface, I found it error-prone to have leading whitespace in translatable strings. This is because it is very easy to unintentionally drop them in translations unnoticed.
Fixed all current cases. Added a linter to prevent similar cases in the future.
ACKs for top commit:
furszy:
utACK 856325f
Tree-SHA512: b1ca5effb2db6649e1e99382de79acf3a9f81cc9dad434db5623338489e597897e8addd60c1ab3dcc7506ae62753a7a4ad5a41d7a865f8fcdf94348b54baa7e7
51e4dc49f5 gui: Show error if unrecognized command line args are present (John Moffett)
Pull request description:
Fixes https://github.com/bitcoin-core/gui/issues/741
Starting bitcoin-qt with non-hyphen ("-") arguments causes it to silently ignore any later valid options. For instance, invoking `bitcoin-qt -server=1 foo -regtest` on a fresh install will run `mainnet` instead of `regtest`.
This change makes the client exit with an error message if any such "loose" arguments are encountered. This mirrors how `bitcoind` handles it:
c6287faae4/src/bitcoind.cpp (L127-L132)
However, BIP-21 `bitcoin:` payment URIs are still allowed, but only if they're not followed by any additional options.
ACKs for top commit:
maflcko:
lgtm ACK 51e4dc49f5
hernanmarino:
tested ACK 51e4dc49f5
pablomartin4btc:
tACK 51e4dc49f5
hebasto:
ACK 51e4dc49f5, I have reviewed the code and it looks OK.
Tree-SHA512: 3997a7a9a747314f13e118aee63e8679e00ed832d9c6f115559a4c39c9c4091572207c60e362cb4c19fc8da980d4b0b040050aa70c5ef84a855cb7e3568bbf13
This should cut some include bloat and seems fine to do, because
prevector exists primarily to represent scripts.
Also, add missing includes to script.h and addresstype.h
Since Python 3.9, type hinting has become a little less awkward, as for
collection types one doesn't need to import the corresponding
capitalized types (`Dict`, `List`, `Set`, `Tuple`, ...) anymore, but can
use the built-in types directly. [1] [2]
This commit applies the replacement for all Python scripts (i.e. in the
contrib and test folders) for the basic types:
- typing.Dict -> dict
- typing.List -> list
- typing.Set -> set
- typing.Tuple -> tuple
[1] https://docs.python.org/3.9/whatsnew/3.9.html#type-hinting-generics-in-standard-collections
[2] https://peps.python.org/pep-0585/#implementation for a list of type
Before trying to unlock a wallet, first check if it has private keys
disabled. If so, there is no need to unlock.
Note that such wallets are not expected to occur in typical usage.
However bugs in previous versions allowed such wallets to be created,
and so we need to handle them.
986d7fed05 depends: zeromq 4.3.5 (fanquake)
Pull request description:
First new point release of zeromq in two and a half years. Mostly bug fixes; the project also completed a relicense to the "Mozilla Public License".
See https://github.com/zeromq/libzmq/releases/tag/v4.3.5.
ACKs for top commit:
hebasto:
ACK 986d7fed05, I have reviewed the code and it looks OK.
TheCharlatan:
ACK 986d7fed05
Tree-SHA512: cdd6abfbbe10873c1ca267fed648c2e6ff17a4aff50c414924006e63fa39d501e803f8893a5cd966a2078b5c077f2578e482483e6723ea6f5760f16211d40998
fae379b6b1 build: Bump minimum supported Clang to clang-13 (MarcoFalke)
fab1ef9512 Bump .python-version from 3.9.17 to 3.9.18 (MarcoFalke)
Pull request description:
All supported operating systems ship with clang-13 (or later), so bump the minimum to that and remove now unused workarounds for previous clang bugs.
For reference:
* https://packages.debian.org/bullseye/clang-13
* https://packages.ubuntu.com/jammy/clang (`clang-14`) and https://packages.ubuntu.com/jammy/clang-15
* CentOS-like 8/9 Stream: All Clang versions from 13 to 15
* FreeBSD 12/13: All Clang version from 13 to 16
* OpenSuse Tumbleweed ships with https://software.opensuse.org/package/clang (`clang16`); No idea about OpenSuse Leap
This is for Bitcoin Core 27.0 in 2024 (next year), not the soon upcoming 26.0 next month.
ACKs for top commit:
fanquake:
ACK fae379b6b1
Tree-SHA512: 8ed2b227de39b60d3f004daa4a38ea66fe005988bd977046a40613fba847d88d272925732f24777c00264abb99e25874b05b4b9243868d304eba84b450835ccc
fa25e8b0a1 doc: Recommend lint image build on every call (MarcoFalke)
faf70c1f33 Bump python minimum version to 3.9 (MarcoFalke)
fa8996b930 ci: Bump i686_multiprocess.sh to latest Ubuntu LTS (MarcoFalke)
Pull request description:
All supported operating systems ship with python 3.9 (or later), so bumping the minimum should not cause any issues. A bump will allow new code to use new python 3.9 features.
For reference:
* https://packages.debian.org/bullseye/python3
* https://packages.ubuntu.com/focal/python3.9
* FreeBSD 12/13 also ships with 3.9
* CentOS-like 8/9 also ships with 3.9 (and 3.11)
* OpenSuse Leap also ships with 3.9 (and 3.11) https://software.opensuse.org/package/python311-base
This is for Bitcoin Core 27.0 in 2024 (next year), not the soon upcoming 26.0 next month.
ACKs for top commit:
Sjors:
ACK fa25e8b0a1
jamesob:
ACK fa25e8b0a1 ([`jamesob/ackr/28211.1.MarcoFalke.bump_python_minimum_supp`](https://github.com/jamesob/bitcoin/tree/ackr/28211.1.MarcoFalke.bump_python_minimum_supp))
Tree-SHA512: 86c9f6ac4b5ba94a62ee6a6062dd48a8295d8611a39cdb5829f4f0dbc77aaa1a51edccc7a99275bf699143ad3a6fe826de426d413e5a465e3b0e82b86d10c32e
1ac5584f80 doc: remove release note fragments for 26.x branch (fanquake)
799ce4d050 build: bump version to 26.99 (fanquake)
Pull request description:
26.x has been branched. Delete all release note fragments from this branch.
Bump master version to 26.99.
ACKs for top commit:
stickies-v:
ACK 1ac5584f80
Tree-SHA512: d7d0c26333ed026460fb648ff5cb8f2f3abe150b47dcf011a563b8fcaad6efe59688f3aa2f23c246e003e37d9b612bb9b9f017ad17cf575455e3f73c6948cff8
This commit introduces a helper `create_outpoints` to execute the
`send` RPC and immediately return the target address outpoints as UTXO
dictionary in the common format, making the tests more readable and
avoiding unnecessary duplication.
Currently the shutdown function resets the kernel before the
chainman and scheduler. Invert this order by resetting the kernel
last, since they might rely on the kernel.
No change in behavior, this just moves code which updates transaction state to
a new method so it can be used after offline processes such as wallet
migration.
Also add test to make sure this doesn't get broken in the future.
This was breaking vector<bool> serialization in multiprocess code because
template current deduction guides would make it appear like vector<bool> could
be converted to a span, but then the actual conversion to span would fail.
Coin serialize method segfaults if IsSpent condition is true. This caused
multiprocess code to segfault when serializing the Coin& output argument to of
the Node::getUnspentOutput method if the coin was not found. Segfault could be
triggered by double clicking and viewing transaction details in the GUI
transaction list.
Fix this by replacing Coin& output argument with optional<Coin> return value to
avoid trying to serializing spent coins.
With `queuedTx` owning the `CTransactionRef` shared ptrs, they (and
the managed objects) are entirely allocated on the heap. In
`DisconnectedBlockTransactions::DynamicMemoryUsage`, we account for
the 2 pointers that make up the shared_ptr, but not for the managed
object (CTransaction) or the control block.
Prior to this commit, by calculating the `RecursiveDynamicUsage` on
a `CTransaction` whenever modifying `cachedInnerUsage`, we account
for the dynamic usage of the `CTransaction`, i.e. the `vins` and
`vouts` vectors, but we do not account for the `CTransaction`
object itself, nor for the `CTransactionRef` control block.
This means prior to this commit, `DynamicMemoryUsage` underestimates
dynamic memory usage by not including the `CTransaction` objects and
the shared ptr control blocks.
Fix this by calculating `RecursiveDynamicUsage` on the
`CTransactionRef` instead of the `CTransaction` whenever modifying
`cachedInnerUsage`.
In `AddTransactionsToBlock` description comment we have the asuumption
that callers will never pass multiple transactions with the same txid
We are asserting to assume that does not happen.
Bitcoind nodes send getaddr msgs only to outbound nodes (and ignore those
received by outgoing connections). The python p2p node should mirror
this behavior by not sending a getaddr message when it is not the
initiator of the connection.
It is unclear what the goal of this check is, given that the value may
need to be set lower for the mimimum supported version of compilers that
forgot to bump the value, see
https://github.com/bitcoin/bitcoin/pull/28349#issuecomment-1745143612 .
The minimum supported compiler versions are already documented in
doc/dependencies.md and using an older compiler will already result in a
compile failure, so this check can be removed as redundant. Especially
given that it is only included in one file, where iwyu suggests to
remove it.
Extract the logic for calculating & maintaining inbound connection limits to be
a member within connman for consistency with other maximum connection limits.
Note that we now limit m_max_inbound to 0 and don't call
AttemptToEvictConnection() when we don't have any inbounds.
Previously, nMaxInbound could become negative if the user ran with a low
-maxconnections, which didn't break any logic but didn't make sense.
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
Currently the logic is fragmented between init and connman. Encapsulating this
logic within connman allows for less mental overhead and easier reuse in tests.
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
This checks whether the ARMv8.5 extensions RNDR and RNDRRS
are available and uses them for random entropy purposes.
They are functionally identical to the x86 RDRAND/RDSEED
extensions and are used in a similar manner.
Included a test that checks the functionality of setting
the first param of getnetworkhashps to negative value returns
the average network hashes per second from the last difficulty change.
Co-authored-by: ismaelsadeeq <ask4ismailsadiq@gmail.com>
as it was only needed for GetServicesNames(). This potentially avoids needlessly
compiling the 500 lines of protocol.h in the 35 files other than rpc/net.cpp
that include rpc/util.h.
Drop an unneeded CPubKey forward declaration. The other IWYU suggestions would
require more extensive changes in other files.
Add 3 already-missing include headers in other translation units that are needed
to compile without protocol.h in rpc/util.h, as it includes netaddress.h, which
in turn includes util/strencodings.h.
All code in this repo uses <util/fs.h>, except for a few lines. This is
confusing and potentially dangerous, if the safe <util/fs.h> wrappers
are not used.
as string_view is optimized to be trivially copiable, and in these use cases we
only perform read operations on the passed object.
These utility methods are called by quite a few RPCs and tests, as well as by each other.
$ git grep "ParseHashV\|ParseHashO\|ParseHexV\|ParseHexO" | wc -l
61
It's recommended to restrict the possible application binary interfaces that can be used when setting `MemoryDenyWriteExecute=true` to ensure it cannot be circumvented.
Starting bitcoin-qt with non-dash ("-") arguments causes it to
silently ignore any later valid options. This change makes the
client exit with an error message if any such "loose" arguments
are encountered.
However, allow BIP-21 'bitcoin:' URIs only if no other options
follow.
Constructs like
```cpp
AssertLockNotHeld(m);
LOCK(m);
```
are equivalent to
```cpp
LOCK(m);
```
for non-recursive mutexes, so it is ok to omit `AssertLockNotHeld()` in
such cases.
When calculating a median fee for a confirmation target at a particular
threshold, we analyse buckets in ranges rather than individually in
case some buckets have very little data. This patch ensures the breaks
between ranges are independent of the the confirmation target.
Could be verified with
$ ./configure CC=clang CXX=clang++ CXXFLAGS='-Wthread-safety -Wthread-safety-negative'
$ make clean
$ make 2>&1 | grep m_banned_mutex
TEST_RUNNER_PORT_MIN:"14000"# Must be larger than 12321, which is used for the http cache. See https://cirrus-ci.org/guide/writing-tasks/#http-cache
CI_FAILFAST_TEST_LEAVE_DANGLING:"1"# Cirrus CI does not care about dangling processes and setting this variable avoids killing the CI script itself on error
CCACHE_MAXSIZE:"200M"
CCACHE_DIR:"/tmp/ccache_dir"
CCACHE_NOHASHDIR:"1"# Debug info might contain a stale path if the build dir changes, but this is fine
# https://cirrus-ci.org/guide/persistent-workers/
#
# It is possible to select a specific persistent worker by label. Refer to the
# Cirrus CI docs for more details.
#
# Generally, a persistent worker must run Ubuntu 23.04+ or Debian 12+.
# Specifically,
# - apt-get is required due to PACKAGE_MANAGER_INSTALL
# - podman-docker-4.1+ is required due to the use of `podman` when
# RESTART_CI_DOCKER_BEFORE_RUN is set and 4.1+ due to the bugfix in 4.1
# - To register the persistent worker, open a `screen` session and run:
#
# ```
# RESTART_CI_DOCKER_BEFORE_RUN=1 screen cirrus worker run --labels type=todo_fill_in_type --token todo_fill_in_token
# ```
#
# The following specific types should exist, with the following requirements:
# - small: For an x86_64 machine, recommended to have 2 CPUs and 8 GB of memory.
# - medium: For an x86_64 machine, recommended to have 4 CPUs and 16 GB of memory.
# - mantic: For a machine running the Linux kernel shipped with exaclty Ubuntu Mantic 23.10. The machine is recommended to have 4 CPUs and 16 GB of memory.
# - arm64: For an aarch64 machine, recommended to have 2 CPUs and 8 GB of memory.
skip:$CIRRUS_REPO_FULL_NAME == "bitcoin-core/gui" && $CIRRUS_PR == "" # No need to run on the read-only mirror, unless it is a PR. https://cirrus-ci.org/guide/writing-tasks/#conditional-task-execution
description:For example, “`std::thread`”, “Qt5 GUI and async GUI design” or “basic understanding of Bitcoin mining and the Bitcoin Core RPC interface”.
description:For example, “`std::thread`”, “Qt6 GUI and async GUI design” or “basic understanding of Bitcoin mining and the Bitcoin Core RPC interface”.
value:|
* Compiling Bitcoin Core from source
* Running the C++ unit tests and the Python functional tests
timeout-minutes:360# Use maximum time, see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes. Assuming a worst case time of 1 hour per commit, this leads to a --max-count=6 below.
timeout-minutes:360# Use maximum time, see https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes. Assuming a worst case time of 1 hour per commit, this leads to a --max-count=6 below.
# Can't use ctest here like other jobs as we don't have a CMake build tree.
run:|
./bin/test_bitcoin.exe -l test_suite # Intentionally run sequentially here, to catch test case failures caused by dirty global state from prior test cases.
cmake_dependent_option(ENABLE_IPC "Buildmultiprocessbitcoin-nodeandbitcoin-guiexecutablesinadditiontomonolithicbitcoindandbitcoin-qtexecutables." ON "NOTWIN32" OFF)
cmake_dependent_option(WITH_EXTERNAL_LIBMULTIPROCESS "BuildwithexternallibmultiprocesslibraryinsteadofwithlocalgitsubtreewhenENABLE_IPCisenabled.Thisisnotnormallyrecommended,butcanbeusefulfordevelopinglibmultiprocessitself." OFF "ENABLE_IPC" OFF)
should be used to discuss complicated or controversial consensus or P2P protocol changes before working on
a patch set.
Archives can be found on [https://gnusha.org/pi/bitcoindev/](https://gnusha.org/pi/bitcoindev/).
Contributor Workflow
@@ -80,7 +80,7 @@ facilitates social contribution, easy testing and peer review.
To contribute a patch, the workflow is as follows:
1. Fork repository ([only for the first time](https://docs.github.com/en/get-started/quickstart/fork-a-repo))
1. Fork repository ([only for the first time](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo))
1. Create topic branch
1. Commit patches
@@ -115,13 +115,14 @@ fixes or code moves with actual code changes.
Make sure each individual commit is hygienic: that it builds successfully on its
own without warnings, errors, regressions, or test failures.
This means tests must be updated in the same commit that changes the behavior.
Commit messages should be verbose by default consisting of a short subject line
(50 chars max), a blank line and detailed explanatory text as separate
paragraph(s), unless the title alone is self-explanatory (like "Correct typo
in init.cpp") in which case a single title line is sufficient. Commit messages should be
helpful to people reading your code in the future, so explain the reasoning for
your decisions. Further explanation [here](https://chris.beams.io/posts/git-commit/).
your decisions. Further explanation [here](https://cbea.ms/git-commit/).
If a particular commit references another issue, please add the reference. For
example: `refs #1234` or `fixes #4321`. Using the `fixes` or `closes` keywords
@@ -148,11 +149,11 @@ the pull request affects. Valid areas as:
-`net` or `p2p` for changes to the peer-to-peer network code
-`refactor` for structural changes that do not change behavior
-`rpc`, `rest` or `zmq` for changes to the RPC, REST or ZMQ APIs
-`script` for changes to the scripts and tools
-`contrib` or `cli` for changes to the scripts and tools
-`test`, `qa` or `ci` for changes to the unit tests, QA tests or CI code
-`util` or `lib` for changes to the utils or libraries
-`wallet` for changes to the wallet code
-`build` for changes to the GNU Autotools or MSVC builds
-`build` for changes to CMake
-`guix` for changes to the GUIX reproducible builds
Examples:
@@ -182,7 +183,7 @@ for more information on helping with translations.
### Work in Progress Changes and Requests for Comments
If a pull request is not to be considered for merging (yet), please
prefix the title with [WIP] or use [Tasks Lists](https://docs.github.com/en/github/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#task-lists)
prefix the title with [WIP] or use [Tasks Lists](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#task-lists)
in the body of the pull request to indicate tasks are pending.
### Address Feedback
@@ -401,7 +402,7 @@ about:
- It may be because your code is too complex for all but a few people, and those people
may not have realized your pull request even exists. A great way to find people who
are qualified and care about the code you are touching is the
You can import a key by running the following command with that individual’s fingerprint: `gpg --keyserver hkps://keys.openpgp.org --recv-keys "<fingerprint>"` Ensure that you put quotes around fingerprints containing spaces.
AC_MSG_NOTICE([[We could not detect the boost libraries (version $1 or higher). If you have a staged boost library (still not installed) please specify \$BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.]])
else
AC_MSG_NOTICE([Your boost libraries seems to old (version $_version).])
fi
# execute ACTION-IF-NOT-FOUND (if present):
ifelse([$3], , :, [$3])
else
AC_DEFINE(HAVE_BOOST,,[define if the Boost library is available])
AC_ARG_WITH([qt-incdir],[AS_HELP_STRING([--with-qt-incdir=INC_DIR],[specify qt include path (overridden by pkgconfig)])], [qt_include_path=$withval], [])
AC_ARG_WITH([qt-libdir],[AS_HELP_STRING([--with-qt-libdir=LIB_DIR],[specify qt lib path (overridden by pkgconfig)])], [qt_lib_path=$withval], [])
AC_ARG_WITH([qt-plugindir],[AS_HELP_STRING([--with-qt-plugindir=PLUGIN_DIR],[specify qt plugin path (overridden by pkgconfig)])], [qt_plugin_path=$withval], [])
AC_ARG_WITH([qt-translationdir],[AS_HELP_STRING([--with-qt-translationdir=PLUGIN_DIR],[specify qt translation path (overridden by pkgconfig)])], [qt_translation_path=$withval], [])
AC_ARG_WITH([qt-bindir],[AS_HELP_STRING([--with-qt-bindir=BIN_DIR],[specify qt bin path])], [qt_bin_path=$withval], [])
AC_ARG_WITH([qtdbus],
[AS_HELP_STRING([--with-qtdbus],
[enable DBus support (default is yes if qt is enabled and QtDBus is found, except on Android)])],
[use_dbus=$withval],
[use_dbus=auto])
dnl Android doesn't support D-Bus and certainly doesn't use it for notifications
case $host in
*android*)
if test "$use_dbus" != "yes"; then
use_dbus=no
fi
;;
esac
AC_SUBST(QT_TRANSLATION_DIR,$qt_translation_path)
])
dnl Find Qt libraries and includes.
dnl
dnl BITCOIN_QT_CONFIGURE([MINIMUM-VERSION])
dnl
dnl Outputs: See _BITCOIN_QT_FIND_LIBS
dnl Outputs: Sets variables for all qt-related tools.
Visual Studio 2022 is minimum required to build Bitcoin Core.
Solution and project files to build with `msbuild` or Visual Studio can be found in the `build_msvc` directory.
To build Bitcoin Core from the command-line, it is sufficient to only install the [Visual Studio Build Tools](https://visualstudio.microsoft.com/downloads/) component.
The "Desktop development with C++" workload must be installed as well.
Building with Visual Studio is an alternative to the Linux based [cross-compiler build](../doc/build-windows.md).
Prerequisites
---------------------
To build [dependencies](../doc/dependencies.md) (except for [Qt](#qt)),
the default approach is to use the [vcpkg](https://vcpkg.io) package manager from Microsoft:
To build Bitcoin Core with the GUI, a static build of Qt is required.
1. Download a single ZIP archive of Qt source code from https://download.qt.io/official_releases/qt/ (e.g., [`qt-everywhere-opensource-src-5.15.10.zip`](https://download.qt.io/official_releases/qt/5.15/5.15.10/single/qt-everywhere-opensource-src-5.15.10.zip)), and expand it into a dedicated folder. The following instructions assume that this folder is `C:\dev\qt-source`.
2. Open "x64 Native Tools Command Prompt for VS 2022", and input the following commands:
One could speed up building with [`jom`](https://wiki.qt.io/Jom), a replacement for `nmake` which makes use of all CPU cores.
To build Bitcoin Core without Qt, unload or disable the `bitcoin-qt`, `libbitcoin_qt` and `test_bitcoin-qt` projects.
Building
---------------------
1. Use Python to generate `*.vcxproj` for the Visual Studio 2022 toolchain from Makefile:
```cmd
python build_msvc\msvc-autogen.py
```
2. An optional step is to adjust the settings in the `build_msvc` directory and the `common.init.vcxproj` file. This project file contains settings that are common to all projects such as the runtime library version and target Windows SDK version. The Qt directories can also be set. To specify a non-default path to a static Qt package directory, use the `QTBASEDIR` environment variable.
3. To build from the command-line with the Visual Studio toolchain use:
Alternatively, open the `build_msvc/bitcoin.sln` file in Visual Studio.
Security
---------------------
[Base address randomization](https://learn.microsoft.com/en-us/cpp/build/reference/dynamicbase-use-address-space-layout-randomization) is used to make Bitcoin Core more secure. When building Bitcoin using the `build_msvc` process base address randomization can be disabled by editing `common.init.vcproj` to change `RandomizedBaseAddress` from `true` to `false` and then rebuilding the project.
To check if `bitcoind` has `RandomizedBaseAddress` enabled or disabled run
```
.\dumpbin.exe /headers src/bitcoind.exe
```
If is it enabled then in the output `Dynamic base` will be listed in the `DLL characteristics` under `OPTIONAL HEADER VALUES` as shown below
```
8160 DLL characteristics
High Entropy Virtual Addresses
Dynamic base
NX compatible
Terminal Server Aware
```
This may not disable all stack randomization as versions of windows employ additional stack randomization protections. These protections must be turned off in the OS configuration.
<!-- Sometimes the version in the registry has to .0 suffix, and sometimes it doesn't. Check and add it -->
<WindowsTargetPlatformVersion_10Condition="'$(WindowsTargetPlatformVersion_10)' != '' and !$(WindowsTargetPlatformVersion_10.EndsWith('.0'))">$(WindowsTargetPlatformVersion_10).0</WindowsTargetPlatformVersion_10>
This directory contains scripts for each build step in each build stage.
### Running a Stage Locally
## Running a Stage Locally
Be aware that the tests will be built and run in-place, so please run at your own risk.
If the repository is not a fresh git clone, you might have to clean files from previous builds or test runs first.
The ci needs to perform various sysadmin tasks such as installing packages or writing to the user's home directory.
While it should be fine to run
the ci system locally on you development box, the ci scripts can generally be assumed to have received less review and
the ci system locally on your development box, the ci scripts can generally be assumed to have received less review and
testing compared to other parts of the codebase. If you want to keep the work tree clean, you might want to run the ci
system in a virtual machine with a Linux operating system of your choice.
To allow for a wide range of tested environments, but also ensure reproducibility to some extent, the test stage
requires `bash`, `docker`, and `python3` to be installed. To install all requirements on Ubuntu, run
requires `bash`, `docker`, and `python3` to be installed. To run on different architectures than the host `qemu` is also required. To install all requirements on Ubuntu, run
The files starting with `0n` (`n` greater than 0) are the scripts that are run
in order.
### Cache
## Cache
In order to avoid rebuilding all dependencies for each build, the binaries are
cached and re-used when possible. Changes in the dependency-generator will
cached and reused when possible. Changes in the dependency-generator will
trigger cache-invalidation and rebuilds as necessary.
## Configuring a repository for CI
### Primary repository
To configure the primary repository, follow these steps:
1. Register with [Cirrus Runners](https://cirrus-runners.app/) and purchase runners.
2. Install the Cirrus Runners GitHub app against the GitHub organization.
3. Enable organisation-level runners to be used in public repositories:
1.`Org settings -> Actions -> Runner Groups -> Default -> Allow public repos`
4. Permit the following actions to run:
1. cirruslabs/cache/restore@\*
1. cirruslabs/cache/save@\*
1. docker/setup-buildx-action@\*
1. actions/github-script@\*
### Forked repositories
When used in a fork the CI will run on GitHub's free hosted runners by default.
In this case, due to GitHub's 10GB-per-repo cache size limitations caches will be frequently evicted and missed, but the workflows will run (slowly).
It is also possible to use your own Cirrus Runners in your own fork with an appropriate patch to the `REPO_USE_CIRRUS_RUNNERS` variable in ../.github/workflows/ci.yml
NB that Cirrus Runners only work at an organisation level, therefore in order to use your own Cirrus Runners, *the fork must be within your own organisation*.
exportCI_IMAGE_NAME_TAG="mirror.gcr.io/ubuntu:24.04"# Check that https://packages.ubuntu.com/noble/g++-arm-linux-gnueabihf (version 13.x, similar to guix) can cross-compile
exportCI_IMAGE_PLATFORM="linux/arm64"
exportUSE_BUSY_BOX=true
exportRUN_UNIT_TESTS=true
exportRUN_FUNCTIONAL_TESTS=false
exportGOAL="install"
exportCI_LIMIT_STACK_SIZE=1
# -Wno-psabi is to disable ABI warnings: "note: parameter passing for argument of type ... changed in GCC 7.1"
# This could be removed once the ABI change warning does not show up by default
exportCI_IMAGE_NAME_TAG="docker.io/ubuntu:23.10"# This version will reach EOL in Jul 2024, and can be replaced by "ubuntu:24.04" (or anything else that ships the wanted clang version).
exportCI_IMAGE_NAME_TAG="docker.io/ubuntu:23.10"# This version will reach EOL in Jul 2024, and can be replaced by "ubuntu:24.04" (or anything else that ships the wanted clang version).
exportCI_CONTAINER_CAP="--cap-add SYS_PTRACE"# If run with (ASan + LSan), the container needs access to ptrace (https://github.com/google/sanitizers/issues/764)
# Use minimum supported python3.10 and gcc-11, see doc/dependencies.md
exportPACKAGES="gcc-11 g++-11 python3-zmq"
exportDEP_OPTS="CC=gcc-11 CXX=g++-11"
exportTEST_RUNNER_EXTRA="--previous-releases --coverage --extended --exclude feature_dbcrash"# Run extended tests so that coverage does not fail, but exclude the very slow dbcrash
exportTEST_RUNNER_EXTRA="--previous-releases --coverage --extended --exclude feature_dbcrash"# Run extended tests so that coverage does not fail, but exclude the very slow dbcrash
exportCI_IMAGE_NAME_TAG="docker.io/ubuntu:23.10"# This version will reach EOL in Jul 2024, and can be replaced by "ubuntu:24.04" (or anything else that ships the wanted clang version).
exportCI_IMAGE_NAME_TAG="docker.io/ubuntu:23.10"# This version will reach EOL in Jul 2024, and can be replaced by "ubuntu:24.04" (or anything else that ships the wanted clang version).
exportTEST_RUNNER_EXTRA="--exclude feature_init,rpc_bind,feature_bind_extra"# Excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
# bind tests excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
exportTEST_RUNNER_EXTRA="--exclude feature_init,rpc_bind,feature_bind_extra"# Excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
exportTEST_RUNNER_EXTRA="--exclude rpc_bind,feature_bind_extra"# Excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
exportCI_IMAGE_NAME_TAG="docker.io/amd64/ubuntu:22.04"# Check that Jammy can cross-compile to win64
exportCI_IMAGE_NAME_TAG="mirror.gcr.io/ubuntu:24.04"# Check that https://packages.ubuntu.com/noble/g++-mingw-w64-x86-64-posix (version 13.x, similar to guix) can cross-compile
# Though, exclude those with newlines to avoid parsing problems.
python3 -c 'import os; [print(f"{key}={value}") for key, value in os.environ.items() if "\n" not in value and "HOME" != key and "PATH" != key and "USER" != key]'| tee /tmp/env
# System-dependent env vars must be kept as is. So read them from the container.
docker run --rm "${CI_IMAGE_NAME_TAG}" bash -c "env | grep --extended-regexp '^(HOME|PATH|USER)='"| tee --append /tmp/env
# Env vars during the build can not be changed. For example, a modified
# $MAKEJOBS is ignored in the build process. Use --cpuset-cpus as an
# approximation to respect $MAKEJOBS somewhat, if cpuset is available.
echo"Creating $CI_IMAGE_NAME_TAG container to run in"
DOCKER_BUILDKIT=1 docker build \
# Use buildx unconditionally
# Using buildx is required to properly load the correct driver, for use with registry caching. Neither build, nor BUILDKIT=1 currently do this properly
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.