This fixes an error reported by Antoine Poinsot <darosior@protonmail.com> in
https://github.com/bitcoin-core/libmultiprocess/issues/123 that does not happen
in master, but does happen with https://github.com/bitcoin/bitcoin/pull/10102
applied, where if Ctrl-C is pressed when `bitcoin-node` is started, it is
handled by both `bitcoin-node` and `bitcoin-wallet` processes, causing the
wallet to shutdown abruptly instead of waiting for the node and shutting down
cleanly.
This change fixes the problem by having the wallet process print to stdout when
it receives a Ctrl-C signal but not otherwise react, letting the node shut
everything down cleanly.
Currently this code is not called in unit tests. Calling should make it
possible to write tests for things like IPC exceptions being thrown during
shutdown.
Use EventLoopRef to avoid reference counting bugs and be more exception safe
and deal with removal of addClient/removeClient methods in
https://github.com/bitcoin-core/libmultiprocess/pull/160
A test update is also required due to
https://github.com/bitcoin-core/libmultiprocess/pull/160 to deal with changed
reference count semantics. In IpcPipeTest(), it is now necessary to destroy
the client Proxy object instead of just the client Connection object to
decrease the event loop reference count and allow the loop to exit so the test
does not hang on shutdown.
fa68dcb207 ci: Add missing errexit to lint CI install (MarcoFalke)
fa535a6de7 ci: Allow running CI in worktrees (MarcoFalke)
faf6a04597 ci: Clean UID/GID mismatch (MarcoFalke)
Pull request description:
Fixes#30028 (modulo lint and tidy CI).
The error on current master in a worktree is:
```
$ git worktree add ./main origin/master && cd ./main
$ MAKEJOBS="-j$(nproc)" FILE_ENV="./ci/test/00_setup_env_mac_cross.sh" ./ci/test_run_all.sh
...
+ git config --global ci.base-install-done true
fatal: not a git repository: /root/b-c-ci/.git/worktrees/main
```
So just use a plain file, instead of git. Also, enable pipefail while touching this bash script.
ACKs for top commit:
willcl-ark:
tACK fa68dcb207
Tree-SHA512: 0ce360a80883b4aa655fe8a99c38eb54a465b17c7cdb0a69a2d886ff78da32d6af996412ffc5b0db0322acafa9650619838573484de8243dc41594a04a6e17ec
3473986fe1 contrib: tracing: Correctly read msg type in p2p_monitor.py (David Gumberg)
Pull request description:
This fixes a bug in the contrib tracing script `p2p_monitor.py`. currently the script fails to read the `msg_type` of inbound and outbound messages, which is useful in the per-peer message view.
<details>
<summary>Screenshot of p2p_monitor.py on master</summary>
</details>
<details>
<summary>Screenshot of p2p_monitor.py on this branch</summary>
</details>
ACKs for top commit:
yuvicc:
ACK 3473986fe1
janb84:
ut ACK 3473986fe1
0xB10C:
ACK 3473986fe1
Tree-SHA512: 94da0dc35072933a20ef693024855b3c382fc6f5ae0a3108d092d7aa5a4004df478f5de07b80f675be13e00f3f4596b0f34c49ec1d8d2c38a15797dcf86c2a56
578ea3eedb test: round difficulty and networkhashps (Sjors Provoost)
Pull request description:
Both are rational numbers. Client software should only use them to display information to humans. Followup calculations should use the underlying values such as target.
Therefore it's not necessary to test the handling of these floating point values. Round them down to avoid spurious test failures.
Fixes#32515
ACKs for top commit:
Prabhat1308:
Code Review ACK [`578ea3e`](578ea3eedb)
achow101:
ACK 578ea3eedb
w0xlt:
Code review ACK 578ea3eedb
janb84:
ACK 578ea3eedb
Tree-SHA512: 5fc63c73ad236b7cd55c15da0f1d1e6b45e4289d252147a86717bf77d79f897f42c3e38aa514df6a4a8deca10c87a8710b61b454c533ad56b0daf738365f426c
b184f5c87c test: update BIP340 test vectors and implementation (variable-length messages) (Sebastian Falbesoner)
Pull request description:
This PR updates the Schnorr signatures implementation in the functional test framework to the latest BIP changes (see https://github.com/bitcoin/bips/pull/1446,commit 200f9b26fe0a2f235a2af8b30c4be9f12f6bc9cb) and syncs the [test vectors](https://github.com/bitcoin/bips/blob/master/bip-0340/test-vectors.csv) accordingly. Practically, we probably don't need non-32-bytes message signing/verifying any time soon, but it seems good practice anyways to update.
ACKs for top commit:
stratospher:
ACK b184f5c.
achow101:
ACK b184f5c87c
real-or-random:
utACK b184f5c87c
jonasnick:
utACK b184f5c87c
Tree-SHA512: b566823aa0f1cd7151215178c57551d772b338d022ccb2807a0df2670df6d59c4b63a6fc936708ccf2922c7e59f474f544adaafc4aea731bfd896250c0d45fa6
272cd09b79 log: Use warning level while scanning wallet dir (MarcoFalke)
1777644367 qa, wallet: Verify warning when failing to scan (Hodlinator)
893e51ffeb wallet: Correct dir iteration error handling (Hodlinator)
Pull request description:
Make wallet DB properly detect and report failure to scan wallet directory. Seems to have been broken since moving from Boost to `std::filesystem`.
Found while reviewing: https://github.com/bitcoin/bitcoin/pull/31410#pullrequestreview-2604068753
ACKs for top commit:
achow101:
ACK 272cd09b79
maflcko:
re-ACK 272cd09b79 🍽
rkrux:
tACK 272cd09b79
Tree-SHA512: 969afde2e37f885ed0c823dc36d2dbeaa0378639849c6a26f8ac67b4f1997eea95bbcae6d58aef5b716807210f37eb166c0cda7ba1d6caffd34249970833af3a
cd1ae1b4df fuzz: wallet: remove FundTx from FuzzedWallet (brunoerg)
Pull request description:
`FundTx` was used by the `wallet_notifications` target which we recently removed. So it's now unused and can be removed.
ACKs for top commit:
maflcko:
lgtm ACK cd1ae1b4df
kevkevinpal:
ACK [cd1ae1b](cd1ae1b4df)
dergoegge:
utACK cd1ae1b4df
Tree-SHA512: 909cc4c8a0ac2a5f8844993ccf0e725021932888da3591925799145daf9196eadfcd0ebbc74a44f4a245074ded4cb8c3c099513f109ce2681dceff36b5f74bcc
e285e691b7 test: Fix list index out of range error in feature_bip68_sequence.py (zaidmstrr)
Pull request description:
Fixes [#32334](https://github.com/bitcoin/bitcoin/issues/32334)
The test `feature_bip68_sequence.py` fails with `IndexError: list index out of range` error due to a mismatch between the number of inputs requested (at random) and the number of UTXOs available. The error is reproducible with the randomseed:
```
$ ./build/test/functional/feature_bip68_sequence.py --randomseed 6169832640268785903
```
This PR adds a valid upper bound to randomly select the inputs.
ACKs for top commit:
maflcko:
lgtm ACK e285e691b7
Prabhat1308:
re-ACK [`e285e69`](e285e691b7)
theStack:
ACK e285e691b7
Tree-SHA512: 2e5e19d5db2880915f556ed4444abed94e9ceb1ecee5f857df5616040c850dae682aaa4ade3060c48acb16676df92ba81c3af078c1958965e9e874e7bb489388
fa94fd53c9 doc: Explain how to fetch commits directly (MarcoFalke)
Pull request description:
This is often needed, and works better than the existing refspec documentation, because even commits that have been force-pushed away can be fetched (as long as they are not garbage collected on the remote).
ACKs for top commit:
Sjors:
ACK fa94fd53c9
l0rinc:
ACK fa94fd53c9
willcl-ark:
ACK fa94fd53c9
rkrux:
ACK fa94fd53c9
janb84:
ACK fa94fd53c9
Tree-SHA512: b68c0c612e13f501ad4c1c709502e060b0a2d0eb55ef888c7466e2a10bdf3ca63d81b8bd7927de49cde9e29f0b06f8233d51b99d015ae0b39d556854be542b8a
53a996f122 doc: fix transifex 404s (fanquake)
Pull request description:
https://www.transifex.com/bitcoin/bitcoin/ is now a 404.
ACKs for top commit:
maflcko:
lgtm ACK 53a996f122
hebasto:
ACK 53a996f122, I've verified all the links.
Tree-SHA512: 8e698c83095a3d3a225b0bf2ee9c39ad434b2917ead4271ff39a282cea6283710091d1e8b91edafd280bf356dec2bdbe42981aafe4d64f623a975232c5ca848c
8ee8a951c2 doc: taproot became always active in v24.0 (Sjors Provoost)
Pull request description:
Split from #26201.
ACKs for top commit:
maflcko:
lgtm ACK 8ee8a951c2
janb84:
ACK 8ee8a951c2
Tree-SHA512: 1ac6994c6775ca5423f022d1e02e3d531fb7fa295be9940355b8aa9d173787a8d65945a0cf976ab344bcaa3ea8a0f3aa6f8da851325bf475e59375981b115cab
a18e572328 test: more template verification tests (Sjors Provoost)
10c908808f test: move gbt proposal mode tests to new file (Sjors Provoost)
94959b8dee Add checkBlock to Mining interface (Sjors Provoost)
6077157531 ipc: drop BlockValidationState special handling (Sjors Provoost)
74690f4ed8 validation: refactor TestBlockValidity (Sjors Provoost)
Pull request description:
This PR adds the IPC equivalent of the `getblocktemplate` RPC in `proposal` mode.
In order to do so it has `TestBlockValidity` return error reasons as a string instead of `BlockValidationState`. This avoids complexity in IPC code for handling the latter struct.
The new Mining interface method is used in `miner_tests`.
It's not used by the `getblocktemplate` and `generateblock` RPC calls, see https://github.com/bitcoin/bitcoin/pull/31981#discussion_r2096473337
The `inconclusive-not-best-prevblk` check is moved from RPC
code to `TestBlockValidity`.
Test coverage is increased by `mining_template_verification.py`.
Superseedes #31564
## Background
### Verifying block templates (no PoW)
Stratum v2 allows miners to generate their own block template. Pools may wish (or need) to verify these templates. This typically involves comparing mempools, asking miners to providing missing transactions and then reconstructing the proposed block.[^0] This is not sufficient to ensure a proposed block is actually valid. In some schemes miners could take advantage of incomplete validation[^1].
The Stratum Reference Implementation (SRI), currently the only Stratum v2 implementation, collects all missing mempool transactions, but does not yet fully verify the block.[^2]. It could use the `getblocktemplate` RPC in `proposal` mode, but using IPC is more performant, as it avoids serialising up to 4 MB of transaction data as JSON.
(although SRI could use this PR, the Template Provider role doesn't need it, so this is _not_ part of #31098)
[^0]: https://github.com/stratum-mining/sv2-spec/blob/main/06-Job-Declaration-Protocol.md
[^1]: https://delvingbitcoin.org/t/pplns-with-job-declaration/1099/45?u=sjors
[^2]: https://github.com/stratum-mining/stratum/blob/v1.1.0/roles/jd-server/src/lib/job_declarator/message_handler.rs#L196
ACKs for top commit:
davidgumberg:
reACK a18e572328
achow101:
ACK a18e572328
TheCharlatan:
ACK a18e572328
ryanofsky:
Code review ACK a18e572328 just adding another NONFATAL_UNREACHABLE since last review
Tree-SHA512: 1a6c29f45a1666114f10f55aed155980b90104db27761c78aada4727ce3129e6ae7a522d90a56314bd767bd7944dfa46e85fb9f714370fc83e6a585be7b044f1
rsync --archive will preserve owner and group, which is then required to
be handled by adding a git safe.directory workaround.
Remove the need for the workaround by only preserving permissions during
the recursive rsync copy.
a201a99f8c thread-safety: fix annotations with REVERSE_LOCK (Cory Fields)
aeea5f0ec1 thread-safety: add missing lock annotation (Cory Fields)
832c57a534 thread-safety: modernize thread safety macros (Cory Fields)
Pull request description:
This is one of several PRs to cleanup/modernize our threading primitives.
While replacing the old critical section locks in the mining code with a `REVERSE_LOCK`, I noticed that our thread-safety annotations weren't hooked up to it. This PR gets `REVERSE_LOCK` working properly.
Firstly it modernizes the attributes as-recommended by the [clang docs](https://clang.llvm.org/docs/ThreadSafetyAnalysis.html) (ctrl+f for `USE_LOCK_STYLE_THREAD_SAFETY_ATTRIBUTES`). There's a subtle difference between the old `unlock_function` and new `release_capability`, where our `reverse_lock` only works with the latter. I believe this is an upstream bug. I've [reported and attempted a fix here](https://github.com/llvm/llvm-project/pull/139343), but either way it makes sense to me to modernize.
The second adds a missing annotation pointed out by a fixed `REVERSE_LOCK`. Because clang's thread-safety annotations aren't passed through a reference to `UniqueLock` as one may assume (see [here](https://clang.llvm.org/docs/ThreadSafetyAnalysis.html#no-alias-analysis) for more details), `cs_main` has to be listed explicitly as a requirement.
The last commit actually fixes the `reverse_lock` by making it a `SCOPED_LOCK` and using the pattern [found in a clang test](https://github.com/llvm/llvm-project/blob/main/clang/test/SemaCXX/warn-thread-safety-analysis.cpp#L3126). Though the docs don't describe how to accomplish it, the functionality was added [in this commit](6a68efc959). Due to aliasing issues (see link above), in order to work correctly, the original mutex has to be passed along with the lock, so all existing `REVERSE_LOCK`s have been updated. To ensure that the mutexes actually match, a runtime assertion is added.
ACKs for top commit:
fjahr:
re-ACK a201a99f8c
davidgumberg:
reACK a201a99f8c
theuni:
Ok, done. Those last pushes can be ignored. ACKs on a201a99 are still fresh.
ryanofsky:
Code review ACK a201a99f8c. Just dropping 0065b9673db5da2994b0b07c1d50ebfb19af39d0 and fixing incorrect `reverse_lock::lockname` initialization since last review.
TheCharlatan:
Re-ACK a201a99f8c
Tree-SHA512: 2755fae0c41021976a1a633014a86d927f104ccbc8014c01c06dae89af363f92e5bc5d4276ad6d759302ac4679fe02a543758124d48318074db1c370989af7a7
9dfc61d95f test: detect no external signer connected (Sjors Provoost)
0a4ee93529 wallet: use PSBTError::EXTERNAL_SIGNER_NOT_FOUND (Sjors Provoost)
8ba2f9b7c8 refactor: use util::Result for GetExternalSigner() (Sjors Provoost)
Pull request description:
When attempting to sign a transaction involving an external signer, if the device isn't connected we throw an `std::runtime_error`. This prevents the (mainly GUI) code that's actually supposed to handle this case from running.
This PR returns a `PSBTError::EXTERNAL_SIGNER_NOT_FOUND` instead of throwing.
The first commit is a refactor to have `GetExternalSigner()` return a `util::Result<ExternalSigner>` so the caller can decide how to handle the error. There are two other places where call `GetExternalSigner()` which this PR doesn't change (which I think is fine there).
Before:
After (the translation already exist):
Fixes#32426
Additionally use `LogWarning` instead of `std::cerr` for both a missing signer and failure to sign.
ACKs for top commit:
achow101:
ACK 9dfc61d95f
brunoerg:
code review ACK 9dfc61d95f
Tree-SHA512: 22515f4f0b4f50cb0ef532b729e247f11a68be9c90e384942d4277087b2e76806a1cdaa57fb51d5883dacf0a428e5279674aab37cce8c0d3d7de0f96346b8233
Without proper annotations, clang thinks that mutexes are still held for the
duration of a reverse_lock. This could lead to subtle bugs as
EXCLUSIVE_LOCKS_REQUIRED(foo) passes when it shouldn't.
As mentioned in the docs [0], clang's thread-safety analyzer is unable to deal
with aliases of mutexes, so it is not possible to use the lock's copy of the
mutex for that purpose. Instead, the original mutex needs to be passed back to
the reverse_lock for the sake of thread-safety analysis, but it is not actually
used otherwise.
[0]: https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
c7eaac326a depends: capnp 1.2.0 (fanquake)
Pull request description:
See https://github.com/capnproto/capnproto/compare/release-1.1.0...release-1.2.0. We can drop all the patches we are currently applying.
ACKs for top commit:
Sjors:
ACK c7eaac326a
theStack:
ACK c7eaac326a
ryanofsky:
Code review ACK c7eaac326a. Just checked hashes, compared tarball to git and diffed 1.1.0 and 1.2.0 tarballs which showed only minor and expected changes.
Tree-SHA512: 75085ec96952e9693c67531c3d04cd0d7df580dd1df35ce50dff618b29f651674c17a84e9089c6b7ed230e2b4fd0a7f24e2220e983ec00235db9a9d1ee2d7116
6ecb9fc65f chore: use `std::vector<std::byte>` for `BlockManager::ReadRawBlock()` (Roman Zeyde)
Pull request description:
Following [this comment](https://github.com/bitcoin/bitcoin/pull/32540#discussion_r2135820932), this PR changes `BlockManager::ReadRawBlock()` to accept a `std::vector<std::byte>` instead of `std::vector<uint8_t>`, in order to avoid casts during its invocations.
It also adds a new `SpanReader` constructor to allow reading from a span of `std::byte`s (in addition to span of `uint8_t`).
ACKs for top commit:
l0rinc:
ACK 6ecb9fc65f
maflcko:
re-ACK 6ecb9fc65f
TheCharlatan:
Re-ACK 6ecb9fc65f
Tree-SHA512: b0976c34b8da4fa1e6d805a89de2883f48ba431a71069e8c1ae450f48e425cc41aff1a5d479a7d40312a972aaf1f92e9478a985a14a1357c6b3e564e988d03e5
Additionally this commit gives each test its
own function.
The assert_submitblock helper is absorbed into
assert_template.
Review hint:
git show --color-moved=dimmed-zebra
Comments are expanded.
Return BlockValidationState instead of passing a reference.
Lock Chainman mutex instead of cs_main.
Remove redundant chainparams and pindexPrev arguments.
Drop defaults for checking proof-of-work and merkle root.
The ContextualCheckBlockHeader check is moved to after CheckBlock,
which is more similar to normal validation where context-free checks
are done first.
Validation failure reasons are no longer printed through LogError(),
since it depends on the caller whether this implies an actual bug
in the node, or an externally sourced block that happens to be invalid.
When called from getblocktemplate, via BlockAssembler::CreateNewBlock(),
this method already throws an std::runtime_error if validation fails.
Additionally it moves the inconclusive-not-best-prevblk check from RPC
code to TestBlockValidity.
There is no behavior change when callling getblocktemplate with proposal.
Previously this would return a BIP22ValidationResult which can throw for
state.IsError(). But CheckBlock() and the functions it calls only use
state.IsValid().
The final assert is changed into Assume, with a LogError.
Co-authored-by: <Ryan Ofsky <ryan@ofsky.org>
ce90f0c99f rpc, wallet, refactor: Remove non-descriptor errors (pablomartin4btc)
573bcd75d7 wallet, refactor: Remove unused SetupGeneration (pablomartin4btc)
5431f2dc21 wallet, refactor: Remove Legacy warnings and errors (pablomartin4btc)
Pull request description:
Remove dead code due to legacy wallet support removal.
These changes have no impact on functionality. They are transparent to the end user, as legacy wallets can't be created or loaded anymore, so these checks are no longer reached. The legacy-to-descriptor wallet migration flow is not affected either, as these removals are not part of its process.
ACKs for top commit:
achow101:
ACK ce90f0c99f
rkrux:
utACK ce90f0c99f
Tree-SHA512: 9229ad9dda9ff1dece73b5b15a20d69c6ab1ff2c75b2ec430ddbbaeb3467f6a850f53df527bcb4a8114ccbf1aa9c794462d71a8d516aed6f9a9da74edae16feb
0def84d407 test: Verify parent_desc in RPCs (Ava Chow)
2554cee988 test: Enable default wallet for wallet_descriptor.py (Ava Chow)
3fc9d9f241 wallet, rpc: Push the normalized parent descriptor (Ava Chow)
Pull request description:
Instead of prividing the descriptor string as stored in the db, use the normalized descriptor as is done for getaddressinfo's parent_desc field.
Split from #32489
ACKs for top commit:
Sjors:
re-utACK 0def84d407
rkrux:
ACK 0def84d407
w0xlt:
reACK 0def84d407
Tree-SHA512: 575c5b545d6f0aa7e135696b7a55c004e754fca4dd35dd9cf71b0b45b49a2e86e7b20570e768534d587005953bb893645379ec1ba4f98cfd26811f9c2f17de2d
130a922980 wallet, interfaces: Use BERKELEY_RO in isEncrypted (Ava Chow)
Pull request description:
The GUI uses `WalletLoader::isEncrypted()` to detect whether a wallet file is encrypted so that it knows whether to prompt for a passphrase when migrating a legacy wallet. However, legacy wallets need to be opened with `options.require_format = BERKELEY_RO`. Since this wasn't being provided, following #28710, encrypted legacy wallets could not be migrated.
This fixes the issue by detecting when a wallet file is for a legacy wallet, and re-attempting with `options.require_format = BERKELEY_RO` in that case.
Depends on #32449 for `DatabaseStatus::FAILED_LEGACY_DISABLED`
ACKs for top commit:
davidgumberg:
Tested ACK 130a922980
furszy:
utACK 130a922980
pablomartin4btc:
tACK 130a922980
w0xlt:
Code review ACK 130a922980
rkrux:
utACK 130a922980
Tree-SHA512: aa70defc3b5f41635333a4d83c46ecdb5cd3cb129d590b4c0fe7a5f16e8aeaba1592f932ead242ed5f84524b146d87319154f4a1820bb34d9e80f63d24fc6b20
206bc05e62 test: remove unnecessary m_best_header setting hack in feature_assumeutxo.py (Sebastian Falbesoner)
Pull request description:
The mentioned bug (issue #26245) has been fixed in PR #30666, so the workaround is not needed anymore.
ACKs for top commit:
fjahr:
ACK 206bc05e62
janb84:
ACK 206bc05e62
mzumsande:
utACK 206bc05e62
Tree-SHA512: 92ad6c09c80318a02e00d8c2a4ca1ec8a6caa6aa310916ff705871c91fe5c960c15ed394ead8ce9c7466b592dba176aa87a635b3228ddc93f57082124f04613f
