4becee396f3bda40832138dd1aaa90368ed31857 guix: combine and document enable_werror (fanquake)
Pull request description:
Combine into `hardened-glibc`.
Document why we don't use `--disable-werror` directly.
https://www.gnu.org/software/libc/manual/html_node/Configuring-and-compiling.html
> By default, the GNU C Library is built with -Werror. If you wish
> to build without this option (for example, if building with a
> newer version of GCC than this version of the GNU C Library was
> tested with, so new warnings cause the build with -Werror to fail),
> you can configure with --disable-werror.
ACKs for top commit:
hebasto:
ACK 4becee396f3bda40832138dd1aaa90368ed31857, the diff is correct.
TheCharlatan:
ACK 4becee396f3bda40832138dd1aaa90368ed31857
Tree-SHA512: 8724415f51b4d72d40c4e797faf52c93a81147fb629332b9388ffd7f113f2b16db3b7496bf3063dd978ac629fd5bde3ec7df4f1ff1ed714cb56f316a9334d119
24f26e08cc0db4041c51fe8391b1736b47a13af9 guix: use cmake-minimal for python-lief (fanquake)
43d8173f9911cb2130e52d5ddac44c0e19edccce guix: import LIEF from upstream (0.12.3) (fanquake)
Pull request description:
Update to version 0.12.3.
Retain our PPC64 patch.
Mention when we can drop our local definition.
Also switch to using cmake-minimal (see #27172), which fixes atleast one build failure I've seen on aarch64, where cmake dependencies fail to build. Fix that by using the cmake without all the dependencies we don't actually need:
```bash
The following derivations will be built:
/gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv
/gnu/store/f9zwh1ldy63ga0i5w6cbbqlj6sfq226j-cmake-3.21.4.drv
/gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv
building /gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv...
/ 'check' phasenote: keeping build directory `/tmp/guix-build-python-sphinx-4.2.0.drv-5'
builder for `/gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv' failed with exit code 1
build of /gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv failed
View build log at '/var/log/guix/drvs/3w/g6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv.gz'.
cannot build derivation `/gnu/store/f9zwh1ldy63ga0i5w6cbbqlj6sfq226j-cmake-3.21.4.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv': 1 dependencies couldn't be built
guix environment: error: build of `/gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv' failed
```
ACKs for top commit:
TheCharlatan:
ACK 24f26e08cc0db4041c51fe8391b1736b47a13af9
Tree-SHA512: d4260cdf5121686fd2fa36c1fc85687848eeb26cabaad2c6566feb71a18ea7fb013cfc6353c99f6f74bc89108a9505adce513c1cfa22a0a67450e6a1c451d209
87afcb0029b8dab933c122fb8f7263c2e7272731 depends: fix osx build with clang 16 (Cory Fields)
Pull request description:
Current build (using forced system clang as a test) results in:
> error: unknown argument: '-internal-externc-isystem/opt/clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/include'
For some reason the previous syntax worked with clang 15 and below, but clang 16 requires that the option and value are properly separated.
See [here for an example of upstream using this syntax](https://github.com/llvm/llvm-project/blob/main/clang/test/Driver/crash-report-with-asserts.c#L9).
There is no change in behavior for previous versions.
I'm seeing an additional unrelated problem with linking with system clang, but I'll PR the solution to that separately as it's not as straightforward as this.
ACKs for top commit:
TheCharlatan:
ACK 87afcb0029b8dab933c122fb8f7263c2e7272731
hebasto:
ACK 87afcb0029b8dab933c122fb8f7263c2e7272731
Tree-SHA512: 127037c888c37c6ccd9679e96da34037cc43ccdc07915865a0a5494edb62633e83fc1bd6b1c4bb7a0322f5b59622e10090a31987f38496fb6b306488e9941594
eb1c3adf38cb71c3e6f298a61871738c4919b4a1 depends: qrencode 4.1.1 (fanquake)
Pull request description:
Upgrade to the latest qrencode, and disable some warnings that cause compile failures with newer compilers (clang-15+).
I haven't tested this (from a GUI perspective) at all. This is just "good enough" to keep things compiling, and uses some similar work-arounds as we have with other older packages, i.e bdb.
Note that upstream, libqrencode is effectively unmaintained. No code changes for > 2 years. No responses to issues/PRs. Seems like the author has mostly dropped off of GitHub as well.
This fixes part of #27299.
ACKs for top commit:
TheCharlatan:
Code review ACK eb1c3adf38cb71c3e6f298a61871738c4919b4a1
Tree-SHA512: 898eaac3e9915dfcdc0a011b736fff685a3b46990bd27f6038ef4d3e7cb6a276206438ea50d45908a051ce55c9b0779347d4be1d35271b67f76f409a7dc21fed
Combine into hardened-glibc.
Document why we don't use --disable-werror directly.
https://www.gnu.org/software/libc/manual/html_node/Configuring-and-compiling.html
> By default, the GNU C Library is built with -Werror. If you wish
> to build without this option (for example, if building with a
> newer version of GCC than this version of the GNU C Library was
> tested with, so new warnings cause the build with -Werror to fail),
> you can configure with --disable-werror.
faa0839837e6d2ba5d0ab08fddd497a257d7e075 ci: Cache more stuff in the ci images: msan, iwyu, pip, sdks (MarcoFalke)
Pull request description:
Now that `apt` packages are cached in the ci images, it makes sense to think about caching all other packages as well.
ACKs for top commit:
TheCharlatan:
re-ACK faa0839837e6d2ba5d0ab08fddd497a257d7e075
Tree-SHA512: e2ea491570c6cdcc8522585ae7669c51ab2c0b680ff34067b58727994aa8f2e5c45ba7b76ed27a9c76d788ed155d7aade554dc164f7552fa713c00cc47b722f1
d178082996dc3000f42816f89afcf3fa4d31e159 test: add bech32 decoding support to address_to_scriptpubkey() (ismaelsadeeq)
aac8793c7a2ee7630641dd74be6ba2ead50c2aee test: test_bech32_decode in address.py (ismaelsadeeq)
Pull request description:
[rpc_scantxoutset.py](e695d8536e/test/functional/rpc_scantxoutset.py (L26)) sendtodestination only sends to legacy addresses and scriptPubkeys because [wallet.py](e695d8536e/test/functional/test_framework/wallet.py (L415)) address_to_scriptpubkey does not support conversion of segwit address.
This update enables address_to_scriptpubkey to support the conversion of testnet segwit addresses to scriptPubkeys.
This change will enable [rpc_scantxoutset.py](e695d8536e/test/functional/rpc_scantxoutset.py (L22)) ScantxoutsetTest to have more test coverage by adding more sendtodestination calls with bech32 and bech32m testnet addresses, then test the bech32 and bech32m derivation subsets UTXO amount in [Test extended key derivation](e695d8536e/test/functional/rpc_scantxoutset.py (L84)).
I will add the test coverage in a subsequent Pull request.
ACKs for top commit:
josibake:
ACK d178082996
theStack:
ACK d178082996dc3000f42816f89afcf3fa4d31e159 ✔️
willcl-ark:
ACK d17808299
Tree-SHA512: 312c20ce192c648faf7dd178622700c9b871d755db56c246250e25508c3c19e7b02c0ae901dda11a1794629b9a9429c877168c05e1c4c1dbf41493316e30e7e9
8c47d599b87d6b2d43e7d37ce0aaf4f541535bb9 doc: improve -debuglogfile help to be a bit clearer (jonatack)
20d89d6802b67e2b8c864bedf23673d008e3faab bench: document expected results in logging benchmarks (jonatack)
d8deba8c36a42481b1c1e73009d7c9cc2cb25f70 bench: add LogPrintfCategory and LogPrintLevel benchmarks (Jon Atack)
102b2033493f0d61e9763d094cb8a0017f7e3a10 bench: order the logging benchmark code by output (Jon Atack)
4b3fdbf6fe20dc70c8dbdaacce161bc4e76b9c84 bench: update logging benchmark naming for clarity (Jon Atack)
4684aa8733e831d7858d43bec4271d5d026ba183 bench: allow logging benchmarks to be order-independent (Larry Ruane)
Pull request description:
Update our logging benchmarks for evaluating ongoing work like #25203 and refactoring proposals like #26619 and #26697.
- make the logging benchmarks order-independent (Larry Ruane)
- add missing benchmarks for the `LogPrintLevel` and `LogPrintfCategory` macros that our logging is migrating to; at some later point it should be feasible to drop some of the previous logging benchmarks
- update the logging benchmark naming to be clear which benchmark corresponds to which log macro, and update the ordering to be the same as the output
- add clarifying documentation to the logging benchmarks
- improve the `-debuglogfile` config option help to be clearer; can be tested by running `./src/bitcoind -help | grep -A4 '\-debuglogfile'`
Reviewers can run the logging benchmarks with:
```bash
./src/bench/bench_bitcoin -filter='LogP*.*'
```
ACKs for top commit:
LarryRuane:
ACK 8c47d599b87d6b2d43e7d37ce0aaf4f541535bb9
martinus:
code review & tested ACK 8c47d599b87d6b2d43e7d37ce0aaf4f541535bb9, here are my benchmark results:
achow101:
ACK 8c47d599b87d6b2d43e7d37ce0aaf4f541535bb9
Tree-SHA512: 705f8720c9ceaf14a1945039c7578a0c17a12215cbc44908099af4ac444561c3f95d833c5a91b325cdd4470737d8a01e2da64db2d542dd7c9a3747fbfdbf213e
Because wallets are internally synchronized
through the validation interface, and the
interface dispatches events on a worker thread,
it is possible for a transaction created by the
first wallet to not arrive at the second wallet
before the second wallet attempts to use one of
its outputs. This is because we do not wait for
the BroadcastTransaction callback during the wallet's
"submit to mempool" process. To address this in the
tests, we need to sync the validation queue.
The current `FormatISO8601DateTime` function will
return an empty string if it encounters an error
when converting the `int64_t` seconds since epoch
to a formatted date time. In the unlikely case that happens,
`strStamped.pop_back()` would be undefined behavior.
3566aa7d495bb783bbd135726238d9f2a9e9f80e [net] Remove CNode friends (dergoegge)
3eac5e7cd1eda6ababb9af9cd72dae08d395993d [net] Add CNode helper for send byte accounting (dergoegge)
60441a3432df10f5d7c15c09c9569f27a793625b scripted-diff: [net] Rename CNode process queue members (dergoegge)
6693c499f7d48aaa1f527a1a860b943fc8442241 [net] Make cs_vProcessMsg a non-recursive mutex (dergoegge)
23d93526545c271b0eed2bd468681864c4213cce [net] Make CNode msg process queue members private (dergoegge)
897e342d6ec320c286753daef01d6eb9839e2c4d [net] Encapsulate CNode message polling (dergoegge)
cc5cdf877666c232a94f03faaf430cbeb6968372 [net] Deduplicate marking received message for processing (dergoegge)
ad44aa5c64d4ee5f31c867fda26350ab560575b7 [net] Add connection type getter to CNode (dergoegge)
Pull request description:
We should define clear interfaces between CNode, CConnman and PeerManager. This PR makes a small step in that direction by ending the friendship of CNode, CConnman and ConnmanTestMsg. CNode's message processing queue is made private in the process and its mutex is turned into a non-recursive mutex.
ACKs for top commit:
jnewbery:
utACK 3566aa7d495bb783bbd135726238d9f2a9e9f80e
vasild:
ACK 3566aa7d495bb783bbd135726238d9f2a9e9f80e
theStack:
re-ACK 3566aa7d495bb783bbd135726238d9f2a9e9f80e
brunoerg:
re-ACK 3566aa7d495bb783bbd135726238d9f2a9e9f80e
Tree-SHA512: 26b87da5054e32401b693b2904e9c5f40e35a53937c0b6cf44b8597034ad07bacf27d87cdffc54d3e7ccfebde4231ef30a38d326f88cc18133bbb34688ead567
f3221d373a8623fe4808e00c7a92fbb6e0d6419b test: add wallet too-long-mempool-chain error coverage (furszy)
acf0119d24c9f8fae825093249a46cd38e4a3a91 wallet: return error msg for too-long-mempool-chain failure (furszy)
Pull request description:
Fixes#23144.
We currently return a general "Insufficient funds" from Coin
Selection when we actually skipped unconfirmed UTXOs that
surpassed the mempool ancestors limit.
This PR make the error clearer by returning:
"Unconfirmed UTXOs are available, but spending them creates
a chain of transactions that will be rejected by the mempool"
Also, added an early return from Coin Selection if the sum of
the discarded coins decreases the available balance below the
target amount.
ACKs for top commit:
achow101:
ACK f3221d373a8623fe4808e00c7a92fbb6e0d6419b
S3RK:
Code review ACK f3221d373a8623fe4808e00c7a92fbb6e0d6419b
Xekyo:
ACK f3221d373a8623fe4808e00c7a92fbb6e0d6419b
Tree-SHA512: 13e5824b75ac302280ff894560a4ebf32a74f32fe49ef8281f2bc99c0104b92cef33d3b143c6e131f3a07eafe64533af7fc60abff585142c134b9d6e531a6a66
fa0696e7863af03efbffa026c2060ff2b5720fb1 test: Replace threading with concurrent.futures (MarcoFalke)
Pull request description:
`threading` has no easy way to get the return value or exception once the target function stops. Not checking the return value or exception can make tests more fragile and failures harder to debug.
Fix this by checking the return value (or exception) by wrapping the function execution into a future and calling `result()` on it.
Can be reviewed with `--ignore-all-space`.
(There are still some uses of `threading` around, because some tests do expect an exception to be thrown and caught in the target function)
ACKs for top commit:
ishaanam:
ACK fa0696e7863af03efbffa026c2060ff2b5720fb1
stickies-v:
ACK fa0696e7863af03efbffa026c2060ff2b5720fb1
Tree-SHA512: d9ddf6b3c530cd8c485a030a3c84d4e03d3e9f9ea8240b050afcd566a884f5cabe816ac56910cec9ea9fa299239e5abb99e672dda05a74974f61bb68dc3c1d65
fa18504d5767a40dc9827fb081633219bf251001 rpc: Add submit option to generateblock (MarcoFalke)
fab9a08e145dc5a1d9576bf062473f1095b56a16 refactor: Replace block_hash with block_out (MarcoFalke)
Pull request description:
When submit is turned off, a block can be generated and returned as hex, to be used for further tests. For example, it can be submitted on a different node, on a different interface (like p2p), or just never submitted and be used for other testing purposes.
ACKs for top commit:
instagibbs:
ACK fa18504d5767a40dc9827fb081633219bf251001
TheCharlatan:
tACK fa18504d5767a40dc9827fb081633219bf251001
Tree-SHA512: 1b2ab6b71bb7e155c6482d75f5373f4e77de6446cb16bc2dfd19e7a4075b3a6ad87d7ad7a049a9eed934cb71574acfd27202f54c8bb3b03fac869f2e95db7ee5
8fe27fbed85311bbdcd75136ca13370d368a6f2f ci: Use clang-15 in "tidy" task (Hennadii Stepanov)
Pull request description:
Newer tools usually are better in terms of features and bug fixes.
Requested in https://github.com/bitcoin/bitcoin/pull/26642#issuecomment-1440230390.
Split from https://github.com/bitcoin/bitcoin/pull/26766.
ACKs for top commit:
MarcoFalke:
lgtm ACK 8fe27fbed85311bbdcd75136ca13370d368a6f2f
Tree-SHA512: 62be3307d488fc4f75c40c0fa095aaa091aade2d5fe85296b56751e006c801f9d58c72c5cee8c0a0b1ba1a43804e315a3301c03e6e394bb3f3eb9b763fbb6271
The fs.* files are already part of the libbitcoin_util library. With the
introduction of the fs_helpers.* it makes sense to move fs.* into the
util/ directory as well.
This is an extraction of filesystem related functions from util/system
into their own utility file.
The background of this commit is an ongoing effort to decouple the
libbitcoinkernel library from the ArgsManager defined in system.h.
Moving these functions out of system.h allows including them from a
separate source file without including the ArgsManager definitions from
system.h.
Adds bech32_to_bytes() which can decode a bech32 address and return the
version as an `int` and the payload in bytes.
bech32_to_bytes() is used by the test_bech32_decode unit test to test
decoding of segwit addresses.
faf3f12424fa8558e65fa3f1dd3aa1d0eea8604e refactor: Replace GetTimeMicros by SystemClock (MarcoFalke)
Pull request description:
It is unclear from the name that `GetTimeMicros` returns the system time. Also, it is not using the type-safe `std::chrono` types.
Fix both issues by replacing it with `SystemClock` in the only place it is used.
This refactor should not change behavior.
ACKs for top commit:
willcl-ark:
tACK faf3f1242
john-moffett:
ACK faf3f12424fa8558e65fa3f1dd3aa1d0eea8604e changes, but left a comment for the existing code.
Tree-SHA512: 069e6ef26467a469f128b98a4aeb334f141742befd7880cb3a7d280480e9f0684dc0686fa6a828cdcb3d11943ae5c7f8ad5d9d9dab4c668be85e5d28c78cd489
fae349076db03ddfbf23c5d828368d538b5d52d5 test: Remove unused Check* default constructors (MarcoFalke)
Pull request description:
They are no longer needed after the removal of `swap`, see https://github.com/bitcoin/bitcoin/pull/26749#discussion_r1144532693
Also, flatten a redundant `if` check.
ACKs for top commit:
hebasto:
ACK fae349076db03ddfbf23c5d828368d538b5d52d5
Tree-SHA512: c0bc0c16b5df0f16fc25e18d2414a2a3c4769da1aa30d53f8d267bc2e97dd79a0296db94c1e49cd1ca89bd42275d8c462f7bf47f03f105dfe867ebea6563454b
95ad70ab652ddde7de65f633c36c1378b26a313a test: Default initialize `should_freeze` to `true` (Hennadii Stepanov)
cea50521fe810111a8a3c84ad14f944eafb5b658 refactor: Drop no longer used `swap` member functions (Hennadii Stepanov)
a87fb6bee5a7fb0879b3adea9a29997f1331acb0 clang-tidy: Fix modernize-use-default-member-init in `CScriptCheck` (Hennadii Stepanov)
b4bed5c1f98c0eed18f52fdcea11a420c10ed98d refactor: Drop no longer used `CScriptCheck()` default constructor (Hennadii Stepanov)
d8427cc28e3a9ac3319fb452b16661957c812b8f refactor: Use move semantics in `CCheckQueue::Loop` (Hennadii Stepanov)
9a0b5241396efe3b3ceb3931717c30bb94f99bfb clang-tidy, test: Fix bugprone-use-after-move in `Correct_Queue_range()` (Hennadii Stepanov)
04831fee6dca3eb86cd1d6b9ef879b296263fe35 refactor: Make move semantics explicit for callers (Hennadii Stepanov)
6c2d5972f3544c4f3e987828a99e88f27b62cf87 refactor: Use move semantics in `CCheckQueue::Add` (Hennadii Stepanov)
06820032142a75cc3c5b832045058bc6f6f74786 test, refactor: Avoid `CScriptCheck::swap` in `transaction_tests` (Hennadii Stepanov)
15209d97c6aad7d5c199fe007ad39b91c8ee6562 consensus, refactor: Avoid `CScriptCheck::swap` in `CheckInputScripts` (Hennadii Stepanov)
Pull request description:
This PR makes code more succinct and readable by using move semantics.
ACKs for top commit:
martinus:
re-ACK 95ad70ab652ddde7de65f633c36c1378b26a313a
achow101:
ACK 95ad70ab652ddde7de65f633c36c1378b26a313a
TheCharlatan:
re-ACK 95ad70ab65
MarcoFalke:
re-ACK 95ad70ab652ddde7de65f633c36c1378b26a313a 🚥
Tree-SHA512: adda760891b12d252dc9b823fe7c41eed660364b6fb1a69f17607d7a31eb0bbb82a80d154a7acfaa241b5de37d42a293c2b6e059f26a8e92d88d3a87c99768fb
This also fixes atleast one --no-substitues build failure I've seen,
where cmake dependencies wouldn't build:
```bash
The following derivations will be built:
/gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv
/gnu/store/f9zwh1ldy63ga0i5w6cbbqlj6sfq226j-cmake-3.21.4.drv
/gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv
building /gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv...
/ 'check' phasenote: keeping build directory `/tmp/guix-build-python-sphinx-4.2.0.drv-5'
builder for `/gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv' failed with exit code 1
build of /gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv failed
View build log at '/var/log/guix/drvs/3w/g6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv.gz'.
cannot build derivation `/gnu/store/f9zwh1ldy63ga0i5w6cbbqlj6sfq226j-cmake-3.21.4.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv': 1 dependencies couldn't be built
guix environment: error: build of `/gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv' failed
```
fa67b8181c3ecf94395ecc50fd8acd436f1f8c3a Refactor: Remove unused FlatFilePos::SetNull (MarcoFalke)
Pull request description:
This is unused outside of tests and the default constructor. With C++11, it can be replaced by C++11 member initializers in the default constructor.
Beside removing unused code, this also makes it less fragile in light of uninitialized memory. (See also https://github.com/bitcoin/bitcoin/pull/26296#issuecomment-1477801767)
If new code needs to set this to null, it can use `std::optional`, or in the worst case re-introduce this method.
ACKs for top commit:
dergoegge:
Code review ACK fa67b8181c3ecf94395ecc50fd8acd436f1f8c3a
TheCharlatan:
ACK fa67b8181c3ecf94395ecc50fd8acd436f1f8c3a
john-moffett:
ACK fa67b8181c3ecf94395ecc50fd8acd436f1f8c3a
Tree-SHA512: 465c5e3eb4625405c445695d33e09a1fc5185c7dd1e766ba06034fb093880bfc65441d5334f7d9b20e2e417c2075557d86059f59d9648ca0e62a54c699c029b9
2c3a90f663a61ee147d785167a2902494d81d34d log: on new valid header (James O'Beirne)
e5ce8576349d404c466b2f4cab1ca7bf920904b2 log: net: new header over cmpctblock (James O'Beirne)
Pull request description:
Alternate to #27276.
Devs were [suprised to realize](https://twitter.com/jamesob/status/1637237917201383425) last night that we don't have definitive logging for when a given header was first received.
This logs to the main stream when new headers are received outside of IBD, as well as when headers come in over cmpctblocks. The rationale of not hiding these under log categories is that they may be useful to have widely available when debugging strange network activity, and the marginal volume is modest.
ACKs for top commit:
dergoegge:
Code review ACK 2c3a90f663a61ee147d785167a2902494d81d34d
achow101:
ACK 2c3a90f663a61ee147d785167a2902494d81d34d
Sjors:
tACK 2c3a90f663a61ee147d785167a2902494d81d34d
josibake:
ACK 2c3a90f663
Tree-SHA512: 49fdcbe07799c8adc24143d7e5054a0c93fef120d2e9d5fddbd3b119550d895e2985be6ac10dd1825ea23a6fa5479c1b76d5518c136fbd983fa76c0d39dc354f
