9eac5a0529f869075f0331e40d322c34fc8fc2af [functional test] transaction orphan handling (glozow)
61e77bb901bca038b46064dc4531e12c2ab50673 [test framework] make it easier to fast-forward setmocktime (glozow)
Pull request description:
I was doing some mutation testing (through reckless refactoring) locally and found some specific behaviors in orphan handling that weren't picked up by tests. Adding some of these test cases now can maybe help with reviewing refactors like #28031.
- Parent requests aren't sent immediately. A delay is added and the requests are filtered by AlreadyHaveTx before they are sent, which means you can't use fake orphans to probe precise arrival timing of a tx.
- Parent requests include all that are not AlreadyHaveTx. This means old confirmed parents may be requested.
- The node does not give up on orphans if the peer responds to a parent request with notfound. This means that if a parent is an old confirmed transaction (in which notfound is expected), the orphan should still be resolved.
- Rejected parents can cause an orphan to be dropped, but it depends on the reason and only based on txid.
- Rejected parents can cause an orphan to be rejected too, by both wtxid and txid.
- Requests for orphan parents should be de-duplicated with "regular" txrequest. If a missing parent has the same hash as an in-flight request, it shouldn't be requested.
- Multiple orphans with overlapping parents should not cause duplicated parent requests.
ACKs for top commit:
instagibbs:
reACK 9eac5a0529
dergoegge:
reACK 9eac5a0529f869075f0331e40d322c34fc8fc2af
achow101:
ACK 9eac5a0529f869075f0331e40d322c34fc8fc2af
fjahr:
Code review ACK 9eac5a0529f869075f0331e40d322c34fc8fc2af
Tree-SHA512: 85488dc6a3f62cf0c38e7dfe7839c01215b44b172d1755b18164d41d01038f3a749451241e4eba8b857fd344a445740b21d6382c45977234b21460e3f53b1b2a
51324c9517a21471916ab722c33f9925c6071c55 guix: pre time-machine bump changes (Windows) (fanquake)
Pull request description:
This is some refactoring to the Windows Guix build that facilitates bumping our Guix time-machine. Namely, avoiding `package-with-extra-configure-variable`, which is non-functional in the newer time-machine, see https://issues.guix.gnu.org/64436.
At the same time, consolidate our Windows GCC build into `mingw-w64-base-gcc`.
Rename `gcc-10-remap-guix-store.patch` to avoid changing it whenever GCC changes.
We move the old `building-on` inside `explicit-cross-configure`, so that non-windows builds continue to work. Note that `explicit-cross-configure` will be going away entirely (see #27897), so this is only temporary duplication.
Split out of #27897. Most of the [[WIP] Windows commit](e50d5f8953), minus the gcov change, as it's not-yet needed, and any gexp changes, which need the newer time-machine.
I'll also be splitting out the other changes we can do pre-timemachine bump, for easier review.
Guix Build:
```bash
9cff4df245df0b8e92d102bda786eb13a1f10be58940b98a16e119d9d4859d7f guix-build-51324c9517a2/output/aarch64-linux-gnu/SHA256SUMS.part
dbe6a5fe2e6152cd114b1d549a383384cd3325ed4625ad93f8d824ba457ae9cd guix-build-51324c9517a2/output/aarch64-linux-gnu/bitcoin-51324c9517a2-aarch64-linux-gnu-debug.tar.gz
e36409cd7287e4da62806168ee21ef764e607a11685bb36b7642381060291789 guix-build-51324c9517a2/output/aarch64-linux-gnu/bitcoin-51324c9517a2-aarch64-linux-gnu.tar.gz
0721a4b61ad991ec97d09281eda223ed7bad32cb2b4812e05bc5e64772756be7 guix-build-51324c9517a2/output/arm-linux-gnueabihf/SHA256SUMS.part
f2aa9412ff77bfc3092404533ec669a610c3fa95711b514abd6cd8ae6918aca1 guix-build-51324c9517a2/output/arm-linux-gnueabihf/bitcoin-51324c9517a2-arm-linux-gnueabihf-debug.tar.gz
cda70b7df940dcc1fdeb25e2d3cc6ecce45732fdd741881e2b0384e35897b6d5 guix-build-51324c9517a2/output/arm-linux-gnueabihf/bitcoin-51324c9517a2-arm-linux-gnueabihf.tar.gz
15b5d30336bcaf75db4138c59716af93b9660511222c5f1ad814fc9d7c48c852 guix-build-51324c9517a2/output/arm64-apple-darwin/SHA256SUMS.part
a29b327e4d869629af3625d6b7fe44e029e98ac4fedb265133e0d88e045ef933 guix-build-51324c9517a2/output/arm64-apple-darwin/bitcoin-51324c9517a2-arm64-apple-darwin-unsigned.dmg
b1af35290051da64462987e32f215464c5e2376afbe5b5bff272f74f160a3461 guix-build-51324c9517a2/output/arm64-apple-darwin/bitcoin-51324c9517a2-arm64-apple-darwin-unsigned.tar.gz
6c3815392377a8497cab15f534cfdb749dbe31e39ca2ae7e53a55b37acfe069d guix-build-51324c9517a2/output/arm64-apple-darwin/bitcoin-51324c9517a2-arm64-apple-darwin.tar.gz
efff276305c36ccf2e6eeca56928e6284965d0d08b05565b0bfc94a3dc30cec4 guix-build-51324c9517a2/output/dist-archive/bitcoin-51324c9517a2.tar.gz
0d70106907ac873bd31603b58183ef2cd3b0820ce20b27e3a8d00044b876835c guix-build-51324c9517a2/output/powerpc64-linux-gnu/SHA256SUMS.part
9a6cbc010d08c26c5bc587ed9c42383a6d004ac20426508e67364848aa099154 guix-build-51324c9517a2/output/powerpc64-linux-gnu/bitcoin-51324c9517a2-powerpc64-linux-gnu-debug.tar.gz
e39ca22d186bad00119a740279b046a0040628847bea76db0b5d5345efa2312d guix-build-51324c9517a2/output/powerpc64-linux-gnu/bitcoin-51324c9517a2-powerpc64-linux-gnu.tar.gz
945ab751f65d5333b259fd169b280a5fabd35cae89a8f777bc6019f2ac011073 guix-build-51324c9517a2/output/powerpc64le-linux-gnu/SHA256SUMS.part
1c33b4385d5759e4a38cd42e6af99c1224788d03e15b8abd3df48caef3ebcb0a guix-build-51324c9517a2/output/powerpc64le-linux-gnu/bitcoin-51324c9517a2-powerpc64le-linux-gnu-debug.tar.gz
3b4d0f6d874d600095b90eb19ad568ab08c105762c4ba285c6b1a4dc8d9fc2db guix-build-51324c9517a2/output/powerpc64le-linux-gnu/bitcoin-51324c9517a2-powerpc64le-linux-gnu.tar.gz
6d4f508256747841f9b099370cc7da2a97687f7b182683fce0369b524388e952 guix-build-51324c9517a2/output/riscv64-linux-gnu/SHA256SUMS.part
34a474abaf833664448f3643d572eef284037535685af3f3928d9f859e7353e6 guix-build-51324c9517a2/output/riscv64-linux-gnu/bitcoin-51324c9517a2-riscv64-linux-gnu-debug.tar.gz
b91a2209d360fe97d3d04cee4ff52271d45c2e47044e4f196dde4585cf61a273 guix-build-51324c9517a2/output/riscv64-linux-gnu/bitcoin-51324c9517a2-riscv64-linux-gnu.tar.gz
88a40ee878a6340a4c5dd4bb82effaff814328e6b6f1345c5824170ba7498279 guix-build-51324c9517a2/output/x86_64-apple-darwin/SHA256SUMS.part
0714092f76e32a1e8484651aed349ccdb5acc38f54dc2ac3ae3031111b5050eb guix-build-51324c9517a2/output/x86_64-apple-darwin/bitcoin-51324c9517a2-x86_64-apple-darwin-unsigned.dmg
ab66d04ea3e9304155e66b0e2899b8325fd609c3fed27c74fe1b53f1b9efea15 guix-build-51324c9517a2/output/x86_64-apple-darwin/bitcoin-51324c9517a2-x86_64-apple-darwin-unsigned.tar.gz
222b688ef72b657c0701c8b4660240833e7c516fbbdcc4df8e9a006f5aa9b3bb guix-build-51324c9517a2/output/x86_64-apple-darwin/bitcoin-51324c9517a2-x86_64-apple-darwin.tar.gz
340e73e1697ae5cc1984771ce6e5caf22361ba4d063cbc4073236b210f1754dc guix-build-51324c9517a2/output/x86_64-linux-gnu/SHA256SUMS.part
a967753c192b8cb28cc644ce9d239da88ef8faaebc2094e21b7580616a0b8415 guix-build-51324c9517a2/output/x86_64-linux-gnu/bitcoin-51324c9517a2-x86_64-linux-gnu-debug.tar.gz
7d77b14a2c94091a2d68dd9cdb0d878348da323c67eeeb5600273814db1b0e10 guix-build-51324c9517a2/output/x86_64-linux-gnu/bitcoin-51324c9517a2-x86_64-linux-gnu.tar.gz
fbfb10b13b7bc5a1ed33578dcdd9483360cad47fd744ac99a9aecdf2a8ac14c2 guix-build-51324c9517a2/output/x86_64-w64-mingw32/SHA256SUMS.part
925bf5f88e907ddc4567acecfc634ef3570b0576499c6427225a52e1f6209d1e guix-build-51324c9517a2/output/x86_64-w64-mingw32/bitcoin-51324c9517a2-win64-debug.zip
b3d19a25df2dbd913814e74471da585e629187a19a6afede5890a7df2dd0101f guix-build-51324c9517a2/output/x86_64-w64-mingw32/bitcoin-51324c9517a2-win64-setup-unsigned.exe
96b03fa66e294e95c0cee5d4c9869ac377e739d2da2e87c0c9d35eddd25e6ceb guix-build-51324c9517a2/output/x86_64-w64-mingw32/bitcoin-51324c9517a2-win64-unsigned.tar.gz
95d7f52622c534e40dceb89f150b3f2e0061c56768bcf7399aac3f24b937983b guix-build-51324c9517a2/output/x86_64-w64-mingw32/bitcoin-51324c9517a2-win64.zip
```
ACKs for top commit:
TheCharlatan:
ACK 51324c9517a21471916ab722c33f9925c6071c55
Tree-SHA512: 4d26ea2b587e4ea2bf76861925437b178376cda7373012f4a6d48ccb3606c20b035b94cba6248ac8d9e0c9e974e9eace818043d312c67d5c5e5771300a839c44
fadf671fa539e76d000a08b92109ea5503d5f3ac Refactor: Remove confusing static_cast (MarcoFalke)
faeea1ab586e30ca3b980999a264f2a3b4769143 refactor: Add missing includes (MarcoFalke)
Pull request description:
It seems confusing to use `static_cast<uint160>(bla)` to call the constructor of `uint160`. The normal and common way to call a constructor is by simply calling it. (`uint160{bla}`).
Do this, and also drop the constructor completely where the existing `const&` reference is enough.
Also, add missing includes while touching the file.
ACKs for top commit:
vincenzopalazzo:
ACK fadf671fa5
TheCharlatan:
ACK fadf671fa539e76d000a08b92109ea5503d5f3ac
Tree-SHA512: 8fb9a72203a6461b1f4b38bb90943ca25a92b218fc87da2022b90802e7747350e3668a13db3189201ad30e2e39a51d6658fed4aad176fd52cecc1c7f972c3134
2222e15771838ffcb68adb0f825ce488dce5cab2 test: Support riscv64 in get_previous_releases.py (MarcoFalke)
Pull request description:
To test: `test/get_previous_releases.py -b -t /tmp/prev_releases v0.18.1`
On master: `Not sure which binary to download for riscv64-unknown-linux-gnu`
Here: (pass)
ACKs for top commit:
fanquake:
ACK 2222e15771838ffcb68adb0f825ce488dce5cab2
Tree-SHA512: 18dc9a6c65f78adb5f7fc09e57db34c6b544071cb7bb3fa2846c86a23202e37d6ea1c5aca9acc1c2040b7d2b97bb93840a8a949f81f71fe6f01c395d2894739d
fa6286891fa4164510e4fbf4bc214ce3033b2d1b Remove unused includes from wallet.cpp (MarcoFalke)
fa8fdbe22932a4717d2bc4060269da9bff228728 Remove unused includes from blockfilter.h (MarcoFalke)
fad8c36aa9011c3f7b1183f8380577e16a2167a6 move-only: Create src/kernel/mempool_removal_reason.h (MarcoFalke)
fa5760880094c4e4238249f6d1837cd74383cc3a Remove unused includes from txmempool.h (MarcoFalke)
Pull request description:
This makes compilation of wallet.cpp use a few % less memory and time, locally.
Created in the context of https://github.com/bitcoin/bitcoin/issues/28109, but I don't think it is enough to actually fix this problem.
ACKs for top commit:
hebasto:
ACK fa6286891fa4164510e4fbf4bc214ce3033b2d1b, I have reviewed the code and it looks OK.
Tree-SHA512: 06f1120af2a8ef3368dbd9ae747acda88ace2507bd261bcc10341d476a0b3d71c8485377ea6c108b47df3e4c13b7f75a15f486bafa6a8466303168dde16ebbc8
452c094449de00f3640e6e0763366e7603375825 test: fix 'unknown named parameter' test in `wallet_basic` (brunoerg)
Pull request description:
This PR removes loop when testing an unknown named parameter. They don't have any effect.
ACKs for top commit:
jonatack:
ACK 452c094449de00f3640e6e0763366e7603375825
theStack:
re-ACK 452c094449de00f3640e6e0763366e7603375825
Tree-SHA512: cf1a37d738bb6fdf9817e7b1d33bc69643dae61e3dbfae5c1e9f26220c55db6f134018dd9a1c65c13869ee58bcb6f3337c5999aabf2614d3126fbc01270705e8
ee5a0369cc4305da7b3d26f37677de05ad797e51 test: ensure acceptstalefeeestimates is supported only on regtest chain (ismaelsadeeq)
22d5d4b2b2486feaef981e96f0321f020617f082 tx fees, policy: doc: update and delete unnecessary comment (ismaelsadeeq)
Pull request description:
This PR Follow up comments from [#27622](https://github.com/bitcoin/bitcoin/pull/27622)
It test that the new `regtest-only` option `acceptstalefeeestimates` is not supported on [main, signet and test chains](https://github.com/bitcoin/bitcoin/pull/27622/files#r1235218268), removes an unnecessary [comment](https://github.com/bitcoin/bitcoin/pull/27622/files#r1235204323), and update fee estimator `MAXFILEAGE` [description comment](https://github.com/bitcoin/bitcoin/pull/27622/files#r1233887314).
ACKs for top commit:
jonatack:
ACK ee5a0369cc4305da7b3d26f37677de05ad797e51
glozow:
utACK ee5a0369cc4305da7b3d26f37677de05ad797e51
Tree-SHA512: 4755f25b08db62f37614ea768272b12580ee0d481fb7fa339379901a6132c66828777c6747d3fe67490ceace3a6ff248bf13bdf65720f6e5ba8642eb762acd3c
241d6ca34c60d9003a27bb7960cebfb66366753b ci: Disable cache save for pull requests in GitHub Actions (Hennadii Stepanov)
Pull request description:
This PR disable cache save for pull requests in GitHub Actions.
Otherwise, multiple pull requests fill GitHub Actions cache quota shortly.
See a discussion [here](https://github.com/bitcoin/bitcoin/pull/28187#discussion_r1295459732).
---
**NOTE** for the maintainers with "owner" permissions.
This PR needs the `actions/cache/restore@*` and `actions/cache/save@*` acrions to be explicitly allowed in the repository's Actions permissions.
ACKs for top commit:
MarcoFalke:
lgtm ACK 241d6ca34c60d9003a27bb7960cebfb66366753b
Tree-SHA512: a7786c7ec99bfa6991bf6ae08fd7ed3546e8c5d083a1b2bae7638f6f31e77fdf2cf4fc69d85834faf87f98db1f4a82026ce1dc5fc1bc6650e8bf1c09bf7e90f5
94a98fbd1d14a4ea10629b917771d50f3191e944 assumeutxo cleanup: Move IsInitialBlockDownload & NotifyHeaderTip to ChainstateManager (Ryan Ofsky)
Pull request description:
This change makes `IsInitialBlockDownload` and `NotifyHeaderTip` functions no longer tied to individual `Chainstate` objects. It makes them work with the `ChainstateManager` object instead so code is simpler and it is no longer possible to call them incorrectly with an inactive `Chainstate`.
This change also makes `m_cached_finished_ibd` caching easier to reason about, because now there is only one cached value instead of two (for background and snapshot chainstates) so the cached IBD state now no longer gets reset when a snapshot is loaded.
There should be no change in behavior because these functions were always called on the active `ChainState` objects.
These changes were discussed previously https://github.com/bitcoin/bitcoin/pull/27746#discussion_r1246868905 and https://github.com/bitcoin/bitcoin/pull/27746#discussion_r1237552792 as possible followups for that PR.
ACKs for top commit:
MarcoFalke:
re-ACK 94a98fbd1d14a4ea10629b917771d50f3191e944 🐺
naumenkogs:
ACK 94a98fbd1d14a4ea10629b917771d50f3191e944
dergoegge:
reACK 94a98fbd1d14a4ea10629b917771d50f3191e944
Tree-SHA512: 374d6e5c9bbc7564c143f634bd709a4e8f4a42c8d77e7a8554c832acdcf60fa2a134f3ea10827db1a1e0191006496329c0ebf5c64f3ab868398c3722bb7ff56f
This change makes IsInitialBlockDownload and NotifyHeaderTip functions no
longer tied to individual Chainstate objects. It makes them work with the
ChainstateManager object instead so code is simpler and it is no longer
possible to call them incorrectly with an inactive Chainstate.
This change also makes m_cached_finished_ibd caching easier to reason about,
because now there is only one cached value instead of two (for background and
snapshot chainstates) so the cached IBD state now no longer gets reset when a
snapshot is loaded.
There should be no change in behavior because these functions were always
called on the active ChainState objects.
These changes were discussed previously
https://github.com/bitcoin/bitcoin/pull/27746#discussion_r1246868905 and
https://github.com/bitcoin/bitcoin/pull/27746#discussion_r1237552792 as
possible followups for that PR.
fa56d17a4b868f42fa45bea0d1f0c78de75e7838 ci: Add missing amd64 to win64-cross task (MarcoFalke)
Pull request description:
Currently the task will fail if run on non-`x86_64`.
Fix this by adding the missing `amd64`, similar to
7bf078f2b7/ci/test/00_setup_env_i686_multiprocess.sh (L11)
ACKs for top commit:
hebasto:
ACK fa56d17a4b868f42fa45bea0d1f0c78de75e7838
Tree-SHA512: faab1c5b945283b7e8d080bbcc8e9379c480cf6973506149ace5990cb4d04673f83f4bc36d08d5b4e9cb17a86fdbe23ac97ef4eab0e842616b367b8138229c58
fa968ef6a32bb66ca9fa3b84807f1c960f6a9833 ci: Add missing ${CI_RETRY_EXE} before curl (MarcoFalke)
Pull request description:
GitHub is frequently down and this is causing many intermittent issues. For example, from today: https://cirrus-ci.com/task/5740122163904512?logs=ci#L398
Try to fix it with a retry.
ACKs for top commit:
hebasto:
ACK fa968ef6a32bb66ca9fa3b84807f1c960f6a9833
Tree-SHA512: e9a1e51af37ec718ca776f20d8b5394680a9b059fb2a22505ccb6781472e4b072694e7d01661ede5a87ef1f58a9143e29d032b6a5d13d8be3b7d46eeff061563
Split out of #27897. This is some refactoring to the Windows Guix build
that facilitates bumping our Guix time-machine. Namely, avoiding
`package-with-extra-configure-variable`, which is non-functional in the
newer time-machine, see https://issues.guix.gnu.org/64436.
At the same time, consolidate our Windows GCC build into mingw-w64-base-gcc.
Rename `gcc-10-remap-guix-store.patch` to avoid changing it whenever GCC changes.
We move the old `building-on` inside `explicit-cross-configure`, so that
non-windows builds continue to work. Note that `explicit-cross-configure`
will be going away entirely (see #27897).
fa60fa3b0cba4a30726af8e0e9d1e84e14849eda bitcoin-tidy: Apply bitcoin-unterminated-logprintf to spkm as well (MarcoFalke)
faa11434fe38aa82892802adb6d879d112ae1675 refactor: Enable all clang-tidy plugin bitcoin tests (MarcoFalke)
fa6dc57760e0a04dbb2e365ca7ad9fd8171ebfdb refactor: Enforce C-str fmt strings in WalletLogPrintf() (MarcoFalke)
fa244f3321de7884f530bb38493a8d0a0cec86ab doc: Fix bitcoin-unterminated-logprintf tidy comments (MarcoFalke)
Pull request description:
All fmt functions only accept a raw C-string as argument.
There should never be a need to pass a format string that is not a compile-time string literal, so disallow it in `WalletLogPrintf()` to avoid accidentally introducing it.
Apart from consistency, this also fixes the clang-tidy plugin bug https://github.com/bitcoin/bitcoin/pull/26296#discussion_r1286821141.
ACKs for top commit:
theuni:
ACK fa60fa3b0cba4a30726af8e0e9d1e84e14849eda
Tree-SHA512: fa6f4984c50f9b34e850bdfee7236706af586e512d866cc869cf0cdfaf9aa707029c210ca72d91f85e75fcbd8efe0d77084701de8c3d2004abfd7e46b6fa9072
57cc136282c38825e97bbf85728df4bdf1ccc648 crypto: make ChaCha20::SetKey wipe buffer (Pieter Wuille)
da0ec62e34cc56bf8990e28c6ec12683d4752305 tests: miscellaneous hex / std::byte improvements (Pieter Wuille)
bdcbc8594c208f11e7d5221700bfa7f7a874aec9 fuzz: support std::byte in Consume{Fixed,Variable}LengthByteVector (Pieter Wuille)
7d1cd932342e74421ae927800eeada14f504b944 crypto: require key on ChaCha20 initialization (Pieter Wuille)
44c11769a83b90ca6b8af086d6fa69ff7ac1c3ae random: simplify FastRandomContext::randbytes using fillrand (Pieter Wuille)
3da636e08b781fa2f7c1c23bb937015185732a75 crypto: refactor ChaCha20 classes to use Span<std::byte> interface (Pieter Wuille)
Pull request description:
This modernizes the ChaCha20 and ChaCha20Aligned interfaces to be `Span<std::byte>` based, and other improvements.
* Modifies all functions and constructors of `ChaCha20` and `ChaCha20Aligned` to be `Span<std::byte>` based (aligning them with `FSChaCha20`, `AEADChaCha20Poly1305`, and `FSChaCha20Poly1305`)
* Remove default constructors, to make sure all call sites provide a key (suggested in https://github.com/bitcoin/bitcoin/pull/26153#discussion_r1129313162)
* Wipe key material on rekey for security (suggested in https://github.com/bitcoin/bitcoin/pull/26153#discussion_r1267164605)
* Use `HexStr` on byte vectors in tests (suggested in https://github.com/bitcoin/bitcoin/pull/27993#discussion_r1262023316)
* Support `std::byte` vectors in `ConsumeRandomLengthByteVector` and `ConsumeFixedLengthByteVector`, and use it (suggested in https://github.com/bitcoin/bitcoin/pull/27993#discussion_r1265337111)
* And a few more.
While related, I don't see this as a necessary for BIP324.
ACKs for top commit:
stratospher:
ACK 57cc136.
theStack:
re-ACK 57cc136282c38825e97bbf85728df4bdf1ccc648
Tree-SHA512: 361da4ff003c8465a32eeac0983a8a6f047dbbf5b400168b409c8e3234e79d577fc854e0764389446585da3e12b964c94dd67fc0c9c1d1d092cec296121e05d4
239431444216850b63ecf01c3b5c5d6d24230d08 rpc: remove one more quote from non-string oneline description (Martin Zumsande)
Pull request description:
This fixes a silent conflict between https://github.com/bitcoin/bitcoin/pull/28123 (which removed all `\"options\"`) and https://github.com/bitcoin/bitcoin/pull/27460 (which added a new one).
It should fix the current CI failures.
ACKs for top commit:
ajtowns:
utACK 239431444216850b63ecf01c3b5c5d6d24230d08
MarcoFalke:
lgtm ACK 239431444216850b63ecf01c3b5c5d6d24230d08
jonatack:
ACK 239431444216850b63ecf01c3b5c5d6d24230d08
hebasto:
ACK 239431444216850b63ecf01c3b5c5d6d24230d08
Tree-SHA512: feb0c2b936a77be45d9c65aa7d738277b2266b5153665fee3b1413045de521195dc7d5efa2fc8b37b22f16e9b8d0ee8de25bfd151a428666122b31f64056557a
This removes unused includes, primitives/block found manually, and the
others by iwyu:
blockfilter.h should remove these lines:
- #include <serialize.h> // lines 16-16
- #include <undo.h> // lines 18-18
... and move them to where they are really needed.
This was found by IWYU:
txmempool.h should remove these lines:
- #include <random.h> // lines 29-29
- class CBlockIndex; // lines 43-43
- class Chainstate; // lines 45-45
Also, move the stdlib section to the right place. Can be reviewed with:
--color-moved=dimmed-zebra
fa263877691a7babb08a83f5f977390a0ba64729 ci: Refactor: Remove CI_USE_APT_INSTALL (MarcoFalke)
Pull request description:
Seems odd to use `CI_USE_APT_INSTALL == no` as an alias for `CI_OS_NAME == macos`. Fix this by removing the alias.
Also, for github CI:
* restore MAKEJOBS to the same value as in cirrus.yml.
* remove cirrus-only PACKAGE_MANAGER_INSTALL.
* remove redundant TEST_RUNNER_TIMEOUT_FACTOR
* Add M1 link
ACKs for top commit:
hebasto:
ACK fa263877691a7babb08a83f5f977390a0ba64729.
Tree-SHA512: e235aa70abd60738a9ad1531284a94e2122c9c7a22c2514ede437b49da5c06b2597fba7fccf615541fb3adb4e1f8076aa8c6047f926393191a629713554ab000
0080b5650e0bf125612e62b591f348076e9ad5d7 ci: Ensure that only a single workflow processes `github.ref` at a time (Hennadii Stepanov)
Pull request description:
This PR ensures that only a single workflow processes any push or pull request at a time.
A new push will be queued (including the master branch).
For a new pull request update, the previous in-progress one will be cancelled.
Address https://github.com/bitcoin/bitcoin/pull/28187#discussion_r1295144563.
ACKs for top commit:
dergoegge:
utACK 0080b5650e0bf125612e62b591f348076e9ad5d7
Tree-SHA512: d03f1678c93c817c1c3be5e75171bfe85d494f99a9aab7113c9438e0c820e950edd5c10199cccd54868e6dc50217bb3aa5601b23bc88ad4927c001031e917513
5e3e83b005518659a69916c373b808da27e51791 RPC/Mining: Document template_request better for getblocktemplate (Luke Dashjr)
de319c61759952318364fbcb28c47f0959d89d0e RPC/rpcdoccheck: Error if a oneline_description has a quote for a non-string (Luke Dashjr)
7c61e9df90579ed42a30016e52355e437733b128 Bugfix: RPC: Remove quotes from non-string oneline descriptions (Luke Dashjr)
Pull request description:
Various JSON Object parameters had a `oneline_description` with quote characters. Fix those, and extend `rpcdoccheck` to detect them.
Also, slightly improve GBT's oneline description for template_request.
ACKs for top commit:
MarcoFalke:
review ACK 5e3e83b005518659a69916c373b808da27e51791
Tree-SHA512: 363d1669a661d0acfc19fddb57e777d781c7246f330cf62160e77dde10a6adcb0249db748127067da1afe1b7d17c71cf611d9fdc3664d6bf5b3f30105637769a
fa748c6f2ac2f9cac7ce42fd745ed3c3eae093b7 test: Fix intermittent issue in mining_getblocktemplate_longpoll.py (MarcoFalke)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/26962
Wait for the thread to have started and the RPC to have reached the node before continuing. Otherwise the test may run into a race.
For example:
```
test 2023-06-23T13:10:29.245000Z TestFramework (INFO): Test that introducing a new transaction into the mempool will terminate the longpoll
node0 2023-06-23T13:10:29.245712Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
node0 2023-06-23T13:10:29.245915Z [httpworker.3] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getblocktemplate user=__cookie__
node0 2023-06-23T13:10:29.252594Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
node0 2023-06-23T13:10:29.254545Z [httpworker.2] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getblockchaininfo user=__cookie__
node0 2023-06-23T13:10:29.256530Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
node0 2023-06-23T13:10:29.256741Z [httpworker.1] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=sendrawtransaction user=__cookie__
node0 2023-06-23T13:10:29.258033Z [httpworker.1] [validationinterface.cpp:213] [TransactionAddedToMempool] [validation] Enqueuing TransactionAddedToMempool: txid=38335600f2465c0f8bb2b86d5830a34851d86fa879800c0e1434ddfc78c42898 wtxid=c033cd3efd301c369d66cf759769159609471bd4f9efb3ee30e7209e57b74778
node0 2023-06-23T13:10:29.258263Z [httpworker.1] [txmempool.cpp:660] [check] [mempool] Checking mempool with 1 transactions and 1 inputs
node0 2023-06-23T13:10:29.258542Z [scheduler] [validationinterface.cpp:213] [operator()] [validation] TransactionAddedToMempool: txid=38335600f2465c0f8bb2b86d5830a34851d86fa879800c0e1434ddfc78c42898 wtxid=c033cd3efd301c369d66cf759769159609471bd4f9efb3ee30e7209e57b74778
node0 2023-06-23T13:10:29.259549Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
node0 2023-06-23T13:10:29.259745Z [httpworker.0] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=decoderawtransaction user=__cookie__
node0 2023-06-23T13:10:29.261066Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:52690
node0 2023-06-23T13:10:29.261803Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
node0 2023-06-23T13:10:29.262770Z [httpworker.2] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getblocktemplate user=__cookie__
```
(`sendrawtransaction` is called before `getblocktemplate`)
ACKs for top commit:
jamesob:
Github ACK fa748c6f2a
theStack:
ACK fa748c6f2ac2f9cac7ce42fd745ed3c3eae093b7
Tree-SHA512: c67d9ec7c56e8a22c1a26a3c3d4d4a4bcc17e4282cad0d66561ba2abd6e92240cb028369b4edc6077ea34e8736c0294f6066381979aee22a6166580cea43729a
3388e523a129ad9c7aef418c9f57491f8c2d9df8 Rework receive buffer pushback (Pieter Wuille)
Pull request description:
See https://github.com/ElementsProject/elements/issues/1233. There, it has been observed that if both sides of a P2P connection have a significant amount of data to send, a stall can occur, where both try to drain their own send queue before trying to receive. The same issue seems to apply to the current Bitcoin Core codebase, though I don't know whether it's a frequent issue for us.
The core issue is that whenever our optimistic send fails to fully send a message, we do subsequently not even select() for receiving; if it then turns out that sending is not possible either, no progress is made at all. To address this, the solution used in this PR is to still select() for both sending and receiving when an optimistic send fails, but skip receiving if sending succeeded, and (still) doesn't fully drain the send queue.
This is a significant reduction in how aggressive the "receive pushback" mechanism is, because now it will only mildly push back while sending progress is made; if the other side stops receiving entirely, the pushback disappears. I don't think that's a serious problem though:
* We still have a pushback mechanism at the application buffer level (when the application receive buffer overflows, receiving is paused until messages in the buffer get processed; waiting on our own net_processing thread, not on the remote party).
* There are cases where the existing mechanism is too aggressive; e.g. when the send queue is non-empty, but tiny, and can be sent with a single send() call. In that case, I think we'd prefer to still receive within the same processing loop of the network thread.
ACKs for top commit:
ajtowns:
ACK 3388e523a129ad9c7aef418c9f57491f8c2d9df8
naumenkogs:
ACK 3388e523a129ad9c7aef418c9f57491f8c2d9df8
mzumsande:
Tested ACK 3388e523a129ad9c7aef418c9f57491f8c2d9df8
Tree-SHA512: 28960feb3cd2ff3dfb39622510da62472612f88165ea98fc9fb844bfcb8fa3ed3633f83e7bd72bdbbbd37993ef10181b2e1b34836ebb8f0d83fd1c558921ec17
91d924ede1b421df31c895f4f43359e453a09ca5 Rename script/standard.{cpp/h} to script/solver.{cpp/h} (Andrew Chow)
bacdb2e208531124e85ed2d4ea2a4b508fbb5088 Clean up script/standard.{h/cpp} includes (Andrew Chow)
f3c9078b4cddec5581e52de5c216ae53984ec130 Clean up things that include script/standard.h (Andrew Chow)
8bbe257bac751859a272ddf52dc0328c1b5a1ede MOVEONLY: Move datacarrier defaults to policy.h (Andrew Chow)
7a172c76d2361fc3cdf6345590e26c79a7821672 Move CTxDestination to its own file (Andrew Chow)
145f36ec81e79d2e391847520364c2420ef0e0e8 Move Taproot{SpendData/Builder} to signingprovider.{h/cpp} (Andrew Chow)
86ea8bed5473f400f7a93fcc455393a574a2f319 Move CScriptID to script.{h/cpp} (Andrew Chow)
b81ebff0d99c45c071b999796b8ae3f0f2517b22 Remove ScriptHash from CScriptID constructor (Andrew Chow)
cba69dda3da0e4fa39cff5ce4dc81d1242fe651b Move MANDATORY_SCRIPT_VERIFY_FLAGS from script/standard.h to policy/policy.h (Anthony Towns)
Pull request description:
Some future work needs to touch things in script/standard.{h/cpp}, however it is unclear if it is safe to do so as they are included in several different places that could effect standardness and consensus. It contains a mix of policy parameters, consensus parameters, and utilities only used by the wallet. This PR breaks up the various components and renames the files to clearly separate everything.
* `CTxDestination` is moved to a new file `src/addresstype.{cpp/h}`
* `TaprootSpendData` and `TaprootBuilder` (and their utility functions and structs) are moved to `SigningProvider` as these are used only during signing.
* `CScriptID` is moved to `script/script.h` to be next to `CScript`.
* `MANDATORY_SCRIPT_VERIFY_FLAGS` is moved to `interpreter.h`
* The parameters `DEFAULT_ACCEPT_DATACARRIER` and `MAX_OP_RETURN_RELAY` are moved to `policy.h`
* `standard.{cpp/h}` is renamed to `solver.{cpp/h}` since that's all that's left in the file after the above moves
ACKs for top commit:
Sjors:
ACK 91d924ede1b421df31c895f4f43359e453a09ca5
ajtowns:
ACK 91d924ede1b421df31c895f4f43359e453a09ca5
MarcoFalke:
ACK 91d924ede1b421df31c895f4f43359e453a09ca5 😇
murchandamus:
ACK 91d924ede1b421df31c895f4f43359e453a09ca5
darosior:
Code review ACK 91d924ede1b421df31c895f4f43359e453a09ca5.
theStack:
Code-review ACK 91d924ede1b421df31c895f4f43359e453a09ca5
Tree-SHA512: d347439890c652081f6a303d99b2bde6c371c96e7f4127c5db469764a17d39981f19884679ba883e28b733fde6142351dd8288c7bc61c379b7eefe7fa7acca1a
faaa0794b24f250f787bc9b9605270108ea101a0 refactor: Remove PERSISTENT_WORKER_* yaml templates (MarcoFalke)
fa1d8955f69d3934f975f42eb04b5a3fc0d8aa35 ci: Move tidy to persistent worker (MarcoFalke)
Pull request description:
Cirrus CI will be capping the free compute soon. For now, switch more tasks to persistent worker, as recommended by Cirrus CI.
(See slightly related discussion in https://github.com/bitcoin/bitcoin/issues/28098)
Also, add more docs.
ACKs for top commit:
hebasto:
re-ACK faaa0794b24f250f787bc9b9605270108ea101a0
Tree-SHA512: d83032eeeda7869969aa8504ed5e88089f896da850f97dfb799c4d4f64e6cb9da7973eec9a97b07f646613d1dabd2308dc0055ab6e1062d18bd34a201fcaf6db
fb02ba3c5f5bcd96b5e3622ef001b8e57ce63fc0 mempool_entry: improve struct packing (Anthony Towns)
1a118062fbc4ec8f645f4ec4298d869a869c3344 net_processing: Clean up INVENTORY_BROADCAST_MAX constants (Anthony Towns)
6fa49937e488d0924044786c76b42324b659f351 test: Check tx from disconnected block is immediately requestable (glozow)
e4ffabbffacc4b890d393aafcc8286916ef887d8 net_processing: don't add txids to m_tx_inventory_known_filter (Anthony Towns)
6ec1809d33bfc42b80cb6f35625dccd56be8d507 net_processing: drop m_recently_announced_invs bloom filter (Anthony Towns)
a70beafdb22564043dc24fc98133fdadbaf77d8a validation: when adding txs due to a block reorg, allow immediate relay (Anthony Towns)
1e9684f39fba909b3501e9402d5b61f4bf744ff2 mempool_entry: add mempool entry sequence number (Anthony Towns)
Pull request description:
This PR replaces the `m_recently_announced_invs` bloom filter with a simple sequence number tracking the mempool state when we last considered sending an INV message to a node. This saves 33kB per peer (or more if we raise the rate at which we relay transactions over the network, in which case we would need to increase the size of the bloom filter proportionally).
The philosophy here (compare with #18861 and #19109) is that we consider the rate limiting on INV messages to only be about saving bandwidth and not protecting privacy, and therefore after you receive an INV message, it's immediately fair game to request any transaction that was in the mempool at the time the INV message was sent. We likewise consider the BIP 133 feefilter and BIP 37 bloom filters to be bandwidth optimisations here, and treat transactions as requestable if they would have been announced without those filters. Given that philosophy, tracking the timestamp of the last INV message and comparing that against the mempool entry time allows removal of each of `m_recently_announced_invs`, `m_last_mempool_req` and `UNCONDITIONAL_RELAY_DELAY` and associated logic.
ACKs for top commit:
naumenkogs:
ACK fb02ba3c5f5bcd96b5e3622ef001b8e57ce63fc0
amitiuttarwar:
review ACK fb02ba3c5f5
glozow:
reACK fb02ba3c5f5bcd96b5e3622ef001b8e57ce63fc0
Tree-SHA512: cbba5ee04c86df26b6057f3654c00a2b45ec94d354f4f157a769cecdaa0b509edaac02b3128afba39b023e82473fc5e28c915a787f84457ffe66638c6ac9c2d4
fa6e5d3eeffebf81b5d7ca99bf7b5e70356516ab ci: Avoid error on macOS native (MarcoFalke)
fa193f5dfc937a7ff8e12b9ffd21861046a46489 ci: Fix macOS-cross SDK rsync (MarcoFalke)
Pull request description:
This should fix the macOS-cross build on Cirrus CI containers.
Locally this was already working, because the SDK was cached in
`/ci_container_base/` in the image, which is also the folder used for a
later CI run.
However, on Cirrus CI, when using an image *and* a custom `BASE_ROOT_DIR`,
the SDK will not be found in `/ci_base_install/`, nor in `BASE_ROOT_DIR`.
Fix this by normalizing *all* folders to `/ci_container_base/`.
ACKs for top commit:
hebasto:
ACK fa6e5d3eeffebf81b5d7ca99bf7b5e70356516ab
Tree-SHA512: 8312f7e72c3638caa6804e39206d3563ba1703204d53ce63de22e0a16a71e1e143ec00fac6b43ebfc0653c7b74160472c04e95e2d694c8c0965e7dc39e627d39
769f5b15f207ce6d1067672ea5e195541c97de6b test: check backup from `migratewallet` can be successfully restored (brunoerg)
Pull request description:
`migratewallet` migrates the wallet to a descriptor one. During the process, it generates a backup file of the wallet in case of an incorrect migration. This PR adds test to check if the backup file can be successfully restored.
ACKs for top commit:
achow101:
ACK 769f5b15f207ce6d1067672ea5e195541c97de6b
MarcoFalke:
lgtm ACK 769f5b15f207ce6d1067672ea5e195541c97de6b
Tree-SHA512: 94c50b34fbd47c4d3cc34b94e9e7903bc233608c7f50f45c161669996fd5f5b7d8f9a4e6a3437b9151d66a76af833f3f1ca28e44ecb63b5a8f391f6d6be0e39f
