38854 Commits

Author SHA1 Message Date
glozow
a84dade1f9
Merge bitcoin/bitcoin#28157: test doc: tests acceptstalefeeestimates option is only supported on regtest chain
ee5a0369cc4305da7b3d26f37677de05ad797e51 test: ensure acceptstalefeeestimates is supported only on regtest chain (ismaelsadeeq)
22d5d4b2b2486feaef981e96f0321f020617f082 tx fees, policy: doc: update and delete unnecessary comment (ismaelsadeeq)

Pull request description:

  This PR Follow up comments from [#27622](https://github.com/bitcoin/bitcoin/pull/27622)

  It test that the new `regtest-only` option `acceptstalefeeestimates` is not supported on [main, signet and test chains](https://github.com/bitcoin/bitcoin/pull/27622/files#r1235218268), removes an unnecessary [comment](https://github.com/bitcoin/bitcoin/pull/27622/files#r1235204323), and update fee estimator  `MAXFILEAGE` [description comment](https://github.com/bitcoin/bitcoin/pull/27622/files#r1233887314).

ACKs for top commit:
  jonatack:
    ACK ee5a0369cc4305da7b3d26f37677de05ad797e51
  glozow:
    utACK ee5a0369cc4305da7b3d26f37677de05ad797e51

Tree-SHA512: 4755f25b08db62f37614ea768272b12580ee0d481fb7fa339379901a6132c66828777c6747d3fe67490ceace3a6ff248bf13bdf65720f6e5ba8642eb762acd3c
2023-08-22 09:17:12 +01:00
brunoerg
452c094449 test: fix 'unknown named parameter' test in wallet_basic
Fixes loop when testing an unknown named parameter.
2023-08-21 20:59:15 -03:00
fanquake
ded6873340
Merge bitcoin/bitcoin#28292: ci: Disable cache save for pull requests in GitHub Actions
241d6ca34c60d9003a27bb7960cebfb66366753b ci: Disable cache save for pull requests in GitHub Actions (Hennadii Stepanov)

Pull request description:

  This PR disable cache save for pull requests in GitHub Actions.

  Otherwise, multiple pull requests fill GitHub Actions cache quota shortly.

  See a discussion [here](https://github.com/bitcoin/bitcoin/pull/28187#discussion_r1295459732).

  ---

  **NOTE** for the maintainers with "owner" permissions.

  This PR needs the `actions/cache/restore@*` and `actions/cache/save@*` acrions to be explicitly allowed in the repository's Actions permissions.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 241d6ca34c60d9003a27bb7960cebfb66366753b

Tree-SHA512: a7786c7ec99bfa6991bf6ae08fd7ed3546e8c5d083a1b2bae7638f6f31e77fdf2cf4fc69d85834faf87f98db1f4a82026ce1dc5fc1bc6650e8bf1c09bf7e90f5
2023-08-21 16:59:33 +01:00
Hennadii Stepanov
241d6ca34c
ci: Disable cache save for pull requests in GitHub Actions
Otherwise, multiple pull requests fill GitHub Actions cache quota
shortly.
2023-08-21 11:26:11 +01:00
fanquake
723f1c669f
Merge bitcoin/bitcoin#28218: refactor: Make IsInitialBlockDownload & NotifyHeaderTip not require a Chainstate
94a98fbd1d14a4ea10629b917771d50f3191e944 assumeutxo cleanup: Move IsInitialBlockDownload & NotifyHeaderTip to ChainstateManager (Ryan Ofsky)

Pull request description:

  This change makes `IsInitialBlockDownload` and `NotifyHeaderTip` functions no longer tied to individual `Chainstate` objects. It makes them work with the `ChainstateManager` object instead so code is simpler and it is no longer possible to call them incorrectly with an inactive `Chainstate`.

  This change also makes `m_cached_finished_ibd` caching easier to reason about, because now there is only one cached value instead of two (for background and snapshot chainstates) so the cached IBD state now no longer gets reset when a snapshot is loaded.

  There should be no change in behavior because these functions were always called on the active `ChainState` objects.

  These changes were discussed previously https://github.com/bitcoin/bitcoin/pull/27746#discussion_r1246868905 and https://github.com/bitcoin/bitcoin/pull/27746#discussion_r1237552792 as possible followups for that PR.

ACKs for top commit:
  MarcoFalke:
    re-ACK 94a98fbd1d14a4ea10629b917771d50f3191e944 🐺
  naumenkogs:
    ACK 94a98fbd1d14a4ea10629b917771d50f3191e944
  dergoegge:
    reACK 94a98fbd1d14a4ea10629b917771d50f3191e944

Tree-SHA512: 374d6e5c9bbc7564c143f634bd709a4e8f4a42c8d77e7a8554c832acdcf60fa2a134f3ea10827db1a1e0191006496329c0ebf5c64f3ab868398c3722bb7ff56f
2023-08-21 10:55:35 +01:00
ismaelsadeeq
ee5a0369cc test: ensure acceptstalefeeestimates is supported only on regtest chain 2023-08-21 07:21:34 +01:00
Ryan Ofsky
94a98fbd1d assumeutxo cleanup: Move IsInitialBlockDownload & NotifyHeaderTip to ChainstateManager
This change makes IsInitialBlockDownload and NotifyHeaderTip functions no
longer tied to individual Chainstate objects. It makes them work with the
ChainstateManager object instead so code is simpler and it is no longer
possible to call them incorrectly with an inactive Chainstate.

This change also makes m_cached_finished_ibd caching easier to reason about,
because now there is only one cached value instead of two (for background and
snapshot chainstates) so the cached IBD state now no longer gets reset when a
snapshot is loaded.

There should be no change in behavior because these functions were always
called on the active ChainState objects.

These changes were discussed previously
https://github.com/bitcoin/bitcoin/pull/27746#discussion_r1246868905 and
https://github.com/bitcoin/bitcoin/pull/27746#discussion_r1237552792 as
possible followups for that PR.
2023-08-18 12:52:30 -04:00
MarcoFalke
fa8e89d5e4
ci: Remove distro-name from task name
The exact distro name should not be important. Also, it is easy to find
out, if needed. Thus, remove it to avoid bloat and maintenance overhead
having to keep it in sync.
2023-08-18 18:06:04 +02:00
MarcoFalke
fad006fa0a
ci: Switch remaining tasks to self-hosted
This allows to drop unused templates, such as
cirrus_ephemeral_worker_template_env, or container_depends_template.

Also, ccache_cache, previous_releases_cache, and
base_depends_built_cache can be dropped, because the caching is done in
container volumes on the self-hosted runners.
2023-08-18 18:05:59 +02:00
fanquake
9b066da8af
Merge bitcoin/bitcoin#28295: ci: Add missing amd64 to win64-cross task
fa56d17a4b868f42fa45bea0d1f0c78de75e7838 ci: Add missing amd64 to win64-cross task (MarcoFalke)

Pull request description:

  Currently the task will fail if run on non-`x86_64`.

  Fix this by adding the missing `amd64`, similar to

  7bf078f2b7/ci/test/00_setup_env_i686_multiprocess.sh (L11)

ACKs for top commit:
  hebasto:
    ACK fa56d17a4b868f42fa45bea0d1f0c78de75e7838

Tree-SHA512: faab1c5b945283b7e8d080bbcc8e9379c480cf6973506149ace5990cb4d04673f83f4bc36d08d5b4e9cb17a86fdbe23ac97ef4eab0e842616b367b8138229c58
2023-08-18 14:42:01 +01:00
fanquake
93e8bc22bf
Merge bitcoin/bitcoin#28296: ci: Add missing ${CI_RETRY_EXE} before curl
fa968ef6a32bb66ca9fa3b84807f1c960f6a9833 ci: Add missing ${CI_RETRY_EXE} before curl (MarcoFalke)

Pull request description:

  GitHub is frequently down and this is causing many intermittent issues. For example, from today: https://cirrus-ci.com/task/5740122163904512?logs=ci#L398

  Try to fix it with a retry.

ACKs for top commit:
  hebasto:
    ACK fa968ef6a32bb66ca9fa3b84807f1c960f6a9833

Tree-SHA512: e9a1e51af37ec718ca776f20d8b5394680a9b059fb2a22505ccb6781472e4b072694e7d01661ede5a87ef1f58a9143e29d032b6a5d13d8be3b7d46eeff061563
2023-08-18 13:59:18 +01:00
MarcoFalke
fa56d17a4b
ci: Add missing amd64 to win64-cross task
Also, do the same for android, which also fails.
2023-08-18 14:23:17 +02:00
MarcoFalke
fa968ef6a3
ci: Add missing ${CI_RETRY_EXE} before curl 2023-08-18 14:11:40 +02:00
fanquake
51324c9517
guix: pre time-machine bump changes (Windows)
Split out of #27897. This is some refactoring to the Windows Guix build
that facilitates bumping our Guix time-machine. Namely, avoiding
`package-with-extra-configure-variable`, which is non-functional in the
newer time-machine, see https://issues.guix.gnu.org/64436.

At the same time, consolidate our Windows GCC build into mingw-w64-base-gcc.
Rename `gcc-10-remap-guix-store.patch` to avoid changing it whenever GCC changes.

We move the old `building-on` inside `explicit-cross-configure`, so that
non-windows builds continue to work. Note that `explicit-cross-configure`
will be going away entirely (see #27897).
2023-08-18 12:02:26 +01:00
fanquake
7bf078f2b7
Merge bitcoin/bitcoin#28237: refactor: Enforce C-str fmt strings in WalletLogPrintf()
fa60fa3b0cba4a30726af8e0e9d1e84e14849eda bitcoin-tidy: Apply bitcoin-unterminated-logprintf to spkm as well (MarcoFalke)
faa11434fe38aa82892802adb6d879d112ae1675 refactor: Enable all clang-tidy plugin bitcoin tests (MarcoFalke)
fa6dc57760e0a04dbb2e365ca7ad9fd8171ebfdb refactor: Enforce C-str fmt strings in WalletLogPrintf() (MarcoFalke)
fa244f3321de7884f530bb38493a8d0a0cec86ab doc: Fix bitcoin-unterminated-logprintf tidy comments (MarcoFalke)

Pull request description:

  All fmt functions only accept a raw C-string as argument.

  There should never be a need to pass a format string that is not a compile-time string literal, so disallow it in `WalletLogPrintf()` to avoid accidentally introducing it.

  Apart from consistency, this also fixes the clang-tidy plugin bug https://github.com/bitcoin/bitcoin/pull/26296#discussion_r1286821141.

ACKs for top commit:
  theuni:
    ACK fa60fa3b0cba4a30726af8e0e9d1e84e14849eda

Tree-SHA512: fa6f4984c50f9b34e850bdfee7236706af586e512d866cc869cf0cdfaf9aa707029c210ca72d91f85e75fcbd8efe0d77084701de8c3d2004abfd7e46b6fa9072
2023-08-18 11:38:38 +01:00
fanquake
5eb669024f
Merge bitcoin/bitcoin#28100: crypto: more Span<std::byte> modernization & follow-ups
57cc136282c38825e97bbf85728df4bdf1ccc648 crypto: make ChaCha20::SetKey wipe buffer (Pieter Wuille)
da0ec62e34cc56bf8990e28c6ec12683d4752305 tests: miscellaneous hex / std::byte improvements (Pieter Wuille)
bdcbc8594c208f11e7d5221700bfa7f7a874aec9 fuzz: support std::byte in Consume{Fixed,Variable}LengthByteVector (Pieter Wuille)
7d1cd932342e74421ae927800eeada14f504b944 crypto: require key on ChaCha20 initialization (Pieter Wuille)
44c11769a83b90ca6b8af086d6fa69ff7ac1c3ae random: simplify FastRandomContext::randbytes using fillrand (Pieter Wuille)
3da636e08b781fa2f7c1c23bb937015185732a75 crypto: refactor ChaCha20 classes to use Span<std::byte> interface (Pieter Wuille)

Pull request description:

  This modernizes the ChaCha20 and ChaCha20Aligned interfaces to be `Span<std::byte>` based, and other improvements.

  * Modifies all functions and constructors of `ChaCha20` and `ChaCha20Aligned` to be `Span<std::byte>` based (aligning them with `FSChaCha20`, `AEADChaCha20Poly1305`, and `FSChaCha20Poly1305`)
  * Remove default constructors, to make sure all call sites provide a key (suggested in https://github.com/bitcoin/bitcoin/pull/26153#discussion_r1129313162)
  * Wipe key material on rekey for security (suggested in https://github.com/bitcoin/bitcoin/pull/26153#discussion_r1267164605)
  * Use `HexStr` on byte vectors in tests (suggested in https://github.com/bitcoin/bitcoin/pull/27993#discussion_r1262023316)
  * Support `std::byte` vectors in `ConsumeRandomLengthByteVector` and `ConsumeFixedLengthByteVector`, and use it (suggested in https://github.com/bitcoin/bitcoin/pull/27993#discussion_r1265337111)
  * And a few more.

  While related, I don't see this as a necessary for BIP324.

ACKs for top commit:
  stratospher:
    ACK 57cc136.
  theStack:
    re-ACK 57cc136282c38825e97bbf85728df4bdf1ccc648

Tree-SHA512: 361da4ff003c8465a32eeac0983a8a6f047dbbf5b400168b409c8e3234e79d577fc854e0764389446585da3e12b964c94dd67fc0c9c1d1d092cec296121e05d4
2023-08-18 11:19:34 +01:00
fanquake
e4a855c4e0
Merge bitcoin/bitcoin#28289: rpc: remove one more quote from non-string oneline description
239431444216850b63ecf01c3b5c5d6d24230d08 rpc: remove one more quote from non-string oneline description (Martin Zumsande)

Pull request description:

  This fixes a silent conflict between https://github.com/bitcoin/bitcoin/pull/28123 (which removed all `\"options\"`) and https://github.com/bitcoin/bitcoin/pull/27460 (which added a new one).

  It should fix the current CI failures.

ACKs for top commit:
  ajtowns:
    utACK 239431444216850b63ecf01c3b5c5d6d24230d08
  MarcoFalke:
    lgtm ACK 239431444216850b63ecf01c3b5c5d6d24230d08
  jonatack:
    ACK 239431444216850b63ecf01c3b5c5d6d24230d08
  hebasto:
    ACK 239431444216850b63ecf01c3b5c5d6d24230d08

Tree-SHA512: feb0c2b936a77be45d9c65aa7d738277b2266b5153665fee3b1413045de521195dc7d5efa2fc8b37b22f16e9b8d0ee8de25bfd151a428666122b31f64056557a
2023-08-18 10:01:22 +01:00
MarcoFalke
fa6286891f
Remove unused includes from wallet.cpp
This removes unused includes, such as undo.h or txmempool.h from
wallet.cpp.

Also, add missing ones, according to IWYU.
2023-08-18 08:20:43 +02:00
kevkevin
9a84200cfc
doc, refactor: Changing -torcontrol help to specify that a default port is used
Right now when we get the help for -torcontrol it says that there is a
default ip and port we dont specify if there is a specified ip that we
would also use port 9051 as default
2023-08-17 23:58:47 -05:00
Reese Russell
6e8f6468cb removed StrFormatInternalBug quote delimitation 2023-08-18 04:04:06 +00:00
Martin Zumsande
2394314442 rpc: remove one more quote from non-string oneline description
This fixes a silent conflict betwen #28123 and #27460
2023-08-17 16:18:56 -04:00
Pieter Wuille
57cc136282 crypto: make ChaCha20::SetKey wipe buffer 2023-08-17 15:37:41 -04:00
Pieter Wuille
da0ec62e34 tests: miscellaneous hex / std::byte improvements 2023-08-17 15:31:56 -04:00
Pieter Wuille
bdcbc8594c fuzz: support std::byte in Consume{Fixed,Variable}LengthByteVector 2023-08-17 15:31:56 -04:00
Pieter Wuille
7d1cd93234 crypto: require key on ChaCha20 initialization 2023-08-17 15:31:27 -04:00
Pieter Wuille
44c11769a8 random: simplify FastRandomContext::randbytes using fillrand 2023-08-17 15:26:38 -04:00
Pieter Wuille
3da636e08b crypto: refactor ChaCha20 classes to use Span<std::byte> interface 2023-08-17 15:26:34 -04:00
MarcoFalke
fa8fdbe229
Remove unused includes from blockfilter.h
This removes unused includes, primitives/block found manually, and the
others by iwyu:

blockfilter.h should remove these lines:
- #include <serialize.h>  // lines 16-16
- #include <undo.h>  // lines 18-18
2023-08-17 18:28:15 +02:00
Anthony Towns
1b09cc5959 Make post-p2sh consensus rules mandatory for tx relay 2023-08-18 00:59:27 +10:00
Anthony Towns
69c31bc748 doc, policy: Clarify comment on STANDARD_SCRIPT_VERIFY_FLAGS 2023-08-18 00:57:59 +10:00
MarcoFalke
fad8c36aa9
move-only: Create src/kernel/mempool_removal_reason.h
This is needed for a future commit. Can be reviewed with:
--color-moved=dimmed-zebra
2023-08-17 16:26:20 +02:00
MarcoFalke
fa57608800
Remove unused includes from txmempool.h
... and move them to where they are really needed.

This was found by IWYU:

txmempool.h should remove these lines:
- #include <random.h>  // lines 29-29
- class CBlockIndex;  // lines 43-43
- class Chainstate;  // lines 45-45

Also, move the stdlib section to the right place. Can be reviewed with:
--color-moved=dimmed-zebra
2023-08-17 16:25:31 +02:00
MarcoFalke
fadf671fa5
Refactor: Remove confusing static_cast 2023-08-17 15:55:07 +02:00
MarcoFalke
faeea1ab58
refactor: Add missing includes 2023-08-17 15:55:01 +02:00
fanquake
6ce5e8f475
Merge bitcoin/bitcoin#28278: ci: Refactor: Remove CI_USE_APT_INSTALL
fa263877691a7babb08a83f5f977390a0ba64729 ci: Refactor: Remove CI_USE_APT_INSTALL (MarcoFalke)

Pull request description:

  Seems odd to use `CI_USE_APT_INSTALL == no` as an alias for `CI_OS_NAME == macos`. Fix this by removing the alias.

  Also, for github CI:
  * restore MAKEJOBS to the same value as in cirrus.yml.
  * remove cirrus-only PACKAGE_MANAGER_INSTALL.
  * remove redundant TEST_RUNNER_TIMEOUT_FACTOR
  * Add M1 link

ACKs for top commit:
  hebasto:
    ACK fa263877691a7babb08a83f5f977390a0ba64729.

Tree-SHA512: e235aa70abd60738a9ad1531284a94e2122c9c7a22c2514ede437b49da5c06b2597fba7fccf615541fb3adb4e1f8076aa8c6047f926393191a629713554ab000
2023-08-17 14:17:40 +01:00
fanquake
de197c19e4
Merge bitcoin/bitcoin#28282: ci: Ensure that only a single workflow processes github.ref at a time
0080b5650e0bf125612e62b591f348076e9ad5d7 ci: Ensure that only a single workflow processes `github.ref` at a time (Hennadii Stepanov)

Pull request description:

  This PR ensures that only a single workflow processes any push or pull request at a time.

  A new push will be queued (including the master branch).

  For a new pull request update, the previous in-progress one will be cancelled.

  Address https://github.com/bitcoin/bitcoin/pull/28187#discussion_r1295144563.

ACKs for top commit:
  dergoegge:
    utACK 0080b5650e0bf125612e62b591f348076e9ad5d7

Tree-SHA512: d03f1678c93c817c1c3be5e75171bfe85d494f99a9aab7113c9438e0c820e950edd5c10199cccd54868e6dc50217bb3aa5601b23bc88ad4927c001031e917513
2023-08-17 14:03:11 +01:00
fanquake
ecb20563b6
Merge bitcoin/bitcoin#28123: Bugfix: RPC: Remove quotes from non-string oneline descriptions
5e3e83b005518659a69916c373b808da27e51791 RPC/Mining: Document template_request better for getblocktemplate (Luke Dashjr)
de319c61759952318364fbcb28c47f0959d89d0e RPC/rpcdoccheck: Error if a oneline_description has a quote for a non-string (Luke Dashjr)
7c61e9df90579ed42a30016e52355e437733b128 Bugfix: RPC: Remove quotes from non-string oneline descriptions (Luke Dashjr)

Pull request description:

  Various JSON Object parameters had a `oneline_description` with quote characters. Fix those, and extend `rpcdoccheck` to detect them.

  Also, slightly improve GBT's oneline description for template_request.

ACKs for top commit:
  MarcoFalke:
    review ACK 5e3e83b005518659a69916c373b808da27e51791

Tree-SHA512: 363d1669a661d0acfc19fddb57e777d781c7246f330cf62160e77dde10a6adcb0249db748127067da1afe1b7d17c71cf611d9fdc3664d6bf5b3f30105637769a
2023-08-17 13:58:31 +01:00
fanquake
6d473bad22
Merge bitcoin/bitcoin#27941: test: Fix intermittent issue in mining_getblocktemplate_longpoll.py
fa748c6f2ac2f9cac7ce42fd745ed3c3eae093b7 test: Fix intermittent issue in mining_getblocktemplate_longpoll.py (MarcoFalke)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/26962

  Wait for the thread to have started and the RPC to have reached the node before continuing. Otherwise the test may run into a race.

  For example:

  ```
   test  2023-06-23T13:10:29.245000Z TestFramework (INFO): Test that introducing a new transaction into the mempool will terminate the longpoll
   node0 2023-06-23T13:10:29.245712Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
   node0 2023-06-23T13:10:29.245915Z [httpworker.3] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getblocktemplate user=__cookie__
   node0 2023-06-23T13:10:29.252594Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
   node0 2023-06-23T13:10:29.254545Z [httpworker.2] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getblockchaininfo user=__cookie__
   node0 2023-06-23T13:10:29.256530Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
   node0 2023-06-23T13:10:29.256741Z [httpworker.1] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=sendrawtransaction user=__cookie__
   node0 2023-06-23T13:10:29.258033Z [httpworker.1] [validationinterface.cpp:213] [TransactionAddedToMempool] [validation] Enqueuing TransactionAddedToMempool: txid=38335600f2465c0f8bb2b86d5830a34851d86fa879800c0e1434ddfc78c42898 wtxid=c033cd3efd301c369d66cf759769159609471bd4f9efb3ee30e7209e57b74778
   node0 2023-06-23T13:10:29.258263Z [httpworker.1] [txmempool.cpp:660] [check] [mempool] Checking mempool with 1 transactions and 1 inputs
   node0 2023-06-23T13:10:29.258542Z [scheduler] [validationinterface.cpp:213] [operator()] [validation] TransactionAddedToMempool: txid=38335600f2465c0f8bb2b86d5830a34851d86fa879800c0e1434ddfc78c42898 wtxid=c033cd3efd301c369d66cf759769159609471bd4f9efb3ee30e7209e57b74778
   node0 2023-06-23T13:10:29.259549Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
   node0 2023-06-23T13:10:29.259745Z [httpworker.0] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=decoderawtransaction user=__cookie__
   node0 2023-06-23T13:10:29.261066Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:52690
   node0 2023-06-23T13:10:29.261803Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
   node0 2023-06-23T13:10:29.262770Z [httpworker.2] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getblocktemplate user=__cookie__
  ```

  (`sendrawtransaction` is called before `getblocktemplate`)

ACKs for top commit:
  jamesob:
    Github ACK fa748c6f2a
  theStack:
    ACK fa748c6f2ac2f9cac7ce42fd745ed3c3eae093b7

Tree-SHA512: c67d9ec7c56e8a22c1a26a3c3d4d4a4bcc17e4282cad0d66561ba2abd6e92240cb028369b4edc6077ea34e8736c0294f6066381979aee22a6166580cea43729a
2023-08-17 13:30:49 +01:00
fanquake
0a55bcd299
Merge bitcoin/bitcoin#27981: Fix potential network stalling bug
3388e523a129ad9c7aef418c9f57491f8c2d9df8 Rework receive buffer pushback (Pieter Wuille)

Pull request description:

  See https://github.com/ElementsProject/elements/issues/1233. There, it has been observed that if both sides of a P2P connection have a significant amount of data to send, a stall can occur, where both try to drain their own send queue before trying to receive. The same issue seems to apply to the current Bitcoin Core codebase, though I don't know whether it's a frequent issue for us.

  The core issue is that whenever our optimistic send fails to fully send a message, we do subsequently not even select() for receiving; if it then turns out that sending is not possible either, no progress is made at all. To address this, the solution used in this PR is to still select() for both sending and receiving when an optimistic send fails, but skip receiving if sending succeeded, and (still) doesn't fully drain the send queue.

  This is a significant reduction in how aggressive the "receive pushback" mechanism is, because now it will only mildly push back while sending progress is made; if the other side stops receiving entirely, the pushback disappears. I don't think that's a serious problem though:
  * We still have a pushback mechanism at the application buffer level (when the application receive buffer overflows, receiving is paused until messages in the buffer get processed; waiting on our own net_processing thread, not on the remote party).
  * There are cases where the existing mechanism is too aggressive; e.g. when the send queue is non-empty, but tiny, and can be sent with a single send() call. In that case, I think we'd prefer to still receive within the same processing loop of the network thread.

ACKs for top commit:
  ajtowns:
    ACK 3388e523a129ad9c7aef418c9f57491f8c2d9df8
  naumenkogs:
    ACK 3388e523a129ad9c7aef418c9f57491f8c2d9df8
  mzumsande:
    Tested ACK 3388e523a129ad9c7aef418c9f57491f8c2d9df8

Tree-SHA512: 28960feb3cd2ff3dfb39622510da62472612f88165ea98fc9fb844bfcb8fa3ed3633f83e7bd72bdbbbd37993ef10181b2e1b34836ebb8f0d83fd1c558921ec17
2023-08-17 13:15:42 +01:00
MarcoFalke
fa26387769
ci: Refactor: Remove CI_USE_APT_INSTALL 2023-08-17 13:55:18 +02:00
fanquake
7ef2d4ee4d
Merge bitcoin/bitcoin#28244: Break up script/standard.{h/cpp}
91d924ede1b421df31c895f4f43359e453a09ca5 Rename script/standard.{cpp/h} to script/solver.{cpp/h} (Andrew Chow)
bacdb2e208531124e85ed2d4ea2a4b508fbb5088 Clean up script/standard.{h/cpp} includes (Andrew Chow)
f3c9078b4cddec5581e52de5c216ae53984ec130 Clean up things that include script/standard.h (Andrew Chow)
8bbe257bac751859a272ddf52dc0328c1b5a1ede MOVEONLY: Move datacarrier defaults to policy.h (Andrew Chow)
7a172c76d2361fc3cdf6345590e26c79a7821672 Move CTxDestination to its own file (Andrew Chow)
145f36ec81e79d2e391847520364c2420ef0e0e8 Move Taproot{SpendData/Builder} to signingprovider.{h/cpp} (Andrew Chow)
86ea8bed5473f400f7a93fcc455393a574a2f319 Move CScriptID to script.{h/cpp} (Andrew Chow)
b81ebff0d99c45c071b999796b8ae3f0f2517b22 Remove ScriptHash from CScriptID constructor (Andrew Chow)
cba69dda3da0e4fa39cff5ce4dc81d1242fe651b Move MANDATORY_SCRIPT_VERIFY_FLAGS from script/standard.h to policy/policy.h (Anthony Towns)

Pull request description:

  Some future work needs to touch things in script/standard.{h/cpp}, however it is unclear if it is safe to do so as they are included in several different places that could effect standardness and consensus. It contains a mix of policy parameters, consensus parameters, and utilities only used by the wallet. This PR breaks up the various components and renames the files to clearly separate everything.

  * `CTxDestination` is moved to a new file `src/addresstype.{cpp/h}`
  * `TaprootSpendData` and `TaprootBuilder` (and their utility functions and structs) are moved to `SigningProvider` as these are used only during signing.
  * `CScriptID` is moved to `script/script.h` to be next to `CScript`.
  * `MANDATORY_SCRIPT_VERIFY_FLAGS` is moved to `interpreter.h`
  * The parameters `DEFAULT_ACCEPT_DATACARRIER` and `MAX_OP_RETURN_RELAY` are moved to `policy.h`
  * `standard.{cpp/h}` is renamed to `solver.{cpp/h}` since that's all that's left in the file after the above moves

ACKs for top commit:
  Sjors:
    ACK 91d924ede1b421df31c895f4f43359e453a09ca5
  ajtowns:
    ACK 91d924ede1b421df31c895f4f43359e453a09ca5
  MarcoFalke:
    ACK 91d924ede1b421df31c895f4f43359e453a09ca5 😇
  murchandamus:
    ACK 91d924ede1b421df31c895f4f43359e453a09ca5
  darosior:
    Code review ACK 91d924ede1b421df31c895f4f43359e453a09ca5.
  theStack:
    Code-review ACK 91d924ede1b421df31c895f4f43359e453a09ca5

Tree-SHA512: d347439890c652081f6a303d99b2bde6c371c96e7f4127c5db469764a17d39981f19884679ba883e28b733fde6142351dd8288c7bc61c379b7eefe7fa7acca1a
2023-08-17 12:54:16 +01:00
Hennadii Stepanov
0080b5650e
ci: Ensure that only a single workflow processes github.ref at a time 2023-08-17 11:49:14 +01:00
fanquake
d78ff380a2
Merge bitcoin/bitcoin#28214: ci: Move tidy to persistent worker
faaa0794b24f250f787bc9b9605270108ea101a0 refactor: Remove PERSISTENT_WORKER_* yaml templates (MarcoFalke)
fa1d8955f69d3934f975f42eb04b5a3fc0d8aa35 ci: Move tidy to persistent worker (MarcoFalke)

Pull request description:

  Cirrus CI will be capping the free compute soon. For now, switch more tasks to persistent worker, as recommended by Cirrus CI.

  (See slightly related discussion in https://github.com/bitcoin/bitcoin/issues/28098)

  Also, add more docs.

ACKs for top commit:
  hebasto:
    re-ACK faaa0794b24f250f787bc9b9605270108ea101a0

Tree-SHA512: d83032eeeda7869969aa8504ed5e88089f896da850f97dfb799c4d4f64e6cb9da7973eec9a97b07f646613d1dabd2308dc0055ab6e1062d18bd34a201fcaf6db
2023-08-17 11:21:04 +01:00
ismaelsadeeq
22d5d4b2b2 tx fees, policy: doc: update and delete unnecessary comment 2023-08-17 11:09:14 +01:00
fanquake
a62f5ee86c
Merge bitcoin/bitcoin#27675: p2p: Drop m_recently_announced_invs bloom filter
fb02ba3c5f5bcd96b5e3622ef001b8e57ce63fc0 mempool_entry: improve struct packing (Anthony Towns)
1a118062fbc4ec8f645f4ec4298d869a869c3344 net_processing: Clean up INVENTORY_BROADCAST_MAX constants (Anthony Towns)
6fa49937e488d0924044786c76b42324b659f351 test: Check tx from disconnected block is immediately requestable (glozow)
e4ffabbffacc4b890d393aafcc8286916ef887d8 net_processing: don't add txids to m_tx_inventory_known_filter (Anthony Towns)
6ec1809d33bfc42b80cb6f35625dccd56be8d507 net_processing: drop m_recently_announced_invs bloom filter (Anthony Towns)
a70beafdb22564043dc24fc98133fdadbaf77d8a validation: when adding txs due to a block reorg, allow immediate relay (Anthony Towns)
1e9684f39fba909b3501e9402d5b61f4bf744ff2 mempool_entry: add mempool entry sequence number (Anthony Towns)

Pull request description:

  This PR replaces the `m_recently_announced_invs` bloom filter with a simple sequence number tracking the mempool state when we last considered sending an INV message to a node. This saves 33kB per peer (or more if we raise the rate at which we relay transactions over the network, in which case we would need to increase the size of the bloom filter proportionally).

  The philosophy here (compare with #18861 and #19109) is that we consider the rate limiting on INV messages to only be about saving bandwidth and not protecting privacy, and therefore after you receive an INV message, it's immediately fair game to request any transaction that was in the mempool at the time the INV message was sent. We likewise consider the BIP 133 feefilter and BIP 37 bloom filters to be bandwidth optimisations here, and treat transactions as requestable if they would have been announced without those filters. Given that philosophy, tracking the timestamp of the last INV message and comparing that against the mempool entry time allows removal of each of `m_recently_announced_invs`, `m_last_mempool_req` and `UNCONDITIONAL_RELAY_DELAY` and associated logic.

ACKs for top commit:
  naumenkogs:
    ACK fb02ba3c5f5bcd96b5e3622ef001b8e57ce63fc0
  amitiuttarwar:
    review ACK fb02ba3c5f5
  glozow:
    reACK fb02ba3c5f5bcd96b5e3622ef001b8e57ce63fc0

Tree-SHA512: cbba5ee04c86df26b6057f3654c00a2b45ec94d354f4f157a769cecdaa0b509edaac02b3128afba39b023e82473fc5e28c915a787f84457ffe66638c6ac9c2d4
2023-08-17 10:52:06 +01:00
MarcoFalke
fa60fa3b0c
bitcoin-tidy: Apply bitcoin-unterminated-logprintf to spkm as well 2023-08-16 14:56:37 +02:00
MarcoFalke
faa11434fe
refactor: Enable all clang-tidy plugin bitcoin tests
This makes it easier to add new ones without having to modify this file
every time.
2023-08-16 14:48:06 +02:00
fanquake
60d3e4b0cd
Merge bitcoin/bitcoin#28273: ci: Fix macOS-cross SDK rsync
fa6e5d3eeffebf81b5d7ca99bf7b5e70356516ab ci: Avoid error on macOS native (MarcoFalke)
fa193f5dfc937a7ff8e12b9ffd21861046a46489 ci: Fix macOS-cross SDK rsync (MarcoFalke)

Pull request description:

  This should fix the macOS-cross build on Cirrus CI containers.

  Locally this was already working, because the SDK was cached in
  `/ci_container_base/` in the image, which is also the folder used for a
  later CI run.

  However, on Cirrus CI, when using an image *and* a custom `BASE_ROOT_DIR`,
  the SDK will not be found in `/ci_base_install/`, nor in `BASE_ROOT_DIR`.

  Fix this by normalizing *all* folders to `/ci_container_base/`.

ACKs for top commit:
  hebasto:
    ACK fa6e5d3eeffebf81b5d7ca99bf7b5e70356516ab

Tree-SHA512: 8312f7e72c3638caa6804e39206d3563ba1703204d53ce63de22e0a16a71e1e143ec00fac6b43ebfc0653c7b74160472c04e95e2d694c8c0965e7dc39e627d39
2023-08-16 13:10:58 +01:00
fanquake
72304ccf1e
Merge bitcoin/bitcoin#28257: test: check backup from migratewallet can be successfully restored
769f5b15f207ce6d1067672ea5e195541c97de6b test: check backup from `migratewallet` can be successfully restored (brunoerg)

Pull request description:

  `migratewallet` migrates the wallet to a descriptor one. During the process, it generates a backup file of the wallet in case of an incorrect migration. This PR adds test to check if the backup file can be successfully restored.

ACKs for top commit:
  achow101:
    ACK 769f5b15f207ce6d1067672ea5e195541c97de6b
  MarcoFalke:
    lgtm ACK 769f5b15f207ce6d1067672ea5e195541c97de6b

Tree-SHA512: 94c50b34fbd47c4d3cc34b94e9e7903bc233608c7f50f45c161669996fd5f5b7d8f9a4e6a3437b9151d66a76af833f3f1ca28e44ecb63b5a8f391f6d6be0e39f
2023-08-16 12:56:09 +01:00
MarcoFalke
fa6e5d3eef
ci: Avoid error on macOS native
This avoids "mkdir: /ci_container_base: Read-only file system"
2023-08-16 10:30:51 +02:00