Commit Graph

279 Commits

Author SHA1 Message Date
Lőrinc
743ac30e34 Add std::optional support to Boost's equality check
Also moved the operators to the bottom of the file since they're less important and to group them together.

Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: stickies-v <stickies-v@protonmail.com>
2024-09-09 21:29:44 +02:00
merge-script
c0cbe26a86 Merge bitcoin/bitcoin#30748: test: Pin and document TEST_DIR_PATH_ELEMENT, SeedRand::FIXED_SEED
fa84f9decd test: Pin and document TEST_DIR_PATH_ELEMENT (MarcoFalke)
2222f7a874 test: Rename SeedRand::SEED to FIXED_SEED for clarity (MarcoFalke)

Pull request description:

  Two small test changes:

  * A refactor to update the name and documentation around `SeedRand::FIXED_SEED`.
  * A change to extract and document `TEST_DIR_PATH_ELEMENT`, and to change its value to better match the `TMPDIR_PREFIX` in functional tests. The value previously included `PACKAGE_NAME`, which is cute, but doesn't explain why it was used (to include a space). So just use `test_common bitcoin` to achieve the same with less effort.

ACKs for top commit:
  hodlinator:
    ACK fa84f9decd
  ryanofsky:
    Code review ACK fa84f9decd

Tree-SHA512: eb35d6598bb08f9b996e3a4762d8f26b2441c0ca00780798e473015af735dfc9997120895a922b94d4b6ada45adadba4a686e9cf9c285ddf688848e764c64840
2024-09-06 09:42:02 +01:00
MarcoFalke
fa84f9decd test: Pin and document TEST_DIR_PATH_ELEMENT 2024-09-02 09:28:52 +02:00
MarcoFalke
2222f7a874 test: Rename SeedRand::SEED to FIXED_SEED for clarity 2024-08-29 09:38:05 +02:00
Hodlinator
50bc017040 refactor: Hand-replace some ParseHex -> ""_hex
The following scripted-diff commit will replace ParseHex("...") with "..."_hex_u8, but this replacement will not work in cases where vectors are needed instead of arrays, and is not ideal in cases where std::byte is accepted.

For example, it is currently necessary to use _hex_v_u8 when calling CScript operator<< because that operator does not currently support std::array or std::byte.

Conversely, it is incorrect to use _hex_v instead of _hex in net_processing.cpp for the MakeAndPushMessage argument, because if the argument is a std::vector it is considered variable-length and serialized with a size prefix, but if the argument is a std::array or Span is it considered fixed length and serialized without a prefix.

By the same logic, it is also safe to change the NUMS_H constant in pubkey.cpp from a std::vector to std::array because it is never serialized.
2024-08-28 19:11:59 +02:00
MarcoFalke
fa0fe08eca scripted-diff: [test] Use g_rng/m_rng directly
-BEGIN VERIFY SCRIPT-

 # Use m_rng in unit test files
 ren() { sed -i "s:\<$1\>:$2:g" $( git grep -l "$1" src/test/*.cpp src/wallet/test/*.cpp src/test/util/setup_common.cpp ) ; }
 ren InsecureRand32                m_rng.rand32
 ren InsecureRand256               m_rng.rand256
 ren InsecureRandBits              m_rng.randbits
 ren InsecureRandRange             m_rng.randrange
 ren InsecureRandBool              m_rng.randbool
 ren g_insecure_rand_ctx           m_rng
 ren g_insecure_rand_ctx_temp_path g_rng_temp_path

-END VERIFY SCRIPT-
2024-08-26 11:19:52 +02:00
MarcoFalke
fa899fb7aa fuzz: Speed up utxo_snapshot fuzz target
This speeds up the fuzz target, which allows "valid" inputs. It does not
affect the "INVALID" fuzz target.
2024-08-20 07:54:04 +02:00
MarcoFalke
fa386642b4 fuzz: Speed up utxo_snapshot by lazy re-init
The re-init is expensive, so skip it if there is no need.

Also, add an even faster fuzz target utxo_snapshot_invalid, which does
not need any re-init at all.
2024-08-16 08:06:59 +02:00
MarcoFalke
fae8c73d9e test: Disallow fee_estimator construction in ChainTestingSetup
It is expensive to construct, and only one test uses it.

Fix both issues by disallowing the construction and moving it to the
single test that uses it.
2024-08-13 10:30:44 +02:00
Hodlinator
73e3fa10b4 doc + test: Correct uint256 hex string endianness
Follow-up to #30436.

uint256 string representation was wrongfully documented as little-endian due to them being reversed by GetHex() etc, and base_blob::Compare() giving most significance to the beginning of the internal array. They are closer to "big-endian", but this commit tries to be even more precise than that.

uint256_tests.cpp - Avoid using variable from the left side of the condition in the right side.

setup_common.cpp - Skip needless ArithToUint256-conversion.
2024-08-03 21:59:54 +02:00
merge-script
bee23ce9ec Merge bitcoin/bitcoin#30399: test: Add arguments for creating a slimmer TestingSetup
f46b220256 fuzz: Use BasicTestingSetup for coins_view target (TheCharlatan)
9e2a723d5d test: Add arguments for creating a slimmer setup (TheCharlatan)

Pull request description:

  This adds arguments to some of the testing setup constructors for creating an environment without networking and a validation interface. This is useful for improving the performance of the utxo snapshot fuzz test,  which constructs a new TestingSetup on each iteration.

  Using this slimmed down `TestingSetup` in future might also make the tests a bit faster when run in aggregate.

ACKs for top commit:
  maflcko:
    review ACK f46b220256
  dergoegge:
    utACK f46b220256

Tree-SHA512: 9dc62512b127b781fc9e2d8ef2b5a9b06ebb927a8294b6d872001c553984a7eb1f348e0257b32435b34b5505b5d0323f73bdd572a673da272d3e1e8538ab49d6
2024-07-25 13:53:50 +01:00
Ryan Ofsky
7cc00bfc86 Merge bitcoin/bitcoin#30436: fix: Make TxidFromString() respect string_view length
09ce3501fa fix: Make TxidFromString() respect string_view length (Hodlinator)
01e314ce0a refactor: Change base_blob::SetHex() to take std::string_view (Hodlinator)
2f5577dc2e test: uint256 - Garbage suffixes and zero padding (Hodlinator)
f11f816800 refactor: Make uint256_tests no longer use deprecated BOOST_CHECK() (Hodlinator)
f0eeee2dc1 test: Add test for TxidFromString() behavior (Ryan Ofsky)

Pull request description:

  ### Problem

  Prior to this, `TxidFromString()` was passing `string_view::data()` into `uint256S()` which meant it would only receive the a naked `char*` pointer and potentially scan past the `string_view::length()` until it found a null terminator (or some other non-hex character).

  Appears to have been a fully dormant bug as callers were either passing a string literal or `std::string` directly to `TxidFromFromString()`, meaning a null terminator always existed at `pointer[length()]`. Bug existed since original merge of `TxidFromString()`.

  ### Solution

  Make `uint256S()` (and `base_blob::SetHex()`) take and operate on `std::string_view` instead of `const char*` and have `TxidFromString()` pass that in.

  (PR was prompted by comment in https://github.com/bitcoin/bitcoin/pull/30377#issuecomment-2208857200 (referring to https://github.com/bitcoin/bitcoin/pull/28922#discussion_r1404437378)).

ACKs for top commit:
  maflcko:
    re-ACK 09ce3501fa 🕓
  paplorinc:
    ACK 09ce3501fa
  ryanofsky:
    Code review ACK 09ce3501fa. I think the current code changes are about as small as you could make to fix the bug without introducing a string copy, and the surrounding test improvements are all very nice and welcome.

Tree-SHA512: c2c10551785fb6688d1e2492ba42a8eee4c19abbe8461bb0774d56a70c23cd6b0718d2641632890bee880c06202dee148126447dd2264eaed4f5fee7e1bcb581
2024-07-23 14:19:27 -04:00
Hodlinator
f11f816800 refactor: Make uint256_tests no longer use deprecated BOOST_CHECK() 2024-07-23 14:15:39 +02:00
TheCharlatan
9e2a723d5d test: Add arguments for creating a slimmer setup
Adds more testing options for creating an environment without networking
and a validation interface. This is useful for improving the performance
of the utxo snapshot fuzz test, which constructs a new TestingSetup on
each iteration.
2024-07-19 13:37:31 +02:00
merge-script
ff827a8f46 Merge bitcoin/bitcoin#30407: test: [refactor] Pass TestOpts
fa690c8e53 test: [refactor] Pass TestOpts (MarcoFalke)

Pull request description:

  Currently optional test context setup settings are passed by adding a new optional argument to the constructors. For example `extra_args`. This is problematic, because:

  * Adding more optional settings in the future requires touching all affected constructors, increasing their verbosity.
  * Setting only a later option requires setting the earlier ones.
  * Clang-tidy named args passed to `std::make_unique` are not checked.

  Fix all issues by adding a new struct `TestOpts`, which holds all options. Notes:

  * The chain type is not an option in the struct for now, because the default values vary.
  * The struct holds all possible test options globally. Not all fields may be used by all constructors. Albeit harmless, it is up to the test author to not set a field that is unused.

ACKs for top commit:
  kevkevinpal:
    utACK [fa690c8](fa690c8e53)
  dergoegge:
    utACK fa690c8e53
  TheCharlatan:
    Nice, ACK fa690c8e53

Tree-SHA512: 8db8efa5dff854a73757d3f454f8f902e41bb4358f5f9bae29dbb3e251e20ee93489605de51d0822ba31d97835cd15526a29c075278dd6a8bbde26134feb4f49
2024-07-15 17:21:55 +01:00
Ryan Ofsky
94d56b9def Merge bitcoin/bitcoin#30141: kernel: De-globalize validation caches
606a7ab862 kernel: De-globalize signature cache (TheCharlatan)
66d74bfc45 Expose CSignatureCache class in header (TheCharlatan)
021d38822c kernel: De-globalize script execution cache hasher (TheCharlatan)
13a3661aba kernel: De-globalize script execution cache (TheCharlatan)
ab14d1d6a4 validation: Don't error if maxsigcachesize exceeds uint32::max (TheCharlatan)

Pull request description:

  The validation caches are currently setup independently from where the rest of the validation code is initialized. This makes their ownership semantics unclear. There is also no clear enforcement on when and in what order they need to be initialized. The caches are always initialized in the `BasicTestingSetup` although a number of tests don't actually need them.

  Solve this by moving the caches from global scope into the `ChainstateManager` class. This simplifies the usage of the kernel library by no longer requiring manual setup of the caches prior to using the `ChainstateManager`. Tests that need to access the caches can instantiate them independently.

  ---
  This pull request is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).

ACKs for top commit:
  stickies-v:
    re-ACK 606a7ab862
  glozow:
    reACK 606a7ab
  ryanofsky:
    Code review ACK 606a7ab862. Just small formatting, include, and static_assert changes since last review.

Tree-SHA512: e7f3ee41406e3b233832bb67dc3a63c4203b5367e5daeed383df9cb590f227fcc62eae31311029c077d5e81b273a37a88a364db3dee2efe91bb3b9c9ddc8a42e
2024-07-08 12:14:12 -04:00
MarcoFalke
fa690c8e53 test: [refactor] Pass TestOpts 2024-07-08 16:11:15 +02:00
TheCharlatan
606a7ab862 kernel: De-globalize signature cache
Move its ownership to the ChainstateManager class.

Next to simplifying usage of the kernel library by no longer requiring
manual setup of the cache prior to using validation code, it also slims
down the amount of memory allocated by BasicTestingSetup.

Use this opportunity to make SignatureCache RAII styled

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-07-05 09:03:04 +02:00
TheCharlatan
13a3661aba kernel: De-globalize script execution cache
Move its ownership to the ChainstateManager class.

Next to simplifying usage of the kernel library by no longer requiring
manual setup of the cache prior to using validation code, it also slims
down the amount of memory allocated by BasicTestingSetup.
2024-07-04 22:39:37 +02:00
Pieter Wuille
97e16f5704 tests: make fuzz tests (mostly) deterministic with fixed seed 2024-07-01 12:39:57 -04:00
Pieter Wuille
810cdf6b4e tests: overhaul deterministic test randomness
The existing code provides two randomness mechanisms for test purposes:
- g_insecure_rand_ctx (with its wrappers InsecureRand*), which during tests is
  initialized using either zeros (SeedRand::ZEROS), or using environment-provided
  randomness (SeedRand::SEED).
- g_mock_deterministic_tests, which controls some (but not all) of the normal
  randomness output if set, but then makes it extremely predictable (identical
  output repeatedly).

Replace this with a single mechanism, which retains the SeedRand modes to control
all randomness. There is a new internal deterministic PRNG inside the random
module, which is used in GetRandBytes() when in test mode, and which is also used
to initialize g_insecure_rand_ctx. This means that during tests, all random numbers
are made deterministic. There is one exception, GetStrongRandBytes(), which even
in test mode still uses the normal PRNG state.

This probably opens the door to removing a lot of the ad-hoc "deterministic" mode
functions littered through the codebase (by simply running relevant tests in
SeedRand::ZEROS mode), but this isn't done yet.
2024-07-01 10:26:46 -04:00
Sjors Provoost
64ebb0f971 Always pass options to BlockAssembler constructor
This makes the options argument for BlockAssembler constructor mandatory,
dropping implicit use of ArgsManager. The caller i.e. the Mining
interface implementation now handles this.

In a future Stratum v2 change the Options object needs to be
mofified after arguments have been processed. Specifically
the pool communicates how many extra bytes it needs for
its own outputs (payouts, extra commitments, etc). This will need
to be substracted from what the user set as -blockmaxweight.

Such a change can be implemented in createNewBlock, after
ApplyArgsManOptions.
2024-06-18 18:47:51 +02:00
stickies-v
260f8da71a refactor: remove warnings globals 2024-06-13 11:20:49 +01:00
Ava Chow
891e4bf374 Merge bitcoin/bitcoin#28339: validation: improve performance of CheckBlockIndex
5bc2077e8f validation: allow to specify frequency for -checkblockindex (Martin Zumsande)
d5a631b959 validation: improve performance of CheckBlockIndex (Martin Zumsande)
32c80413fd bench: add benchmark for checkblockindex (Martin Zumsande)

Pull request description:

  `CheckBlockIndex() ` are consistency checks that are currently enabled by default on regtest.

  The function is rather slow, which is annoying if you
  * attempt to run it on other networks, especially if not fully synced
  * want to generate a long chain on regtest and see block generation slow down because you forgot to disable `-checkblockindex` or don't know it existed.

  One reason why it's slow is that in order to be able to traverse the block tree depth-first from genesis, it inserts pointers to all block indices into a `std::multimap` - for which inserts and lookups become slow once there are hundred thousands of entries.
  However, typically the block index is mostly chain-like with just a few forks so a multimap isn't really needed for the most part. This PR suggests to store the block indices of the chain ending in the best header in a vector instead, and store only the rest of the indices in a multimap. This does not change the actual consistency checks that are being performed for each index, just the way the block index tree is stored and traversed.

  This adds a bit of complication to make sure each block is visited (note that there are asserts that check it), making sure that the two containers are traversed correctly, but it speeds up the function considerably:

  On master, a single invocation of `CheckBlockIndex` takes ~1.4s on mainnet for me (4.9s on testnet which has >2.4 million blocks).
  With this branch, the runtime goes down to ~0.27s (0.85s on testnet).This is a speedup by a factor ~5.

ACKs for top commit:
  achow101:
    ACK 5bc2077e8f
  furszy:
    ACK 5bc2077e8f
  ryanofsky:
    Code review ACK 5bc2077e8f. Just added suggested assert and simplification since last review

Tree-SHA512: 6b9c3e3e5069d6152b45a09040f962380d114851ff0f9ff1771cf8cad7bb4fa0ba25cd787ceaa3dfa5241fb249748e2ee6987af0ccb24b786a5301b2836f8487
2024-06-11 16:41:44 -04:00
Ava Chow
2251460f3e Merge bitcoin/bitcoin#28830: [refactor] Check CTxMemPool options in ctor
09ef322acc [[refactor]] Check CTxMemPool options in constructor (TheCharlatan)

Pull request description:

  The tests should run the same checks on the mempool options that the init code also applies. The downside to this patch is that the log line may now be printed more than once in the for loop.

  This was originally noticed here https://github.com/bitcoin/bitcoin/pull/25290#discussion_r900272797.

ACKs for top commit:
  stickies-v:
    re-ACK 09ef322acc . Fixed unreachable assert and updated docstring, and also added an exception for "-maxmempool must be at least " in the `tx_pool` fuzz test, which makes sense when looking at how the mempool options are constructed in `SetMempoolConstraints`.
  achow101:
    ACK 09ef322acc
  ryanofsky:
    Code review ACK 09ef322acc. Just fuzz test error checking fix and updated comment since last review

Tree-SHA512: eb3361411c2db70be17f912e3b14d9cb9c60fb0697a1eded952c3b7e8675b7d783780d45c52e091931d1d80fe0f0280cee98dd57a3100def13af20259d9d1b9e
2024-06-11 15:24:49 -04:00
Ryan Ofsky
804f09dfa1 kernel: Add less confusing reindex options
Drop confusing kernel options:

  BlockManagerOpts::reindex
  ChainstateLoadOptions::reindex
  ChainstateLoadOptions::reindex_chainstate

Replacing them with more straightforward options:

  ChainstateLoadOptions::wipe_block_tree_db
  ChainstateLoadOptions::wipe_chainstate_db

Having two options called "reindex" which did slightly different things
was needlessly confusing (one option wiped the block tree database, and
the other caused block files to be rescanned). Also the previous set of
options did not allow rebuilding the block database without also
rebuilding the chainstate database, when it should be possible to do
those independently.
2024-06-07 19:17:11 +02:00
TheCharlatan
09ef322acc [[refactor]] Check CTxMemPool options in constructor
This ensures that the tests run the same checks on the mempool options
that the init code also applies.
2024-05-17 23:37:25 +02:00
TheCharlatan
b47bd95920 kernel: De-globalize fReindex
fReindex is one of the last remaining globals exposed by the kernel
library, so move it into the blockstorage class to reduce the amount of
global mutable state and make the kernel library a bit less awkward to
use.
2024-05-16 11:28:46 +02:00
Ava Chow
f5fc3190fb Merge bitcoin/bitcoin#29086: refactor: Simply include CTxMemPool::Options in CTxMemPool directly rather than duplicating definition
cc67d33fda refactor: Simply include CTxMemPool::Options in CTxMemPool directly rather than duplicating definition (Luke Dashjr)

Pull request description:

  Instead of duplicating mempool options two places, just include the Options struct directly on the CTxMemPool

ACKs for top commit:
  achow101:
    ACK cc67d33fda
  kristapsk:
    cr utACK cc67d33fda
  jonatack:
    ACK cc67d33fda

Tree-SHA512: 9deb5ea6f85eeb1c7e04536cded65303b0ec459936a97e4f257aff2c50b0984a4ddbf69a4651f48455b9c80200a1fd24e9c74926874fdd9be436bbbe406251ce
2024-05-14 20:00:34 -04:00
TheCharlatan
41eba5bd71 kernel: Remove key module from kernel library
The key module's functionality is not used by the kernel library, but
currently kernel users are still required to initialize the key module's
`secp256k1_context_sign` global as part of the `kernel::Context` through
`ECC_Start`.
2024-05-09 15:56:08 +02:00
Luke Dashjr
cc67d33fda refactor: Simply include CTxMemPool::Options in CTxMemPool directly rather than duplicating definition 2024-05-06 20:34:10 +00:00
MarcoFalke
dddd40ba82 scripted-diff: Add IWYU pragma keep to bitcoin-config.h includes
-BEGIN VERIFY SCRIPT-
 perl -0777 -pi -e 's/#if defined\(HAVE_CONFIG_H\)\n#include <config\/bitcoin-config.h>.*\n#endif.*\n/#include <config\/bitcoin-config.h> \/\/ IWYU pragma: keep\n/g' $( git grep -l '#include <config/bitcoin-config.h>' )
-END VERIFY SCRIPT-
2024-05-01 08:33:04 +02:00
Martin Zumsande
5bc2077e8f validation: allow to specify frequency for -checkblockindex
This makes it similar to -checkaddrman and -checkmempool, which
also allow to run the check occasionally instead of always / never.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-04-26 13:31:28 -04:00
Fabian Jahr
8f39aaae41 refactor: Remove hooking code for urlDecode
The point of this was to be able to build bitcoin-tx and bitcoin-wallet without libevent, see #18504.

Now that we use our own implementation of urlDecode this is not needed anymore.

Co-authored-by: stickies-v <stickies-v@protonmail.com>
2024-04-24 23:23:38 +02:00
Ava Chow
5ebb406357 Merge bitcoin/bitcoin#26564: test: test_bitcoin: allow -testdatadir=<datadir>
d27e2d87b9 test: test_bitcoin: allow -testdatadir=<datadir> (Larry Ruane)

Pull request description:

  This backward-compatible change would help with code review, testing, and debugging. When `test_bitcoin` runs, it creates a working or data directory within `/tmp/test_common_Bitcoin\ Core/`, named as a long random (hex) string.

  This small patch does three things:

  - If the (new) argument `-testdatadir=<datadir>` is given, use `<datadir>/test_temp/<test-name>/datadir` as the working directory
  - When the test starts, remove `<datadir>/test_temp/<test-name>/datadir` if it exists from an earlier run (currently, it's presumed not to exist due to the long random string)
  - Don't delete the working directory at the end of the test if a custom data directory is being used

  Example usage, which will remove, create, use `/somewhere/test_temp/getarg_tests/boolarg`, and leave it afterward:
  ```
  $ test_bitcoin --run_test=getarg_tests/boolarg -- -testdatadir=/somewhere
  Running 1 test case...
  Test directory (will not be deleted): "/somewhere/test_temp/getarg_tests/boolarg/datadir"

  *** No errors detected
  $ ls -l /somewhere/test_temp/getarg_tests/boolarg/datadir
  total 8
  drwxrwxr-x 2 larry larry 4096 Feb 22 10:28 blocks
  -rw-rw-r-- 1 larry larry 1273 Feb 22 10:28 debug.log
  ```
  (A relative pathname also works.)

  This change affects only `test_bitcoin`; it could also be applied to `test_bitcoin-qt` but that's slightly more involved so I'm skipping that for now.

  The rationale for this change is that, when running the test using the debugger, it's often useful to watch `debug.log` as the test runs and inspect some of the other files (I've looked at the generated `blknnnn.dat` files for example). Currently, that requires figuring out where the test's working directory is since it changes on every test run. Tests can be run with `-printtoconsole=1` to show debug logging to the terminal, but it's nice to keep `debug.log` continuously open in an editor, for example.

  Even if not using a debugger, it's sometimes helpful to see `debug.log` and other artifacts after the test completes.

  Similar functionality is already possible with the functional tests using the `--tmpdir=` and `--nocleanup` arguments.

ACKs for top commit:
  davidgumberg:
    ACK d27e2d87b9
  tdb3:
    re-ACK for d27e2d87b9
  achow101:
    ACK d27e2d87b9
  cbergqvist:
    ACK d27e2d87b95b7982c05b4c88e463cc9626ab9f0a! (Already did some testing with `fs::remove()` to make sure it was compatible with the `util::Lock/UnlockDirectory` implementation).
  marcofleon:
    ACK d27e2d87b9. I ran all the tests with my previous open file limit and no errors were detected. Also ran some individual tests with no, relative, and absolute paths and everything looks good.
  furszy:
    ACK d27e2d8

Tree-SHA512: a8f535f34a48b6699cb440f97f5562ec643f3bfba4ea685768980b871fc8b6e1135f70fc05dbe19aa2c8bacb1ddeaff212d63473605a7422ff76332b3a6b1f68
2024-03-11 07:03:02 -04:00
Ava Chow
c07935bcf5 Merge bitcoin/bitcoin#28960: kernel: Remove dependency on CScheduler
d5228efb53 kernel: Remove dependency on CScheduler (TheCharlatan)
06069b3913 scripted-diff: Rename MainSignals to ValidationSignals (TheCharlatan)
0d6d2b650d scripted-diff: Rename SingleThreadedSchedulerClient to SerialTaskRunner (TheCharlatan)
4abde2c4e3 [refactor] Make MainSignals RAII styled (TheCharlatan)
84f5c135b8 refactor: De-globalize g_signals (TheCharlatan)
473dd4b97a [refactor] Prepare for g_signals de-globalization (TheCharlatan)
3fba3d5dee [refactor] Make signals optional in mempool and chainman (TheCharlatan)

Pull request description:

  By defining a virtual interface class for the scheduler client, users of the kernel can now define their own event consuming infrastructure, without having to spawn threads or rely on the scheduler design.

  Removing `CScheduler` also allows removing the thread and exception modules from the kernel library.

  To make the `CMainSignals` class easier to use from a kernel library perspective, remove its global instantiation and adopt RAII practices.

  Renames `CMainSignals` to `ValidationSignals`, which more accurately describes its purpose and scope.

  Also make the `ValidationSignals` in the `ChainstateManager` and CTxMemPool` optional. This could be useful in the future for using or testing these classes without having to instantiate any form of signal handling.

  ---

  This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587). It improves the kernel API and removes two modules from the kernel library.

ACKs for top commit:
  maflcko:
    re-ACK d5228efb53 🌄
  ryanofsky:
    Code review ACK d5228efb53. Just comment change since last review.
  vasild:
    ACK d5228efb53
  furszy:
    diff ACK d5228ef

Tree-SHA512: e93a5f10eb6182effb84bb981859a7ce750e466efd8171045d8d9e7fe46e4065631d9f6f533c5967c4d34c9bb7d7a67e9f4593bd4c5b30cd7b3bbad7be7b331b
2024-03-08 20:58:04 -05:00
Larry Ruane
d27e2d87b9 test: test_bitcoin: allow -testdatadir=<datadir>
Specifying this argument overrides the path location for test_bitcoin;
it becomes <datadir>/test_common_Bitcoin Core/<testname>/datadir. Also,
this directory isn't removed after the test completes. This can make it
easier for developers to study the results of a test (see the state of
the data directory after the test runs), and also (for example) have an
editor open on debug.log to monitor it across multiple test runs instead
of having to re-open a different pathname each time.

Example usage (note the "--" is needed):

test_bitcoin --run_test=getarg_tests/boolarg -- \
-testdatadir=/somewhere/mydatadir

This will create (if necessary) and use the data directory:

/somewhere/mydatadir/test_common_Bitcoin Core/getarg_tests/boolarg/datadir

Co-authored-by: furszy <mfurszy@protonmail.com>
2024-03-07 10:11:45 -07:00
TheCharlatan
d5228efb53 kernel: Remove dependency on CScheduler
By defining a virtual interface class for the scheduler client, users of
the kernel can now define their own event consuming infrastructure,
without having to spawn threads or rely on the scheduler design.

Removing CScheduler also allows removing the thread and
exception modules from the kernel library.
2024-02-16 17:12:52 +01:00
TheCharlatan
06069b3913 scripted-diff: Rename MainSignals to ValidationSignals
-BEGIN VERIFY SCRIPT-
s() { git grep -l "$1" src | xargs sed -i "s/$1/$2/g"; }

s 'CMainSignals'    'ValidationSignals'
s 'MainSignalsImpl' 'ValidationSignalsImpl'
-END VERIFY SCRIPT-
2024-02-15 14:45:51 +01:00
TheCharlatan
4abde2c4e3 [refactor] Make MainSignals RAII styled 2024-02-15 14:43:12 +01:00
TheCharlatan
84f5c135b8 refactor: De-globalize g_signals 2024-02-15 14:37:01 +01:00
TheCharlatan
3fba3d5dee [refactor] Make signals optional in mempool and chainman
This is done in preparation for the next two commits, where the
CMainSignals are de-globalized.

This avoids adding new constructor arguments to the ChainstateManager
and CTxMemPool classes over the next two commits.

This could also allow future tests that are only interested in the
internal behaviour of the classes to forgo instantiating the signals.
2024-02-15 13:28:45 +01:00
TheCharlatan
9d1dbbd4ce scripted-diff: Fix bitcoin_config_h includes
-BEGIN VERIFY SCRIPT-

regex_string='^(?!//).*(AC_APPLE_UNIVERSAL_BUILD|BOOST_PROCESS_USE_STD_FS|CHAR_EQUALS_INT8|CLIENT_VERSION_BUILD|CLIENT_VERSION_IS_RELEASE|CLIENT_VERSION_MAJOR|CLIENT_VERSION_MINOR|COPYRIGHT_HOLDERS|COPYRIGHT_HOLDERS_FINAL|COPYRIGHT_HOLDERS_SUBSTITUTION|COPYRIGHT_YEAR|ENABLE_ARM_SHANI|ENABLE_AVX2|ENABLE_EXTERNAL_SIGNER|ENABLE_SSE41|ENABLE_TRACING|ENABLE_WALLET|ENABLE_X86_SHANI|ENABLE_ZMQ|HAVE_BOOST|HAVE_BUILTIN_CLZL|HAVE_BUILTIN_CLZLL|HAVE_BYTESWAP_H|HAVE_CLMUL|HAVE_CONSENSUS_LIB|HAVE_CXX20|HAVE_DECL_BE16TOH|HAVE_DECL_BE32TOH|HAVE_DECL_BE64TOH|HAVE_DECL_BSWAP_16|HAVE_DECL_BSWAP_32|HAVE_DECL_BSWAP_64|HAVE_DECL_FORK|HAVE_DECL_FREEIFADDRS|HAVE_DECL_GETIFADDRS|HAVE_DECL_HTOBE16|HAVE_DECL_HTOBE32|HAVE_DECL_HTOBE64|HAVE_DECL_HTOLE16|HAVE_DECL_HTOLE32|HAVE_DECL_HTOLE64|HAVE_DECL_LE16TOH|HAVE_DECL_LE32TOH|HAVE_DECL_LE64TOH|HAVE_DECL_PIPE2|HAVE_DECL_SETSID|HAVE_DECL_STRERROR_R|HAVE_DEFAULT_VISIBILITY_ATTRIBUTE|HAVE_DLFCN_H|HAVE_DLLEXPORT_ATTRIBUTE|HAVE_ENDIAN_H|HAVE_EVHTTP_CONNECTION_GET_PEER_CONST_CHAR|HAVE_FDATASYNC|HAVE_GETENTROPY_RAND|HAVE_GETRANDOM|HAVE_GMTIME_R|HAVE_INTTYPES_H|HAVE_LIBADVAPI32|HAVE_LIBCOMCTL32|HAVE_LIBCOMDLG32|HAVE_LIBGDI32|HAVE_LIBIPHLPAPI|HAVE_LIBKERNEL32|HAVE_LIBOLE32|HAVE_LIBOLEAUT32|HAVE_LIBSHELL32|HAVE_LIBSHLWAPI|HAVE_LIBUSER32|HAVE_LIBUUID|HAVE_LIBWINMM|HAVE_LIBWS2_32|HAVE_MALLOC_INFO|HAVE_MALLOPT_ARENA_MAX|HAVE_MINIUPNPC_MINIUPNPC_H|HAVE_MINIUPNPC_UPNPCOMMANDS_H|HAVE_MINIUPNPC_UPNPERRORS_H|HAVE_NATPMP_H|HAVE_O_CLOEXEC|HAVE_POSIX_FALLOCATE|HAVE_PTHREAD|HAVE_PTHREAD_PRIO_INHERIT|HAVE_STDINT_H|HAVE_STDIO_H|HAVE_STDLIB_H|HAVE_STRERROR_R|HAVE_STRINGS_H|HAVE_STRING_H|HAVE_STRONG_GETAUXVAL|HAVE_SYSCTL|HAVE_SYSCTL_ARND|HAVE_SYSTEM|HAVE_SYS_ENDIAN_H|HAVE_SYS_PRCTL_H|HAVE_SYS_RESOURCES_H|HAVE_SYS_SELECT_H|HAVE_SYS_STAT_H|HAVE_SYS_SYSCTL_H|HAVE_SYS_TYPES_H|HAVE_SYS_VMMETER_H|HAVE_THREAD_LOCAL|HAVE_TIMINGSAFE_BCMP|HAVE_UNISTD_H|HAVE_VM_VM_PARAM_H|LT_OBJDIR|PACKAGE_BUGREPORT|PACKAGE_NAME|PACKAGE_STRING|PACKAGE_TARNAME|PACKAGE_URL|PACKAGE_VERSION|PTHREAD_CREATE_JOINABLE|QT_QPA_PLATFORM_ANDROID|QT_QPA_PLATFORM_COCOA|QT_QPA_PLATFORM_MINIMAL|QT_QPA_PLATFORM_WINDOWS|QT_QPA_PLATFORM_XCB|QT_STATICPLUGIN|STDC_HEADERS|STRERROR_R_CHAR_P|USE_ASM|USE_BDB|USE_DBUS|USE_NATPMP|USE_QRCODE|USE_SQLITE|USE_UPNP|_FILE_OFFSET_BITS|_LARGE_FILES)'

exclusion_files=":(exclude)src/minisketch :(exclude)src/crc32c :(exclude)src/secp256k1 :(exclude)src/crypto/sha256_arm_shani.cpp :(exclude)src/crypto/sha256_avx2.cpp :(exclude)src/crypto/sha256_sse41.cpp :(exclude)src/crypto/sha256_x86_shani.cpp"

git grep --perl-regexp --files-with-matches "$regex_string" -- '*.cpp' $exclusion_files | xargs git grep -L "bitcoin-config.h" | while read -r file; do line_number=$(awk -v my_file="$file" '/\/\/ file COPYING or https?:\/\/www.opensource.org\/licenses\/mit-license.php\./ {line = NR} /^\/\// && NR == line + 1 {while(getline && /^\/\//) line = NR} END {print line+1}' "$file"); sed -i "${line_number}i\\\\n\#if defined(HAVE_CONFIG_H)\\n#include <config/bitcoin-config.h>\\n\#endif" "$file"; done;

git grep --perl-regexp --files-with-matches "$regex_string" -- '*.h' $exclusion_files | xargs git grep -L "bitcoin-config.h" | while read -r file; do sed -i "/#define.*_H/a \\\\n\#if defined(HAVE_CONFIG_H)\\n#include <config/bitcoin-config.h>\\n\#endif" "$file"; done;

for file in $(git grep --files-with-matches 'bitcoin-config.h' -- '*.cpp' '*.h' $exclusion_files); do if ! grep -q --perl-regexp "$regex_string" $file; then sed -i '/HAVE_CONFIG_H/{N;N;N;d;}' $file; fi; done;

-END VERIFY SCRIPT-

The first command creates a regular expression for matching all bitcoin-config.h symbols in the following form: ^(?!//).*(AC_APPLE_UNIVERSAL_BUILD|BOOST_PROCESS_USE_STD_FS|...|_LARGE_FILES). It was generated with:
./autogen.sh && printf '^(?!//).*(%s)' $(awk '/^#undef/ {print $2}' src/config/bitcoin-config.h.in | paste -sd "|" -)

The second command holds a list of files and directories that should not be processed. These include subtree directories as well as some crypto files that already get their symbols through the makefile.

The third command checks for missing bitcoin-config headers in .cpp files and adds the header if it is missing.

The fourth command checks for missing bitcoin-config headers in .h files and adds the header if it is missing.

The fifth command checks for unneeded bitcoin-config headers in sources files and removes the header if it is unneeded.
2024-02-13 20:10:44 +00:00
MarcoFalke
fad0fafd5a refactor: Fix timedata includes 2024-02-01 13:52:05 +01:00
dergoegge
ff9039f6ea Remove GetAdjustedTime 2024-01-05 17:16:38 +00:00
Ryan Ofsky
6db04be102 Get rid of shutdown.cpp/shutdown.h, use SignalInterrupt directly
This change is mostly a refectoring that removes some code and gets rid of an
unnecessary layer of indirection after #27861

But it is not a pure refactoring since StartShutdown, AbortShutdown, and
WaitForShutdown functions used to abort on failure, and the replacement code
logs or returns errors instead.
2023-12-04 15:39:15 -04:00
Ryan Ofsky
feeb7b816a refactor: Remove calls to StartShutdown from KernelNotifications
Use SignalInterrupt object instead. There is a slight change in behavior here
because the previous StartShutdown code used to abort on failure and the
new code logs errors instead.
2023-12-04 15:39:15 -04:00
Ryan Ofsky
73133c36aa refactor: Add NodeContext::shutdown member
Add NodeContext::shutdown variable and start using it to replace the
kernel::Context::interrupt variable. The latter can't easily be removed right
away but will be removed later in this PR.

Moving the interrupt object from the kernel context to the node context
increases flexibility of the kernel API so it is possible to use multiple
interrupt objects, or avoid creating one if one is not needed. It will also
allow getting rid of the kernel::g_context global later in this PR, replacing
it with a private SignalInterrupt instance in init.cpp

There is no change in behavior in this commit outside of unit tests. In unit
tests there should be no visible change either, but internally now each test
has its own interrupt variable so the variable will be automatically reset
between tests.
2023-12-04 15:39:15 -04:00
Andrew Chow
498994b6f5 Merge bitcoin/bitcoin#26762: bugfix: Make CCheckQueue RAII-styled (attempt 2)
5b3ea5fa2e refactor: Move `{MAX,DEFAULT}_SCRIPTCHECK_THREADS` constants (Hennadii Stepanov)
6e17b31680 refactor: Make `CCheckQueue` non-copyable and non-movable explicitly (Hennadii Stepanov)
8111e74653 refactor: Drop unneeded declaration (Hennadii Stepanov)
9cf89f7a5b refactor: Make `CCheckQueue` constructor start worker threads (Hennadii Stepanov)
d03eaacbcf Make `CCheckQueue` destructor stop worker threads (Hennadii Stepanov)
be4ff3060b Move global `scriptcheckqueue` into `ChainstateManager` class (Hennadii Stepanov)

Pull request description:

  This PR:
  - makes `CCheckQueue` RAII-styled
  - gets rid of the global `scriptcheckqueue`
  - fixes https://github.com/bitcoin/bitcoin/issues/25448

  The previous attempt was in https://github.com/bitcoin/bitcoin/pull/18731.

ACKs for top commit:
  martinus:
    ACK 5b3ea5fa2e
  achow101:
    ACK 5b3ea5fa2e
  TheCharlatan:
    ACK 5b3ea5fa2e

Tree-SHA512: 45cca846e7ed107e3930149f0b616ddbaf2648d6cde381f815331b861b5d67ab39e154883ae174b8abb1dae485bc904318c50c51e5d6b46923d89de51c5eadb0
2023-11-30 14:28:46 -05:00
MarcoFalke
fae00fe9c2 Remove unused CDataStream 2023-11-30 11:27:54 +01:00