highperfocused/bitcoin
1
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2026-03-12 16:45:40 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge bitcoin/bitcoin#31483: kernel: Move kernel-related cache constants to kernel cache

Browse Source 
2a92702baf init: Use size_t consistently for cache sizes (TheCharlatan)
65cde3621d kernel: Move default cache constants to caches (TheCharlatan)
8826cae285 kernel: Move non-kernel db cache size constants (TheCharlatan)
e758b26b85 kernel: Move kernel-specific cache size options to kernel (TheCharlatan)
d5e2c4a409 fuzz: Add fuzz test for checked and saturating add and left shift (TheCharlatan)
c03a2795a8 util: Add integer left shift helpers (TheCharlatan)
8bd5f8a38c [refactor] init: Simplify coinsdb cache calculation (TheCharlatan)
5db7d4d3d2 doc: Correct docstring describing max block tree db cache (TheCharlatan)

Pull request description:

  Carrying non-kernel related fields in the cache sizes for the indexes is confusing for kernel library users. The cache sizes are set currently with magic numbers in bitcoin-chainstate. The comments for the cache size calculations are not completely clear. The constants for the cache sizes are also currently in `txdb.h`, which is not an ideal place for holding all cache size related constants.

  Solve these things by moving the kernel-specific cache size fields to their own struct and moving the constants to either the node or the kernel cache sizes.

  This slightly changes the way the cache is allocated if (and only if) the txindex and/or blockfilterindex is used. Since they are now given precedence over the block tree db cache, this results in a bit less cache being allocated to the block tree db, coinsdb and coins caches. The effect is negligible though, i.e. cache sizes with default dbcache reported through the logs are:

  master:
  ```
  Cache configuration:
  * Using 2.0 MiB for block index database
  * Using 56.0 MiB for transaction index database
  * Using 49.0 MiB for basic block filter index database
  * Using 8.0 MiB for chain state database
  * Using 335.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
  ```

  this PR:
  ```
  Cache configuration:
  * Using 2.0 MiB for block index database
  * Using 56.2 MiB for transaction index database
  * Using 49.2 MiB for basic block filter index database
  * Using 8.0 MiB for chain state database
  * Using 334.5 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
  ```

  ---
  This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).

ACKs for top commit:
  stickies-v:
    re-ACK 2a92702baf
  ryanofsky:
    Code review ACK 2a92702baf. Changes since last review are fixing size options to use size_t instead of int64_t again, simplifying CheckedLeftShift more, and making other minor suggested cleanups
  hodlinator:
    re-ACK 2a92702baf

Tree-SHA512: 98376eaa0660b1b8c096a5ce1f3e7c8c30e7cd6644de36856c2d3e573108cfc9473c93ebb3952b7881047b5ae6c85c5b096e6726f30f35be58b98eca07c8c785
This commit is contained in:
merge-script
2025-01-16 15:04:58 +00:00
parent 335798c496 2a92702baf
commit df8bf65745
18 changed files with 322 additions and 76 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
src/init.cpp
View File

@@ -32,6 +32,7 @@
#include <interfaces/ipc.h>
#include <interfaces/mining.h>
#include <interfaces/node.h>
#include <kernel/caches.h>
#include <kernel/context.h>
#include <key.h>
#include <logging.h>
@@ -121,7 +122,6 @@ using common::ResolveErrMsg;
using node::ApplyArgsManOptions;
using node::BlockManager;
using node::CacheSizes;
using node::CalculateCacheSizes;
using node::ChainstateLoadResult;
using node::ChainstateLoadStatus;
@@ -487,7 +487,7 @@ void SetupServerArgs(ArgsManager& argsman, bool can_listen_ipc)
argsman.AddArg("-conf=<file>", strprintf("Specify path to read-only configuration file. Relative paths will be prefixed by datadir location (only useable from command line, not configuration file) (default: %s)", BITCOIN_CONF_FILENAME), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
argsman.AddArg("-datadir=<dir>", "Specify data directory", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::OPTIONS);
argsman.AddArg("-dbbatchsize", strprintf("Maximum database write batch size in bytes (default: %u)", nDefaultDbBatchSize), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::OPTIONS);
argsman.AddArg("-dbcache=<n>", strprintf("Maximum database cache size <n> MiB (minimum %d, default: %d). Make sure you have enough RAM. In addition, unused memory allocated to the mempool is shared with this cache (see -maxmempool).", nMinDbCache, nDefaultDbCache), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
argsman.AddArg("-dbcache=<n>", strprintf("Maximum database cache size <n> MiB (minimum %d, default: %d). Make sure you have enough RAM. In addition, unused memory allocated to the mempool is shared with this cache (see -maxmempool).", MIN_DB_CACHE >> 20, DEFAULT_DB_CACHE >> 20), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
argsman.AddArg("-includeconf=<file>", "Specify additional configuration file, relative to the -datadir path (only useable from configuration file, not command line)", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
argsman.AddArg("-allowignoredconf", strprintf("For backwards compatibility, treat an unused %s file in the datadir as a warning, not an error.", BITCOIN_CONF_FILENAME), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
argsman.AddArg("-loadblock=<file>", "Imports blocks from external file on startup", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
@@ -1177,7 +1177,7 @@ static ChainstateLoadResult InitAndLoadChainstate(
NodeContext& node,
bool do_reindex,
const bool do_reindex_chainstate,
CacheSizes& cache_sizes,
const kernel::CacheSizes& cache_sizes,
const ArgsManager& args)
{
const CChainParams& chainparams = Params();
@@ -1602,18 +1602,18 @@ bool AppInitMain(NodeContext& node, interfaces::BlockAndHeaderTipInfo* tip_info)
ReadNotificationArgs(args, kernel_notifications);
// cache size calculations
CacheSizes cache_sizes = CalculateCacheSizes(args, g_enabled_filter_types.size());
const auto [index_cache_sizes, kernel_cache_sizes] = CalculateCacheSizes(args, g_enabled_filter_types.size());
LogPrintf("Cache configuration:\n");
LogPrintf("* Using %.1f MiB for block index database\n", cache_sizes.block_tree_db * (1.0 / 1024 / 1024));
LogInfo("Cache configuration:");
LogInfo("* Using %.1f MiB for block index database", kernel_cache_sizes.block_tree_db * (1.0 / 1024 / 1024));
if (args.GetBoolArg("-txindex", DEFAULT_TXINDEX)) {
LogPrintf("* Using %.1f MiB for transaction index database\n", cache_sizes.tx_index * (1.0 / 1024 / 1024));
LogInfo("* Using %.1f MiB for transaction index database", index_cache_sizes.tx_index * (1.0 / 1024 / 1024));
}
for (BlockFilterType filter_type : g_enabled_filter_types) {
LogPrintf("* Using %.1f MiB for %s block filter index database\n",
cache_sizes.filter_index * (1.0 / 1024 / 1024), BlockFilterTypeName(filter_type));
LogInfo("* Using %.1f MiB for %s block filter index database",
index_cache_sizes.filter_index * (1.0 / 1024 / 1024), BlockFilterTypeName(filter_type));
}
LogPrintf("* Using %.1f MiB for chain state database\n", cache_sizes.coins_db * (1.0 / 1024 / 1024));
LogInfo("* Using %.1f MiB for chain state database", kernel_cache_sizes.coins_db * (1.0 / 1024 / 1024));
assert(!node.mempool);
assert(!node.chainman);
@@ -1626,7 +1626,7 @@ bool AppInitMain(NodeContext& node, interfaces::BlockAndHeaderTipInfo* tip_info)
node,
do_reindex,
do_reindex_chainstate,
cache_sizes,
kernel_cache_sizes,
args);
if (status == ChainstateLoadStatus::FAILURE && !do_reindex && !ShutdownRequested(node)) {
// suggest a reindex
@@ -1645,7 +1645,7 @@ bool AppInitMain(NodeContext& node, interfaces::BlockAndHeaderTipInfo* tip_info)
node,
do_reindex,
do_reindex_chainstate,
cache_sizes,
kernel_cache_sizes,
args);
}
if (status != ChainstateLoadStatus::SUCCESS && status != ChainstateLoadStatus::INTERRUPTED) {
@@ -1671,12 +1671,12 @@ bool AppInitMain(NodeContext& node, interfaces::BlockAndHeaderTipInfo* tip_info)
// ********************************************************* Step 8: start indexers
if (args.GetBoolArg("-txindex", DEFAULT_TXINDEX)) {
g_txindex = std::make_unique<TxIndex>(interfaces::MakeChain(node), cache_sizes.tx_index, false, do_reindex);
g_txindex = std::make_unique<TxIndex>(interfaces::MakeChain(node), index_cache_sizes.tx_index, false, do_reindex);
node.indexes.emplace_back(g_txindex.get());
}
for (const auto& filter_type : g_enabled_filter_types) {
InitBlockFilterIndex([&]{ return interfaces::MakeChain(node); }, filter_type, cache_sizes.filter_index, false, do_reindex);
InitBlockFilterIndex([&]{ return interfaces::MakeChain(node); }, filter_type, index_cache_sizes.filter_index, false, do_reindex);
node.indexes.emplace_back(GetBlockFilterIndex(filter_type));
}