mirror of
https://github.com/bitcoin/bitcoin.git
synced 2026-01-20 15:19:07 +01:00
168360f4ae
Oversized allocations can cause out-of-memory errors or [heavy swapping](https://github.com/getumbrel/umbrel-os/issues/64#issuecomment-663637321), [grinding the system to a halt](https://x.com/murchandamus/status/1964432335849607224). `LogOversizedDbCache()` now emits a startup warning if the configured `-dbcache` exceeds a cap derived from system RAM, using the same parsing/clamping as cache sizing via CalculateDbCacheBytes(). This isn't meant as a recommended setting, rather a likely upper limit. Note that we're not modifying the set value, just issuing a warning. Also note that the 75% calculation is rounded for the last two numbers since we have to divide first before multiplying, otherwise we wouldn't stay inside size_t on 32-bit systems - and this was simpler than casting back and forth. We could have chosen the remaining free memory for the warning (e.g. warn if free memory is less than 1 GiB), but this is just a heuristic, we assumed that on systems with a lot of memory, other processes are also running, while memory constrained ones run only Core. If total RAM < 2 GiB, cap is `DEFAULT_DB_CACHE` (`450 MiB`), otherwise it's 75% of total RAM. The threshold is chosen to be close to values commonly used in [raspiblitz](https://github.com/raspiblitz/raspiblitz/blob/dev/home.admin/_provision.setup.sh#L98-L115) for common setups: | Total RAM | `dbcache` (MiB) | raspiblitz % | proposed cap (MiB) | |----------:|----------------:|-------------:|-------------------:| | 1 GiB | 512 | 50.0% | 450* | | 2 GiB | 1536 | 75.0% | 1536 | | 4 GiB | 2560 | 62.5% | 3072 | | 8 GiB | 4096 | 50.0% | 6144 | | 16 GiB | 4096 | 25.0% | 12288 | | 32 GiB | 4096 | 12.5% | 24576 | [Umbrel issues](https://github.com/getumbrel/umbrel-os/issues/64#issuecomment-663816367) also mention 75% being the upper limit. Starting `bitcoind` on an 8 GiB rpi4b with a dbcache of 7 GiB: > ./build/bin/bitcoind -dbcache=7000 warns now as follows: ``` 2025-09-07T17:24:29Z [warning] A 7000 MiB dbcache may be too large for a system memory of only 7800 MiB. 2025-09-07T17:24:29Z Cache configuration: 2025-09-07T17:24:29Z * Using 2.0 MiB for block index database 2025-09-07T17:24:29Z * Using 8.0 MiB for chain state database 2025-09-07T17:24:29Z * Using 6990.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space) ``` Besides the [godbolt](https://godbolt.org/z/EPsaE3xTj) reproducers for the new total memory method, we also tested the warnings manually on: - [x] Apple M4 Max, macOS 15.6.1 - [x] Intel Core i9-9900K, Ubuntu 24.04.2 LTS - [x] Raspberry Pi 4 Model B, Armbian Linux 6.12.22-current-bcm2711 - [x] Intel Xeon x64, Windows 11 Home Version 24H2, OS Build 26100.4351 Co-authored-by: stickies-v <stickies-v@protonmail.com> Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com> Co-authored-by: w0xlt <woltx@protonmail.com>
66 lines
2.5 KiB
C++
66 lines
2.5 KiB
C++
// Copyright (c) 2021-2022 The Bitcoin Core developers
|
|
// Distributed under the MIT software license, see the accompanying
|
|
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
|
|
|
#include <node/caches.h>
|
|
|
|
#include <common/args.h>
|
|
#include <common/system.h>
|
|
#include <index/txindex.h>
|
|
#include <kernel/caches.h>
|
|
#include <logging.h>
|
|
#include <node/interface_ui.h>
|
|
#include <tinyformat.h>
|
|
#include <util/byte_units.h>
|
|
|
|
#include <algorithm>
|
|
#include <string>
|
|
|
|
// Unlike for the UTXO database, for the txindex scenario the leveldb cache make
|
|
// a meaningful difference: https://github.com/bitcoin/bitcoin/pull/8273#issuecomment-229601991
|
|
//! Max memory allocated to tx index DB specific cache in bytes.
|
|
static constexpr size_t MAX_TX_INDEX_CACHE{1024_MiB};
|
|
//! Max memory allocated to all block filter index caches combined in bytes.
|
|
static constexpr size_t MAX_FILTER_INDEX_CACHE{1024_MiB};
|
|
//! Maximum dbcache size on 32-bit systems.
|
|
static constexpr size_t MAX_32BIT_DBCACHE{1024_MiB};
|
|
|
|
namespace node {
|
|
size_t CalculateDbCacheBytes(const ArgsManager& args)
|
|
{
|
|
if (auto db_cache{args.GetIntArg("-dbcache")}) {
|
|
if (*db_cache < 0) db_cache = 0;
|
|
const uint64_t db_cache_bytes{SaturatingLeftShift<uint64_t>(*db_cache, 20)};
|
|
constexpr auto max_db_cache{sizeof(void*) == 4 ? MAX_32BIT_DBCACHE : std::numeric_limits<size_t>::max()};
|
|
return std::max<size_t>(MIN_DB_CACHE, std::min<uint64_t>(db_cache_bytes, max_db_cache));
|
|
}
|
|
return DEFAULT_DB_CACHE;
|
|
}
|
|
|
|
CacheSizes CalculateCacheSizes(const ArgsManager& args, size_t n_indexes)
|
|
{
|
|
size_t total_cache{CalculateDbCacheBytes(args)};
|
|
|
|
IndexCacheSizes index_sizes;
|
|
index_sizes.tx_index = std::min(total_cache / 8, args.GetBoolArg("-txindex", DEFAULT_TXINDEX) ? MAX_TX_INDEX_CACHE : 0);
|
|
total_cache -= index_sizes.tx_index;
|
|
if (n_indexes > 0) {
|
|
size_t max_cache = std::min(total_cache / 8, MAX_FILTER_INDEX_CACHE);
|
|
index_sizes.filter_index = max_cache / n_indexes;
|
|
total_cache -= index_sizes.filter_index * n_indexes;
|
|
}
|
|
return {index_sizes, kernel::CacheSizes{total_cache}};
|
|
}
|
|
|
|
void LogOversizedDbCache(const ArgsManager& args) noexcept
|
|
{
|
|
if (const auto total_ram{GetTotalRAM()}) {
|
|
const size_t db_cache{CalculateDbCacheBytes(args)};
|
|
if (ShouldWarnOversizedDbCache(db_cache, *total_ram)) {
|
|
InitWarning(bilingual_str{tfm::format(_("A %zu MiB dbcache may be too large for a system memory of only %zu MiB."),
|
|
db_cache >> 20, *total_ram >> 20)});
|
|
}
|
|
}
|
|
}
|
|
} // namespace node
|