Ava Chow d155fc12a0 Merge bitcoin/bitcoin#32414: validation: periodically flush dbcache during reindex-chainstate ...

c1e554d3e5 refactor: consolidate 3 separate locks into one block (Andrew Toth)
41479ed1d2 test: add test for periodic flush inside ActivateBestChain (Andrew Toth)
84820561dc validation: periodically flush dbcache during reindex-chainstate (Andrew Toth)

Pull request description:

  After #30611 we periodically do a non-erasing flush of the dbcache to disk roughly every hour during IBD.
  The intention was to also do this periodic flush during reindex-chainstate, so we would not risk losing progress during a system failure when reindexing with a high dbcache value.

  It was discovered that reindex-chainstate does not perform a PERIODIC flush until it has already reached the tip. Since reindexing to tip usually happens within 24 hours, this behaviour was unnoticed with the previous periodic flush interval. Note that reindex-chainstate still does IF_NEEDED flushes during `ConnectBlock`, so this also would not be noticed when running with a lower dbcache value.

  This patch moves the PERIODIC flush from after the outer loop in `ActivateBestChain` to inside the outer loop after we release `cs_main`. This will periodically flush during IBD, reindex-chainstate, and steady state.

ACKs for top commit:
  l0rinc:
    ACK c1e554d3e5
  achow101:
    ACK c1e554d3e5
  sipa:
    utACK c1e554d3e5

Tree-SHA512: c447ad03e16c9978b8ed2c285b38e1b4c56e7778ab93b6f64435116f47b8931017f5f56ab53eb61656693146aaced776f666af573a41ab28e8f2b6d8657fa756

2025-12-11 11:56:01 -08:00

..

bench

optimization: migrate SipHashUint256 to PresaltedSipHasher

2025-12-09 17:16:15 +01:00

common

scripted-diff: LogPrintLevel(*,BCLog::Level::*,*) -> LogError()/LogWarning()

2025-12-09 10:44:33 +01:00

compat

net: remove unnecessary casts in socket operations

2025-09-16 06:26:01 -04:00

consensus

Merge bitcoin/bitcoin#33805: merkle: migrate path arg to reference and drop unused args

2025-12-10 15:28:50 -08:00

crc32c

Update crc32c subtree to latest upstream master

2025-08-07 09:27:25 +01:00

crypto

refactor: extract shared SipHash state into SipHashState

2025-12-09 17:17:47 +01:00

index

Merge bitcoin/bitcoin#32882: index: remove unnecessary locator cleaning in BaseIndex::Init()

2025-12-03 13:47:17 +00:00

init

init: add exe name to bitcoind, bitcoin-node -version output to be able to distinguish these in tests

2025-09-17 04:57:09 -04:00

interfaces

Improve comments for getTransactionAncestry to reference cluster counts instead of descendants

2025-12-01 10:53:22 -05:00

ipc

refactor: Enable clang-tidy bugprone-unused-return-value

2025-12-06 13:06:28 +01:00

kernel

Add util::Expected (std::expected)

2025-12-06 13:06:21 +01:00

leveldb

Update leveldb subtree to latest master

2025-10-16 13:49:49 +01:00

logging

scripted-diff: LogPrintf -> LogInfo

2025-12-04 19:52:49 +01:00

minisketch

Update minisketch subtree to latest upstream

2025-10-13 12:33:23 +01:00

node

scripted-diff: LogPrintLevel(*,BCLog::Level::*,*) -> LogError()/LogWarning()

2025-12-09 10:44:33 +01:00

policy

Merge bitcoin/bitcoin#33960: log: Use more severe log level (warn/err) where appropriate

2025-12-02 13:35:16 +00:00

primitives

refactor: Remove all operator!= definitions

2025-11-03 14:39:48 +01:00

qt

Revert "gui, qt: brintToFront workaround for Wayland"

2025-11-21 12:38:03 +00:00

rpc

Add util::Expected (std::expected)

2025-12-06 13:06:21 +01:00

script

Merge bitcoin/bitcoin#33961: script: Add a separate ScriptError for empty pubkeys encountered in Tapscript

2025-12-09 08:01:49 -08:00

secp256k1

Update secp256k1 subtree to latest master

2025-10-15 09:49:11 +01:00

support

refactor: Remove all operator!= definitions

2025-11-03 14:39:48 +01:00

test

Merge bitcoin/bitcoin#32414: validation: periodically flush dbcache during reindex-chainstate

2025-12-11 11:56:01 -08:00

univalue

scripted-diff: change whitespace to spaces in univalue

2025-11-11 11:12:50 +00:00

util

optimization: migrate SipHashUint256 to PresaltedSipHasher

2025-12-09 17:16:15 +01:00

wallet

Merge bitcoin/bitcoin#34006: Add util::Expected (std::expected)

2025-12-08 20:11:51 -05:00

zmq

ci: Update Clang in "tidy" job

2025-10-28 15:33:57 +00:00

.clang-format

Merge bitcoin/bitcoin#33917: clang-format: Set Bitcoin Core IncludeCategories

2025-11-25 12:24:26 +00:00

.clang-tidy

refactor: Enable clang-tidy bugprone-unused-return-value

2025-12-06 13:06:28 +01:00

addrdb.cpp

log: Properly log warnings with warn loglevel in addrdb

2025-07-10 11:24:40 +02:00

addrdb.h

…

addresstype.cpp

policy: Add OP_1 <0x4e73> as a standard output type

2024-07-30 14:06:58 -04:00

addresstype.h

refactor: deduplicate anchor witness program bytes (0x4e,0x73)

2025-03-23 21:58:39 +01:00

addrman_impl.h

Merge bitcoin/bitcoin#30568: addrman: change internal id counting to int64_t

2024-09-20 12:55:22 -04:00

addrman.cpp

log: Use LogError for fatal errors

2025-11-27 14:33:57 +01:00

addrman.h

addrman: cap the max_pct to not exceed the maximum number of addresses

2024-11-11 12:47:53 -03:00

arith_uint256.cpp

Add a fuzz test for Num3072 multiplication and inversion

2025-01-09 10:11:46 -05:00

arith_uint256.h

refactor(headerssync): Extract test constants ahead of breakup into functions

2025-09-12 22:28:41 +02:00

attributes.h

…

banman.cpp

log: [refactor] Use info level for init logs

2025-07-25 09:50:50 +02:00

banman.h

doc: remove For ... comments

2025-05-19 16:40:33 +01:00

base58.cpp

clang-tidy: Apply modernize-deprecated-headers

2025-06-03 15:13:54 +02:00

base58.h

scripted-diff: Bump copyright headers after std::span changes

2025-03-12 19:46:54 +01:00

bech32.cpp

scripted-diff: Bump copyright headers after include changes

2025-06-03 15:13:57 +02:00

bech32.h

scripted-diff: Bump copyright headers after include changes

2025-06-03 15:13:57 +02:00

bip324.cpp

refactor: Sort includes of touched source files

2025-06-03 19:56:55 +02:00

bip324.h

scripted-diff: Bump copyright headers after std::span changes

2025-03-12 19:46:54 +01:00

bitcoin-chainstate.cpp

kernel: Add pure kernel bitcoin-chainstate

2025-11-04 08:32:13 +01:00

bitcoin-cli-res.rc

doc, windows: CompanyName "Bitcoin" => "Bitcoin Core project"

2025-06-10 15:14:24 +02:00

bitcoin-cli.cpp

refactor: Enable clang-tidy bugprone-unused-return-value

2025-12-06 13:06:28 +01:00

bitcoin-res.rc

doc, windows: CompanyName "Bitcoin" => "Bitcoin Core project"

2025-06-10 15:14:24 +02:00

bitcoin-tx-res.rc

doc, windows: CompanyName "Bitcoin" => "Bitcoin Core project"

2025-06-10 15:14:24 +02:00

bitcoin-tx.cpp

bitcoin-tx: Reject + sign in MutateTxDel*

2025-05-17 09:36:25 +02:00

bitcoin-util-res.rc

doc, windows: CompanyName "Bitcoin" => "Bitcoin Core project"

2025-06-10 15:14:24 +02:00

bitcoin-util.cpp

refactor: Use TranslateFn type consistently

2025-01-15 12:15:40 +01:00

bitcoin-wallet-res.rc

doc, windows: CompanyName "Bitcoin" => "Bitcoin Core project"

2025-06-10 15:14:24 +02:00

bitcoin-wallet.cpp

Remove no longer necessary WinCmdLineArgs class

2025-10-06 19:41:35 +01:00

bitcoin.cpp

Merge bitcoin/bitcoin#32821: rpc: Handle -named argument parsing where '=' character is used

2025-11-18 14:06:54 -08:00

bitcoind-res.rc

doc, windows: CompanyName "Bitcoin" => "Bitcoin Core project"

2025-06-10 15:14:24 +02:00

bitcoind.cpp

Remove no longer necessary WinCmdLineArgs class

2025-10-06 19:41:35 +01:00

blockencodings.cpp

optimization: cache PresaltedSipHasher in CBlockHeaderAndShortTxIDs

2025-12-09 17:16:17 +01:00

blockencodings.h

optimization: cache PresaltedSipHasher in CBlockHeaderAndShortTxIDs

2025-12-09 17:16:17 +01:00

blockfilter.cpp

refactor: increase string_view usage

2025-10-02 12:53:55 +01:00

blockfilter.h

refactor: increase string_view usage

2025-10-02 12:53:55 +01:00

chain.cpp

move-only: Move CBlockFileInfo to kernel namespace

2025-10-28 16:08:44 +01:00

chain.h

move-only: Move CBlockFileInfo to kernel namespace

2025-10-28 16:08:44 +01:00

chainparams.cpp

log: [refactor] Use info level for init logs

2025-07-25 09:50:50 +02:00

chainparams.h

…

chainparamsbase.cpp

scripted-diff: Bump copyright headers after include changes

2025-06-03 15:13:57 +02:00

chainparamsbase.h

net, init: derive default onion port if a user specified a -port

2024-11-14 13:41:02 -05:00

chainparamsseeds.h

contrib: update fixed seeds

2025-09-03 11:23:30 +01:00

checkqueue.h

validation: use a lock for CCheckQueueControl

2025-05-19 22:06:48 +00:00

clientversion.cpp

refactor: Avoid -W*-whitespace in git archive

2025-11-13 09:33:36 +01:00

clientversion.h

windows: Use predefined RC_INVOKED macro instead of custom one

2025-05-29 11:18:36 +01:00

cluster_linearize.h

depgraph: add memory usage control (feature)

2025-10-11 17:25:09 -04:00

CMakeLists.txt

Merge bitcoin/bitcoin#33774: cmake: Move IPC tests to ipc/test

2025-12-04 13:51:38 +00:00

coins.cpp

Merge bitcoin/bitcoin#33602: [IBD] coins: reduce lookups in dbcache layer propagation

2025-12-10 15:02:25 -08:00

coins.h

validation: don't reallocate cache for short-lived CCoinsViewCache

2025-11-02 21:40:16 +01:00

compressor.cpp

…

compressor.h

scripted-diff: Bump copyright headers after std::span changes

2025-03-12 19:46:54 +01:00

core_io.h

rpc: add "ischange: true" in wallet gettransaction decoded tx output

2025-10-29 12:09:19 -04:00

core_memusage.h

…

core_read.cpp

refactor: starts/ends_with changes for clang-tidy 20

2025-04-22 13:16:54 +01:00

core_write.cpp

Merge bitcoin/bitcoin#32517: rpc: add "ischange: true" to decoded tx outputs in wallet gettransaction response

2025-11-10 08:58:34 -08:00

cuckoocache.h

doc: remove For ... comments

2025-05-19 16:40:33 +01:00

dbwrapper.cpp

log: Use LogError for fatal errors

2025-11-27 14:33:57 +01:00

dbwrapper.h

refactor: inline constant return value of CDBWrapper::Erase and BlockTreeDB::WriteReindexing

2025-08-13 15:47:48 -07:00

deploymentinfo.cpp

scripted-diff: rename gbt_force and gbt_force_name

2025-04-30 14:51:21 +02:00

deploymentinfo.h

Move mapFlagNames and FormatScriptFlags logic to script/interpreter.h

2025-08-14 10:17:30 +10:00

deploymentstatus.cpp

scripted-diff: modernize outdated trait patterns - types

2025-02-21 10:41:27 +01:00

deploymentstatus.h

versionbits: Simplify VersionBitsCache API

2025-01-21 00:38:03 +10:00

dummywallet.cpp

log: [refactor] Use info level for init logs

2025-07-25 09:50:50 +02:00

external_signer.cpp

refactor: Remove Span operator==, Use std::ranges::equal

2024-08-13 07:44:31 +02:00

external_signer.h

…

flatfile.cpp

log: Use LogError for fatal errors

2025-11-27 14:33:57 +01:00

flatfile.h

Merge bitcoin/bitcoin#33771: refactor: C++20 operators

2025-12-08 16:46:03 +00:00

hash.cpp

scripted-diff: Bump copyright headers after std::span changes

2025-03-12 19:46:54 +01:00

hash.h

scripted-diff: Bump copyright headers after std::span changes

2025-03-12 19:46:54 +01:00

headerssync.cpp

headerssync: Make HeadersSyncState more flexible and move constants

2025-09-12 22:28:41 +02:00

headerssync.h

headerssync: Make HeadersSyncState more flexible and move constants

2025-09-12 22:28:41 +02:00

httprpc.cpp

log: Use LogWarning for non-critical logs

2025-11-27 14:33:59 +01:00

httprpc.h

…

httpserver.cpp

refactor: Enable clang-tidy bugprone-unused-return-value

2025-12-06 13:06:28 +01:00

httpserver.h

rpc: increase the defaults for -rpcthreads and -rpcworkqueue

2024-11-04 17:08:21 +01:00

i2p.cpp

scripted-diff: LogPrintLevel(*,BCLog::Level::*,*) -> LogError()/LogWarning()

2025-12-09 10:44:33 +01:00

i2p.h

fuzz: make it possible to mock (fuzz) CThreadInterrupt

2025-06-09 14:17:33 +02:00

indirectmap.h

…

init.cpp

Merge bitcoin/bitcoin#34031: net: Remove "tor" as a network specification

2025-12-10 11:51:01 +00:00

init.h

refactor: introduce a more general LockDirectories for init

2025-01-16 21:06:21 +00:00

key_io.cpp

scripted-diff: Bump copyright headers after include changes

2025-06-03 15:13:57 +02:00

key_io.h

…

key.cpp

Merge bitcoin/bitcoin#29675: wallet: Be able to receive and spend inputs involving MuSig2 aggregate keys

2025-10-14 16:25:52 -04:00

key.h

sign: Add CreateMuSig2PartialSig

2025-09-30 11:06:43 -07:00

logging.cpp

scripted-diff: LogPrintf -> LogInfo

2025-12-04 19:52:49 +01:00

logging.h

refactor: Remove unused LogPrintf

2025-12-04 19:53:06 +01:00

mapport.cpp

scripted-diff: LogPrintLevel(*,BCLog::Level::*,*) -> LogError()/LogWarning()

2025-12-09 10:44:33 +01:00

mapport.h

mapport: turn -natpmp to on by default

2025-07-17 12:43:04 -04:00

memusage.h

memusage: Add DynamicUsage for std::string

2024-11-04 18:46:40 +01:00

merkleblock.cpp

refactor: Convert RPCs and merkleblock from uint256 to Txid

2025-08-11 15:53:34 +01:00

merkleblock.h

refactor: Move transaction_identifier.h to primitives

2025-08-11 16:47:51 +01:00

musig.cpp

sign: Add CreateMuSig2AggregateSig

2025-09-30 11:06:43 -07:00

musig.h

Merge bitcoin/bitcoin#29675: wallet: Be able to receive and spend inputs involving MuSig2 aggregate keys

2025-10-14 16:25:52 -04:00

net_permissions.cpp

util: move fees.h and error.h to common/messages.h

2024-05-16 10:16:08 -05:00

net_permissions.h

scripted-diff: modernize outdated trait patterns - types

2025-02-21 10:41:27 +01:00

net_processing.cpp

Merge bitcoin/bitcoin#34008: log: don't rate-limit "new peer" with -debug=net

2025-12-09 11:16:12 -08:00

net_processing.h

Add util::Expected (std::expected)

2025-12-06 13:06:21 +01:00

net_types.cpp

scripted-diff: LogPrintf -> LogInfo

2025-12-04 19:52:49 +01:00

net_types.h

tidy: modernize-use-equals-default

2024-07-08 11:12:01 +02:00

net.cpp

scripted-diff: LogPrintLevel(*,BCLog::Level::*,*) -> LogError()/LogWarning()

2025-12-09 10:44:33 +01:00

net.h

net: fix use-after-free with v2->v1 reconnection logic

2025-11-26 15:51:51 -05:00

netaddress.cpp

refactor: increase string_view usage

2025-10-02 12:53:55 +01:00

netaddress.h

refactor: Remove all operator!= definitions

2025-11-03 14:39:48 +01:00

netbase.cpp

Merge bitcoin/bitcoin#34033: scripted-diff: Unify error and warning log formatting

2025-12-11 09:00:18 -08:00

netbase.h

test: Add unit test coverage for Init and Shutdown code

2025-08-04 13:38:26 -04:00

netgroup.cpp

scripted-diff: LogPrintf -> LogInfo

2025-12-04 19:52:49 +01:00

netgroup.h

…

netmessagemaker.h

…

noui.cpp

log: [refactor] Use info level for init logs

2025-07-25 09:50:50 +02:00

noui.h

log: [refactor] Use info level for init logs

2025-07-25 09:50:50 +02:00

outputtype.cpp

refactor: increase string_view usage

2025-10-02 12:53:55 +01:00

outputtype.h

refactor: increase string_view usage

2025-10-02 12:53:55 +01:00

pow.cpp

fuzz: enable running fuzz test cases in Debug mode

2025-04-22 17:11:24 +10:00

pow.h

scripted-diff: Bump copyright headers after include changes

2025-06-03 15:13:57 +02:00

prevector.h

refactor: Prefer <=> over multiple relational operators

2025-11-03 14:39:50 +01:00

protocol.cpp

scripted-diff: Replace GenTxidVariant with GenTxid

2025-07-08 20:00:51 +01:00

protocol.h

scripted-diff: Replace GenTxidVariant with GenTxid

2025-07-08 20:00:51 +01:00

psbt.cpp

psbt: MuSig2 data in Fill/FromSignatureData

2025-09-30 11:06:43 -07:00

psbt.h

Merge bitcoin/bitcoin#32419: psbt: clarify PSBT, PSBTInput, PSBTOutput unserialization flows

2025-11-17 14:08:20 -08:00

pubkey.cpp

pubkey: Return tweaks from BIP32 derivation

2025-09-30 11:06:09 -07:00

pubkey.h

refactor: Remove all operator!= definitions

2025-11-03 14:39:48 +01:00

random.cpp

log: [refactor] Use info level for init logs

2025-07-25 09:50:50 +02:00

random.h

random: add fixed-size std::array generation

2025-07-16 13:19:18 -07:00

randomenv.cpp

clang-tidy: Remove no longer needed NOLINT

2025-11-04 17:38:45 +00:00

randomenv.h

…

rest.cpp

refactor: Use STR_INTERNAL_BUG macro where possible

2025-11-06 14:19:25 +01:00

rest.h

…

scheduler.cpp

thread-safety: fix annotations with REVERSE_LOCK

2025-06-16 18:09:14 +00:00

scheduler.h

…

semaphore_grant.h

threading: semaphore: move CountingSemaphoreGrant to its own header

2025-05-10 03:31:30 +00:00

serialize.h

refactor: Return uint64_t from GetSerializeSize

2025-10-30 17:51:53 +01:00

signet.cpp

script/interpreter: introduce script_verify_flags typename

2025-08-14 10:17:32 +10:00

signet.h

…

span.h

scripted-diff: Bump copyright headers after std::span changes

2025-03-12 19:46:54 +01:00

streams.cpp

refactor: Add AutoFile::size

2025-11-14 01:17:38 +02:00

streams.h

refactor: Add AutoFile::size

2025-11-14 01:17:38 +02:00

sync.cpp

log: Use LogError for fatal errors

2025-11-27 14:33:57 +01:00

sync.h

threading: remove obsolete critsect macros

2025-08-22 14:25:39 +00:00

threadsafety.h

thread-safety: modernize thread safety macros

2025-05-08 20:13:46 +00:00

tinyformat.h

refactor: Introduce struct to hold a runtime format string

2025-01-15 12:16:08 +01:00

torcontrol.cpp

scripted-diff: LogPrintf -> LogInfo

2025-12-04 19:52:49 +01:00

torcontrol.h

config: allow setting -proxy per network

2025-05-13 12:09:40 +02:00

txdb.cpp

scripted-diff: LogPrintLevel(*,BCLog::Level::*,*) -> LogError()/LogWarning()

2025-12-09 10:44:33 +01:00

txdb.h

refactor: Extract default batch size into kernel

2025-08-28 10:09:32 -07:00

txgraph.cpp

txgraph: drop move assignment operator

2025-11-25 07:36:50 -05:00

txgraph.h

txgraph: drop move assignment operator

2025-11-25 07:36:50 -05:00

txmempool.cpp

scripted-diff: LogPrintf -> LogInfo

2025-12-04 19:52:49 +01:00

txmempool.h

Simplify removeRecursive

2025-11-30 10:57:48 -05:00

txrequest.cpp

refactor: miscellaneous GenTxid followups

2025-07-23 15:01:18 +01:00

txrequest.h

refactor: miscellaneous GenTxid followups

2025-07-23 15:01:18 +01:00

uint256.cpp

refactor: Remove SetHexDeprecated

2025-04-09 15:59:59 +01:00

uint256.h

refactor: Remove all operator!= definitions

2025-11-03 14:39:48 +01:00

undo.h

…

validation.cpp

Merge bitcoin/bitcoin#32414: validation: periodically flush dbcache during reindex-chainstate

2025-12-11 11:56:01 -08:00

validation.h

doc: clarify UpdateUncommittedBlockStructures

2025-10-31 11:47:28 +01:00

validationinterface.cpp

kernel: improve BlockChecked ownership semantics

2025-08-01 15:12:37 +02:00

validationinterface.h

kernel: improve BlockChecked ownership semantics

2025-08-01 15:12:37 +02:00

versionbits_impl.h

tests: refactor versionbits fuzz test

2025-01-21 00:38:03 +10:00

versionbits.cpp

scripted-diff: rename gbt_force and gbt_force_name

2025-04-30 14:51:21 +02:00

versionbits.h

scripted-diff: rename gbt_force and gbt_force_name

2025-04-30 14:51:21 +02:00

walletinitinterface.h

tidy: modernize-use-equals-default

2024-07-08 11:12:01 +02:00