mirror of https://github.com/bitcoin/bitcoin.git synced 2026-04-13 17:18:00 +02:00

Files

merge-script 1c2f164d34 Merge bitcoin/bitcoin#34253 : validation: cache tip recency for lock-free IsInitialBlockDownload()

557b41a38c validation: make `IsInitialBlockDownload()` lock-free (Lőrinc)
b9c0ab3b75 chain: add `CChain::IsTipRecent` helper (Lőrinc)
8d531c6210 validation: invert `m_cached_finished_ibd` to `m_cached_is_ibd` (Lőrinc)
8be54e3b19 test: cover IBD exit conditions (Lőrinc)

Pull request description:

  This PR is a follow-up to the stale #32885.

  ### Problem
  `ChainstateManager::IsInitialBlockDownload()` currently acquires `cs_main` internally, even though most existing call sites already hold the lock. This becomes relevant for proposals like #34054, which would call `IsInitialBlockDownload()` from the scheduler thread without holding `cs_main`, potentially introducing lock contention.

  ### Fix
  Make `ChainstateManager::IsInitialBlockDownload()` lock-free by caching its result in a single atomic `m_cached_is_ibd` (true while in IBD, latched to false on exit).
  Move the IBD exit checks out of `IsInitialBlockDownload()` (reader-side) into a new `ChainstateManager::UpdateIBDStatus()` (writer-side, called under cs_main).

  Call UpdateIBDStatus() at strategic points where IBD exit conditions may change, after active chain tip updates in `ConnectTip()`, `DisconnectTip()`, and `LoadChainTip()`, and after `ImportBlocks()` returns.

  With this, `IsInitialBlockDownload()` becomes a lock-free atomic read, avoiding internal `cs_main` acquisition on hot paths.

  ### Testing and Benchmarks
  This isn't strictly an optimization (though some usecases might benefit from it), so rather as a sanity check I ran a reindex-chainstate and an `AssumeUTXO` load (without background validation).

  <details>
  <summary>assumeutxo load | 910000 blocks | dbcache 4500 | i9-ssd | x86_64 | Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz | 16 cores | 62Gi RAM | xfs | SSD</summary>

  ```
  COMMITS="595504a43209bead162da54a204df7d140a25f0e 63e822b637f67242e3689adedc0155b34100e651"; \
  CC=gcc; CXX=g++; \
  BASE_DIR="/mnt/my_storage"; DATA_DIR="$BASE_DIR/ShallowBitcoinData"; LOG_DIR="$BASE_DIR/logs"; UTXO_SNAPSHOT_PATH="$BASE_DIR/utxo-910000.dat"; \
  (echo ""; for c in $COMMITS; do git fetch -q origin $c && git log -1 --pretty='%h %s' $c || exit 1; done; echo "") && \
  for DBCACHE in 4500; do \
    (echo "assumeutxo load | 910000 blocks | dbcache ${DBCACHE} | $(hostname) | $(uname -m) | $(lscpu | grep 'Model name' | head -1 | cut -d: -f2 | xargs) | $(nproc) cores | $(free -h | awk '/^Mem:/{print $2}') RAM | $(df -T $BASE_DIR | awk 'NR==2{print $2}') | $(lsblk -no ROTA $(df --output=source $BASE_DIR | tail -1) | grep -q 0 && echo SSD || echo HDD)";) &&\
    hyperfine \
    --sort command \
    --runs 3 \
    --export-json "$BASE_DIR/assumeutxo-$(sed -E 's/(\w{8})\w+ ?/\1-/g;s/-$//'<<<"$COMMITS")-$DBCACHE-$CC-$(date +%s).json" \
    --parameter-list COMMIT ${COMMITS// /,} \
    --prepare "killall -9 bitcoind 2>/dev/null; rm -rf $DATA_DIR/blocks $DATA_DIR/chainstate $DATA_DIR/chainstate_snapshot $DATA_DIR/debug.log; git clean -fxd; git reset --hard {COMMIT} && \
               cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo && ninja -C build bitcoind bitcoin-cli -j2 && \
               ./build/bin/bitcoind -datadir=$DATA_DIR -stopatheight=1 -printtoconsole=0; sleep 20 && \
               ./build/bin/bitcoind -datadir=$DATA_DIR -daemon -blocksonly -connect=0 -dbcache=$DBCACHE -printtoconsole=0; sleep 20" \
     --conclude "build/bin/bitcoin-cli -datadir=$DATA_DIR stop || true; killall bitcoind || true; sleep 10; \
                 echo '{COMMIT} | dbcache=$DBCACHE | chainstate: $(find $DATA_DIR/chainstate_snapshot -type f 2>/dev/null | wc -l) files, $(du -sb $DATA_DIR/chainstate_snapshot 2>/dev/null | cut -f1) bytes' >> $DATA_DIR/debug.log; \
                 cp $DATA_DIR/debug.log $LOG_DIR/debug-assumeutxo-{COMMIT}-dbcache-$DBCACHE-$(date +%s).log" \
      "COMPILER=$CC DBCACHE=$DBCACHE ./build/bin/bitcoin-cli -datadir=$DATA_DIR -rpcclienttimeout=0 loadtxoutset $UTXO_SNAPSHOT_PATH"; \
  done

  595504a432 Merge bitcoin/bitcoin#34236: Add sedited to trusted-keys
  63e822b637 validation: make `IsInitialBlockDownload()` lock-free

  assumeutxo load | 910000 blocks | dbcache 4500 | i9-ssd | x86_64 | Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz | 16 cores | 62Gi RAM | xfs | SSD

  Benchmark 1: COMPILER=gcc DBCACHE=4500 ./build/bin/bitcoin-cli -datadir=/mnt/my_storage/ShallowBitcoinData -rpcclienttimeout=0 loadtxoutset /mnt/my_storage/utxo-910000.dat (COMMIT = 595504a432)
    Time (mean ± σ):     418.452 s ±  0.461 s    [User: 0.001 s, System: 0.001 s]
    Range (min … max):   418.070 s … 418.964 s    3 runs

  Benchmark 2: COMPILER=gcc DBCACHE=4500 ./build/bin/bitcoin-cli -datadir=/mnt/my_storage/ShallowBitcoinData -rpcclienttimeout=0 loadtxoutset /mnt/my_storage/utxo-910000.dat (COMMIT = 63e822b637f67242e3689adedc0155b34100e651)
    Time (mean ± σ):     415.994 s ±  0.294 s    [User: 0.001 s, System: 0.001 s]
    Range (min … max):   415.788 s … 416.330 s    3 runs

  Relative speed comparison
          1.01 ±  0.00  COMPILER=gcc DBCACHE=4500 ./build/bin/bitcoin-cli -datadir=/mnt/my_storage/ShallowBitcoinData -rpcclienttimeout=0 loadtxoutset /mnt/my_storage/utxo-910000.dat (COMMIT = 595504a432)
          1.00          COMPILER=gcc DBCACHE=4500 ./build/bin/bitcoin-cli -datadir=/mnt/my_storage/ShallowBitcoinData -rpcclienttimeout=0 loadtxoutset /mnt/my_storage/utxo-910000.dat (COMMIT = 63e822b637f67242e3689adedc0155b34100e651)
  ```

  </details>

  <details>
  <summary>2026-01-12 | reindex-chainstate | 931139 blocks | dbcache 4500 | i9-ssd | x86_64 | Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz | 16 cores | 62Gi RAM | SSD</summary>

  ```
  for DBCACHE in 4500; do \
    COMMITS="595504a43209bead162da54a204df7d140a25f0e 63e822b637f67242e3689adedc0155b34100e651"; \
    STOP=931139; CC=gcc; CXX=g++; \
    BASE_DIR="/mnt/my_storage"; DATA_DIR="$BASE_DIR/BitcoinData"; LOG_DIR="$BASE_DIR/logs"; \
    (echo ""; for c in $COMMITS; do git fetch -q origin $c && git log -1 --pretty='%h %s' $c || exit 1; done) && \
    (echo "" && echo "$(date -I) | reindex-chainstate | ${STOP} blocks | dbcache ${DBCACHE} | $(hostname) | $(uname -m) | $(lscpu | grep 'Model name' | head -1 | cut -d: -f2 | xargs) | $(nproc) cores | $(free -h | awk '/^Mem:/{print $2}') RAM | SSD"; echo "") &&\
    hyperfine \
      --sort command \
      --runs 1 \
      --export-json "$BASE_DIR/rdx-$(sed -E 's/(\w{8})\w+ ?/\1-/g;s/-$//'<<<"$COMMITS")-$STOP-$DBCACHE-$CC.json" \
      --parameter-list COMMIT ${COMMITS// /,} \
      --prepare "killall -9 bitcoind 2>/dev/null; rm -f $DATA_DIR/debug.log; git clean -fxd; git reset --hard {COMMIT} && \
        cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DENABLE_IPC=OFF && ninja -C build bitcoind -j1 && \
        ./build/bin/bitcoind -datadir=$DATA_DIR -stopatheight=$STOP -dbcache=1000 -printtoconsole=0; sleep 20; rm -f $DATA_DIR/debug.log" \
      --conclude "killall bitcoind || true; sleep 5; grep -q 'height=0' $DATA_DIR/debug.log && grep -q 'Disabling script verification at block #1' $DATA_DIR/debug.log && grep -q 'height=$STOP' $DATA_DIR/debug.log; \
                  cp $DATA_DIR/debug.log $LOG_DIR/debug-{COMMIT}-$(date +%s).log" \
      "COMPILER=$CC ./build/bin/bitcoind -datadir=$DATA_DIR -stopatheight=$STOP -dbcache=$DBCACHE -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0";
  done

  595504a432 Merge bitcoin/bitcoin#34236: Add sedited to trusted-keys
  63e822b637 validation: make `IsInitialBlockDownload()` lock-free

  2026-01-12 | reindex-chainstate | 931139 blocks | dbcache 4500 | i9-ssd | x86_64 | Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz | 16 cores | 62Gi RAM | SSD

  Benchmark 1: COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=931139 -dbcache=4500 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 595504a432)
    Time (abs ≡):        17187.310 s               [User: 33104.415 s, System: 937.548 s]

  Benchmark 2: COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=931139 -dbcache=4500 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 63e822b637f67242e3689adedc0155b34100e651)
    Time (abs ≡):        17240.300 s               [User: 33164.803 s, System: 976.485 s]

  Relative speed comparison
          1.00          COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=931139 -dbcache=4500 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 595504a432)
          1.00          COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=931139 -dbcache=4500 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 63e822b637f67242e3689adedc0155b34100e651)
  ```

  </details>

ACKs for top commit:
  sedited:
    ACK 557b41a38c
  sipa:
    utACK 557b41a38c
  mzumsande:
    Code Review ACK 557b41a38c

Tree-SHA512: 174015b9785846fc3375bd9d6e4ef91de47ffb659a94d645c49d333a33a32986d5c3cb2eb5a0a7245f96580ed6fea4ba5b7f93cac7e42e3b225f0a01538a2e5c

2026-01-29 11:51:41 +01:00

blockfilter.cpp

…

blockfilter.h

…

chainstate.h

…

cluster_linearize.h

…

CMakeLists.txt

…

coins.cpp

…

coins.h

…

coverage.cpp

…

coverage.h

…

json.cpp

…

json.h

…

logging.cpp

…

logging.h

…

mining.cpp

…

mining.h

…

net.cpp

…

net.h

…

poolresourcetester.h

…

random.cpp

…

random.h

…

README.md

…

script.cpp

…

script.h

…

setup_common.cpp

…

setup_common.h

…

str.cpp

…

str.h

…

time.cpp

…

time.h

…

transaction_utils.cpp

…

transaction_utils.h

…

txmempool.cpp

…

txmempool.h

…

validation.cpp

…

validation.h

…

README.md

Test library

This contains files for the test library, which is used by the test binaries (unit tests, benchmarks, fuzzers, gui tests).

Generally, the files in this folder should be well-separated modules. New code should be added to existing modules or (when in doubt) a new module should be created.

The utilities in here are compiled into a library, which does not hold any state. However, the main file setup_common defines the common test setup for all test binaries. The test binaries will handle the global state when they instantiate the BasicTestingSetup (or one of its derived classes).