highperfocused/bitcoin
1
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-11-11 06:28:31 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge bitcoin/bitcoin#25740: assumeutxo: background validation completion

Browse Source 
2b373fe49d docs: update assumeutxo.md (James O'Beirne)
87a1108c81 test: add snapshot completion unittests (James O'Beirne)
d70919a88f refactor: make MempoolMutex() public (James O'Beirne)
7300ced9de log: add LoadBlockIndex() message for assumedvalid blocks (James O'Beirne)
d96c59cc5c validation: add ChainMan logic for completing UTXO snapshot validation (James O'Beirne)
f2a4f3376f move-only-ish: init: factor out chainstate initialization (James O'Beirne)
637a90b973 add Chainstate::HasCoinsViews() (James O'Beirne)
c29f26b47b validation: add CChainState::m_disabled and ChainMan::isUsable (James O'Beirne)
5ee22cdafd add ChainstateManager.GetSnapshot{BaseHeight,BaseBlock}() (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11) (parent PR: https://github.com/bitcoin/bitcoin/pull/15606)

  Part two of replacing https://github.com/bitcoin/bitcoin/pull/24232.

  ---

  When a user activates a snapshot, the serialized UTXO set data is used to create an "assumed-valid" chainstate, which becomes active in an attempt to get the node to network tip as quickly as possible. Simultaneously in the background, the already-existing chainstate continues "conventional" IBD to both accumulate full block data and serve as a belt-and-suspenders to validate the assumed-valid chainstate.

  Once the background chainstate's tip reaches the base block of the snapshot used, we set `m_stop_use` on that chainstate and immediately take the hash of its UTXO set; we verify that this matches the assumeutxo value in the source code. Note that while we ultimately want to remove this background chainstate, we don't do so until the following initialization process, when we again check the UTXO set hash of the background chainstate, and if it continues to match, we remove the (now unnecessary) background chainstate, and move the (previously) assumed-valid chainstate into its place. We then reinitialize the chainstate in the normal way.

  As noted in previous comments, we could do the filesystem operations "inline" immediately when the background validation completes, but that's basically just an optimization that saves disk space until the next restart. It didn't strike me as worth the risk of moving chainstate data around on disk during runtime of the node, though maybe my concerns are overblown.

  The final result of this completion process is a fully-validated chain, where the only evidence that the user synced using assumeutxo is the existence of a `base_blockhash` file in the `chainstate` directory.

ACKs for top commit:
  achow101:
    ACK 2b373fe49d

Tree-SHA512: a204e1d6e6932dd83c799af3606b01a9faf893f04e9ee1a36d63f2f1ccfa9118bdc1c107d86976aa0312814267e6a42074bf3e2bf1dead4b2513efc6d955e13d
This commit is contained in:
Andrew Chow
2023-03-07 18:36:51 -05:00
parent fc037c8c83 2b373fe49d
commit d5e4f9a439
5 changed files with 737 additions and 90 deletions
Download Patch File Download Diff File Expand all files Collapse all files

119
src/validation.h
View File

@@ -24,6 +24,7 @@
#include <policy/packages.h>
#include <policy/policy.h>
#include <script/script_error.h>
#include <shutdown.h>
#include <sync.h>
#include <txdb.h>
#include <txmempool.h> // For CTxMemPool::cs
@@ -493,6 +494,19 @@ protected:
//! Manages the UTXO set, which is a reflection of the contents of `m_chain`.
std::unique_ptr<CoinsViews> m_coins_views;
//! This toggle exists for use when doing background validation for UTXO
//! snapshots.
//!
//! In the expected case, it is set once the background validation chain reaches the
//! same height as the base of the snapshot and its UTXO set is found to hash to
//! the expected assumeutxo value. It signals that we should no longer connect
//! blocks to the background chainstate. When set on the background validation
//! chainstate, it signifies that we have fully validated the snapshot chainstate.
//!
//! In the unlikely case that the snapshot chainstate is found to be invalid, this
//! is set to true on the snapshot chainstate.
bool m_disabled GUARDED_BY(::cs_main) {false};
public:
//! Reference to a BlockManager instance which itself is shared across all
//! Chainstate instances.
@@ -560,15 +574,15 @@ public:
CCoinsViewCache& CoinsTip() EXCLUSIVE_LOCKS_REQUIRED(::cs_main)
{
AssertLockHeld(::cs_main);
assert(m_coins_views->m_cacheview);
return *m_coins_views->m_cacheview.get();
Assert(m_coins_views);
return *Assert(m_coins_views->m_cacheview);
}
//! @returns A reference to the on-disk UTXO set database.
CCoinsViewDB& CoinsDB() EXCLUSIVE_LOCKS_REQUIRED(::cs_main)
{
AssertLockHeld(::cs_main);
return m_coins_views->m_dbview;
return Assert(m_coins_views)->m_dbview;
}
//! @returns A pointer to the mempool.
@@ -582,12 +596,15 @@ public:
CCoinsViewErrorCatcher& CoinsErrorCatcher() EXCLUSIVE_LOCKS_REQUIRED(::cs_main)
{
AssertLockHeld(::cs_main);
return m_coins_views->m_catcherview;
return Assert(m_coins_views)->m_catcherview;
}
//! Destructs all objects related to accessing the UTXO set.
void ResetCoinsViews() { m_coins_views.reset(); }
//! Does this chainstate have a UTXO set attached?
bool HasCoinsViews() const { return (bool)m_coins_views; }
//! The cache size of the on-disk coins view.
size_t m_coinsdb_cache_size_bytes{0};
@@ -667,6 +684,12 @@ public:
* May not be called with cs_main held. May not be called in a
* validationinterface callback.
*
* Note that if this is called while a snapshot chainstate is active, and if
* it is called on a background chainstate whose tip has reached the base block
* of the snapshot, its execution will take *MINUTES* while it hashes the
* background UTXO set to verify the assumeutxo value the snapshot was activated
* with. `cs_main` will be held during this time.
*
* @returns true unless a system error occurred
*/
bool ActivateBestChain(
@@ -745,6 +768,12 @@ public:
std::string ToString() EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
//! Indirection necessary to make lock annotations work with an optional mempool.
RecursiveMutex* MempoolMutex() const LOCK_RETURNED(m_mempool->cs)
{
return m_mempool ? &m_mempool->cs : nullptr;
}
private:
bool ActivateBestChainStep(BlockValidationState& state, CBlockIndex* pindexMostWork, const std::shared_ptr<const CBlock>& pblock, bool& fInvalidFound, ConnectTrace& connectTrace) EXCLUSIVE_LOCKS_REQUIRED(cs_main, m_mempool->cs);
bool ConnectTip(BlockValidationState& state, CBlockIndex* pindexNew, const std::shared_ptr<const CBlock>& pblock, ConnectTrace& connectTrace, DisconnectedBlockTransactions& disconnectpool) EXCLUSIVE_LOCKS_REQUIRED(cs_main, m_mempool->cs);
@@ -758,12 +787,6 @@ private:
void CheckForkWarningConditions() EXCLUSIVE_LOCKS_REQUIRED(cs_main);
void InvalidChainFound(CBlockIndex* pindexNew) EXCLUSIVE_LOCKS_REQUIRED(cs_main);
//! Indirection necessary to make lock annotations work with an optional mempool.
RecursiveMutex* MempoolMutex() const LOCK_RETURNED(m_mempool->cs)
{
return m_mempool ? &m_mempool->cs : nullptr;
}
/**
* Make mempool consistent after a reorg, by re-adding or recursively erasing
* disconnected block transactions from the mempool, and also removing any
@@ -788,9 +811,37 @@ private:
std::chrono::microseconds m_last_write{0};
std::chrono::microseconds m_last_flush{0};
/**
* In case of an invalid snapshot, rename the coins leveldb directory so
* that it can be examined for issue diagnosis.
*/
void InvalidateCoinsDBOnDisk() EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
friend ChainstateManager;
};
enum class SnapshotCompletionResult {
SUCCESS,
SKIPPED,
// Expected assumeutxo configuration data is not found for the height of the
// base block.
MISSING_CHAINPARAMS,
// Failed to generate UTXO statistics (to check UTXO set hash) for the background
// chainstate.
STATS_FAILED,
// The UTXO set hash of the background validation chainstate does not match
// the one expected by assumeutxo chainparams.
HASH_MISMATCH,
// The blockhash of the current tip of the background validation chainstate does
// not match the one expected by the snapshot chainstate.
BASE_BLOCKHASH_MISMATCH,
};
/**
* Provides an interface for creating and interacting with one or two
* chainstates: an IBD chainstate generated by downloading blocks, and
@@ -860,10 +911,6 @@ private:
//! that call.
Chainstate* m_active_chainstate GUARDED_BY(::cs_main) {nullptr};
//! If true, the assumed-valid chainstate has been fully validated
//! by the background validation chainstate.
bool m_snapshot_validated GUARDED_BY(::cs_main){false};
CBlockIndex* m_best_invalid GUARDED_BY(::cs_main){nullptr};
//! Internal helper for ActivateSnapshot().
@@ -889,6 +936,22 @@ private:
/** Most recent headers presync progress update, for rate-limiting. */
std::chrono::time_point<std::chrono::steady_clock> m_last_presync_update GUARDED_BY(::cs_main) {};
//! Returns nullptr if no snapshot has been loaded.
const CBlockIndex* GetSnapshotBaseBlock() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
//! Return the height of the base block of the snapshot in use, if one exists, else
//! nullopt.
std::optional<int> GetSnapshotBaseHeight() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
//! Return true if a chainstate is considered usable.
//!
//! This is false when a background validation chainstate has completed its
//! validation of an assumed-valid chainstate, or when a snapshot
//! chainstate has been found to be invalid.
bool IsUsable(const Chainstate* const cs) const EXCLUSIVE_LOCKS_REQUIRED(::cs_main) {
return cs && !cs->m_disabled;
}
public:
using Options = kernel::ChainstateManagerOpts;
@@ -976,6 +1039,18 @@ public:
[[nodiscard]] bool ActivateSnapshot(
AutoFile& coins_file, const node::SnapshotMetadata& metadata, bool in_memory);
//! Once the background validation chainstate has reached the height which
//! is the base of the UTXO snapshot in use, compare its coins to ensure
//! they match those expected by the snapshot.
//!
//! If the coins match (expected), then mark the validation chainstate for
//! deletion and continue using the snapshot chainstate as active.
//! Otherwise, revert to using the ibd chainstate and shutdown.
SnapshotCompletionResult MaybeCompleteSnapshotValidation(
std::function<void(bilingual_str)> shutdown_fnc =
[](bilingual_str msg) { AbortNode(msg.original, msg); })
EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
//! The most-work chain.
Chainstate& ActiveChainstate() const;
CChain& ActiveChain() const EXCLUSIVE_LOCKS_REQUIRED(GetMutex()) { return ActiveChainstate().m_chain; }
@@ -1000,7 +1075,10 @@ public:
std::optional<uint256> SnapshotBlockhash() const;
//! Is there a snapshot in use and has it been fully validated?
bool IsSnapshotValidated() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main) { return m_snapshot_validated; }
bool IsSnapshotValidated() const EXCLUSIVE_LOCKS_REQUIRED(::cs_main)
{
return m_snapshot_chainstate && m_ibd_chainstate && m_ibd_chainstate->m_disabled;
}
/**
* Process an incoming block. This only returns after the best known valid
@@ -1080,6 +1158,17 @@ public:
Chainstate& ActivateExistingSnapshot(CTxMemPool* mempool, uint256 base_blockhash)
EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
//! If we have validated a snapshot chain during this runtime, copy its
//! chainstate directory over to the main `chainstate` location, completing
//! validation of the snapshot.
//!
//! If the cleanup succeeds, the caller will need to ensure chainstates are
//! reinitialized, since ResetChainstates() will be called before leveldb
//! directories are moved or deleted.
//!
//! @sa node/chainstate:LoadChainstate()
bool ValidatedSnapshotCleanup() EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
~ChainstateManager();
};