highperfocused/bitcoin
1
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-11-12 06:58:57 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge bitcoin/bitcoin#26533: prune: scan and unlink already pruned block files on startup

Browse Source 
3141eab9c6 test: add functional test for ScanAndUnlinkAlreadyPrunedFiles (Andrew Toth)
e252909e56 test: add unit test for ScanAndUnlinkAlreadyPrunedFiles (Andrew Toth)
77557dda4a prune: scan and unlink already pruned block files on startup (Andrew Toth)

Pull request description:

  There are a few cases where we can mark a block and undo file as pruned in our block index, but not actually remove the files from disk.
  1. If we call `FindFilesToPrune` or `FindFilesToPruneManual` and crash before `UnlinkPrunedFiles`.
  2. If on Windows there is an open file handle to the file somewhere else when calling `fs::remove` in `UnlinkPrunedFiles` (https://en.cppreference.com/w/cpp/filesystem/remove, https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-deletefilew#remarks). This could be from another process, or if we are calling `ReadBlockFromDisk`/`ReadRawBlockFromDisk` without having a lock on `cs_main` (which has been allowed since ccd8ef65f9).

  This PR mitigates this by scanning all pruned block files on startup after `LoadBlockIndexDB` and unlinking them again.

ACKs for top commit:
  achow101:
    ACK 3141eab9c6
  pablomartin4btc:
    re-ACK with added functional test 3141eab9c6.
  furszy:
    Code review ACK 3141eab9
  theStack:
    Code-review ACK 3141eab9c6

Tree-SHA512: 6c73bc57838ad1b7e5d441af3c4d6bf4c61c4382e2b86485e57fbb74a61240710c0ceeceb8b4834e610ecfa3175c6955c81ea4b2285fee11ca6383f472979d8d
This commit is contained in:
Andrew Chow
2023-02-28 09:54:06 -05:00
parent 519ec2650e 3141eab9c6
commit bb136aaf2c
6 changed files with 130 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
src/node/blockstorage.cpp
View File

@@ -371,6 +371,23 @@ bool BlockManager::LoadBlockIndexDB(const Consensus::Params& consensus_params)
return true;
}
void BlockManager::ScanAndUnlinkAlreadyPrunedFiles()
{
AssertLockHeld(::cs_main);
if (!m_have_pruned) {
return;
}
std::set<int> block_files_to_prune;
for (int file_number = 0; file_number < m_last_blockfile; file_number++) {
if (m_blockfile_info[file_number].nSize == 0) {
block_files_to_prune.insert(file_number);
}
}
UnlinkPrunedFiles(block_files_to_prune);
}
const CBlockIndex* BlockManager::GetLastCheckpoint(const CCheckpointData& data)
{
const MapCheckpoints& checkpoints = data.mapCheckpoints;
@@ -556,11 +573,14 @@ uint64_t BlockManager::CalculateCurrentUsage()
void UnlinkPrunedFiles(const std::set<int>& setFilesToPrune)
{
std::error_code ec;
for (std::set<int>::iterator it = setFilesToPrune.begin(); it != setFilesToPrune.end(); ++it) {
FlatFilePos pos(*it, 0);
fs::remove(BlockFileSeq().FileName(pos));
fs::remove(UndoFileSeq().FileName(pos));
LogPrint(BCLog::BLOCKSTORE, "Prune: %s deleted blk/rev (%05u)\n", __func__, *it);
const bool removed_blockfile{fs::remove(BlockFileSeq().FileName(pos), ec)};
const bool removed_undofile{fs::remove(UndoFileSeq().FileName(pos), ec)};
if (removed_blockfile || removed_undofile) {
LogPrint(BCLog::BLOCKSTORE, "Prune: %s deleted blk/rev (%05u)\n", __func__, *it);
}
}
}