mirror of
https://github.com/bitcoin/bitcoin.git
synced 2026-07-24 07:08:44 +02:00
Merge bitcoin/bitcoin#35634: txospenderindex: use zero-byte entry values
113402286edoc: add txospenderindex release note (Lőrinc)ce06878288index: shrink txospenderindex value markers (Lőrinc) Pull request description: **Problem:** `TxoSpenderIndex` values are never read: `FindSpender()` only uses the key, which already encodes the spender's disk position, but the values are written as single-byte `\0`(instead of empty values). **Fix:** Write zero-byte markers for new `txospenderindex` entries instead. Existing indexes stay readable because both formats use the same keys and the value is ignored. Rebuilding is only needed to shrink old entries. **Related work:** This complements #35568, which saves another ~4.5 GB by dropping bloom filters. A #35531-style rewrite could let existing indexes reclaim both savings without a full rebuild. **Reproducer:** Automated tests weren't added, but the manual reproducer below builds both commits, checks old/new format compatibility through `gettxspendingprevout`, and prints rebuilt index sizes. On my mainnet datadir this PR shrank `txospenderindex` by ~3.4 GB (from 89G to 86G). <details><summary>Script + sample output</summary> ```bash BEFORE="89b4000ae06c72c5a14ee05ad70d3aece3f1b382" AFTER="1be799ef858c33158a302857b084e169b5bf8c1f" DATA_DIR="/mnt/my_storage/BitcoinData" LOG="${DATA_DIR}/debug.log" OUT='[{"txid":"0437cd7f8525ceed2324359c2d0ba26006d92d856a9c20fa0241106ee5a597c9","vout":0}]' SPEND="f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16"; \ for b in before after; do [ -x ./build-$b/bin/bitcoin-cli ] && ./build-$b/bin/bitcoin-cli -datadir="${DATA_DIR}" stop >/dev/null 2>&1 || true; done; sleep 10; \ git reset --hard >/dev/null 2>&1 && git clean -fxd >/dev/null 2>&1 && (git fetch origin "$BEFORE" "$AFTER" >/dev/null 2>&1 || true) && \ for c in before:$BEFORE after:$AFTER; do git checkout ${c#*:} >/dev/null 2>&1 && cmake -B build-${c%:*} -DCMAKE_BUILD_TYPE=Release >/dev/null 2>&1 && cmake --build build-${c%:*} -j --target bitcoind bitcoin-cli >/dev/null 2>&1; done && \ for b in before after; do ./build-$b/bin/bitcoin-cli -datadir="${DATA_DIR}" stop >/dev/null 2>&1 || true; done; sleep 10; \ start_node() { [ "$2" = wipe ] && rm -rf "${DATA_DIR}/indexes/txospenderindex" "${LOG}"; : > "${LOG}"; ./build-"$1"/bin/bitcoind -datadir="${DATA_DIR}" -txospenderindex=1 -connect=0 -printtoconsole=0 & pid=$!; while ! grep -Fq 'txospenderindex is enabled at height' "${LOG}" 2>/dev/null; do kill -0 "$pid" 2>/dev/null || { tail -100 "${LOG}"; return 1; }; sleep 5; done; }; \ check_spend() { result="$(./build-"$1"/bin/bitcoin-cli -datadir="${DATA_DIR}" gettxspendingprevout "$OUT" '{"mempool_only":false}')"; echo "$2: $result"; echo "$result" | grep -q "$SPEND"; }; \ stop_node() { ./build-"$1"/bin/bitcoin-cli -datadir="${DATA_DIR}" stop >/dev/null; wait "$pid"; }; \ echo "prevout: 0437cd7f8525ceed2324359c2d0ba26006d92d856a9c20fa0241106ee5a597c9:0 -> ${SPEND}" && \ start_node before wipe && check_spend before "old binary reads old-format index" && du -sh "${DATA_DIR}/indexes/txospenderindex"; stop_node before && \ start_node after && check_spend after "new binary reads old-format index"; stop_node after && \ start_node after wipe && check_spend after "new binary reads new-format index" && du -sh "${DATA_DIR}/indexes/txospenderindex"; stop_node after && \ start_node before && check_spend before "old binary reads new-format index"; stop_node before ``` Expected result, trimmed to the relevant compatibility and size lines: ```text prevout: 0437cd7f8525ceed2324359c2d0ba26006d92d856a9c20fa0241106ee5a597c9:0 -> f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16 old binary reads old-format index: [ ... "spendingtxid": "f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16" ... ] 89G /mnt/my_storage/BitcoinData/indexes/txospenderindex new binary reads old-format index: [ ... "spendingtxid": "f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16" ... ] new binary reads new-format index: [ ... "spendingtxid": "f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16" ... ] 86G /mnt/my_storage/BitcoinData/indexes/txospenderindex old binary reads new-format index: [ ... "spendingtxid": "f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16" ... ] ``` </details> ACKs for top commit: davidgumberg: crACK113402286esedited: ACK113402286eekzyis: utACK113402286eandrewtoth: ACK113402286eTree-SHA512: 07945b9470670115e27a1ef51e64164da7b91e550fdcb59952bfc98d862264b405e8fd8c3267c42718a39823f853057390d031d936d6e0b2172ec07e36cfdd67
This commit is contained in:
9
doc/release-notes-35634.md
Normal file
9
doc/release-notes-35634.md
Normal file
@@ -0,0 +1,9 @@
|
||||
Indexes
|
||||
-------
|
||||
|
||||
- The transaction output spender index (`-txospenderindex`) now stores one byte
|
||||
less per spender entry for newly indexed blocks. Existing indexes remain
|
||||
compatible and no action is required. To apply the same saving to entries
|
||||
indexed before upgrading, stop the node, delete the
|
||||
`<datadir>/indexes/txospenderindex/` directory, and restart with
|
||||
`-txospenderindex` enabled to rebuild it.
|
||||
@@ -23,15 +23,17 @@
|
||||
#include <util/fs.h>
|
||||
#include <validation.h>
|
||||
|
||||
#include <cstddef>
|
||||
#include <cstdio>
|
||||
#include <exception>
|
||||
#include <ios>
|
||||
#include <span>
|
||||
#include <string>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
/* The database is used to find the spending transaction of a given utxo.
|
||||
* For every input of every transaction it stores a key that is a pair(siphash(input outpoint), transaction location on disk) and an empty value.
|
||||
* For every input of every transaction it stores a key that is a pair(siphash(input outpoint), transaction location on disk) and a zero-byte value.
|
||||
* To find the spending transaction of an outpoint, we perform a range query on siphash(outpoint), and for each returned key load the transaction
|
||||
* and return it if it does spend the provided outpoint.
|
||||
*/
|
||||
@@ -91,8 +93,9 @@ void TxoSpenderIndex::WriteSpenderInfos(const std::vector<std::pair<COutPoint, C
|
||||
CDBBatch batch(*m_db);
|
||||
for (const auto& [outpoint, pos] : items) {
|
||||
DBKey key(CreateKey(m_siphash_key, outpoint, pos));
|
||||
// key is hash(spent outpoint) | disk pos, value is empty
|
||||
batch.Write(key, "");
|
||||
// The key encodes the spent outpoint hash and disk position. The value is only a marker.
|
||||
// Older entries may contain serialized empty strings; FindSpender() reads only keys.
|
||||
batch.Write(key, std::span<const std::byte>{});
|
||||
}
|
||||
m_db->WriteBatch(batch);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user