laanwj 7008087548 Merge bitcoin/bitcoin#24410: [kernel 2a/n] Split hashing/index GetUTXOStats codepaths, decouple from coinstatsindex ...

664a14ba7c coinstats: Move GetUTXOStats to rpc/blockchain (Carl Dong)
f100687566 kernel: Use ComputeUTXOStats in validation (Carl Dong)
faa52387e8 style-only: Rearrange using decls after scripted-diff (Carl Dong)
f329a9298c scripted-diff: Move src/kernel/coinstats to kernel:: (Carl Dong)
0e54456f04 Use only kernel/coinstats.h in index/coinstatsindex.h (Carl Dong)
80970985c9 coinstats: Split node/coinstats.h to kernel/coinstats.h (Carl Dong)
35f73ce4b2 coinstats: Move hasher codepath to kernel/coinstats (Carl Dong)
b7634fe02b Move logic from LookupUTXOStatsWithIndex to CoinStatsIndex::LookUpStats (Carl Dong)
1352e410a5 coinstats: Separate hasher/index lookup codepaths (Carl Dong)
524463daf6 coinstats: Return purely out-param CCoinsStats (Carl Dong)
46eb9fc56a coinstats: Extract index_requested in-member to in-param (Carl Dong)
a789f3f2b8 coinstats: Extract hash_type in-member to in-param (Carl Dong)
102294898d includes: Remove rpc/util.h -> node/coinstats.h (Carl Dong)
0848db9c35 fuzz: Remove useless GetUTXOStats fuzz case (Carl Dong)
52b1939993 kernel: Remove unnecessary blockfilter{index,}.cpp (Carl Dong)

Pull request description:

  Part of: #24303
  Depends on: #24322

  The `GetUTXOStats` function has 2 codepaths:
    - One which queries the `CoinStatsIndex` for the UTXO hash
    - One which actually performs the hashing

  For `libbitcoinkernel`, the only place where we call `GetUTXOStats` is in `PopulateAndValidateSnapshots`, which uses the `SHA256D` hash, and is therefore unable to use the `CoinStatsIndex` since that only provides `MuHash` hashes. Not that I think indices necessarily belong in `libbitcoinkernel` anyway.

  This PR separates these 2 aforementioned codepaths of `GetUTXOStats`, uses the hashing codepath in `PopulateAndValidateSnapshots`, and removes the need to link in `index/coinstatsindex.cpp` and `node/coinstats.cpp`.

  -----

  Logistically, this PR:
  - Extracts out the `index_requested` and `hash_type` members of `CoinStats`, which served as "in-params" to `GetUTXOStats` embedded within the `CoinStats` struct. This allows `CoinStats` to only consist of "out-param" members, and be returned by `GetUTXOStats` without needing to be an "in-out" param
  - Introduce the purely virtual `UTXOHashers` class, with 3 implementations: `SHA256DHasher`, `MuHashHasher`, and `NullHasher`. These replace the existing template-based polymorphism.
  - Split `GetUTXOStats` into:
      - `CalculateUTXOStatsWithHasher(UTXOHasher&, ...)`, and
      - `LookupUTXOStatsWithIndex(CoinStatsIndex&, ...)`
  - Use `CalculateUTXOStatsWithHasher` directly where appropriate (`src/validation.cpp` and `src/fuzz`)
  - Move `GetUTXOStats` to `rpc/blockchain`, which is the only place that depends on `GetUTXOStats`'s weird fallback behaviour
  - Move `LookupUTXOStatsWithIndex` to `index/coinstatsindex`

  Code organization:
  - `src/`
    - `kernel/` → only contains the hashing codepath
      - `coinstats.cpp` → hashing codepath implementations
      - `coinstats.h` → header for `kernel/coinstats.cpp`
    - `index/` → only contains the index codepath
      - `coinstatsindex.cpp` → index codepath implementations
      - `coinstatsindex.h`
    - `validation.cpp` → only uses the hashing codepath
    - `rpc/blockchain.cpp` → uses both the hashing and index codepath, old `GetUTXOStats` fallback logic moved here as static
    - `test/fuzz/coins_view.cpp` → only uses the hashing codepath

  TODOs:
  - [x] Commit messages could be fleshed out more

  Would love any feedback!

ACKs for top commit:
  laanwj:
    Code review ACK 664a14ba7c

Tree-SHA512: 18722c7bd279174d2d1881fec33ea04a9b261aae1c12e998cf434ef297d8ded47de69c526c8033a2ba7abc93ba3d2ff5faf4ce05e8888c725c31cf885ce3ef73

2022-05-24 14:43:00 +02:00

..

addition_overflow.cpp

util: Add SaturatingAdd helper

2022-02-21 14:32:53 +01:00

addrman.cpp

[net] Move asmap into NetGroupManager

2022-04-20 14:29:29 +01:00

asmap_direct.cpp

…

asmap.cpp

[netgroupman] Add GetMappedAS() and GetGroup()

2022-04-20 14:35:52 +01:00

autofile.cpp

…

banman.cpp

…

base_encode_decode.cpp

Use std::string_view throughout util strencodings/string

2022-04-27 14:13:39 +02:00

bech32.cpp

…

block_header.cpp

…

block.cpp

…

blockfilter.cpp

…

bloom_filter.cpp

…

buffered_file.cpp

…

chain.cpp

…

checkqueue.cpp

test, bench: make prevector and checkqueue swap member functions noexcept

2022-04-28 20:34:43 +02:00

coins_view.cpp

fuzz: Remove useless GetUTXOStats fuzz case

2022-05-20 16:33:24 -04:00

connman.cpp

[net] Move asmap into NetGroupManager

2022-04-20 14:29:29 +01:00

crypto_aes256.cpp

…

crypto_aes256cbc.cpp

…

crypto_chacha20_poly1305_aead.cpp

…

crypto_chacha20.cpp

…

crypto_common.cpp

…

crypto_diff_fuzz_chacha20.cpp

Don't use zero as null pointer constant (-Wzero-as-null-pointer-constant)

2022-04-26 10:41:45 +01:00

crypto_hkdf_hmac_sha256_l32.cpp

…

crypto_poly1305.cpp

…

crypto.cpp

…

cuckoocache.cpp

…

decode_tx.cpp

…

descriptor_parse.cpp

…

deserialize.cpp

[net] Move asmap into NetGroupManager

2022-04-20 14:29:29 +01:00

eval_script.cpp

…

fee_rate.cpp

…

fees.cpp

…

flatfile.cpp

…

float.cpp

…

fuzz.cpp

Remove unused GetTimeSeconds

2022-05-11 16:39:23 +02:00

fuzz.h

…

FuzzedDataProvider.h

…

golomb_rice.cpp

…

hex.cpp

util: Add ParseHex<std::byte>() helper

2022-04-27 19:53:17 +02:00

http_request.cpp

Use std::string_view throughout util strencodings/string

2022-04-27 14:13:39 +02:00

i2p.cpp

…

integer.cpp

…

key_io.cpp

…

key.cpp

…

kitchen_sink.cpp

…

load_external_block_file.cpp

…

locale.cpp

…

merkleblock.cpp

…

message.cpp

…

miniscript_decode.cpp

fuzz: add a fuzz target for Miniscript decoding from Script

2022-03-17 14:09:09 +01:00

minisketch.cpp

…

muhash.cpp

…

multiplication_overflow.cpp

…

net_permissions.cpp

…

net.cpp

[net processing] Move tx relay data to Peer

2022-03-18 11:35:56 +00:00

netaddress.cpp

net: remove unused CNetAddr::GetHash()

2022-02-11 15:21:52 +01:00

netbase_dns_lookup.cpp

…

node_eviction.cpp

scripted-diff: rename TxRelay members

2022-03-18 11:35:58 +00:00

p2p_transport_serialization.cpp

…

parse_hd_keypath.cpp

…

parse_iso8601.cpp

…

parse_numbers.cpp

…

parse_script.cpp

…

parse_univalue.cpp

…

policy_estimator_io.cpp

…

policy_estimator.cpp

…

pow.cpp

…

prevector.cpp

test, bench: make prevector and checkqueue swap member functions noexcept

2022-04-28 20:34:43 +02:00

primitives_transaction.cpp

…

process_message.cpp

…

process_messages.cpp

…

protocol.cpp

…

psbt.cpp

Make DecodeBase{32,64} always return vector, not string

2022-04-27 14:12:55 +02:00

random.cpp

…

rbf.cpp

…

rolling_bloom_filter.cpp

…

rpc.cpp

Merge bitcoin/bitcoin#23508: Add getdeploymentinfo RPC

2022-01-28 08:46:03 +01:00

script_assets_test_minimizer.cpp

scripted-diff: Use getInt<T> over get_int/get_int64

2022-05-18 19:15:03 +02:00

script_bitcoin_consensus.cpp

…

script_descriptor_cache.cpp

…

script_flags.cpp

…

script_format.cpp

refactor: merge ScriptPubKeyToUniv & ScriptToUniv into one function

2022-03-30 20:00:23 +01:00

script_interpreter.cpp

…

script_ops.cpp

…

script_sigcache.cpp

…

script_sign.cpp

refactor: Change * to & in MutableTransactionSignatureCreator

2022-05-04 11:49:29 +02:00

script.cpp

fuzz: Split script formatting from script fuzz target

2022-02-15 12:19:34 +01:00

scriptnum_ops.cpp

…

secp256k1_ec_seckey_import_export_der.cpp

…

secp256k1_ecdsa_signature_parse_der_lax.cpp

…

signature_checker.cpp

refactor: use C++11 default initializers

2022-05-17 17:18:58 +01:00

signet.cpp

…

socks5.cpp

…

span.cpp

…

spanparsing.cpp

…

string.cpp

fuzz: SplitString with multiple separators

2022-05-04 07:34:48 +02:00

strprintf.cpp

…

system.cpp

…

timedata.cpp

…

torcontrol.cpp

…

transaction.cpp

refactor: use named args when ScriptToUniv or TxToUniv are invoked

2022-03-30 20:00:27 +01:00

tx_in.cpp

…

tx_out.cpp

…

tx_pool.cpp

Do not pass CChainParams& to BlockAssembler constructor

2022-05-18 18:46:07 +02:00

txrequest.cpp

…

util.cpp

net: add new method Sock::SetSockOpt() that wraps setsockopt()

2022-04-15 09:14:49 +02:00

util.h

refactor: Remove defunct attributes.h includes

2022-05-21 13:54:33 -05:00

utxo_snapshot.cpp

validation: remove redundant CChainParams params from ChainstateManager methods

2022-05-10 12:09:33 +10:00

validation_load_mempool.cpp

…

versionbits.cpp

…