highperfocused/bitcoin
1
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-07-28 14:53:03 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge bitcoin/bitcoin#28368: Fee Estimator updates from Validation Interface/CScheduler thread

Browse Source 
91504cbe0d rpc: `SyncWithValidationInterfaceQueue` on fee estimation RPC's (ismaelsadeeq)
714523918b tx fees, policy: CBlockPolicyEstimator update from `CValidationInterface` notifications (ismaelsadeeq)
dff5ad3b99 CValidationInterface: modify the parameter of `TransactionAddedToMempool` (ismaelsadeeq)
91532bd382 tx fees, policy: update `CBlockPolicyEstimator::processBlock` parameter (ismaelsadeeq)
bfcd401368 CValidationInterface, mempool: add new callback to `CValidationInterface` (ismaelsadeeq)
0889e07987 tx fees, policy: cast with static_cast instead of C-Style cast (ismaelsadeeq)
a0e3eb7549 tx fees, policy: bugfix: move `removeTx` into reason != `BLOCK` condition (ismaelsadeeq)

Pull request description:

  This is an attempt to  #11775

  This Pr will enable fee estimator to listen to ValidationInterface notifications to process new transactions added and removed from the mempool.

  This PR includes the following changes:

  - Added a new callback to the Validation Interface `MempoolTransactionsRemovedForConnectedBlock`, which notifies listeners about the transactions that have been removed due to a new block being connected, along with the height at which the transactions were removed.
  - Modified the `TransactionAddedToMempool` callback parameter to include additional information about the transaction needed for fee estimation.
  - Updated `CBlockPolicyEstimator` to process transactions using` CTransactionRef` instead of `CTxMempoolEntry.`
  - Implemented the `CValidationInterface` interface in `CBlockPolicyEstimater` and overridden the `TransactionAddedToMempool`, `TransactionRemovedFromMempool`, and `MempoolTransactionsRemovedForConnectedBlock` methods to receive updates from their notifications.

  Prior to this PR, the fee estimator updates from the mempool, i.e whenever a new block is connected all transactions in the block that are in our mempool are going to be removed using the `removeForBlock` function in `txmempool.cpp`.

  This removal triggered updates to the fee estimator. As a result, the fee estimator would block mempool's `cs` until it finished updating every time a new block was connected.
  Instead of being blocked only on mempool tx removal, we were blocking on both tx removal and fee estimator updating.
  If we want to further improve fee estimation, or add heavy-calulation steps to it, it is currently not viable as we would be slowing down block relay in the process

  This PR is smaller in terms of the changes made compared to #11775, as it focuses solely on enabling fee estimator updates from the validationInterface/cscheduler thread notifications.

  I have not split the validation interface because, as I understand it, the rationale behind the split in #11775 was to have `MempoolInterface` signals come from the mempool and `CValidationInterface` events come from validation. I believe this separation can be achieved in a separate refactoring PR when the need arises.

  Also left out some commits from #11775
  - Some refactoring which are no longer needed.
  - Handle reorgs much better in fee estimator.
  - Track witness hash malleation in fee estimator

  I believe they are a separate change that can come in a follow-up after this.

ACKs for top commit:
  achow101:
    ACK 91504cbe0d
  TheCharlatan:
    Re-ACK 91504cbe0d
  willcl-ark:
    ACK 91504cbe0d

Tree-SHA512: 846dfb9da57a8a42458827b8975722d153907fe6302ad65748d74f311e1925557ad951c3d95fe71fb90ddcc8a3710c45abb343ab86b88780871cb9c38c72c7b1
This commit is contained in:
Andrew Chow
2023-12-01 14:58:14 -05:00
parent 18bed148af 91504cbe0d
commit a97a89244e
20 changed files with 307 additions and 123 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
src/test/fuzz/package_eval.cpp
View File

@@ -55,13 +55,13 @@ struct OutpointsUpdater final : public CValidationInterface {
explicit OutpointsUpdater(std::set<COutPoint>& r)
: m_mempool_outpoints{r} {}
void TransactionAddedToMempool(const CTransactionRef& tx, uint64_t /* mempool_sequence */) override
void TransactionAddedToMempool(const NewMempoolTransactionInfo& tx, uint64_t /* mempool_sequence */) override
{
// for coins spent we always want to be able to rbf so they're not removed
// outputs from this tx can now be spent
for (uint32_t index{0}; index < tx->vout.size(); ++index) {
m_mempool_outpoints.insert(COutPoint{tx->GetHash(), index});
for (uint32_t index{0}; index < tx.info.m_tx->vout.size(); ++index) {
m_mempool_outpoints.insert(COutPoint{tx.info.m_tx->GetHash(), index});
}
}
@@ -85,10 +85,10 @@ struct TransactionsDelta final : public CValidationInterface {
explicit TransactionsDelta(std::set<CTransactionRef>& a)
: m_added{a} {}
void TransactionAddedToMempool(const CTransactionRef& tx, uint64_t /* mempool_sequence */) override
void TransactionAddedToMempool(const NewMempoolTransactionInfo& tx, uint64_t /* mempool_sequence */) override
{
// Transactions may be entered and booted any number of times
m_added.insert(tx);
m_added.insert(tx.info.m_tx);
}
void TransactionRemovedFromMempool(const CTransactionRef& tx, MemPoolRemovalReason reason, uint64_t /* mempool_sequence */) override
@@ -121,7 +121,6 @@ CTxMemPool MakeMempool(FuzzedDataProvider& fuzzed_data_provider, const NodeConte
mempool_opts.expiry = std::chrono::hours{fuzzed_data_provider.ConsumeIntegralInRange<unsigned>(0, 999)};
nBytesPerSigOp = fuzzed_data_provider.ConsumeIntegralInRange<unsigned>(1, 999);
mempool_opts.estimator = nullptr;
mempool_opts.check_ratio = 1;
mempool_opts.require_standard = fuzzed_data_provider.ConsumeBool();

21
src/test/fuzz/policy_estimator.cpp
View File

@@ -44,9 +44,16 @@ FUZZ_TARGET(policy_estimator, .init = initialize_policy_estimator)
return;
}
const CTransaction tx{*mtx};
block_policy_estimator.processTransaction(ConsumeTxMemPoolEntry(fuzzed_data_provider, tx), fuzzed_data_provider.ConsumeBool());
const CTxMemPoolEntry& entry = ConsumeTxMemPoolEntry(fuzzed_data_provider, tx);
const auto tx_info = NewMempoolTransactionInfo(entry.GetSharedTx(), entry.GetFee(),
entry.GetTxSize(), entry.GetHeight(),
/* m_from_disconnected_block */ false,
/* m_submitted_in_package */ false,
/* m_chainstate_is_current */ true,
/* m_has_no_mempool_parents */ fuzzed_data_provider.ConsumeBool());
block_policy_estimator.processTransaction(tx_info);
if (fuzzed_data_provider.ConsumeBool()) {
(void)block_policy_estimator.removeTx(tx.GetHash(), /*inBlock=*/fuzzed_data_provider.ConsumeBool());
(void)block_policy_estimator.removeTx(tx.GetHash());
}
},
[&] {
@@ -61,15 +68,15 @@ FUZZ_TARGET(policy_estimator, .init = initialize_policy_estimator)
const CTransaction tx{*mtx};
mempool_entries.emplace_back(CTxMemPoolEntry::ExplicitCopy, ConsumeTxMemPoolEntry(fuzzed_data_provider, tx));
}
std::vector<const CTxMemPoolEntry*> ptrs;
ptrs.reserve(mempool_entries.size());
std::vector<RemovedMempoolTransactionInfo> txs;
txs.reserve(mempool_entries.size());
for (const CTxMemPoolEntry& mempool_entry : mempool_entries) {
ptrs.push_back(&mempool_entry);
txs.emplace_back(mempool_entry);
}
block_policy_estimator.processBlock(fuzzed_data_provider.ConsumeIntegral<unsigned int>(), ptrs);
block_policy_estimator.processBlock(txs, fuzzed_data_provider.ConsumeIntegral<unsigned int>());
},
[&] {
(void)block_policy_estimator.removeTx(ConsumeUInt256(fuzzed_data_provider), /*inBlock=*/fuzzed_data_provider.ConsumeBool());
(void)block_policy_estimator.removeTx(ConsumeUInt256(fuzzed_data_provider));
},
[&] {
block_policy_estimator.FlushUnconfirmed();

5
src/test/fuzz/tx_pool.cpp
View File

@@ -59,9 +59,9 @@ struct TransactionsDelta final : public CValidationInterface {
explicit TransactionsDelta(std::set<CTransactionRef>& r, std::set<CTransactionRef>& a)
: m_removed{r}, m_added{a} {}
void TransactionAddedToMempool(const CTransactionRef& tx, uint64_t /* mempool_sequence */) override
void TransactionAddedToMempool(const NewMempoolTransactionInfo& tx, uint64_t /* mempool_sequence */) override
{
Assert(m_added.insert(tx).second);
Assert(m_added.insert(tx.info.m_tx).second);
}
void TransactionRemovedFromMempool(const CTransactionRef& tx, MemPoolRemovalReason reason, uint64_t /* mempool_sequence */) override
@@ -123,7 +123,6 @@ CTxMemPool MakeMempool(FuzzedDataProvider& fuzzed_data_provider, const NodeConte
CTxMemPool::Options mempool_opts{MemPoolOptionsForTest(node)};
// ...override specific options for this specific fuzz suite
mempool_opts.estimator = nullptr;
mempool_opts.check_ratio = 1;
mempool_opts.require_standard = fuzzed_data_provider.ConsumeBool();