Merge bitcoin/bitcoin#30479: validation: Add eligible ancestors of reconsidered block to setBlockIndexCandidates

8cc3ac6c23 validation: Don't use IsValid() to filter for invalid blocks (Martin Zumsande)
86d98b94e5 test: verify that ancestors of a reconsidered block can become the chain tip (stratospher)
3c39a55e64 validation: Add ancestors of reconsiderblock to setBlockIndexCandidates (Martin Zumsande)

Pull request description:

  When we call `reconsiderblock` for some block,  `Chainstate::ResetBlockFailureFlags` puts the descendants of that block into `setBlockIndexCandidates` (if they meet the criteria, i.e. have more work than the tip etc.), but never put any ancestors into the set even though we do clear their failure flags.

  I think that this is wrong, because `setBlockIndexCandidates` should always contain all eligible indexes that have at least as much work as the current tip, which can include ancestors of the reconsidered block. This is being checked by `CheckBlockIndex()`, which could fail if it was invoked after `ActivateBestChain` connects a block and releases `cs_main`:
  ``` diff
  diff --git a/src/validation.cpp b/src/validation.cpp
  index 7b04bd9a5b..ff0c3c9f58 100644
  --- a/src/validation.cpp
  +++ b/src/validation.cpp
  @@ -3551,6 +3551,7 @@ bool Chainstate::ActivateBestChain(BlockValidationState& state, std::shared_ptr<
               }
           }
           // When we reach this point, we switched to a new tip (stored in pindexNewTip).
  +        m_chainman.CheckBlockIndex();
   
           if (exited_ibd) {
               // If a background chainstate is in use, we may need to rebalance our
  ```
  makes `rpc_invalidateblock.py` fail on master.

  Even though we don't currently have a `CheckBlockIndex()` in that place, after `cs_main` is released other threads could invoke it, which is happening in the rare failures of #16444 where an invalid header received from another peer could trigger a `CheckBlockIndex()` call that would fail.

  Fix this by adding eligible ancestors to `setBlockIndexCandidates` in `Chainstate::ResetBlockFailureFlags` (also simplifying that function a bit).

  Fixes #16444

ACKs for top commit:
  achow101:
    ACK 8cc3ac6c23
  TheCharlatan:
    Re-ACK 8cc3ac6c23
  stratospher:
    reACK 8cc3ac6.

Tree-SHA512: 53f27591916246be4093d64b86a0494e55094abd8c586026b1247e4a36747bc3d6dbe46dc26ee4a22f47b8eb0d9699d13e577dee0e7198145f3c9b11ab2a30b7

test/functional/rpc_invalidateblock.py

@@ -85,6 +85,23 @@ class InvalidateTest(BitcoinTestFramework):
         self.wait_until(lambda: self.nodes[0].getblockcount() == 4, timeout=5)
         self.wait_until(lambda: self.nodes[1].getblockcount() == 4, timeout=5)
 
+        self.log.info("Verify that ancestors can become chain tip candidates when we reconsider blocks")
+        # Invalidate node0's current chain (1' -> 2' -> 3' -> 4') so that we don't reorg back to it in this test
+        badhash = self.nodes[0].getblockhash(1)
+        self.nodes[0].invalidateblock(badhash)
+        # Reconsider the tip so that node0's chain becomes this chain again : 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> header 7
+        self.nodes[0].reconsiderblock(tip)
+        blockhash_3 = self.nodes[0].getblockhash(3)
+        blockhash_4 = self.nodes[0].getblockhash(4)
+        blockhash_6 = self.nodes[0].getblockhash(6)
+        assert_equal(self.nodes[0].getbestblockhash(), blockhash_6)
+
+        # Invalidate block 4 so that chain becomes : 1 -> 2 -> 3
+        self.nodes[0].invalidateblock(blockhash_4)
+        assert_equal(self.nodes[0].getbestblockhash(), blockhash_3)
+        assert_equal(self.nodes[0].getblockchaininfo()['blocks'], 3)
+        assert_equal(self.nodes[0].getblockchaininfo()['headers'], 3)
+
         self.log.info("Verify that we reconsider all ancestors as well")
         blocks = self.generatetodescriptor(self.nodes[1], 10, ADDRESS_BCRT1_UNSPENDABLE_DESCRIPTOR, sync_fun=self.no_op)
         assert_equal(self.nodes[1].getbestblockhash(), blocks[-1])
@@ -111,6 +128,14 @@ class InvalidateTest(BitcoinTestFramework):
         # Should report consistent blockchain info
         assert_equal(self.nodes[1].getblockchaininfo()["headers"], self.nodes[1].getblockchaininfo()["blocks"])
 
+        # Reconsider the header
+        self.nodes[0].reconsiderblock(block.hash)
+        # Since header doesn't have block data, it can't be chain tip
+        # Check if it's possible for an ancestor (with block data) to be the chain tip
+        assert_equal(self.nodes[0].getbestblockhash(), blockhash_6)
+        assert_equal(self.nodes[0].getblockchaininfo()['blocks'], 6)
+        assert_equal(self.nodes[0].getblockchaininfo()['headers'], 7)
+
         self.log.info("Verify that invalidating an unknown block throws an error")
         assert_raises_rpc_error(-5, "Block not found", self.nodes[1].invalidateblock, "00" * 32)
         assert_equal(self.nodes[1].getbestblockhash(), blocks[-1])