59ff17e5af4e382cbe16f183767beef1bdcd9131 miner: adjust clock to timewarp rule (Sjors Provoost)
e929054e12210353812f440c685a23329e7040f7 Add timewarp attack mitigation test (Sjors Provoost)
e85f386c4b157b7d1ac16aface9bd2c614e62b46 consensus: enable BIP94 on regtest (Sjors Provoost)
dd154b05689c60fad45df0df6d31cec12e09ab21 consensus: lower regtest nPowTargetTimespan to 144 (Sjors Provoost)
Pull request description:
Because #30647 reduced the timewarp attack threshold from 7200s to 600s, our miner code will fail to propose a block template (on testnet4) if the last block of the previous period has a timestamp two hours in the future. This PR fixes that and also adds a test.
The non-test changes in the last commit should be in v28, otherwise miners have to patch it themselves. If necessary I can split that out into a separate PR, but I prefer to get the tests in as well.
In order to add the test, we activate BIP94 on regtest.
In order for the test to run faster, we reduce its difficulty retarget period to 144, the same number that's already used for softfork activation logic. Regtest does not actually adjust its difficulty, so this change has no effect (except for `getnetworkhashps`, see commit).
An alternative approach would be to run this test on testnet4, by hardcoding its first 2015 in the test suite. But since the timewarp mitigation is a serious candidate for a future mainnet softfork, it seems better to just deploy it on regtest.
The next commits add a test and fix the miner code.
The `MAX_TIMEWARP` constant is moved to `consensus.h` so both validation and miner code have access to it.
ACKs for top commit:
achow101:
ACK 59ff17e5af4e382cbe16f183767beef1bdcd9131
fjahr:
ACK 59ff17e5af4e382cbe16f183767beef1bdcd9131
glozow:
ACK 59ff17e5af4e382cbe16f183767beef1bdcd9131
Tree-SHA512: 50af9fdcba9b0d5c57e1efd5feffd870bd11b5318f1f8b0aabf684657f2d33ab108d5f00b1475fe0d38e8e0badc97249ef8dda20c7f47fcc1698bc1008798830
917e70a6206c62c4c492fa922425fc8e00d3f328 test: assumeutxo: check that UTXO-querying RPCs operate on snapshot chainstate (Sebastian Falbesoner)
Pull request description:
Inspired by some manual testing I did for #28553, this PR checks that RPCs which explicitly query the UTXO set database (i.e. `gettxoutsetinfo`, `scantxoutset` and `gettxout`) operate on the snapshot chainstate as expected.
ACKs for top commit:
fjahr:
utACK 917e70a6206c62c4c492fa922425fc8e00d3f328
achow101:
ACK 917e70a6206c62c4c492fa922425fc8e00d3f328
tdb3:
ACK 917e70a6206c62c4c492fa922425fc8e00d3f328
Tree-SHA512: 40ecd1c5dd879234df1667fa5444a1fbbee9b7c456f597dc982d1a2bce46fe9107711b005ab829e570ef919a4914792f72f342d71d92bad2ae9434b5e68d5bd3
Handle the Block height out of range error gracefully by checking if
the node has synchronized to or beyond the required block height,
otherwise without this validation the node would keep the network
disabled if the user selected that option.
Provide a user-friendly message if the block height is out of range
and exit the script cleanly.
fa899fb7aa8a14acecadd8936ad5824fa0f697ff fuzz: Speed up utxo_snapshot fuzz target (MarcoFalke)
fa386642b4dfd88f74488c288c7886494d69f4ed fuzz: Speed up utxo_snapshot by lazy re-init (MarcoFalke)
fa645c7a861ffa83a53a459263b6a620defe31f9 fuzz: Remove unused DataStream object (MarcoFalke)
fae8c73d9e4eba4603447bb52b6e3e760fbf15f8 test: Disallow fee_estimator construction in ChainTestingSetup (MarcoFalke)
Pull request description:
Two commits to speed up unit and fuzz tests.
Can be tested by running the fuzz target and looking at the time it took, or by looking at the flamegraph. For example:
```
FUZZ=utxo_snapshot perf record -g --call-graph dwarf ./src/test/fuzz/fuzz -runs=100
hotspot ./perf.data
ACKs for top commit:
TheCharlatan:
Re-ACK fa899fb7aa8a14acecadd8936ad5824fa0f697ff
marcofleon:
Re ACK fa899fb7aa8a14acecadd8936ad5824fa0f697ff
brunoerg:
ACK fa899fb7aa8a14acecadd8936ad5824fa0f697ff
Tree-SHA512: d3a771bb12d7ef491eee61ca47325dd1cea5c20b6ad42554babf13ec98d03bef8e7786159d077e59cc7ab8112495037b0f6e55edae65b871c7cf1708687cf717
The flags SECP256K1_CONTEXT_{SIGN,VERIFY} have been deprecated since
libsecp256k1 version 0.2 (released in December 2022), with the
recommendation to use SECP256K1_CONTEXT_NONE instead.
This currently has no effect due to fPowNoRetargeting,
except for the getnetworkhashps when called with -1.
It will when the next commit enforces the timewarp attack mitigation on regtest.
16e95bda86302af20cfb314a2c0252256d01f750 Move maximum timewarp attack threshold back to 600s from 7200s (Matt Corallo)
Pull request description:
In 6bfa26048dbafb91e9ca63ea8d3960271e798098 the testnet4 timewarp attack fix block time variation was increased from the Great Consensus Cleanup value of 600s to 7200s on the thesis that this allows miners to always create blocks with the current time. Sadly, doing so does allow for some nonzero inflation, even if not a huge amount.
While it could be that some hardware ignores the timestamp provided to it over Stratum and forces the block header timestamp to the current time, I'm not aware of any such hardware, and it would also likely suffer from random invalid blocks due to relying on NTP anyway, making its existence highly unlikely.
This leaves the only concern being pools, but most of those rely on work generated by Bitcoin Core (in one way or another, though when spy mining possibly not), and it seems likely that they will also not suffer any lost work. While its possible that a pool does generate invalid work due to spy mining or otherwise custom logic, it seems unlikely that a substantial portion of hashrate would do so, making the difference somewhat academic (any pool that screws this up will only do so once and the network would come out just fine).
Further, while we may end up deciding these assumptions were invalid and we should instead use 7200s, it seems prudent to try with the value we "want" on testnet4, giving us the ability to learn if the compatibility concerns are an issue before we go to mainnet.
ACKs for top commit:
fjahr:
tACK 16e95bda86302af20cfb314a2c0252256d01f750
achow101:
ACK 16e95bda86302af20cfb314a2c0252256d01f750
murchandamus:
crACK 16e95bda86302af20cfb314a2c0252256d01f750
Tree-SHA512: ae46d03b728b6e23cb6ace64c9813bc01c01e38dd7f159cf0fab53b331ef84b3b811edab225453ccdfedb53b242f55b0efd69829782657490fe393d24dacbeb2
6ed424f2db609f9f39ec1d1da2077c7616f3a0c2 wallet: fix, detect blank legacy wallets in IsLegacy (furszy)
Pull request description:
Blank legacy wallets do not have active SPKM. They can only be
detected by checking the descriptors' flag or the db format.
This enables the migration of blank legacy wallets in the GUI.
To test this:
1) Create a blank legacy wallet.
2) Try to migrate it using the GUI's toolbar "Migrate Wallet" button.
-> In master: The button will be disabled because `CWallet::IsLegacy()` returns false for blank legacy wallet.
-> In this PR: the button will be enabled, allowing the migration of legacy wallets.
ACKs for top commit:
achow101:
ACK 6ed424f2db609f9f39ec1d1da2077c7616f3a0c2
tdb3:
ACK 6ed424f2db609f9f39ec1d1da2077c7616f3a0c2
glozow:
ACK 6ed424f2db609f9f39ec1d1da2077c7616f3a0c2
Tree-SHA512: c06c4c4c2e546ccb033287b9aa3aee4ca36b47aeb2fac6fbed5de774b65caef9c818fc8dfdaac6ce78839b2d5d642a5632a5b44c5e889ea169ced80ed50501a7
The seeders now produce onion and i2p seeds, so there is no need to keep these
in the manual list.
Although should also be produced, there are not enough
good ones detected by the seeder, so we keep the manual seeds for them.
faa1b9b0e6de7d213699fbdbc2e25a2a81c35cdc test: add functional test for XORed block/undo files (`-blocksxor`) (Sebastian Falbesoner)
6b3676be3e5b6e407876031791172f441b359295 test: refactor: move `read_xor_key`/`util_xor` helpers to util module (Sebastian Falbesoner)
Pull request description:
This PR adds a dedicated functional test for XORed block data/undo file support (bitcoind option `-blocksxor`, see PR #28052). In order to verify that the XOR pattern has been applied, the {blk,rev}*.dat files are rewritten un-XORed manually by the test while the node is shut down; the node is then started again with `-blocksxor=0`, and both the data and undo files are verified via the `verifychain` RPC (with checklevel=2). Note that starting bitcoind with `-blocksxor=0` fails if a xor key is present already, which is also tested explicitly.
Fixes#30599.
ACKs for top commit:
glozow:
ACK faa1b9b0e6d
maflcko:
ACK faa1b9b0e6de7d213699fbdbc2e25a2a81c35cdc
ismaelsadeeq:
Tested ACK faa1b9b0e6de7d213699fbdbc2e25a2a81c35cdc
Tree-SHA512: e1df106f6b4e3ba67eca108e36d762f1b991673b881934b84cd36946496a09ce9c329c1363c36aa29409137ae4881e2d177e651359686511632ddf2870f7ca8e
The re-init is expensive, so skip it if there is no need.
Also, add an even faster fuzz target utxo_snapshot_invalid, which does
not need any re-init at all.
f550a8e035b4603787273ea250f403f6f0be453f Rename ReleaseWallet to FlushAndDeleteWallet (furszy)
64e736d79efc7201768244fc297084f70c0bebc1 wallet: WaitForDeleteWallet, do not expect thread safety (Ryan Ofsky)
8872b4a6ca91a83bf8d5a118fb808c043b9e879d wallet: rename UnloadWallet to WaitForDeleteWallet (furszy)
5d15485aafefdc759ba97e039bb1b9ccac267358 wallet: unload, notify GUI as soon as possible (furszy)
Pull request description:
Coming from #29073.
Applied ryanofsky suggested changes on https://github.com/bitcoin/bitcoin/issues/29073#issuecomment-2274237242 with few modifications coming from https://github.com/bitcoin/bitcoin/pull/18338#issuecomment-605060348.
The only point I did not tackle from https://github.com/bitcoin/bitcoin/pull/18338#issuecomment-605060348 is:
> * Move log print and flush out of ReleaseWallet into CWallet destructor
Because it would mean every `CWallet` object would flush data to disk during destruction. Which is not necessary for wallet tool utilities and unit tests.
ACKs for top commit:
achow101:
ACK f550a8e035b4603787273ea250f403f6f0be453f
ryanofsky:
Code review ACK f550a8e035b4603787273ea250f403f6f0be453f. Just a simple rename since last review
ismaelsadeeq:
Re-ACK f550a8e035b4603787273ea250f403f6f0be453f
Tree-SHA512: e2eb69bf36883c514f601f4838ae6a41113996b9559abf8dc2b46e16bbcdad401195ac0f2b9d1fb55a10e78bb8ea9953788a168c80474e3f101350d208cb3bd2
99eeb51bf65fa00ee863f32d70f478bb9db0e351 [doc] mention bip94 support (glozow)
Pull request description:
Followup to #29775, noticed while looking at #30604 and #30647. See [release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#before-every-major-and-minor-release).
ACKs for top commit:
maflcko:
ACK 99eeb51bf65fa00ee863f32d70f478bb9db0e351
fjahr:
ACK 99eeb51bf65fa00ee863f32d70f478bb9db0e351
tdb3:
ACK 99eeb51bf65fa00ee863f32d70f478bb9db0e351
Tree-SHA512: 95838d3ace7e5d7b1a2481f2d7bd82902081713e6e89dbf21e0dad16d1cf5295e0c1cfda1f03af72304a5844743d24769f5fa04d4dc9f02f36462ef0ae82a552
The crawlers are not guaranteed to output nodes in a random order, so
shuffle the ips list after parsing to break any biasing that may be
caused by the output order.
Update the user agent regex to match all 3 digits of the version number,
not just the first 2 digits.
Also updates it to include 24.2, 25.2, 26.1, 27.0, 27.1, 27.99, 28.0 and
28.99.
1610643c8b37a9f674b236cfa79abf8f8aaf1410 chainparams: add mainnet assumeutxo param at height 840_000 (Sjors Provoost)
Pull request description:
This adds snapshot parameters for mainnet block 840,000.
You can generate the snapshot yourself using `./contrib/devtools/utxo_snapshot.sh` or download my torrent:
* torrent: `magnet:?xt=urn:btih:596c26cc709e213fdfec997183ff67067241440c&dn=utxo-840000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`
It would be a good idea to test:
1. That you can produce the same snapshot file, sha256 sum:
```
dc4bb43d58d6a25e91eae93eb052d72e3318bd98ec62a5d0c11817cefbba177b utxo-840000.dat
```
2. That the snapshot works
ACKs for top commit:
fjahr:
re-ACK 1610643c8b37a9f674b236cfa79abf8f8aaf1410
achow101:
ACK 1610643c8b37a9f674b236cfa79abf8f8aaf1410
theStack:
Tested ACK 1610643c8b37a9f674b236cfa79abf8f8aaf1410
mzumsande:
tested ACK 1610643c8b37a9f674b236cfa79abf8f8aaf1410
willcl-ark:
tACK 1610643c8b37a9f674b236cfa79abf8f8aaf1410
Tree-SHA512: 581d8e86379bb044324f04f8559dd0a8946b6e2b145d5f25b38727b30b8cf13d6ac3c8777ff06554d3cf1a072809f7b5fbd693239868578f25dceafe5ba5f57c
9b297555207b4ea54bc0051f09c7084797aa9def Deduplicate list of chain strings in RPC help texts (Martin Saposnic)
Pull request description:
As mentioned in issue https://github.com/bitcoin/bitcoin/issues/30645:
Many command line parameter and RPC help texts currently contain the list of chain/network names hardcoded ("main, test, testnet4, signet, regtest"), which is error-prone as it can easily happen to miss an instance if the list ever changes again.
This PR deduplicates the list of possible chain/network strings in RPC/parameter help texts, and it creates a macro `LIST_CHAIN_NAMES` in src/chainparamsbase.h. In the future, there is only 1 place where that list of possible values lives, so maintainability is improved and errors are avoided.
All three places where this change impacts:
```
./bitcoin-cli --help
./bitcoin-cli help getblockchaininfo
./bitcoin-cli help getmininginfo
```
They all return the correct string `"main, test, testnet4, signet, regtest"`
See https://github.com/bitcoin/bitcoin/pull/30642#discussion_r1714711575
ACKs for top commit:
maflcko:
lgtm ACK 9b297555207b4ea54bc0051f09c7084797aa9def
achow101:
ACK 9b297555207b4ea54bc0051f09c7084797aa9def
MarnixCroes:
ACK 9b297555207b4ea54bc0051f09c7084797aa9def
theStack:
ACK 9b297555207b4ea54bc0051f09c7084797aa9def
danielabrozzoni:
ACK 9b297555207b4ea54bc0051f09c7084797aa9def
Tree-SHA512: 1e961bcbe40b0f17a87a2437eb4ba1bb89468fd1b5a39599d72a00ef75cb4009e7d2f05d0a621bb904fecf681c55b8a219fcfe4d44d5d27f27cdda20882b1323
770b0348c0abe2f63b56cc78c7b7a0e6b4057ce1 qt: Update translation source file for v28.0 string freeze (Hennadii Stepanov)
Pull request description:
This PR updates the `src/qt/locale/bitcoin_en.xlf` translation source file according to the [Release schedule for 28.0](https://github.com/bitcoin/bitcoin/issues/29891).
Note for reviewers: it is expected to get a zero diff after running `make -C src translate` locally.
ACKs for top commit:
stickies-v:
re-ACK 770b0348c0abe2f63b56cc78c7b7a0e6b4057ce1
pablomartin4btc:
re-ACK 770b0348c0abe2f63b56cc78c7b7a0e6b4057ce1
Tree-SHA512: 11dd26c470411aefc2e4f897c605162027a00e2a0ab1dcec9a1784c053349a3feaeedda7b649476ff528231801629e0ef342a48430ef54a4ec75ac1548c56d4f
8f2522d242961ceb9e79672aa43e856863a1a6dd gui: Use menu for wallet migration (Ava Chow)
d56a450bf5172e2c3f4b9a2786e71268019e1277 gui: Use wallet name for wallet migration rather than WalletModel (Ava Chow)
c3918583dd5fcd9001136da2192e02e092128901 gui: don't remove wallet manually before migration (furszy)
bfba63880fbb1108b73540faeb0620ba24b8cdd0 gui: Consolidate wallet display name to GUIUtil function (Ava Chow)
28fc562f2692af4f37f918d4ae31c4d115e03aee wallet, interfaces: Include database format in listWalletDir (Ava Chow)
Pull request description:
Currently the Migrate Wallet menu item can only be used to migrate the currently loaded wallet. This is not suitable for the future when legacy wallets can no longer be loaded at all, but should still be able to be migrated. This PR changes that menu item into a menu list like Open Wallet and lets users migrate any legacy wallet in their wallet directory regardless of the wallets loaded.
One issue I ran into was dealing with encrypted wallets. Ideally, we would detect whether a wallet is encrypted, and prompt the user for their passphrase at that time. However, that's actually difficult to do in the GUI since migration will unload the wallet if it was already loaded, and reload it without connecting it to any signals or interfaces. Only then can it detect whether a wallet is encrypted, but then there is no `WalletModel` or even an `interfaces::Wallet` that the GUI could use to unlock it via a callback.
To deal with this, I've opted to just add a button to the migration dialog box that has the user enter their passphrase first, along with instructional text to use that button if their wallet was encrypted. If the user enters the wrong passphrase or clicked the other button that does not prompt for the passphrase, migration will fail with a message indicating that the passphrase was incorrect.
ACKs for top commit:
hebasto:
ACK 8f2522d242961ceb9e79672aa43e856863a1a6dd.
furszy:
ACK 8f2522d
Tree-SHA512: a0e3b70dbfcacb89617956510ebcea94cad8617a987c68fe39fa16ac1721190b7cf7afc156c39b9032920cfb67b5d4ca28791681f5021d92d16acc691387afa1
Once legacy wallets can no longer be loaded, we need to be able to
migrate them without loading. Thus we should use a menu that lists the
wallets in the wallet directory instead of an action which migrates the
currently loaded wallet.
In 6bfa26048dbafb91e9ca63ea8d3960271e798098 the testnet4 timewarp
attack fix block time variation was increased from the Great
Consensus Cleanup value of 600s to 7200s on the thesis that this
allows miners to always create blocks with the current time. Sadly,
doing so does allow for some nonzero inflation, even if not a huge
amount.
While it could be that some hardware ignores the timestamp provided
to it over Stratum and forces the block header timestamp to the
current time, I'm not aware of any such hardware, and it would also
likely suffer from random invalid blocks due to relying on NTP
anyway, making its existence highly unlikely.
This leaves the only concern being pools, but most of those rely on
work generated by Bitcoin Core (in one way or another, though when
spy mining possibly not), and it seems likely that they will also
not suffer any lost work. While its possible that a pool does
generate invalid work due to spy mining or otherwise custom logic,
it seems unlikely that a substantial portion of hashrate would do
so, making the difference somewhat academic (any pool that screws
this up will only do so once and the network would come out just
fine).
Further, while we may end up deciding these assumptions were
invalid and we should instead use 7200s, it seems prudent to try
with the value we "want" on testnet4, giving us the ability to
learn if the compatibility concerns are an issue before we go to
mainnet.
