e1d5dd732d5dc641faf1dde316275c84b6bb224b test: check xor.dat recreated when missing (tdb3)
d1610962bf1ff14df45c57cc1d2e075f71fcd19a test: add null block xor key (tdb3)
1ad999b9da39b60e16c51f9813f4fd39b7bdc2b9 refactor: lift NUM_XOR_BYTES (tdb3)
d8399584dd59b3954a0bea393b2de350a732055e refactor: move read_xor_key() to TestNode (tdb3)
d43948c3ef610c383176bf9b389697973bd0ad64 refactor: use unlink rather than os.remove (tdb3)
c8176f758b5991c3797c32ee519d32c14b444991 test: add blocks_key_path (tdb3)
Pull request description:
Builds on PR #30657.
Refactors `read_xor_key()` from `util.py` to `test_node.py` (comment https://github.com/bitcoin/bitcoin/pull/30657#discussion_r1723358327)
Adds a check that `xor.dat` is created when missing (comment https://github.com/bitcoin/bitcoin/pull/30657#discussion_r1717724161)
Help states:
```
-blocksxor
Whether an XOR-key applies to blocksdir *.dat files. The created XOR-key
will be zeros for an existing blocksdir or when `-blocksxor=0` is
set, and random for a freshly initialized blocksdir. (default: 1)
```
ACKs for top commit:
maflcko:
ACK e1d5dd732d5dc641faf1dde316275c84b6bb224b
achow101:
ACK e1d5dd732d5dc641faf1dde316275c84b6bb224b
theStack:
re-ACK e1d5dd732d5dc641faf1dde316275c84b6bb224b
brunoerg:
reACK e1d5dd732d5dc641faf1dde316275c84b6bb224b
Tree-SHA512: 325912ef646ec88e0a58e1ece263a2b04cbc06497e8fe5fcd603e509e80c6bcf84b09dd52dfac60e23013f07fc2b2f6db851ed0598649c3593f452c4a1424bd9
fa5aeab3cb18405ecf8a1401d89539b924a618f6 test: Avoid duplicate curl call in get_previous_releases.py (MarcoFalke)
Pull request description:
Seems odd having to translate `404` to "Binary tag was not found". Also, it seems odd to write a for-loop over a list with one item.
Fix both issues by just using a single call to `curl --fail ...`.
Can be tested with: `test/get_previous_releases.py -b v99.99.99`
Before:
```
Releases directory: releases
Fetching: https://bitcoincore.org/bin/bitcoin-core-99.99.99/bitcoin-99.99.99-x86_64-linux-gnu.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 286k 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
Binary tag was not found
```
After:
```
Releases directory: releases
Fetching: https://bitcoincore.org/bin/bitcoin-core-99.99.99/bitcoin-99.99.99-x86_64-linux-gnu.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 286k 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
curl: (22) The requested URL returned error: 404
ACKs for top commit:
fanquake:
ACK fa5aeab3cb18405ecf8a1401d89539b924a618f6
brunoerg:
utACK fa5aeab3cb18405ecf8a1401d89539b924a618f6
tdb3:
tested ACK fa5aeab3cb18405ecf8a1401d89539b924a618f6
Tree-SHA512: d5d31e0bccdd9de9b4a8ecf2e69348f4e8cee773050c8259b61db1ce5de73f6fbfffbe8c4d2571f7bef2de29cb42fd244573deebfbec614e487e76ef41681b9c
Removes dependency on unsafe and deprecated uint256S.
This makes parsing more strict, by returning an error
when the input contains non-hex characters, or when it
contains more than 64 hex digits.
Also make feature_assumevalid.py more robust by using CBlock.hash
which is guaranteed to be 64 characters long, as opposed to the
variable-length hex(CBlock.sha256)
Removes dependency on unsafe and deprecated uint256S.
This makes parsing more strict, by returning an error
when the input contains more than 64 hex digits.
cccc5bfd35a008adf08d99ed463fe00d6a6f29c0 test: Enable detect_leaks=1 in ASAN_OPTIONS explicitly (MarcoFalke)
Pull request description:
It should be enabled by default, but being explicit can't hurt.
ACKs for top commit:
fanquake:
ACK cccc5bfd35a008adf08d99ed463fe00d6a6f29c0
Tree-SHA512: ed284abd05c7a99c30b509844aa75785a5ccb506d8296a71347b4c328750a6a4ed1f87e7a3ec36ab17f27b467c033cc8ca5eb5e2b951f2ae7473327c5eb1ddae
59ff17e5af4e382cbe16f183767beef1bdcd9131 miner: adjust clock to timewarp rule (Sjors Provoost)
e929054e12210353812f440c685a23329e7040f7 Add timewarp attack mitigation test (Sjors Provoost)
e85f386c4b157b7d1ac16aface9bd2c614e62b46 consensus: enable BIP94 on regtest (Sjors Provoost)
dd154b05689c60fad45df0df6d31cec12e09ab21 consensus: lower regtest nPowTargetTimespan to 144 (Sjors Provoost)
Pull request description:
Because #30647 reduced the timewarp attack threshold from 7200s to 600s, our miner code will fail to propose a block template (on testnet4) if the last block of the previous period has a timestamp two hours in the future. This PR fixes that and also adds a test.
The non-test changes in the last commit should be in v28, otherwise miners have to patch it themselves. If necessary I can split that out into a separate PR, but I prefer to get the tests in as well.
In order to add the test, we activate BIP94 on regtest.
In order for the test to run faster, we reduce its difficulty retarget period to 144, the same number that's already used for softfork activation logic. Regtest does not actually adjust its difficulty, so this change has no effect (except for `getnetworkhashps`, see commit).
An alternative approach would be to run this test on testnet4, by hardcoding its first 2015 in the test suite. But since the timewarp mitigation is a serious candidate for a future mainnet softfork, it seems better to just deploy it on regtest.
The next commits add a test and fix the miner code.
The `MAX_TIMEWARP` constant is moved to `consensus.h` so both validation and miner code have access to it.
ACKs for top commit:
achow101:
ACK 59ff17e5af4e382cbe16f183767beef1bdcd9131
fjahr:
ACK 59ff17e5af4e382cbe16f183767beef1bdcd9131
glozow:
ACK 59ff17e5af4e382cbe16f183767beef1bdcd9131
Tree-SHA512: 50af9fdcba9b0d5c57e1efd5feffd870bd11b5318f1f8b0aabf684657f2d33ab108d5f00b1475fe0d38e8e0badc97249ef8dda20c7f47fcc1698bc1008798830
917e70a6206c62c4c492fa922425fc8e00d3f328 test: assumeutxo: check that UTXO-querying RPCs operate on snapshot chainstate (Sebastian Falbesoner)
Pull request description:
Inspired by some manual testing I did for #28553, this PR checks that RPCs which explicitly query the UTXO set database (i.e. `gettxoutsetinfo`, `scantxoutset` and `gettxout`) operate on the snapshot chainstate as expected.
ACKs for top commit:
fjahr:
utACK 917e70a6206c62c4c492fa922425fc8e00d3f328
achow101:
ACK 917e70a6206c62c4c492fa922425fc8e00d3f328
tdb3:
ACK 917e70a6206c62c4c492fa922425fc8e00d3f328
Tree-SHA512: 40ecd1c5dd879234df1667fa5444a1fbbee9b7c456f597dc982d1a2bce46fe9107711b005ab829e570ef919a4914792f72f342d71d92bad2ae9434b5e68d5bd3
This currently has no effect due to fPowNoRetargeting,
except for the getnetworkhashps when called with -1.
It will when the next commit enforces the timewarp attack mitigation on regtest.
Previous rpcauth behavior was to sometimes
ignore empty -rpcauth= settings, and other times
treat them as errors.
Empty rpcauth is now consistently treated
as an error and prevents bitcoind from starting.
Updates associated test cases.
Also updates to non-deprecated logging macro.
Co-Authored-By: Luke Dashjr <luke-jr+git@utopios.org>
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
77ff0ec1f185b818b30877de2bedc1750319e6c4 contrib: support reading XORed blocks in linearize-data.py script (Sebastian Falbesoner)
Pull request description:
This PR is a small follow-up for #28052, adding support for the block linearization script to handle XORed blocksdir *.dat files. Note that if no xor.dat file exists, the XOR pattern is set to all-zeros, in order to still support blockdirs that have been created with versions earlier than 28.x.
Partly fixes issue #30599.
ACKs for top commit:
achow101:
ACK 77ff0ec1f185b818b30877de2bedc1750319e6c4
tdb3:
ACK 77ff0ec1f185b818b30877de2bedc1750319e6c4
hodlinator:
ACK 77ff0ec1f185b818b30877de2bedc1750319e6c4
Tree-SHA512: 011eb02e2411de373cbbf4b26db4640fc693a20be8c2430529fba6e36a3a3abfdfdc3b005d330f9ec2846bfad9bfbf34231c574ba99289ef37dd51a68e6e7f3d
6b2dcba07670f04f32c0dc3a2c86fd805c85f12d wallet: List sqlite wallets with empty string name (Ava Chow)
3ddbdd1815c676a88345b3b0e55a551d2a569e28 wallet: Ignore .bak files when listing wallet files (Ava Chow)
Pull request description:
When the default wallet is migrated, we do not rename the wallet so we end up having a descriptor wallet with the empty string as its name and the wallet.dat file in the root of the walletdir. This is supposed to be an unsupported configuration and there is no other way to achieve this (other than file copying), but the wallet loading code does not disallow loading such wallets. However `listwalletdir` does not currently list the default wallet if it is sqlite. This is confusing to users, so change `listwalletdir` to include these wallets.
Additionally, the migration of the default wallet, and of any plain wallet files in the walletdir, produces a backup file in the walletdir itself. Since these backups are a BDB file, `listwalletdir` will detect them as being another wallet that we could open, but this is erroneous and could lead to confusion and potentially funds loss if both the backup and the migrated wallet are in use simultaneously. To reduce the likelihood of this issue, don't list these wallets in `listwalletdir`.
***
Possibly we could have more stringent checks on loading to resolve these issues, but I'm concerned that that will just confuse users and gratuitously break things that already worked.
Since the original intent was to disallow default wallets for sqlite/descriptors, a possible alternative would be to prevent people from loading such wallets and change migration to rename those wallets. However, given that this behavior with migrating default wallets has existed since default wallet migration was fixed, I think that making such a change would be confusing and break things for no good reason. Although perhaps we should still do the renaming.
For the backups, we could also change loading to refuse to load any wallet named with `.bak` (or `.legacy.bak`) as such wallets can still be loaded by giving the path to them directly, which some users may do to "restore" the backup. However restricting what can be loaded based on filename seems a little heavyhanded. It wouldn't be funds loss though since the correct way to restore the backup is with `restorewallet`.
ACKs for top commit:
fjahr:
Code review ACK 6b2dcba07670f04f32c0dc3a2c86fd805c85f12d
furszy:
Code ACK 6b2dcba076
glozow:
ACK 6b2dcba07670f04f32c0dc3a2c86fd805c85f12d
Tree-SHA512: 0b033f6ed55830f8a054afea3fb2cf1fa82a94040053ebfaf123bda36c99f45d3f01a2aec4ed02fed9c61bb3d320b047ed892d7f6644b5a356a7bc5974b10cff
00618e8745192d209c23e3ae873c077e58168957 assumeutxo: Drop block height from metadata (Fabian Jahr)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/30514 which has more context and shows how the issue can be reproduced. Since the value in question is removed, there is no test to add to reproduce anything.
This is an alternative approach to #30516 with much of the [code being suggested there](https://github.com/bitcoin/bitcoin/pull/30516#discussion_r1689146902).
ACKs for top commit:
maflcko:
re-ACK 00618e8745192d209c23e3ae873c077e58168957 🎌
achow101:
ACK 00618e8745192d209c23e3ae873c077e58168957
theStack:
Code-review ACK 00618e8745192d209c23e3ae873c077e58168957
ismaelsadeeq:
Re-ACK 00618e8745192d209c23e3ae873c077e58168957
mzumsande:
ACK 00618e8745192d209c23e3ae873c077e58168957
Tree-SHA512: db9575247bae838ad7742a27a216faaf55bb11e022f9afdd05752bb09bbf9614717d0ad64304ff5722a16bf41d8dea888af544e4ae26dcaa528c1add0269a4a8
Although it is not explicitly possible to create a default wallet with
descriptors, it is possible to migrate a default wallet and have it end
up being a default wallet with descriptors. These wallets should be
listed by ListDatabases so that it appears in wallet directory listings
to avoid user confusion.
Migration creates backup files in the wallet directory with .bak as the
extension. This pollutes the output of listwalletdir with backup files
that most users should not need to care about.
6bfa26048dbafb91e9ca63ea8d3960271e798098 testnet: Add timewarp attack prevention for Testnet4 (Fabian Jahr)
0100907ca168c53e8fe044bdda396f308825162c testnet: Add Testnet4 difficulty adjustment rules fix (Fabian Jahr)
74a04f9e7ad6a16988149cc3438b9ce13c91cdb9 testnet: Introduce Testnet4 (Fabian Jahr)
Pull request description:
To supplement the [ongoing conceptual discussion about a testnet reset](https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/9yCPo3uUBwAJ) I have drafted a move to v4 including a fix to the difficulty adjustment mechanism, which was part of the motivation that started the discussion.
Conceptual considerations:
- The conceptual discussion about doing a testnet4 or softforking the fix into testnet3 is outside of the scope of this PR and I would ask reviewers to contribute their opinions on this on the ML instead. However, I am happy to adapt this PR to a softfork change on testnet3 if there is consensus for that instead.
- The difficulty adjustment fix suggested here touches the `CalculateNextWorkRequired` function and uses the same logic used in `GetNextWorkRequired` to find the last previous block that was not mined with difficulty 1 under the exceptionf. An alternative fix briefly mentioned on the mailing list by Jameson Lopp would be to "restrict the special testnet minimum difficulty rule so that it can't be triggered on the block right before a difficulty retarget". That would also fix the issue but I find my suggestion here a bit more elegant.
ACKs for top commit:
jsarenik:
tACK 6bfa26048dba
achow101:
ACK 6bfa26048dbafb91e9ca63ea8d3960271e798098
murchandamus:
tACK 6bfa26048dbafb91e9ca63ea8d3960271e798098
Tree-SHA512: 0b8b69a621406a944da5be551b863d065358ba94d85dd3b80d83c412660e230ee93b27316081fbee9b4851cc4ff8585db64c7dfa26cb5148ac835663f2712c3d
Refactor satoshi_round function to accept different rounding modes.
Updated call site to use the revised `satoshi_round` function.
Co-authored-by: Kate Salazar <52637275+katesalazar@users.noreply.github.com>
e9de0a76b99fd4708295e1178f6c079db92e7f36 doc: release note for 30212 (willcl-ark)
87b188052528e97729a85d9701864eaff1ea5ec6 rpc: clarify ALREADY_IN_CHAIN rpc errors (willcl-ark)
Pull request description:
Closes: #19363
Renaming this error improves clarity around the returned error both internally and externally when a transactions' outputs are already found in the utxo set (`TransactionError::ALREADY_IN_CHAIN -> TransactionError::ALREADY_IN_UTXO_SET`)
ACKs for top commit:
tdb3:
ACK e9de0a76b99fd4708295e1178f6c079db92e7f36
ismaelsadeeq:
ACK e9de0a76b99fd4708295e1178f6c079db92e7f36
ryanofsky:
Code review ACK e9de0a76b99fd4708295e1178f6c079db92e7f36.
Tree-SHA512: 7d2617200909790340951fe56a241448f9ce511900777cb2a712e8b9c0778a27d1f912b460f82335844224f1abb4322bc898ca076440959edade55c082a09237
fa895c72832f9555b52d5bb1dba1093f73de3136 mingw: Document mode wbx workaround (MarcoFalke)
fa359255fe6b4de5f26784bfc147dbfb58bef116 Add -blocksxor boolean option (MarcoFalke)
fa7f7ac040a9467c307b20e77dc47c87d7377ded Return XOR AutoFile from BlockManager::Open*File() (MarcoFalke)
Pull request description:
Currently the *.dat files in the blocksdir store the data received from remote peers as-is. This may be problematic when a program other than Bitcoin Core tries to interpret them by accident. For example, an anti-virus program or other program may scan them and move them into quarantine, or delete them, or corrupt them. This may cause Bitcoin Core to fail a reorg, or fail to reply to block requests (via P2P, RPC, REST, ...).
Fix this, similar to https://github.com/bitcoin/bitcoin/pull/6650, by rolling a random XOR pattern over the dat files when writing or reading them.
Obviously this can only protect against programs that accidentally and unintentionally are trying to mess with the dat files. Any program that intentionally wants to mess with the dat files can still trivially do so.
The XOR pattern is only applied when the blocksdir is freshly created, and there is an option to disable it (on creation), so that people can disable it, if needed.
ACKs for top commit:
achow101:
ACK fa895c72832f9555b52d5bb1dba1093f73de3136
TheCharlatan:
Re-ACK fa895c72832f9555b52d5bb1dba1093f73de3136
hodlinator:
ACK fa895c72832f9555b52d5bb1dba1093f73de3136
Tree-SHA512: c92a6a717da83bc33a9b8671a779eeefde2c63b192362ba1d71e6535ee31d08e2802b74acc908345197de9daac6930e4771595ee25b09acd5a67f7ea34854720
