8fee5355ee1d2f2b410c548bac9e85f2a21a696d guix: fix suggested fake date for openssl -1.1.1l (Sjors Provoost)
Pull request description:
Using `2020-10-01` as the fake timestamp will cause many test failures with `/gnu/store/bfirgq65ndhf63nn4q6vlkbha9zd931q-openssl-1.1.1l.drv`. I didn't investigate why, but I guess because it's _before_ the test certificates were created. They expired in June 2022. I tried a month before that, which worked.
Also fixes layout of instructions.
ACKs for top commit:
achow101:
ACK 8fee5355ee1d2f2b410c548bac9e85f2a21a696d
maflcko:
review ACK 8fee5355ee1d2f2b410c548bac9e85f2a21a696d
Tree-SHA512: df5dd3aa961e25bd57d0b8b73daeb3ec76856b06e35277f24b6b19be81774512228f75e2b779afa8ea92fcc39beb869f43e0c57fba19ad16a82812e7c0bea38b
77ff0ec1f185b818b30877de2bedc1750319e6c4 contrib: support reading XORed blocks in linearize-data.py script (Sebastian Falbesoner)
Pull request description:
This PR is a small follow-up for #28052, adding support for the block linearization script to handle XORed blocksdir *.dat files. Note that if no xor.dat file exists, the XOR pattern is set to all-zeros, in order to still support blockdirs that have been created with versions earlier than 28.x.
Partly fixes issue #30599.
ACKs for top commit:
achow101:
ACK 77ff0ec1f185b818b30877de2bedc1750319e6c4
tdb3:
ACK 77ff0ec1f185b818b30877de2bedc1750319e6c4
hodlinator:
ACK 77ff0ec1f185b818b30877de2bedc1750319e6c4
Tree-SHA512: 011eb02e2411de373cbbf4b26db4640fc693a20be8c2430529fba6e36a3a3abfdfdc3b005d330f9ec2846bfad9bfbf34231c574ba99289ef37dd51a68e6e7f3d
401cc4ec70d67ba2aa0e078d2fab214e1c40742c fuzz: improve scriptpubkeyman target (brunoerg)
Pull request description:
Fixes#30541
This PR aims to improve `scriptpubkeyman` target to avoid timeouts. The input provided in #30541 takes too much time to run because it basically calls only `MarkUnusedAddresses` (300 times * number of spks). The following changes were made to improve it:
- Reduce keypool size.
- When calling `MarkUnusedAddresses`, do it with one of the spks per iteration.
- Remove the specific `AddDescriptorKey` call since it is already covered with `AddWalletDescriptor`.
- Limit number of iterations to a reasonable value.
ACKs for top commit:
maflcko:
lgtm ACK 401cc4ec70d67ba2aa0e078d2fab214e1c40742c
achow101:
ACK 401cc4ec70d67ba2aa0e078d2fab214e1c40742c
Tree-SHA512: 941812bc6d991dd03675a2974ce1b839494ca7f6e6d8a22c689d4bf4fed2dac5491246998f19cb15dbff516fdd8eeda27e7628c3206d45f57dc292bc05624a5c
15aa7d023688700a47997b92108de95f2d864f5a gui, qt: brintToFront workaround for Wayland (pablomartin4btc)
Pull request description:
There are known issues around handling windows focus in `Wayland` ([this one specific](https://bugs.kde.org/show_bug.cgi?id=462574) in KDE but also in [gnome](https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/730)).
The idea is that the workaround will be executed if `bitcoin-qt` is running using `Wayland` platform (e.g.: `QT_QPA_PLATFORM=wayland ./src/qt/bitcoin-qt -regtest`), since the workaround behaviour looks like re-opening the window again (which I tried to fix by moving the window to the original position and/ or re-setting the original geometry without success) while in `X11` (not sure in Mac) the current `GUIUtil::brintToFront` actually sets the focus to the desired window, keeping its original position as expected, and I didn't want to change that (`X11` behaviour).
The solution was [initially discussed](https://github.com/bitcoin-core/gui/pull/817#issuecomment-2256158902) with hebasto in #817.
ACKs for top commit:
hebasto:
ACK 15aa7d023688700a47997b92108de95f2d864f5a.
Tree-SHA512: 141d6cc4a618026e551627b9f4cc284285980db02a54a7b19c7de91e8c5adccf0c1d67380625146b5413e58c59f39c9e944ed5ba68cb8644f67647518918b6f7
6b2dcba07670f04f32c0dc3a2c86fd805c85f12d wallet: List sqlite wallets with empty string name (Ava Chow)
3ddbdd1815c676a88345b3b0e55a551d2a569e28 wallet: Ignore .bak files when listing wallet files (Ava Chow)
Pull request description:
When the default wallet is migrated, we do not rename the wallet so we end up having a descriptor wallet with the empty string as its name and the wallet.dat file in the root of the walletdir. This is supposed to be an unsupported configuration and there is no other way to achieve this (other than file copying), but the wallet loading code does not disallow loading such wallets. However `listwalletdir` does not currently list the default wallet if it is sqlite. This is confusing to users, so change `listwalletdir` to include these wallets.
Additionally, the migration of the default wallet, and of any plain wallet files in the walletdir, produces a backup file in the walletdir itself. Since these backups are a BDB file, `listwalletdir` will detect them as being another wallet that we could open, but this is erroneous and could lead to confusion and potentially funds loss if both the backup and the migrated wallet are in use simultaneously. To reduce the likelihood of this issue, don't list these wallets in `listwalletdir`.
***
Possibly we could have more stringent checks on loading to resolve these issues, but I'm concerned that that will just confuse users and gratuitously break things that already worked.
Since the original intent was to disallow default wallets for sqlite/descriptors, a possible alternative would be to prevent people from loading such wallets and change migration to rename those wallets. However, given that this behavior with migrating default wallets has existed since default wallet migration was fixed, I think that making such a change would be confusing and break things for no good reason. Although perhaps we should still do the renaming.
For the backups, we could also change loading to refuse to load any wallet named with `.bak` (or `.legacy.bak`) as such wallets can still be loaded by giving the path to them directly, which some users may do to "restore" the backup. However restricting what can be loaded based on filename seems a little heavyhanded. It wouldn't be funds loss though since the correct way to restore the backup is with `restorewallet`.
ACKs for top commit:
fjahr:
Code review ACK 6b2dcba07670f04f32c0dc3a2c86fd805c85f12d
furszy:
Code ACK 6b2dcba076
glozow:
ACK 6b2dcba07670f04f32c0dc3a2c86fd805c85f12d
Tree-SHA512: 0b033f6ed55830f8a054afea3fb2cf1fa82a94040053ebfaf123bda36c99f45d3f01a2aec4ed02fed9c61bb3d320b047ed892d7f6644b5a356a7bc5974b10cff
055bc05792ff5d5b084563044818ebec12bfd748 policy/feerate.h: avoid constraint self-dependency (Matt Whitlock)
138f8671569f7ebb8c84e9d80c44cddeda9e3845 add missing #include <cstdint> for GCC 15 (Matt Whitlock)
Pull request description:
#30612 with changes made.
GCC 15 introduces three build failures:
* Two are related to missing includes. You can't use `uint16_t` et al. without including `<cstdint>`.
* The third is harder to understand but easy to fix. GCC changed something about the way templates are instantiated when checking type constraints, and now there is a dependency loop while checking `std::optional<CFeeRate>`. This manifests as the following compile-time mess:
```
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/format:48,
from /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/chrono_io.h:39,
from /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/chrono:3362,
from ./util/time.h:9,
from ./primitives/block.h:12,
from ./blockencodings.h:8,
from blockencodings.cpp:5:
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits: In substitution of 'template<class _Up> requires !(is_same_v<std::optional<_Tp>, typename std::remove_cvref<_It2>::type>) && (is_constructible_v<_Tp, const _Up&>) && (__construct_from_contained_value<_Up, typename std::remove_cv< <template-parameter-1-1> >::type>) constexpr std::optional<CFeeRate>::optional(const std::optional<_Tp>&) [with _Up = CFeeRate]':
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits:1140:25: required by substitution of 'template<class _Tp, class ... _Args> using std::__is_constructible_impl = std::__bool_constant<__is_constructible(_Tp, _Args ...)> [with _Tp = CFeeRate; _Args = {std::optional<CFeeRate>&}]'
1140 | = __bool_constant<__is_constructible(_Tp, _Args...)>;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits:1145:12: required from 'struct std::is_constructible<CFeeRate, std::optional<CFeeRate>&>'
1145 | struct is_constructible
| ^~~~~~~~~~~~~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits:178:35: required by substitution of 'template<class ... _Bn> std::__detail::__first_t<std::integral_constant<bool, false>, typename std::enable_if<(!(bool)(_Bn::value)), void>::type ...> std::__detail::__or_fn(int) [with _Bn = {std::is_constructible<CFeeRate, std::optional<CFeeRate>&>, std::is_convertible<std::optional<CFeeRate>&, CFeeRate>, std::is_constructible<CFeeRate, std::optional<CFeeRate> >, std::is_convertible<std::optional<CFeeRate>, CFeeRate>, std::is_constructible<CFeeRate, const std::optional<CFeeRate>&>, std::is_convertible<const std::optional<CFeeRate>&, CFeeRate>, std::is_constructible<CFeeRate, const std::optional<CFeeRate> >, std::is_convertible<const std::optional<CFeeRate>, CFeeRate>}]'
178 | __enable_if_t<!bool(_Bn::value)>...>;
| ^~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/type_traits:196:41: required from 'struct std::__or_<std::is_constructible<CFeeRate, std::optional<CFeeRate>&>, std::is_convertible<std::optional<CFeeRate>&, CFeeRate>, std::is_constructible<CFeeRate, std::optional<CFeeRate> >, std::is_convertible<std::optional<CFeeRate>, CFeeRate>, std::is_constructible<CFeeRate, const std::optional<CFeeRate>&>, std::is_convertible<const std::optional<CFeeRate>&, CFeeRate>, std::is_constructible<CFeeRate, const std::optional<CFeeRate> >, std::is_convertible<const std::optional<CFeeRate>, CFeeRate> >'
196 | : decltype(__detail::__or_fn<_Bn...>(0))
| ~~~~~~~~~~~~~~~~~~~~~~~~~^~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/optional:824:45: required from 'constexpr const bool std::optional<CFeeRate>::__construct_from_contained_value<CFeeRate, CFeeRate>'
824 | = !__converts_from_optional<_Tp, _From>::value;
| ^~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/optional:884:7: required by substitution of 'template<class _Up> requires !(is_same_v<std::optional<_Tp>, typename std::remove_cvref<_It2>::type>) && (is_constructible_v<_Tp, const _Up&>) && (__construct_from_contained_value<_Up, typename std::remove_cv< <template-parameter-1-1> >::type>) constexpr std::optional<CFeeRate>::optional(const std::optional<_Tp>&) [with _Up = CFeeRate]'
884 | && __construct_from_contained_value<_Up>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./validation.h:164:41: required from here
164 | return MempoolAcceptResult(state);
| ^
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/optional:886:2: required by the constraints of 'template<class _Tp> template<class _Up> requires !(is_same_v<std::optional<_Tp>, typename std::remove_cvref<_It2>::type>) && (is_constructible_v<_Tp, const _Up&>) && (__construct_from_contained_value<_Up, typename std::remove_cv< <template-parameter-1-1> >::type>) constexpr std::optional<_Tp>::optional(const std::optional<_From>&)'
/usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/optional:884:14: error: satisfaction of atomic constraint '__construct_from_contained_value<_Up, typename std::remove_cv< <template-parameter-1-1> >::type> [with _Tp = _Tp; _Up = _Up]' depends on itself
884 | && __construct_from_contained_value<_Up>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
It is easiest to solve this by changing the `static_assert` in the explicit `CFeeRate` constructor to a SFINAE by using a type constraint on the function template parameter.
We already [downstreamed](https://github.com/gentoo/gentoo/pull/38015) these fixes in Gentoo.
ACKs for top commit:
stickies-v:
ACK 055bc05792ff5d5b084563044818ebec12bfd748
Tree-SHA512: ce9cb27bcd9b0f4bbc80951e45cf7127112dcb7f9937bcb0167b362026d35beecb1255354746de0aac82e03c41eaccbe26acbfe0ddff2ee1e5a8634673f4f4ba
ed83974bb411ab5ebe3eef28f0ac995ce07936cd doc: Add note about distro's g++-mingw-w64-x86-64-posix version (Hennadii Stepanov)
Pull request description:
This PR stems from a requirement for the `g++` minimum supported version [being >= 11](https://github.com/bitcoin/bitcoin/pull/29091):
- https://packages.ubuntu.com/noble/g++-mingw-w64-x86-64-posix
- https://packages.debian.org/bookworm/g++-mingw-w64-x86-64-posix
ACKs for top commit:
m3dwards:
ACK ed83974bb411ab5ebe3eef28f0ac995ce07936cd
maflcko:
review-only ACK ed83974bb411ab5ebe3eef28f0ac995ce07936cd
Tree-SHA512: a4c4d5239df5540b2dac922c2834c51337ec67310d596620bea02fe54334917e83da8954107c6a3fdd08143d1eac6a2cd4b0520ad023d7ccad8bcbdc4e03a7c0
4c2cc63d84dc1c8bae1b620f34188b7c20b70019 Revert "depends: Fetch miniupnpc sources from an alternative website" (sunerok)
Pull request description:
This reverts commit 21b8a14d37c19ce292d5529597e0d52338db48a9.
started fresh fanquake
ACKs for top commit:
fanquake:
ACK 4c2cc63d84dc1c8bae1b620f34188b7c20b70019 - please don't @ mention anyone in your pull request descriptions, otherwise they are likely to just get spammed with notifications.
Tree-SHA512: ab3e6cc65e4531b6c129a6bc663ebc9859e0de1d4b745b29834344cc8aa83ea182b0a5d1fae14e609a85bbbe24c34f42502fb25b53a15b38dd63396913ea4682
032ebe5be4dfc3eb3d3693f1284fac6458bacbf3 ci: Silent Homebrew's reinstall warnings (Hennadii Stepanov)
Pull request description:
Homebrew's warnings are quite noisy on the master branch:
This PR silents them to allow us to focus on other CI infra warnings once they happen.
Similar to https://github.com/bitcoin-core/secp256k1/pull/1578.
ACKs for top commit:
willcl-ark:
utACK 032ebe5be4dfc3eb3d3693f1284fac6458bacbf3
tdb3:
cr ut ACK 032ebe5be4dfc3eb3d3693f1284fac6458bacbf3
Tree-SHA512: ddcdb689b96d07186cea98b76b49bbca47c621a0784d4ef5487a3da20f9c29dad77da720dd9b7d25ed286b94811969d52ebe9c260632eeee0e9f2f335021a404
16d82611812de4e91e7950fe6d31484cc7a9c937 doc: Drop no longer needed workaround for WSL (Hennadii Stepanov)
Pull request description:
This PR effectively reverts commit 4f890ba6bc8caba5394c7a5388d7f07959ced78b from https://github.com/bitcoin/bitcoin/pull/11437, which fixed some build issues on WSL seven years ago.
Testing the current master branch @ 31a3ff55154bf15fb35b157c3f67ec05408ecdf9 on Windows 11 + WSL using Ubuntu 24.04 or Debian images, I noticed that the workaround is no longer required. Moreover, it doesn't affect the build process at all, which means the hashes of the built packages in depends remain the same and the `configure` log in the main build system remains the same as well.
ACKs for top commit:
sipsorcery:
utACK 16d82611812de4e91e7950fe6d31484cc7a9c937.
Tree-SHA512: 703a2ac4647125c91aad47131e2723fd05af30b0cfae5677a26d3e89a77c2779e0197584208f3b378ed64dd7305512d9064fd073ec06517f86e9905af4ec8838
fa5755b0a8536b844fdccfecf386c1baab24f1c9 doc: rpc: Use "output script" consistently (2/2) (MarcoFalke)
Pull request description:
Small follow-up to https://github.com/bitcoin/bitcoin/pull/30408 to fixup the RPCs that were forgotten.
ACKs for top commit:
theStack:
lgtm ACK fa5755b0a8536b844fdccfecf386c1baab24f1c9
Tree-SHA512: f1fc0aabb59017da216d6fe0f08a2274336d04db332ad6ce3d9608cd6f03667be1c76423f24a489ac8e7d536011a129dca752ab64b4621b7bc1d4d53f68602e4
49d569cb1fdd62a9da8dff51dccaf4680fe3d0eb p2p: For assumeutxo, download snapshot chain before background chain (Martin Zumsande)
7a885518d57c6eb818ebef5fd04a575f324ee8b2 p2p: Restrict downloading of blocks for snapshot chain (Martin Zumsande)
Pull request description:
After loading a snapshot, `pindexLastCommonBlock` is usually already set to some block for existing peers. That means we'd continue syncing the background chain from those peers instead of prioritising the snapshot chain, which defeats the purpose of doing assumeutxo in the first place. Only existing peers are affected by this bug.
ACKs for top commit:
fjahr:
re-ACK 49d569cb1fdd62a9da8dff51dccaf4680fe3d0eb
achow101:
ACK 49d569cb1fdd62a9da8dff51dccaf4680fe3d0eb
Sjors:
tACK 49d569cb1fdd62a9da8dff51dccaf4680fe3d0eb
Tree-SHA512: 0eaebe1c29a8510d5ced57e14c09b128ccb34b491692815291df68bf12e2a15b52b1e7bf8d9f34808904e7f7bc20f70b0ad0f7e14df93bbdf456bd12cc02a5d2
00618e8745192d209c23e3ae873c077e58168957 assumeutxo: Drop block height from metadata (Fabian Jahr)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/30514 which has more context and shows how the issue can be reproduced. Since the value in question is removed, there is no test to add to reproduce anything.
This is an alternative approach to #30516 with much of the [code being suggested there](https://github.com/bitcoin/bitcoin/pull/30516#discussion_r1689146902).
ACKs for top commit:
maflcko:
re-ACK 00618e8745192d209c23e3ae873c077e58168957 🎌
achow101:
ACK 00618e8745192d209c23e3ae873c077e58168957
theStack:
Code-review ACK 00618e8745192d209c23e3ae873c077e58168957
ismaelsadeeq:
Re-ACK 00618e8745192d209c23e3ae873c077e58168957
mzumsande:
ACK 00618e8745192d209c23e3ae873c077e58168957
Tree-SHA512: db9575247bae838ad7742a27a216faaf55bb11e022f9afdd05752bb09bbf9614717d0ad64304ff5722a16bf41d8dea888af544e4ae26dcaa528c1add0269a4a8
Although it is not explicitly possible to create a default wallet with
descriptors, it is possible to migrate a default wallet and have it end
up being a default wallet with descriptors. These wallets should be
listed by ListDatabases so that it appears in wallet directory listings
to avoid user confusion.
Migration creates backup files in the wallet directory with .bak as the
extension. This pollutes the output of listwalletdir with backup files
that most users should not need to care about.
92c1d7d1f8664fe2d259cc609c87f603609b2b60 validation: Use MAX_TIMEWARP constant as testnet4 timewarp defense delta (Fabian Jahr)
4b2fad502e5c264012e94d2915476f4dfcbd192d doc: Add release notes for 29775 (Fabian Jahr)
f7cc97313b91483ed4fed298919ecb16054931ee doc: Align deprecation warnings (Fabian Jahr)
1163b08378a50a9be00ced434d55f1b04bc9dea6 chainparams: Add initial minimum chain work for Testnet4 (Fabian Jahr)
Pull request description:
This completes follow-ups left open in #29775.
- Adds release notes
- Addresses the [misalignment](https://github.com/bitcoin/bitcoin/pull/29775#discussion_r1706982102) in deprecation warnings and hints at the intention to remove support for Testnet3.
- Adds initial minimum chainwork for Testnet4.
- Use the `MAX_TIMEWARP` constant as the timewarp defense delta, equal to `MAX_FUTURE_BLOCK_TIME`.
ACKs for top commit:
Sjors:
ACK 92c1d7d1f8664fe2d259cc609c87f603609b2b60
achow101:
ACK 92c1d7d1f8664fe2d259cc609c87f603609b2b60
tdb3:
re ACK 92c1d7d1f8664fe2d259cc609c87f603609b2b60
Tree-SHA512: 7ebdac7809f96231f75ca62706af59cd1ed27f713a4c7be5e2ad69fae95832b146b3ea23c712fb03b412da1deda7e8a5dae55bb2bbd2dcfd9f926e85c2a72666
2925bd537cbd8c70594e23f6c4298b7101f7f73d refactor: use c++20 std::views::reverse instead of reverse_iterator.h (stickies-v)
Pull request description:
C++20 introduces [`std::ranges::views::reverse`](https://en.cppreference.com/w/cpp/ranges/reverse_view), which allows us to drop our own `reverse_iterator.h` implementation and also makes it easier to chain views (even though I think we currently don't use this).
ACKs for top commit:
achow101:
ACK 2925bd537cbd8c70594e23f6c4298b7101f7f73d
maflcko:
ACK 2925bd537cbd8c70594e23f6c4298b7101f7f73d 🎷
Tree-SHA512: 567666ec44af5d1beb7a271836bcc89c4c577abc77f522fcc18bc6d4de516ae9b0df766d0bfa6dd217569e6878331c2aee1d9815620860375e3510dad7fed476
Enhanced efficiency and readability of CCoinsViewCache::FetchCoin by replacing separate find() and emplace() calls with a single try_emplace(), reducing map lookups and potential insertions.
589db872e116779ab9cae693171ac8a8c02d9923 validation: don't erase coins cache on prune flushes (Andrew Toth)
0e8918755f725b6269ed2be5a0b46f1611233515 Add linked-list test to CCoinsViewCache::SanityCheck (Pieter Wuille)
05cf4e18758618bb493d26014d3a9c89bf28d898 coins: move Sync logic to CoinsViewCacheCursor (Andrew Toth)
7825b8b9aeceb4ff607650cdc9c49e5de9c7719f coins: pass linked list of flagged entries to BatchWrite (Andrew Toth)
a14edada8a051e280af6fedd5130be40247e2d7a test: add cache entry linked list tests (Andrew Toth)
24ce37cb867b95e86d9fd4e50858d64ee8a59abf coins: track flagged cache entries in linked list (Andrew Toth)
58b7ed156d5993b69375bb455b03bd8b17f64fa4 coins: call ClearFlags in CCoinsCacheEntry destructor (Andrew Toth)
8bd3959feaa0e71585bc5e0976f584fb06ee6d14 refactor: require self and sentinel parameters for AddFlags (Andrew Toth)
75f36d241d2a344c5c4ce2c80250bdde91b3295e refactor: add CoinsCachePair alias (Andrew Toth)
f08faeade2f99ae1de6f3c481697541cc16186c7 refactor: move flags to private uint8_t and rename to m_flags (Andrew Toth)
4e4fb4cbabcc10e723534f5f80f3e3cf09f6ca50 refactor: disallow setting flags in CCoinsCacheEntry constructors (Andrew Toth)
8737c0cefa6ec49a4d17d9bef9e5e1a7990af1ac refactor: encapsulate flags setting with AddFlags and ClearFlags (Andrew Toth)
9715d3bf1e4013476539f1523a6b54d2055c32c6 refactor: encapsulate flags get access for all other checks (Andrew Toth)
df34a94e57659c31873c26c86a8de5a032400061 refactor: encapsulate flags access for dirty and fresh checks (Andrew Toth)
Pull request description:
Since https://github.com/bitcoin/bitcoin/pull/17487 we no longer need to clear the coins cache when syncing to disk. A warm coins cache significantly speeds up block connection, and only needs to be fully flushed when nearing the `dbcache` limit.
For frequent pruning flushes there's no need to empty the cache and kill connect block speed. However, simply using `Sync` in place of `Flush` actually slows down a pruned full IBD with a high `dbcache` value. This is because as the cache grows, sync takes longer since every coin in the cache is scanned to check if it's dirty. For frequent prune flushes and a large cache this constant scanning starts to really slow IBD down, and just emptying the cache on every prune becomes faster.
To fix this, we can add two pointers to each cache entry and construct a doubly linked list of dirty entries. We can then only iterate through all dirty entries on each `Sync`, and simply clear the pointers after.
With this approach a full IBD with `dbcache=16384` and `prune=550` was 32% faster than master. For default `dbcache=450` speedup was ~9%. All benchmarks were run with `stopatheight=800000`.
| | prune | dbcache | time | max RSS | speedup |
|-----------:|----------:|------------:|--------:|-------------:|--------------:|
| master | 550 | 16384 | 8:52:57 | 2,417,464k | - |
| branch | 550 | 16384 | 6:01:00 | 16,216,736k | 32% |
| branch | 550 | 450 | 8:05:08 | 2,818,072k | 8.8% |
| master | 10000 | 5000 | 8:19:59 | 2,962,752k | - |
| branch | 10000 | 5000| 5:56:39 | 6,179,764k | 28.8% |
| master | 0 | 16384 | 4:51:53 | 14,726,408k | - |
| branch | 0 | 16384 | 4:43:11 | 16,526,348k | 2.7% |
| master | 0 | 450 | 7:08:07 | 3,005,892k | - |
| branch | 0 | 450 | 6:57:24 | 3,013,556k |2.6%|
While the 2 pointers add memory to each cache entry, it did not slow down IBD. For non-pruned IBD results were similar for this branch and master. When I performed the initial IBD, the full UTXO set could be held in memory when using the max `dbcache` value. For non-pruned IBD with max `dbcache` to tip ended up using 12% more memory, but it was also 2.7% faster somehow. For smaller `dbcache` values the `dbcache` limit is respected so does not consume more memory, and the potentially more frequent flushes were not significant enough to cause any slowdown.
For reviewers, the commits in order do the following:
First 4 commits encapsulate all accesses to `flags` on cache entries, and then the 5th makes `flags` private.
Commits `refactor: add CoinsCachePair alias` to `coins: call ClearFlags in CCoinsCacheEntry destructor` create the linked list head nodes and cache entry self references and pass them into `AddFlags`.
Commit `coins: track flagged cache entries in linked list` actually adds the entries into a linked list when they are flagged DIRTY or FRESH and removes them from the linked list when they are destroyed or the flags are cleared manually. However, the linked list is not yet used anywhere.
Commit `test: add cache entry linked list tests` adds unit tests for the linked list.
Commit `coins: pass linked list of flagged entries to BatchWrite` uses the linked list to iterate through DIRTY entries instead of using the entire coins cache.
Commit `validation: don't erase coins cache on prune flushes` uses `Sync` instead of `Flush` for pruning flushes, so the cache is no longer cleared.
Inspired by [this comment](https://github.com/bitcoin/bitcoin/pull/15265#issuecomment-457720636).
Fixes https://github.com/bitcoin/bitcoin/issues/11315.
ACKs for top commit:
paplorinc:
ACK 589db872e116779ab9cae693171ac8a8c02d9923
sipa:
reACK 589db872e116779ab9cae693171ac8a8c02d9923
achow101:
ACK 589db872e116779ab9cae693171ac8a8c02d9923
mzumsande:
re-ACK 589db872e116779ab9cae693171ac8a8c02d9923
Tree-SHA512: 23b2bc01c83edacb5b39aa60bb0b766de9a74ce17f0c59bf13b97b4328a7b758ad9aff6581c3ca88e2973f7658380651530d497444f48d6e22ea0bfc51cc921d
6bfa26048dbafb91e9ca63ea8d3960271e798098 testnet: Add timewarp attack prevention for Testnet4 (Fabian Jahr)
0100907ca168c53e8fe044bdda396f308825162c testnet: Add Testnet4 difficulty adjustment rules fix (Fabian Jahr)
74a04f9e7ad6a16988149cc3438b9ce13c91cdb9 testnet: Introduce Testnet4 (Fabian Jahr)
Pull request description:
To supplement the [ongoing conceptual discussion about a testnet reset](https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/9yCPo3uUBwAJ) I have drafted a move to v4 including a fix to the difficulty adjustment mechanism, which was part of the motivation that started the discussion.
Conceptual considerations:
- The conceptual discussion about doing a testnet4 or softforking the fix into testnet3 is outside of the scope of this PR and I would ask reviewers to contribute their opinions on this on the ML instead. However, I am happy to adapt this PR to a softfork change on testnet3 if there is consensus for that instead.
- The difficulty adjustment fix suggested here touches the `CalculateNextWorkRequired` function and uses the same logic used in `GetNextWorkRequired` to find the last previous block that was not mined with difficulty 1 under the exceptionf. An alternative fix briefly mentioned on the mailing list by Jameson Lopp would be to "restrict the special testnet minimum difficulty rule so that it can't be triggered on the block right before a difficulty retarget". That would also fix the issue but I find my suggestion here a bit more elegant.
ACKs for top commit:
jsarenik:
tACK 6bfa26048dba
achow101:
ACK 6bfa26048dbafb91e9ca63ea8d3960271e798098
murchandamus:
tACK 6bfa26048dbafb91e9ca63ea8d3960271e798098
Tree-SHA512: 0b8b69a621406a944da5be551b863d065358ba94d85dd3b80d83c412660e230ee93b27316081fbee9b4851cc4ff8585db64c7dfa26cb5148ac835663f2712c3d
1f93e3c360f3da38a02fe6c01551f424a2d63dc9 add deprecation warning for mempoolfullrbf (glozow)
4400c979a309ed03c7ed55644122622eb7def089 [doc] update documentation for new mempoolfullrbf default (glozow)
Pull request description:
Followup to #30493. Update bips.md and policy/*.md to reflect new default rules around signaling requirements in RBF.
Also, log a warning when `-mempoolfullrbf=0` that this config option is deprecated and will be removed in a future release.
ACKs for top commit:
petertodd:
ACK 1f93e3c360f3da38a02fe6c01551f424a2d63dc9
instagibbs:
ACK 1f93e3c360
tdb3:
ACK 1f93e3c360f3da38a02fe6c01551f424a2d63dc9
Tree-SHA512: f60a9524f15cfaa4c10c40b6f62b787d3f9865aac48ca883def30efac4f8a118f1359532f1b209ea34e201f0b1c92398abc8bc1e439e6b60910cc7f75c51e9ae
ec973dd19719541dbcd6f3a6facf6f5dd7cf439c refactor: remove un-tested early returns (josibake)
72a5822d43d47431b2838ebfcb1f2e21210f5ccb tests: add tests for KeyPair (josibake)
cebb08b121ce8c4c5e68bd043b8668c106e31169 refactor: move SignSchnorr to KeyPair (josibake)
c39fd39ba868253b5118db2e1ac1461d29f0b4ce crypto: add KeyPair wrapper class (josibake)
5d507a0091da1b6c013b00b6c76e19dd4d3b93a7 tests: add key tweak smoke test (josibake)
f14900b6e4eac26ae5f1c0badfa176d895851c97 bench: add benchmark for signing with a taptweak (josibake)
Pull request description:
Broken out from #28201
---
The wallet returns an untweaked internal key for taproot outputs. If the output commits to a tree of scripts, this key needs to be tweaked with the merkle root. Even if the output does not commit to a tree of scripts, BIP341/342 recommend commiting to a hash of the public key.
Previously, this logic for applying the taptweak was implemented in the `CKey::SignSchnorr` method.
This PR moves introduces a KeyPair class which wraps a `secp256k1_keypair` type and refactors SignSchnorr to use this new KeyPair. The KeyPair class is created with an optional merkle_root argument and the logic from BIP341 is applied depending on the state of the merkle_root argument.
The motivation for this refactor is to be able to use the tap tweak logic outside of signing, e.g. in silent payments when retrieving the private key (see #28201).
Outside of silent payments, since we almost always convert a `CKey` to a `secp256k1_keypair` when doing anything with taproot keys, it seems generally useful to have a way to model this type in our code base.
ACKs for top commit:
paplorinc:
ACK ec973dd19719541dbcd6f3a6facf6f5dd7cf439c - will happily reack if you decide to apply @ismaelsadeeq's suggestions
ismaelsadeeq:
Code review ACK ec973dd19719541dbcd6f3a6facf6f5dd7cf439c
itornaza:
trACK ec973dd19719541dbcd6f3a6facf6f5dd7cf439c
theStack:
Code-review ACK ec973dd19719541dbcd6f3a6facf6f5dd7cf439c
Tree-SHA512: 34947e3eac39bd959807fa21b6045191fc80113bd650f6f08606e4bcd89aa17d6afd48dd034f6741ac4ff304b104fa8c1c1898e297467edcf262d5f97425da7b
6714276d72244c2e2dbe9617f1341ba7fc06c0cc miniscript: Use `ToIntegral` instead of `ParseInt64` (brunoerg)
Pull request description:
Currently, miniscript code uses `ParseInt64` function for `after`, `older`, `multi` and `thresh` fragments. It means that a leading `+` or whitespace, among other things, are accepted into the fragments. However, these cases are not useful and cause Bitcoin Core to behave differently compared to other miniscript implementations (see https://github.com/brunoerg/bitcoinfuzz/issues/34). This PR fixes it.
ACKs for top commit:
achow101:
ACK 6714276d72244c2e2dbe9617f1341ba7fc06c0cc
tdb3:
cr ACK 6714276d72244c2e2dbe9617f1341ba7fc06c0cc
danielabrozzoni:
tACK 6714276d72244c2e2dbe9617f1341ba7fc06c0cc
darosior:
utACK 6714276d72244c2e2dbe9617f1341ba7fc06c0cc
Tree-SHA512: d9eeb93f380f346d636513eeaf26865285e7b0907b8ed258fe1e02153a9eb69d484c82180eb1c78b0ed77ad5f0e5b244be6672c2f890b1d9fddc9e844bee6dde
