76cdeb7b06 wallet: test: Failed migration cleanup (David Gumberg)
9405e915e7 test: coverage for migration failure when last sync is beyond prune height (furszy)
5e8ad98163 wallet: migration, fix watch-only and solvables wallets names (furszy)
a7e2d106db wallet: improve post-migration logging (furszy)
9ea84c08d7 test: restorewallet, coverage for existing dirs, unnamed wallet and prune failure (furszy)
833848e9b8 test: add coverage for unnamed wallet migration failure (furszy)
a074d36254 wallet: fix unnamed wallet migration failure (furszy)
d91f56e1e3 wallet: RestoreWallet failure, erase only what was created (furszy)
cc324aa2be wallettool: do not use fs::remove_all in createfromdump cleanup (Ava Chow)
01c04d32aa wallet: introduce method to return all db created files (furszy)
abaf1e37a7 refactor: remove sqlite dir path back-and-forth conversion (furszy)
Pull request description:
Backports:
* #34215
* #34156
* #34226
* 2 required commits from #31423
Note that this backport is unclean and several changes have to be made to most commits to accommodate BDB and the differences in migration cleanup behavior.
ACKs for top commit:
furszy:
Code review ACK 76cdeb7b06
brunoerg:
light code review ACK 76cdeb7b06 + backported the functional tests without the fixes and all of them failed accordingly.
glozow:
light review ACK 76cdeb7b06.
Tree-SHA512: 432268117783fc9a221d895a6f6601b6a2a5031c76d1443cf804cc1d486b40fcded982409d548acd1c01a13c7b378b840fcc3fbe823d6ba5ffc4ebe017d4e13c
Refactor a common way to perform the failed migration test that exists
for default wallets, and add relative-path wallets and absolute-path
wallets.
Github-Pull: 34226
Rebased-From: eeaf28dbe0
Because the default wallet has no name, the watch-only and solvables
wallets created during migration end up having no name either.
This fixes it by applying the same prefix name we use for the backup
file for an unnamed default wallet.
Before: watch-only wallet named "_watchonly"
After: watch-only wallet named "default_wallet_watchonly"
Github-Pull: bitcoin/bitcoin#34156
Rebased-From: 82caa8193a
Right now, after migration the last message users see is "migration completed",
but the migration isn't actually finished yet. We still need to load the new wallets
to ensure consistency, and if that fails, the migration will be rolled back. This
can be confusing for users.
This change logs the post-migration loading step and if a wallet fails to load and
the migration will be rolled back.
Github-Pull: bitcoin/bitcoin#34156
Rebased-From: d70b159c42
The first test verifies that restoring into an existing empty directory
or a directory with no .dat db files succeeds, while restoring into a
dir with a .dat file fails.
The second test covers restoring into the default unnamed wallet
(wallet.dat), which also implicitly exercises the recovery path used
after a failed migration.
The third test covers failure during restore on a prune node. When
the wallet last sync was beyond the pruning height.
Github-Pull: bitcoin/bitcoin#34156
Rebased-From: f011e0f068
Verifies that a failed migration of the unnamed (default) wallet
does not erase the main /wallets/ directory, and also that the
backup file exists.
Github-Pull: bitcoin/bitcoin#34156
Rebased-From: 36093bde63
When migrating any legacy unnamed wallet, a failed migration would
cause the cleanup logic to remove its parent directory. Since this
type of legacy wallet lives directly in the main '/wallets/' folder,
this resulted in unintentionally erasing all wallets, including the
backup file.
To be fully safe, we will no longer call `fs::remove_all`. Instead,
we only erase the individual db files we have created, leaving
everything else intact. The created wallets parent directories are
erased only if they are empty.
As part of this last change, `RestoreWallet` was modified to allow
an existing directory as the destination, since we no longer remove
the original wallet directory (we only remove the files we created
inside it). This also fixes the restore of top-level default wallets
during failures, which were failing due to the directory existence
check that always returns true for the /wallets/ directory.
This bug started after:
f6ee59b6e2
Previously, the `fs::copy_file` call was failing for top-level wallets,
which prevented the `fs::remove_all` call from being reached.
Github-Pull: bitcoin/bitcoin#34156
Rebased-From: f4c7e28e80
Track what RestoreWallet creates so only those files and directories
are removed during a failure and nothing else. Preexisting paths
must be left untouched.
Note:
Using fs::remove_all() instead of fs::remove() in RestoreWallet does
not cause any problems currently, but the change is necessary for the
next commit which extends RestoreWallet to work with existing directories,
which may contain files that must not be deleted.
Github-Pull: bitcoin/bitcoin#34156
Rebased-From: 4ed0693a3f
2cf352fd8e doc: document capnproto and libmultiprocess deps (will)
Pull request description:
Closes: #33576
These dependencies are both undocumented, and libmultiprocess has a relatively special requirement in that v6.0 and later are known to not work with v29.x of Bitcoin Core due to https://github.com/bitcoin-core/libmultiprocess/pull/160
ACKs for top commit:
ryanofsky:
Code review ACK 2cf352fd8e. Thanks for making all these changes and for opening the fix originally.
Tree-SHA512: 3839bb7b0bbb23c3fe358960e93f7919953cac315eaed2b214491dd4f6c80ed500c09a618d11408836bddf6d11fc98b1ca3989f552104c49713ebd8859706ac5
These dependencies are both undocumented, and libmultiprocess has a
relatively special requirement in that v6.0 and later are known to not
work with v29.x of Bitcoin Core due to https://github.com/bitcoin-core/libmultiprocess/pull/160
Previously, we would check failing input scripts twice when considering
a transaction for the mempool, in order to distinguish policy failures
from consensus failures. This allowed us both to provide a different
error message and to discourage peers for consensus failures. Because we
are no longer discouraging peers for consensus failures during tx relay,
and because checking a script can be expensive, only do this once.
Also renames non-mandatory-script-verify-flag error to
mempool-script-verify-flag-failed.
NOTE: Backport required additional adjustment in test/functional/feature_block
Github-Pull: #33050
Rebased-From: b29ae9efdf
Do not discourage nodes even when they send us consensus invalid
transactions.
Because we do not discourage nodes for transactions we consider
non-standard, we don't get any DoS protection from this check in
adversarial scenarios, so remove the check entirely both to simplify the
code and reduce the risk of splitting the network due to changes in tx
relay policy.
NOTE: Backport required additional adjustment in test/functional/p2p_invalid_tx
Github-Pull: #33050
Rebased-From: 266dd0e10d
Since it was introduced in 4eb515574e (#18044), the detection of a
stripped witness relies on running the Script checks 3 times. In the worst case, this consists in
running Script validation 3 times for every single input.
Detection of a stripped witness is necessary because in this case wtxid==txid, and the transaction's
wtxid must not be added to the reject filter or it could allow a malicious peer to interfere with
txid-based orphan resolution as used in 1p1c package relay.
However it is not necessary to run Script validation to detect a stripped witness (much less so
doing it 3 times in a row). There are 3 types of witness program: defined program types (Taproot,
P2WPKH, P2WSH), undefined types, and the Pay-to-anchor carve-out.
For defined program types, Script validation with an empty witness will always fail (by consensus).
For undefined program types, Script validation is always going to fail regardless of the witness (by
standardness). For P2A, an empty witness is never going to lead to a failure.
Therefore it holds that we can always detect a stripped witness without re-running Script validation.
However this might lead to more "false positives" (cases where we return witness stripping for an
otherwise invalid transaction) than the existing implementation. For instance a transaction with one
P2PKH input with an invalid signature and one P2WPKH input with its witness stripped. The existing
implementation would treat it as consensus invalid while the implementation in this commit would
always consider it witness stripped.
Github-Pull: #33105
Rebased-From: 27aefac425
We will use this helper in later commits to detect witness stripping without having
to execute every input Script three times in a row.
Github-Pull: #33105
Rebased-From: 2907b58834
A stripped witness is detected as a special case in mempool acceptance to make sure we do not add
the wtxid (which is =txid since witness is stripped) to the reject filter. This is because it may
interfere with 1p1c parent relay which currently uses orphan reconciliation (and originally it was
until wtxid-relay was widely adopted on the network.
This commit adds a test for this special case in the p2p_segwit function test, both when spending
a native segwit output and when spending a p2sh-wrapped segwit output.
Thanks to Eugene Siegel for pointing out the p2sh-wrapped detection did not have test coverage by
finding a bug in a related patch of mine.
Github-Pull: #33105
Rebased-From: eb073209db
When using `docker buildx build` in conjunction with the `gha` backend
cache type, it's important to specify the URL and TOKEN needed to
authenticate.
On Cirrus runners this is working with only `ACTIONS_CACHE_URL` and
`ACTIONS_RUNTIME_TOKEN`, but this is not enough for the GitHub backend.
Fix this by exporting all `ACTIONS_*` variables.
This fixes cache restore/save on forks or where GH-hosted runners are
being used.
Github-Pull: #33508
Rebased-From: bc706955d7
$FILE_ENV has a full relative path already, prepending with ci/test/
results in a non-existent path which means that DEPENDS_HASH was not
actually committing to the test's environment file.
Github-Pull: #33581
Rebased-From: ceeb53adcd
abf4a6eeae build: fix depends Qt download link (fanquake)
Pull request description:
Fix Qt download path, so we wont always hit the fallback.
ACKs for top commit:
hebasto:
ACK abf4a6eeae.
Tree-SHA512: 1157528983ede46c60810eae5c73f4bd81640afcae9afd9aad14c30104e90c52f8e97755f22314a5514bc1de3a92d864398087fe826f1980acc772fd32535a9f
d82fc69829 doc: update release notes for 29.2rc2 (fanquake)
513cef75ee doc: update manual pages for v29.2rc2 (fanquake)
eea16f7de7 build: bump version to v29.2rc2 (fanquake)
6b3c1dbc5c contrib: fix using macdploy script without translations. (amisha)
Pull request description:
It's been 2 weeks since rc1: https://github.com/bitcoin/bitcoin/releases/tag/v29.2rc1.
We've backported more changes:
* #33403
* #33474
* #33482
Lets do `rc2`.
ACKs for top commit:
davidgumberg:
reACK d82fc69
glozow:
ACK d82fc69829
darosior:
utACK d82fc69829. Changes look good to me, but i have not been through the process of regenerating the doc myself.
Tree-SHA512: c829efe89f86c9c76767ffe60a3779ece902ee9e3c8f6b4203562aaf257019484bfa49916ddfabdcabbd1478368d9b80a3f0a15057778aa1984852ea245283a6
QT translations are optional, but the script would error when
'translations_dir' falls back to its default value NULL.
This PR fixes it by moving the set-up of QT translations under
the check for 'translations_dir' presence.
Github-Pull: #33482
Rebased-From: 7b5261f7ef
Not enforcing TRUC topology on reorg was the intended
behavior, but the appropriate bypass argument was not
checked.
This mistake means we could potentially invalidate a long
chain of perfectly incentive-compatible transactions that
were made historically, including subsequent non-TRUC
transactions, all of which may have been very high feerate.
Lastly, it wastes CPU cycles doing topology checks since
this behavior cannot actually enforce the topology in
general for the reorg setting.
Github-Pull: #33504
Rebased-From: 26e71c237d
Using bypass_limits=true is essentially fuzzing part of a
reorg only, and results in TRUC invariants unable to be
checked. Remove most instances of bypassing limits, leaving
one harness able to do so.
Github-Pull: #33504
Rebased-From: bbe8e9063c
A target field was added to the getblock and getblockheader RPC calls in bitcoin#31583, but it mistakingly always used the tip value.
Because regtest does not have difficulty adjustment, a test is added for mainnet instead.
Github-Pull: #33446
Rebased-From: bf7996cbc3
