5a0688a20d88a9641c02436abbd7b49e227f1a37 test: enable reindex readonly test on *BSD and macOS as root (Matthew Zipkin)
Pull request description:
see https://github.com/bitcoin/bitcoin/pull/27850#discussion_r1349505585
OpenBSD and FreeBSD don't have `chattr` but they do have `chflags`, use that method to make the block file immutable for the reindex_readonly test.
Written and tested on a VPS running FreeBSD:
```
FreeBSD freebsd-13-1 13.2-RELEASE-p4 FreeBSD 13.2-RELEASE-p4 GENERIC amd64
```
ACKs for top commit:
maflcko:
re-cr-lgtm-ACK 5a0688a20d88a9641c02436abbd7b49e227f1a37
jonatack:
ACK 5a0688a20d88a9641c02436abbd7b49e227f1a37 tested on macOS only
theStack:
ACK 5a0688a20d88a9641c02436abbd7b49e227f1a37
Tree-SHA512: 8c88d282d09c00355d22c4c504b779f60e420327a5e07bcf80fa77b97fefcb04952af9ceaf439d9033a0a2448cb26a02663fe6bddcd4a74792857cfbaf1c5162
This commit introduces a helper `create_outpoints` to execute the
`send` RPC and immediately return the target address outpoints as UTXO
dictionary in the common format, making the tests more readable and
avoiding unnecessary duplication.
Currently the shutdown function resets the kernel before the
chainman and scheduler. Invert this order by resetting the kernel
last, since they might rely on the kernel.
4814e4063e674ad9b0a5c7e56059cd6a2bf9b764 test: Check tx metadata is migrated to watchonly (Andrew Chow)
d616d30ea5fdfb897f8375ffd8b9f4536ae7835b wallet: Reload watchonly and solvables wallets after migration (Andrew Chow)
118f2d7d70b584eee7b89e58b5cd2d61c59a9bbf wallet: Copy all tx metadata to watchonly wallet (Andrew Chow)
9af87cf3485ce3fac553a284cde37a35d1085c25 test: Check that a failed wallet migration is cleaned up (Andrew Chow)
Pull request description:
Some incomplete/incorrect state as a result of migration can be mitigated/cleaned up by simply restarting the migrated wallets. We already do this for a wallet when it is migrated, but we do not for the new watchonly and solvables wallets that may be created. This PR introduces this behavior, in addition to creating those wallets initially without an attached chain.
While implementing this, I noticed that not all `CWalletTx` metadata was being copied over to the watchonly wallet and so some data, such as time received, was being lost. This PR fixes this as a side effect of not having a chain attached to the watchonly wallet. A test has also been added.
ACKs for top commit:
ishaanam:
light code review ACK 4814e4063e674ad9b0a5c7e56059cd6a2bf9b764
ryanofsky:
Code review ACK 4814e4063e674ad9b0a5c7e56059cd6a2bf9b764. Just implemented the suggested orderpos, copyfrom, and path set comments since last review
furszy:
ACK 4814e406
Tree-SHA512: 0b992430df9f452cb252c2212df8e876613f43564fcd1dc00c6c31fa497adb84dfff6b5ef597590f9b288c5f64cb455f108fcc9b6c9d1fe9eb2c39e7f2c12a89
No change in behavior, this just moves code which updates transaction state to
a new method so it can be used after offline processes such as wallet
migration.
4bfaad4eca01674a9c84a447a17594dc2b9a4c39 chainparams, assumeutxo: Fix signet txoutset hash (Fabian Jahr)
a503cd0f0b55736743bcf8d2c46d271064772bef chainparams, assumeutxo: Fix testnet txoutset hash (Fabian Jahr)
f6213929c519d0e615cacd3d6f479f1517be1662 assumeutxo: Check deserialized coins for out of range values (Fabian Jahr)
66865446a771327be9e972cdaf01154ea1bdff6d docs: Add release notes for #28685 (Fabian Jahr)
cb0336817edc2b6aee2eca818f133841f613a767 scripted-diff: Rename hash_serialized_2 to hash_serialized_3 (Fabian Jahr)
351370a1d211615e3d5b158ccb0400cd79c5c085 coinstats: Fix hash_serialized2 calculation (Fabian Jahr)
Pull request description:
Closes#28675
The last commit demonstrates that theStack's analysis [here](https://github.com/bitcoin/bitcoin/issues/28675#issuecomment-1770389468) seems to be correct. There will be more changes needed for the rest of the test suite but the `feature_assumeutxo.py` with my additional tests pass.
ACKs for top commit:
achow101:
ACK 4bfaad4eca01674a9c84a447a17594dc2b9a4c39
theStack:
Code-review ACK 4bfaad4eca01674a9c84a447a17594dc2b9a4c39
ryanofsky:
Code review ACK 4bfaad4eca01674a9c84a447a17594dc2b9a4c39
Tree-SHA512: 2f6abc92b282f7c5da46391803cf0804d13978d191d541f2509b532c538abccd0a081e46cda23d80d47206a05fa2b5d41b7ab246e6a263db7a7461d6292116ef
03f82087f6ce1c29327f34d12945200494e6956d doc: assumeutxo prune and index notes (Sjors Provoost)
Pull request description:
Based on recent comments on #27596.
ACKs for top commit:
pablomartin4btc:
re ACK 03f82087f6ce1c29327f34d12945200494e6956d
ryanofsky:
ACK 03f82087f6ce1c29327f34d12945200494e6956d. Nice changes, these seem like very helpful notes
Tree-SHA512: fe651b49f4d667400a3655899f27a96dd1eaf67cf9215fb35db5f44fb8c0313e7d541518be6791fec93392df24b909793f3886adb808e53228ed2a291165639d
fa6588737714cf26571657fc216552a4291376da ci: Add missing --external to podman image prune (MarcoFalke)
Pull request description:
This should fix the out-of-space issues seen in CI. For example:
https://cirrus-ci.com/task/6208410429947904?logs=ci#L8613
```
Error: committing container for step {Env:[FILE_ENV=./ci/test/00_setup_env_native_msan.sh PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin] Command:run Args:[bash -c cd /ci_container_base/ && set -o errexit && source ./ci/test/00_setup_env.sh && ./ci/test/01_base_install.sh] Flags:[] Attrs:map[json:true] Message:RUN bash -c cd /ci_container_base/ && set -o errexit && source ./ci/test/00_setup_env.sh && ./ci/test/01_base_install.sh Original:RUN ["bash", "-c", "cd /ci_container_base/ && set -o errexit && source ./ci/test/00_setup_env.sh && ./ci/test/01_base_install.sh"]}: copying layers and metadata for container "8d882455cc157be6a85d9779b45cacf4dd92a37cfb16fad38213f758a830827d": writing blob: adding layer with blob "sha256:371f657e226fef20f4af6fb88a288dd6248c82c2088daca2d53aaacb51b4303a": processing tar file(write /usr/lib/x86_64-linux-gnu/perl/5.34.0/auto/Unicode/Collate/Collate.so: no space left on device): exit status 1
ACKs for top commit:
stickies-v:
utACK fa6588737714cf26571657fc216552a4291376da
Tree-SHA512: a949b957654272b9c9450feebb64397542029019f031a18444b5b403aca899c972b3b163be716bf1bfbd5820430c70a6fec008771c6e13d3d8281ed100df575b
f09bfab4aff04a9cc1ea157b94bfdae19f3465b1 Revert "gui: provide wallet controller context to wallet actions" (Hennadii Stepanov)
Pull request description:
The commit 7066e8996d0ac090535cc97cdcb54a219986460f from https://github.com/bitcoin-core/gui/pull/765 breaks "Open Wallet", "Close Wallet" and "Close All Wallets" items in the File menu (at least on Ubuntu 23.10 + Wayland).
Reverting it to avoid this regression in the 26.0 release.
ACKs for top commit:
furszy:
ACK f09bfab4 for including it in v26.
jarolrod:
ACK f09bfab4aff04a9cc1ea157b94bfdae19f3465b1
Tree-SHA512: fedc621c8e9bf84a263b0c28da53225febe0267d0123830a6192297f38e40726e1613e003b634215e7d16791ba6eab52fb4baab3da9637f6660b6ae1ae98462b
6bdff429ec17eae4138c3af1e21de3ec46f4ab13 build: Include `config/bitcoin-config.h` explicitly in `util/trace.h` (Hennadii Stepanov)
Pull request description:
The `ENABLE_TRACING` macro is expected to be defined in the `config/bitcoin-config.h` header.
Therefore, the current code is error-prone as it depends on whether the `config/bitcoin-config.h` header was included before or not.
This bug was noticed while working on CMake [stuff](https://github.com/hebasto/bitcoin/pull/37).
ACKs for top commit:
fanquake:
ACK 6bdff429ec17eae4138c3af1e21de3ec46f4ab13
Tree-SHA512: 22c4fdeb51628814050eb99a83db4268a4f3106207eeef918a07214bbc52f2b22490f6b05fcb96216f147afa4197c51102503738131e2583e750b6d195747a49
fac36b94ef32567c0f10b605a3a441d11559e56e refactor: Remove CBlockFileInfo::SetNull (MarcoFalke)
Pull request description:
Seems better to use C++11 member initializers and then let the compiler figure out how to construct objects of this class.
ACKs for top commit:
stickies-v:
ACK fac36b94ef32567c0f10b605a3a441d11559e56e
pablomartin4btc:
ACK fac36b94ef32567c0f10b605a3a441d11559e56e
theStack:
LGTM ACK fac36b94ef32567c0f10b605a3a441d11559e56e
Tree-SHA512: aee741c8f668f0e5b658fc83f4ebd196b43fead3dd437afdb0a2dafe092ae3d559332b3d9d61985c92e1a59982d8f24942606e6a98598c6ef7ff43697e858725
fa21535551e300eaa988d209ad64cdc17fd7f66b fuzz: Increase merge -rss_limit_mb (MarcoFalke)
Pull request description:
For some reason, the limit is hit. (Presumably due to `-set_cover_merge=1` eating more memory, or by simply having more fuzz inputs).
Fix it by increasing it for the merge operation.
ACKs for top commit:
dergoegge:
ACK fa21535551e300eaa988d209ad64cdc17fd7f66b
hebasto:
ACK fa21535551e300eaa988d209ad64cdc17fd7f66b, considering the discussion in https://github.com/bitcoin-core/qa-assets/pull/155.
Tree-SHA512: 4fed0f254eccc6fe0b53656bc345ff898b13811dc39387387317d34b521ab77cee03d82b0896dd92d253b7546b6a7e4bdcd478749f47064374ab44ad759ab9ff
The legacy serialization was vulnerable to maleation and is fixed by
adopting the same serialization procedure as was already in use for
MuHash.
This also includes necessary test fixes where the hash_serialized2 was
hardcoded as well as correction of the regtest chainparams.
Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
d3223685b1bb3fb4b4626d2afe4bf753e04f7b0a test: De-dublicate/optimize assumeutxo test for further extensions (Fabian Jahr)
0a576d62fe014f31d352f01873121e84e7971bc9 test: check au file with changed outpoint index (Fabian Jahr)
Pull request description:
Also doing some de-duplications. I kept the second commit separate for now as I am not 100% if this is overdoing it and makes it harder to reason about. But it also makes it easier to add more cases where we change more data.
ACKs for top commit:
maflcko:
lgtm ACK d3223685b1bb3fb4b4626d2afe4bf753e04f7b0a
achow101:
ACK d3223685b1bb3fb4b4626d2afe4bf753e04f7b0a
Tree-SHA512: be950a34b0ed50cb58459df47cff6513df19d834bf81815572cd26b10dee26e6f80866f0c44023cf246aafbbd256e62d23ce903e8b07fdff2297bc7065799bb8
dd4dcbd4cda31f67d014a93340a6d1ba1c245b0f [fuzz] Delete i2p target (dergoegge)
Pull request description:
closes#28665
The target is buggy and doesn't reach basic coverage.
ACKs for top commit:
maflcko:
lgtm ACK dd4dcbd4cda31f67d014a93340a6d1ba1c245b0f
glozow:
ACK dd4dcbd4cda31f67d014a93340a6d1ba1c245b0f, agree it's better to delete this test until somebody wants to write a better one
Tree-SHA512: b6ca6cad1773b1ceb6e5ac0fd501ea615f66507ef811745799deaaa4460f1700d96ae03cf55b740a96ed8cd2283b3d6738cd580ba97f2af619197d6c4414ca21
The `ENABLE_TRACING` macro is expected to be defined in the
`config/bitcoin-config.h` header.
Therefore, the current code is error-prone as it depends on whether the
`config/bitcoin-config.h` header was included before or not.
ec84f999f1408b7f1ff4498f78c33b34c30e934c log: Don't log cache rebalancing in absense of a snapshot chainstate (Fabian Jahr)
Pull request description:
I have noticed that this log now is always printed, even if there is no snapshot chainstate present or even was present. I think this is confusing to users that have never even thought about using assumeutxo since in that case the rebalancing is just ensuring the normal environment with one chainstate. So I suggest we don't log in absence of a snapshot chainstate. We could also think about rewording the message instead but I think this is simpler.
ACKs for top commit:
stickies-v:
utACK ec84f999f1408b7f1ff4498f78c33b34c30e934c
glozow:
concept ACK ec84f999f1408b7f1ff4498f78c33b34c30e934c, don't have opinions other than removing confusing log
theStack:
utACK ec84f999f1408b7f1ff4498f78c33b34c30e934c
Tree-SHA512: 30bbfc648e7c788106f78d52e47a3aa1e1874f65d13743643dc50bcf7f450d8330711ff9fdeac361722542da6051533153829c6d49033227ed315e111afc899f
When migrating, create the watchonly and solvables wallets without a
context. Then unload and reload them after migration completes, as we do
for the actual wallet.
There is also additional handling for a failed reload.
5c8e15c451ec870b9dd4eb843ec6ca3ba64cda4f i2p: destroy the session if we get an unexpected error from the I2P router (Vasil Dimov)
762404a68c114e8831cdfae937627174544b55a7 i2p: also sleep after errors in Accept() (Vasil Dimov)
Pull request description:
### Background
In the `i2p::sam::Session` class:
`Listen()` does:
* if the session is not created yet
* create the control socket and on it:
* `HELLO`
* `SESSION CREATE ID=sessid`
* leave the control socked opened
* create a new socket and on it:
* `HELLO`
* `STREAM ACCEPT ID=sessid`
* read reply (`STREAM STATUS`), `Listen()` only succeeds if it contains `RESULT=OK`
Then a wait starts, for a peer to connect. When connected,
`Accept()` does:
* on the socket from `STREAM ACCEPT` from `Listen()`: read the Base64 identification of the connecting peer
### Problem
The I2P router may be in such a state that this happens in a quick succession (many times per second, see https://github.com/bitcoin/bitcoin/issues/22759#issuecomment-1609907115): `Listen()`-succeeds, `Accept()`-fails.
`Accept()` fails because the I2P router sends something that is not Base64 on the socket: `STREAM STATUS RESULT=I2P_ERROR MESSAGE="Session was closed"`
We only sleep after failed `Listen()` because the assumption was that if `Accept()` fails then the next `Listen()` will also fail.
### Solution
Avoid filling the log with "Error accepting:" messages and sleep also after a failed `Accept()`.
### Extra changes
* Reset the error waiting time after one successful connection. Otherwise the timer will remain high due to problems that have been solved long time in the past.
* Increment the wait time less aggressively.
* Handle the unexpected "Session was closed" message more gracefully (don't log stupid messages like `Cannot decode Base64: "STREAM STATUS...`) and destroy the session right way.
ACKs for top commit:
achow101:
ACK 5c8e15c451ec870b9dd4eb843ec6ca3ba64cda4f
jonatack:
re-ACK 5c8e15c451ec870b9dd4eb843ec6ca3ba64cda4f
Tree-SHA512: 1d47958c50eeae9eefcb668b8539fd092adead93328e4bf3355267819304b99ab41cbe1b5dbedbc3452c2bc389dc8330c0e27eb5ccb880e33dc46930a1592885
0e6f6ebc064c5fb425fc3699efe760ec6cd4b6af net: remove unused CConnman::FindNode(const CSubNet&) (Vasil Dimov)
9482cb780fe04c1f1d9050edd1b8e549e52c86ce netbase: possibly change the result of LookupSubNet() to CJDNS (Vasil Dimov)
53afa68026ffa1313ae4aba3664de7791d23b1c8 net: move MaybeFlipIPv6toCJDNS() from net to netbase (Vasil Dimov)
6e308651c441cbf8763c67cc099c538c333c2872 net: move IsReachable() code to netbase and encapsulate it (Vasil Dimov)
c42ded3d9bda8b273780a4a81490bbf1b9e9c261 fuzz: ConsumeNetAddr(): avoid IPv6 addresses that look like CJDNS (Vasil Dimov)
64d6f77907afd461d9b14ee10ab32335f4454734 net: put CJDNS prefix byte in a constant (Vasil Dimov)
Pull request description:
`LookupSubNet()` would treat addresses that start with `fc` as IPv6 even if `-cjdnsreachable` is set. This creates the following problems where it is called:
* `NetWhitelistPermissions::TryParse()`: otherwise `-whitelist=` fails to white list CJDNS addresses: when a CJDNS peer connects to us, it will be matched against IPv6 `fc...` subnet and the match will never succeed.
* `BanMapFromJson()`: CJDNS bans are stored as just IPv6 addresses in `banlist.json`. Upon reading from disk they have to be converted back to CJDNS, otherwise, after restart, a ban entry like (`fc00::1`, IPv6) would not match a peer (`fc00::1`, CJDNS).
* `RPCConsole::unbanSelectedNode()`: in the GUI the ban entries go through `CSubNet::ToString()` and back via `LookupSubNet()`. Then it must match whatever is stored in `BanMan`, otherwise it is impossible to unban via the GUI.
These were uncovered by https://github.com/bitcoin/bitcoin/pull/26859.
Thus, flip the result of `LookupSubNet()` to CJDNS if the network base address starts with `fc` and `-cjdnsreachable` is set. Since subnetting/masking does not make sense for CJDNS (the address is "random" bytes, like Tor and I2P, there is no hierarchy) treat `fc.../mask` as an invalid `CSubNet`.
To achieve that, `MaybeFlipIPv6toCJDNS()` has to be moved from `net` to `netbase` and thus also `IsReachable()`. In the process of moving `IsReachable()`, `SetReachable()` and `vfLimited[]` encapsulate those in a class.
ACKs for top commit:
jonatack:
Code review ACK 0e6f6ebc064c5fb425fc3699efe760ec6cd4b6af
achow101:
ACK 0e6f6ebc064c5fb425fc3699efe760ec6cd4b6af
mzumsande:
re-ACK 0e6f6ebc064c5fb425fc3699efe760ec6cd4b6af
Tree-SHA512: 4767a60dc882916de4c8b110ce8de208ff3f58daaa0b560e6547d72e604d07c4157e72cf98b237228310fc05c0a3922f446674492e2ba02e990a272d288bd566
With `queuedTx` owning the `CTransactionRef` shared ptrs, they (and
the managed objects) are entirely allocated on the heap. In
`DisconnectedBlockTransactions::DynamicMemoryUsage`, we account for
the 2 pointers that make up the shared_ptr, but not for the managed
object (CTransaction) or the control block.
Prior to this commit, by calculating the `RecursiveDynamicUsage` on
a `CTransaction` whenever modifying `cachedInnerUsage`, we account
for the dynamic usage of the `CTransaction`, i.e. the `vins` and
`vouts` vectors, but we do not account for the `CTransaction`
object itself, nor for the `CTransactionRef` control block.
This means prior to this commit, `DynamicMemoryUsage` underestimates
dynamic memory usage by not including the `CTransaction` objects and
the shared ptr control blocks.
Fix this by calculating `RecursiveDynamicUsage` on the
`CTransactionRef` instead of the `CTransaction` whenever modifying
`cachedInnerUsage`.
b59b31ae0b04054c5cf225dad87046d3771707fc build: Drop redundant qt/bitcoin.cpp (Hennadii Stepanov)
d90ad5a42ec6f48d0e504edc16d41c8ef266cc1d build: Include qt sources for parsing with extract_strings.py (Hennadii Stepanov)
Pull request description:
On master (4fc15d15667d9d9c4fb5515ce73c05b4596298ec) some strings are still untranslated.
This PR fixes this issue.
To verify:
1) `./autogen.sh && ./configure && make -C src translate` _before_ applying this change
2) apply this change
3) `./autogen.sh && ./configure && make -C src translate` _after_ applying this change
The result of `git diff src/qt/bitcoinstrings.cpp`:
```diff
--- a/src/qt/bitcoinstrings.cpp
+++ b/src/qt/bitcoinstrings.cpp
@@ -126,6 +126,7 @@ QT_TRANSLATE_NOOP("bitcoin-core", ""
"You need to rebuild the database using -reindex to go back to unpruned "
"mode. This will redownload the entire blockchain"),
QT_TRANSLATE_NOOP("bitcoin-core", "%s is set very high!"),
+QT_TRANSLATE_NOOP("bitcoin-core", "(press q to shutdown and continue later)"),
QT_TRANSLATE_NOOP("bitcoin-core", "-maxmempool must be at least %d MB"),
QT_TRANSLATE_NOOP("bitcoin-core", "A fatal internal error occurred, see debug.log for details"),
QT_TRANSLATE_NOOP("bitcoin-core", "Cannot resolve -%s address: '%s'"),
@@ -204,6 +205,8 @@ QT_TRANSLATE_NOOP("bitcoin-core", "SQLiteDatabase: Failed to prepare statement t
QT_TRANSLATE_NOOP("bitcoin-core", "SQLiteDatabase: Failed to read database verification error: %s"),
QT_TRANSLATE_NOOP("bitcoin-core", "SQLiteDatabase: Unexpected application id. Expected %u, got %u"),
QT_TRANSLATE_NOOP("bitcoin-core", "Section [%s] is not recognized."),
+QT_TRANSLATE_NOOP("bitcoin-core", "Settings file could not be read"),
+QT_TRANSLATE_NOOP("bitcoin-core", "Settings file could not be written"),
QT_TRANSLATE_NOOP("bitcoin-core", "Signing transaction failed"),
QT_TRANSLATE_NOOP("bitcoin-core", "Specified -walletdir \"%s\" does not exist"),
QT_TRANSLATE_NOOP("bitcoin-core", "Specified -walletdir \"%s\" is a relative path"),
@@ -242,4 +245,5 @@ QT_TRANSLATE_NOOP("bitcoin-core", "User Agent comment (%s) contains unsafe chara
QT_TRANSLATE_NOOP("bitcoin-core", "Verifying blocks…"),
QT_TRANSLATE_NOOP("bitcoin-core", "Verifying wallet(s)…"),
QT_TRANSLATE_NOOP("bitcoin-core", "Wallet needed to be rewritten: restart %s to complete"),
+QT_TRANSLATE_NOOP("bitcoin-core", "press q to shutdown"),
};
```
ACKs for top commit:
ryanofsky:
Code review ACK b59b31ae0b04054c5cf225dad87046d3771707fc. Being able to use `_()` macro in qt would allow simplifying some code, for example replacing repetitive:
TheCharlatan:
ACK b59b31ae0b04054c5cf225dad87046d3771707fc
Tree-SHA512: 13d9d86b487a1b6e718ae96c198a0a927c881bf33df318412793ec9efba3a7e59cfa836204f73f5b53ff4c99edce778c11bffaa88138b80e37b71e36df6b816f
2ce7e31d2d6bf0e4d831e35563bcf90c68c39650 docs: Add reference to total.coverage report (Greg Sanders)
Pull request description:
I couldn't get fuzz coverage to work, so couldn't verify what it would look like if made.
ACKs for top commit:
darosior:
ACK 2ce7e31d2d6bf0e4d831e35563bcf90c68c39650
dergoegge:
ACK 2ce7e31d2d6bf0e4d831e35563bcf90c68c39650
Tree-SHA512: c773ef28f7cc630cb094740805c0d235255e7b0559ac7fed4e8952ade4005335f3787bdd2f8786979e341e914b41c4f3eef02bb4773b15f92d618f1040f53c54
