30a94b1ab9 test, wallet: Remove concurrent writes test (Ava Chow)
b44b7c03fe wallet: Write best block record on unload (Ava Chow)
876a2585a8 wallet: Remove unnecessary database Close step on shutdown (Ava Chow)
98a1a5275c wallet: Remove chainStateFlushed (Ava Chow)
7fd3e1cf0c wallet, bench: Write a bestblock record in WalletMigration (Ava Chow)
6d3a8b195a wallet: Replace chainStateFlushed in loading with SetLastBlockProcessed (Ava Chow)
7bacabb204 wallet: Update best block record after block dis/connect (Ava Chow)
Pull request description:
Implements the idea discussed in https://github.com/bitcoin/bitcoin/pull/29652#issuecomment-2010579484
Currently, `m_last_block_processed` and `m_last_block_processed_height` are not guaranteed to match the block locator stored in the wallet, nor do either of those fields actually represent the last block that the wallet is synced up to. This is confusing and unintuitive.
This PR changes those last block fields to be updated whenever the wallet makes a change to the db for new transaction state found in new blocks. Whenever a block is received that contains a transaction relevant to the wallet, the last block locator will now be written to disk. Furthermore, every block disconnection will now write an updated locator.
To ensure that the locator is relatively recent and loading rescans are fairly quick in the event of unplanned shutdown, it is also now written every 144 blocks (~1 day). Additionally it is now written when the wallet is unloaded so that it is accurate when the wallet is loaded again.
Lastly, the `chainstateFlushed` notification in the wallet is changed to be a no-op. The best block locator record is no longer written when `chainstateFlushed` is received from the node since it should already be mostly up to date.
ACKs for top commit:
rkrux:
ACK 30a94b1ab9
mzumsande:
Code Review ACK 30a94b1ab9
ryanofsky:
Code review ACK 30a94b1ab9. Only changes since last review are using WriteBestBlock method more places and updating comments.
Tree-SHA512: 46117541f8aaf13dde57430e813b4bbbd5e146e2632769675803c8e65a82f149a7cc6026489a127d32684b90124bd2b7c28216dbcfa6a47447300e8f3814e029
fa982f1425 Use WitnessV0KeyHash in TestAddAddressesToSendBook (MarcoFalke)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/32558
Fixes https://github.com/bitcoin-core/gui/issues/874
This fixes a bug introduced in commit fafee85358, which changed the type of the dummy address from `WitnessV0KeyHash` to `PKHash`. It was expected that this is fine, given that this is just a dummy address. However, the base58 characters can include the substring "io", leading to test failures later on.
Fix it by just using `WitnessV0KeyHash` again.
For reference, a passing test could look like:
```
Model contains 2 rows and 2 columns.
--- Model Data ---
Row 0 : "io - new A\tmxgkqJWAwfUwbgzZUsWrG1stKWV6fDn8YH"
Row 1 : "io - new B\tmhsxP2yrYDQiEncT8HzKxQSFSFJmUsudsP"
------------------
```
A failing test could look like:
```
Model contains 3 rows and 2 columns.
--- Model Data ---
Row 0 : "already here (s)\tmyDFZSKDQdPMMoSQgzkDtq2yioo8DA8qCX"
Row 1 : "io - new A\tmsAqQKjMrbxYRDhGXBBJ3yUEQxj5Bf5Njz"
Row 2 : "io - new B\tmtALQiit8dw33kznVfHDgE38ohfgz2Pchc"
------------------
FAIL! : AddressBookTests::addressBookTests() Compared values are not the same
Actual (table_view->model()->rowCount()): 3
Expected (2) : 2
Loc: [qt/test/addressbooktests.cpp(219)]
```
ACKs for top commit:
achow101:
ACK fa982f1425
hebasto:
ACK fa982f1425, I have reviewed the code along with the related changes from https://github.com/bitcoin/bitcoin/pull/32511.
Tree-SHA512: f55d7fe4193a0706e1a3ca1a2c0fbf2f04dc5b177699add00013ec56d64218ac85b80dad6e99f9fde26f4c9fca79f99e68ded057c5862364064404ac06b77176
e49a7274a2 rpc: Avoid join-split roundtrip for user:pass for auth credentials (Vasil Dimov)
98ff38a6f1 rpc: Perform HTTP user:pass split once in `RPCAuthorized` (laanwj)
879a17bcb1 rpc: Store all credentials hashed in memory (laanwj)
4ab9bedee9 rpc: Undeprecate rpcuser/rpcpassword, change message to security warning (laanwj)
Pull request description:
This PR does two things:
### Undeprecate rpcuser/rpcpassword, change message to security warning
Back in 2015, in https://github.com/bitcoin/bitcoin/pull/7044, we added configuration option `rpcauth` for multiple RPC users. At the same time the old settings for single-user configuration `rpcuser` and `rpcpassword` were "soon" to be deprecated.
The main reason for this deprecation is that while `rpcpassword` stores the password in plain text, `rpcauth` stores a hash, so it doesn't appear in the configuration in plain text.
As the options are still in active use, actually removing them is expected to be a hassle to many, and it's not clear that is worth it. As for the security risk, in many kinds of setups (no wallet, containerized, single-user-single-application, local-only, etc) it is an unlikely point of escalation.
In the end, it is good to encourage secure practices, but it is the responsibility of the user. Log a clear warning but remove the deprecation notice (this is also the only place where the options appear as deprecated, they were never marked as such in the -help output).
<hr>
### Store all credentials hashed in memory
This gets rid of the special-casing of `strRPCUserColonPass` by hashing cookies as well as manually provided `-rpcuser`/`-rpcpassword` with a random salt before storing them.
Also take the opportunity to modernize the surrounding code a bit. There should be no end-user visible differences in behavior.
<hr>
Closes#29240.
ACKs for top commit:
1440000bytes:
utACK e49a7274a2
janb84:
reACK e49a7274a2
vasild:
ACK e49a7274a2
Tree-SHA512: 7162848ada4545bc07b5843d1ab6fb7e31fb26de8d6385464b7c166491cd122eac2ec5e70887c414fc136600482df8277dc0cc0541d7b7cf62c4f72e25bb6145
We don't add or maintain these, and they are of little value, as
well as having the effect of polluting diffs.
They are also wrong, i.e DEFAULT_SCRIPTCHECK_THREADS is not in
validation.h.
fa330a5e38 doc: Adjust stale MSVC bug url (MarcoFalke)
Pull request description:
The old url is stale, so use the current one. See https://github.com/bitcoin/bitcoin/pull/32552#issuecomment-2889188342
ACKs for top commit:
hebasto:
ACK fa330a5e38.
Tree-SHA512: eb7813edb85f4bac06807bf2a35caf106a0faeffcbd0a4732c57edac07db00171f720888dbb7c35fae3fd515408ea7cb8e5dcbd3cf393caec23904cb800b8907
86de8c1668 scripted-diff: test: remove 'descriptors=True' argument for `createwallet` calls (Sebastian Falbesoner)
Pull request description:
Descriptor wallets are already created by default [since v23.0](7710a31f0c/doc/release-notes/release-notes-23.0.md (L171)), but since the recent legacy wallet removal the `descriptors` parameter *must* be True for the `createwallet` RPC (see commit 9f04e02ffa), i.e. still passing it wouldn't contain any information for test readers anymore. So simply drop them in the functional tests in order to reduce code bloat. The only exception is calls to older versions, which happens in `wallet_backwards_compatibility.py` and is explicitly excluded in the scripted diff.
ACKs for top commit:
Sjors:
ACK 86de8c1668
maflcko:
lgtm ACK 86de8c1668
Tree-SHA512: 1acfae27bd960aeef9e1cf6e3f042752164a4d6869773c42df4c22c03dde0922993a3220fa14d52e75a0ff1f48c5194932b74a21427efbd496b0aaad7a2eafb2
The watch-only functionality in the GUI was only used for legacy wallets.
Watch-only descriptor wallets do not use this.
The only visible changes of this commit are:
- dropped "Spendable:" label from the overview tab
- column width cache is reset
Logging the wallet version before anything has been read from disk results
in the wrong version being logged.
Also split the last client version logging as it may not always be
present to be logged.
Descriptor wallets are already created by default since v23.0, but
since the recent legacy wallet removal this parameter *must* be True
(see commit 9f04e02ffa), i.e. still
passing it wouldn't contain any information for test readers
anymore. So simply drop them in the functional tests in order to
reduce code bloat.
-BEGIN VERIFY SCRIPT-
sed -i 's/, descriptors=True//g' $(git ls-files -- 'test/functional' ':(exclude)test/functional/wallet_backwards_compatibility.py')
sed -i '/descriptors=True,/d' ./test/functional/mempool_persist.py
-END VERIFY SCRIPT-
b104d44227 test: Remove RPCOverloadWrapper (Ava Chow)
4d32c19516 test: Replace importpubkey (Ava Chow)
fe838dd391 test: Replace usage of addmultisigaddress (Ava Chow)
d314207779 test: Replace usage of importaddress (Ava Chow)
fcc457573f test: Replace importprivkey with wallet_importprivkey (Ava Chow)
94c87bbbd0 test: Remove unnecessary importprivkey from wallet_createwallet (Ava Chow)
Pull request description:
`RPCOverloadWrapper` implemented overloads for legacy wallet only RPCs so that the same function call could be used within tests for both legacy wallets and descriptor wallets. With legacy wallets now removed, there is no need to continue to have these overloads.
For `importaddress`, `addmultisigaddress`, and `importpubkey`, the uses of these are converted to `importdescriptors`.
For `importprivkey`, a new helper function `wallet_imporprivkey` is introduced that does what the overload did. This is mainly to reduce verbosity as `importprivkey` was more widely used throughout the tests.
Some tests that used these RPCs are now also no longer relevant and have been removed.
ACKs for top commit:
Sjors:
ACK b104d44227
pablomartin4btc:
cr ACK b104d44227
rkrux:
ACK b104d44227
w0xlt:
ACK b104d44227
Tree-SHA512: ded2f73829e2ce28466d4a9738eb382783ad990daee5d1859dbc4d354e6f8eec0c483ed5ecb1287fe0dd24ac332065b733a30d71b126b841bd7cd49e9a094b6d
fa1f10a49e doc: Fix minor typos in rpc help (MarcoFalke)
fae840e94b rpc: Reject beginning newline in RPC docs (MarcoFalke)
fa414eda08 scripted-diff: Remove unused leading newline in RPC docs (MarcoFalke)
Pull request description:
It is harmless, but newlines in the beginning read a bit odd ("nReturns"). So just require them to not be present.
The diff is large, but a trivial scripted-diff.
ACKs for top commit:
fanquake:
ACK fa1f10a49e
w0xlt:
ACK fa1f10a49e
Tree-SHA512: 5d2f9632f42ec1c02814d050f223941f436e5b0df426d7d6eb93fdd0ff118d57185af07b271dd73af63735dd17231125826c0c9ce0aad36bc8829c5b050a628c
7015052eba build: remove Wsuggest-override suppression from leveldb build (fanquake)
e2c84b896f Squashed 'src/leveldb/' changes from 4188247086..113db4962b (fanquake)
Pull request description:
Pulls in
* https://github.com/bitcoin-core/leveldb-subtree/pull/51
Remove the related warning suppression.
ACKs for top commit:
l0rinc:
utACK 7015052eba
hebasto:
ACK 7015052eba, I've updated the `leveldb` subtree locally and got zero diff with this branch.
Tree-SHA512: 1ac7c8ecc9025086b429e12c22fc25f654eaf68fc9500b95341fb635cea12e7f80d69298cff120e8557a4f2f5809956a3b158cdb4db745cfa605c0df6f346423
When dealing with URI parts, it seems more consistent to use
corresponding SAFE_CHARS_URI mode in error messages.
Co-Authored-By: stickies-v <stickies-v@protonmail.com>
We need to determine if CJDNS is reachable before we parse any IPv6
addresses (for example, by the -rpcallowip setting) or an RFC4193
address might get flipped to CJDNS, which can not be used with subnets
fa2be605fe ci: Enable feature_init and wallet_reorgsrestore in valgrind task (MarcoFalke)
Pull request description:
The `fork()` isn't needed and in fact makes the forked process not react to signals (like kill or terminate), so just avoid it and run the valgrind process directly in the CI task.
Can be tested with something like:
`env -i HOME="$HOME" PATH="$PATH" USER="$USER" MAKEJOBS="-j$(nproc)" FILE_ENV="./ci/test/00_setup_env_native_valgrind.sh" ./ci/test_run_all.sh`
ACKs for top commit:
fanquake:
ACK fa2be605fe - x86_64, aarch64
Tree-SHA512: 6293447d501191598c08f0cb9fcb4ed91e4cfec11255e702a926346ef8011d6ebc0ca12e9a1c14fa53541318c4e05dee5c96dfe965dcf4cf833c9392158ab883
Currently the per-tx sigops limit standardness check (bounded by
`MAX_STANDARD_TX_SIGOPS_COST`, throwing "bad-txns-too-many-sigops"
if exceeded) is only indirectly tested with the much higher per-block
consensus limit (`MAX_BLOCK_SIGOPS_COST`), i.e. an increase in the
limit would still pass all tests. Refine that by splitting up the invalid
tx template `TooManySigops` in a per-block and a per-tx one.
The involved functional tests taking use of these templates are
`feature_block.py` and `p2p_invalid_txs.py`.
fab5a3c803 test: Remove unused verify_flags suppression (MarcoFalke)
Pull request description:
`static bool verify_flags(unsigned)` was removed in commit 80f8b92f4f
ACKs for top commit:
fanquake:
ACK fab5a3c803
hebasto:
ACK fab5a3c803, I have reviewed the code and it looks OK.
Tree-SHA512: da0cfc6ee253419c0aef316cd9c8366b744231261b755a95618ca0e777c1d95cecba8199db5486fd35079ded89c64c1a9f5b056f01dada4176b815b0d97261b7
8f4ba90b8f build: document why we check for std::system (fanquake)
Pull request description:
It's probably debatable if we support targets like iOS, but for now, document why we are checking for this standard library feature.
Trying to use `std::system` for a `aarch64-darwin-ios` target results in:
```bash
test.cpp:7:10: error: 'system' is unavailable: not available on iOS
7 | std::system("some_command");
| ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdlib.h:203:6: note: 'system' has been explicitly marked unavailable here
203 | int system(const char *) __DARWIN_ALIAS_C(system);
| ^
1 error generated.
```
ACKs for top commit:
Sjors:
ACK 8f4ba90b8f
Tree-SHA512: 219cac205b36004c607194f6956c2ce6153f192bd4349e505b52c4e511e403e195ce0f462ae10c515e67f1e95d4b1636d526c8e4376004044853b574a84df427
516f0689b5 refactor: re-enable UBSan implicit-sign-change in serialize.h (Lőrinc)
5827e93507 refactor: use consistent size type for serialization template parameters (Lőrinc)
Pull request description:
Inspired by https://github.com/bitcoin/bitcoin/pull/32154, the main goal of this PR is to reenable sanitizer checks for `serialize.h` since it's modified in a few other PRs.
ACKs for top commit:
maflcko:
review ACK 516f0689b5🎈
stickies-v:
ACK 516f0689b5, nice cleanup!
Tree-SHA512: 63da9bf1988a5b68e3c053b0ed786b8735f8f75b05763511522d1601b728b55798006e063137447615c266582622642d3226318fa83e488bd363f1756f8811e8
486bc91790 depends: bump to latest config.sub (Sebastian Falbesoner)
6880383427 depends: bump to latest config.guess (Sebastian Falbesoner)
Pull request description:
Noticed that these files were last updated from [upstream](https://git.savannah.gnu.org/gitweb/?p=config.git) quite a while ago (in 2023, see #28781), so bump them again.
Can be verified via e.g.
```
$ git clone https://git.savannah.gnu.org/git/config.git /tmp/config.git
$ diff /tmp/config.git/config.guess ./depends/config.guess
$ diff /tmp/config.git/config.sub ./depends/config.sub
```
ACKs for top commit:
fanquake:
ACK 486bc91790
Tree-SHA512: cbfd21a351a2404e5821610b6ef84d1050ea1a8045da8bfb535ef1ed49b5ad3f4140e8332d1eed545332f96d3117adc531d73aa83e19e7154fe382d041102c93
301993ebf7 init: drop -upnp (fanquake)
Pull request description:
This was slated for removal in `30.0`, so remove it.
ACKs for top commit:
i-am-yuvi:
ACK 301993ebf7
maflcko:
review ACK 301993ebf7
darosior:
tACK 301993ebf7
Tree-SHA512: 635e374c013fa08c4cda7caadc465c89bb376d3ee2c66f67a27e3ed9031844674d3e996169aaffb9b65a67b0d44d92aaec000aaf69abe3dd10fce2f4138f3e27
8f4fed7ec7 symbol-check: Add check for application manifest in Windows binaries (Hennadii Stepanov)
2bb6ab8f1b ci: Add "Get bitcoind manifest" steps to Windows CI jobs (Hennadii Stepanov)
282b4913c7 cmake: Add application manifests when cross-compiling for Windows (Hennadii Stepanov)
Pull request description:
Windows [application manifests ](https://learn.microsoft.com/en-us/windows/win32/sbscs/application-manifests) provide several benefits—such as enhanced security settings, and the ability to set a process-wide code page (required for https://github.com/bitcoin/bitcoin/pull/32380), as well as granular control over supported Windows versions. Most of these benefits lie beyond the scope of this PR and will be evaluated separately.
On the current master branch @ fc6346dbc8, the linker generates and embeds a manifest only when building with MSVC:
```xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
<security>
<requestedPrivileges>
<requestedExecutionLevel level="asInvoker" uiAccess="false"></requestedExecutionLevel>
</requestedPrivileges>
</security>
</trustInfo>
</assembly>
```
However, this manifest fails validation:
```
> mt.exe -nologo -inputresource:build\bin\Release\bitcoind.exe -validate_manifest
mt.exe : general error 10100ba: The manifest is missing the definition identity.
```
This PR unifies manifest embedding for both native and cross-compilation builds.
Here is the change in the manifest on Windows:
```diff
--- bitcoind-master.manifest
+++ bitcoind-pr.manifest
@@ -1,5 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
+ <assemblyIdentity type="win32" name="org.bitcoincore.bitcoind" version="29.99.0.0"></assemblyIdentity>
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
<security>
<requestedPrivileges>
```
which effectively resolves the "missing the definition identity" error.
Finally, “Get bitcoind manifest” steps have been added to the Windows CI jobs to ensure the manifest is embedded and validated.
ACKs for top commit:
sipsorcery:
re-tACK 8f4fed7ec7.
hodlinator:
re-ACK 8f4fed7ec7
davidgumberg:
Reviewed and tested ACK 8f4fed7ec7
Tree-SHA512: 6e2dbdc77083eafdc242410eb89a6678e37b11efd786505dcd7844f0bac8f44d68625e62924a03b26549bdb4aaec5330dc608e6b4d66789f0255092e23aef6cb
