849f20a6d3e437631a07469d3c4af5faa0aa06ed ci: create and use non-root user for docker image (josibake)
Pull request description:
Previously, everything in the ci docker image ran as the root user. This would lead to certain directories (`ci/scratch`, `depends`) being owned by `root` after running the ci locally which would lead to annoying behavior such as subsequent guix builds failing due to `depends/` being owned by root.
This PR adds a non-root user in the container and chowns the mounted working directory. All the `docker exec` commands now run as the non-root user, except for the few that still need to run as root (mainly, installing packages).
To test this I checked out a fresh copy of the repo, applied my changes, ran the CI, and verified all the local file permissions were unchanged after the CI was finished running.
ACKs for top commit:
hebasto:
ACK 849f20a6d3e437631a07469d3c4af5faa0aa06ed, tested on Ubuntu 22.04 by running commands as follows:
Tree-SHA512: 734dca0f36157fce5fab243b4ff657fc17ba980e8e4e4644305f41002ff21bd5cef02c306ea1e0b5c841d4c07c095e8e4be16722e6a38c890717c60a3f5ec62a
31d0067f8bccd6b090bf88bad8472bb5cb1f20a4 doc: test: update/fix TestShell example instructions (Sebastian Falbesoner)
Pull request description:
This PR tackles two issues in the TestShell documentation:
- add missing instruction for creating a wallet prior to the `getnewaddress` call (needed as there is no default wallet created anymore since v0.21)
- fix `generatetoaddress` call syntax (the scripted-diff in commit fa0b916971e5bc23ad6396831940a2899ca05402 only worked for tests using `BitcoinTestFramework`)
ACKs for top commit:
fanquake:
ACK 31d0067f8bccd6b090bf88bad8472bb5cb1f20a4 - current instructions don't work. These do.
Tree-SHA512: d2b7808a06892ad16728cb2b6d4a72b255ad711d27fe98b1de562f80444e7bb25d73296abdde4308162fe3be702864e2f7b7dbbbb000fe54c709951c09e6c730
f362920c2ca85853b24c1cbc1d596dd1ff2fd556 doc: clarify that NetPermissionFlags::Implicit is only about whitelists (Vasil Dimov)
Pull request description:
`NetPermissionFlags::Implicit` applies just to connections from `-whitebind` or `-whitelist`, clarify that in its comment.
ACKs for top commit:
Zero-1729:
crACK f362920c2ca85853b24c1cbc1d596dd1ff2fd556
aureleoules:
ACK f362920c2ca85853b24c1cbc1d596dd1ff2fd556
hernanmarino:
re ACK f362920c2ca85853b24c1cbc1d596dd1ff2fd556
Tree-SHA512: 03f6f8be221c6819bdd0b5b56b69b4e3a6dd25e5ca5a247eeb1261113144b9b74cf064a0b7815317782a0a18365dd3dab97963bd238e9b231dbe7e1cf0395683
0f6cd72237b5c819340e524681b5c95f6a81e0c2 test: Fix intermittent failure in rpc_net.py (Martin Zumsande)
Pull request description:
Fixes#26552.
The problem was that calling `disconnect_p2ps` waits until `self.num_test_p2p_connections() == 0`.
`num_test_p2p_connections()` checks the field `subver` in `getpeerinfo` to distinguish p2p nodes from full nodes.
However, if we are dealing with a p2p connection that has never sent a version, the node has never received the special subversion and the wait is ineffective (we continue even though the disconnection is not yet completed).
Fix this by not using `disconnect_p2ps`.
ACKs for top commit:
MarcoFalke:
ACK 0f6cd72237b5c819340e524681b5c95f6a81e0c2
Tree-SHA512: ebdc78498db6971ae2f9b494dc76b35de46155bf191ce82ee04162592d0d9ec1272901992406d530fa46fb52cd815c4b91350824578292df14986584bc60b90a
b89530483d39f6a6a777df590b87ba2fad8c8b60 util: move threadinterrupt into util (fanquake)
Pull request description:
Alongside thread and threadnames. It's part of libbitcoin_util.
ACKs for top commit:
ryanofsky:
Code review ACK b89530483d39f6a6a777df590b87ba2fad8c8b60. No changes since last review other than rebase
theuni:
ACK b89530483d39f6a6a777df590b87ba2fad8c8b60.
Tree-SHA512: 0421f4d1881ec295272446804b27d16bf63e6b62b272f8bb52bfecde9ae6605e8109ed16294690d3e3ce4b15cc5e7c4046f99442df73adb10bdf069d3fb165aa
fa2d01470a9f1a91f35ed8013635ac47dabd868b test: Use type-safe NodeSeconds for TestMemPoolEntryHelper (MacroFake)
Pull request description:
test-only refactor to drop the deprecated `GetTime` in favour of the type-safe alternative
ACKs for top commit:
aureleoules:
ACK fa2d01470a9f1a91f35ed8013635ac47dabd868b - verified that there is no behavior change
Tree-SHA512: 5b64dae19c7bba9e8d90377c85891bc86f60ffbe67ea28d5ed3bd38f6dc30d3fbfba00bf49a16792922bddf83a52c632b6e5e5d8ffe1619fd9bf63effc60d59a
Running all commands as the root user in the docker image
will change local file permissions in the ci and depends directory.
Add a non-root user to the container and use this user whenever
possible when running docker exec commands.
40bdc8a6e4dcf7faf90f63b0912c18d72436e37f test: remove unused class `NodePongAdd1` (Sebastian Falbesoner)
Pull request description:
This class was introduced in commit fa3365430c5fb57d7c0b5f2bce9fbbe290be93c3 ("net: Use mockable time for ping/pong, add tests"), but actually never used.
ACKs for top commit:
stickies-v:
ACK 40bdc8a6e
Tree-SHA512: b5a6552e4f2e0b7e368a071cc53b9a8e6f5d1950565a9fda8eb1971a01d8be0541d066842723ef44174fe8189925fa36f2defb6d7bf8d104abc77de410cc4c13
0eeb9b0442fb2f2da33c04704eefe6a84d28e981 [fuzz] Move ConsumeNetAddr to fuzz/util/net.h (dergoegge)
291c8697d4be0f38635b67880107e39d3ec585ad [fuzz] Make ConsumeNetAddr produce valid onion addresses (dergoegge)
c9ba3f836e1646875d2f96f1f466f8a83634a6f7 [netaddress] Make OnionToString public (dergoegge)
Pull request description:
The chance that the fuzzer is able to guess a valid onion address is probably slim, as they are Base32 encoded and include a checksum. Right now, any target using `ConsumeNetAddr` would have a hard time uncovering bugs that require valid onion addresses as input.
This PR makes `ConsumeNetAddr` produce valid onion addresses by using the 32 bytes given by the fuzzer as the pubkey for the onion address and forming a valid address according to the torv3 spec.
ACKs for top commit:
vasild:
ACK 0eeb9b0442fb2f2da33c04704eefe6a84d28e981
brunoerg:
ACK 0eeb9b0442fb2f2da33c04704eefe6a84d28e981
Tree-SHA512: 7c687a4d12f9659559be8f0c3cd4265167d1261d419cfd3d503fd7c7f207cc0db745220f02fb1737e4a5700ea7429311cfc0b42e6c15968ce6a85f8813c7e1d8
This class was introduced in commit
fa3365430c5fb57d7c0b5f2bce9fbbe290be93c3 ("net: Use mockable time for
ping/pong, add tests"), but actually never used.
cc597bd56d91fad7a9a1c6e2db130febac38ff34 src/bitcoin-cli.cpp: -getinfo help - grammar correction (@RandyMcMillan)
Pull request description:
Top commit has no ACKs.
Tree-SHA512: a5321968d0d377e1481170b4220a1319bf9040ec198b27c011609a5b7a81e9193500b750980c7de423b8b99655ed0f7772a9621e0b230aa6cc5d7b48167ed4f9
c8dc0e3eaa9e0f956c5177bcb69632beb0d51770 refactor: Inline `CTxMemPoolEntry` class's functions (Hennadii Stepanov)
75bbe594e54402ed248ecf2bdfe54e8283d20fff refactor: Move `CTxMemPoolEntry` class to its own module (Hennadii Stepanov)
Pull request description:
This PR:
- gets rid of the `policy/fees` -> `txmempool` -> `policy/fees` circular dependency
- is an alternative to #13949, which nukes only one circular dependency
ACKs for top commit:
ryanofsky:
Code review ACK c8dc0e3eaa9e0f956c5177bcb69632beb0d51770. Just include and whitespace changes since last review, and there's a moveonly commit now so it's very easy to review
theStack:
Code-review ACK c8dc0e3eaa9e0f956c5177bcb69632beb0d51770
glozow:
utACK c8dc0e3eaa9e0f956c5177bcb69632beb0d51770, agree these changes are an improvement.
Tree-SHA512: 36ece824e6ed3ab1a1e198b30a906c8ac12de24545f840eb046958a17315ac9260c7de26e11e2fbab7208adc3d74918db7a7e389444130f8810548ca2e81af41
6630a1e8448c633e4abaa8f5903f11cac6f433a7 Add warning on first startup if free disk space is less than necessary (Ben Woosley)
Pull request description:
This reworks/revives https://github.com/bitcoin/bitcoin/pull/15848 to add a check for low disk space on first startup and issue a warning if disk space is below the expected space required to accommodate the blocks.
This PR was fashioned by a team of developers at the [bitcoin++](https://www.btcplusplus.dev/) conference workshop: "[Let's contribute to Bitcoin Core](https://sched.co/12P6Z)"
Fixes#15813
ACKs for top commit:
achow101:
ACK 6630a1e8448c633e4abaa8f5903f11cac6f433a7
willcl-ark:
tACK 6630a1e8448c633e4abaa8f5903f11cac6f433a7 rebased on master. Warning shows on first start but not on restart after some blocks have been downloaded.
aureleoules:
ACK 6630a1e8448c633e4abaa8f5903f11cac6f433a7
pablomartin4btc:
re-ACK 6630a1e844
hernanmarino:
ReACK 6630a1e844
Tree-SHA512: 0f18acabdf2b514e96e2eea8f304960b952226b83dc91334cf7d1f6355ea2f257aaec0ee38d43ac36435385ecd918333d20657c35a8a7407e7cf2680ccb643bb
fa68d086f33cb4fc13877308d21b6344a804a222 test: Add getpeerinfo test for missing version message (MacroFake)
Pull request description:
There seems to be a lot of discussion about behaviour/code that is completely untested.
Fix this by adding a test. The test documents the current behaviour and helps to detect when the behaviour changes in the future.
ACKs for top commit:
jonatack:
ACK fa68d086f33cb4fc13877308d21b6344a804a222
mzumsande:
Code Review ACK fa68d086f33cb4fc13877308d21b6344a804a222
Tree-SHA512: d092b30d5bdb46712c91a7c5bd2d0c82a0da281f1460967aa4e32c648b15d8d97870ded9565a90af34874eb468aad8b99694a2485af6807994e7cfc05482aa8c
b93beef5ed516af2e652a61c34a563dadb647530 doc: Mac -> macOS in release notes template (fanquake)
2747adb68abc13ad471010f03a723915cd222b35 doc: Add 24.0 release notes (fanquake)
Pull request description:
Same as past releases.
ACKs for top commit:
stickies-v:
ACK b93beef5ed516af2e652a61c34a563dadb647530
Tree-SHA512: c28bc7286f330a6058ae266b238468044439457ff5b9df191232d91dc17b8092facd6ed72accec8bc9db10f055f7bb7e06700cc1ed0bd045fc15f612bc023a46
ac410e6fc0fceb568d4fb3d398724ca859a8be25 log: improve some validation log messages to include hashPrevBlock (Skuli Dulfari)
Pull request description:
When there is an issue with a previous block the current log messages do not indicate hashPrevBlock. Adding it makes debugging easier.
ACKs for top commit:
stickies-v:
ACK ac410e6fc0fceb568d4fb3d398724ca859a8be25
aureleoules:
reACK ac410e6fc0fceb568d4fb3d398724ca859a8be25
theStack:
ACK ac410e6fc0fceb568d4fb3d398724ca859a8be25
Tree-SHA512: d91481321f4474bb4fdf6ad55d1c897437b631b0a12308815c4ac5b053c8a76726e2d93f2aa0701e8cfd48fba7fad19ef5ffca3c67d3aa973dc593df806f1757
8a5014cd8a05b3ab86ae34a47653a82ce11bdf17 Fixes bitcoin#26490 by preventing notifications (John Moffett)
Pull request description:
This is a PR to address https://github.com/bitcoin/bitcoin/issues/26490
The menu bar currently subscribes to window focus change notifications to enable or disable certain menu options in response to the window status.
Notifications are automatically unsubscribed (disconnected in Qt parlance) if the sender is deleted -- in this case, the sender is the QTApplication object (`qApp`). However, MacOS 13 sends a window focus change notification *after* the main window has been destroyed but *before* `qApp` has been fully destroyed.
Since the menu bar is deleted in the main window's destructor, it no longer exists when it receives these notifications (in two different places via lambda expressions). The solution is to pass the main window (`this`) as context when subscribing to the notifications. In this [overloaded version](https://doc.qt.io/qt-5/qobject.html#connect-1) of `connect`, Qt automatically unsubscribes to notifications if the sender OR context (here the main window object) is destroyed. Since the spurious notifications are sent after the main window object is destroyed, this change prevents them from being sent.
Tested on Mac OS 13 and 12 only.
ACKs for top commit:
hebasto:
ACK 8a5014cd8a05b3ab86ae34a47653a82ce11bdf17
Tree-SHA512: 3dff0a252fe0e93dd68cf5503135ecf6a72bcf385ba38407d6021ab77cca323f8bbe58aeca90ec124aa2a22ab9d35b706946179ac3b5d171c96a7010de51a090
Tackles two issues in the TestShell documentation:
- add missing instruction for creating a wallet prior to the
`getnewaddress` call (needed as there is no default wallet created
anymore since v0.21)
- fix `generatetoaddress` call syntax (the scripted-diff in commit
fa0b916971e5bc23ad6396831940a2899ca05402 only worked for tests using
`BitcoinTestFramework`)
7a53033303f25301675fb2d7c7b7032166807910 Fix Transaction Relay tooltip text in Peers details window (Jon Atack)
Pull request description:
as a value of N/A could occur due to a lock or a disconnection race but not during connection setup -- see https://github.com/bitcoin/bitcoin/pull/26457#pullrequestreview-1181641835. Credit to Martin Zumsande for finding this.
ACKs for top commit:
jarolrod:
ACK 7a53033303f25301675fb2d7c7b7032166807910
Tree-SHA512: 031779567e927f05f6fae02394a8c97ba5c45ba9fffd7f1e2c006e152df5f724d92a06f18a4c2540436476eca6b40a3a5cbc4421666cd576439b823668acfcfb
2222ec71fdf573a15bb593fc0dd42d2d28ca5449 util: Move error message formatting of NonFatalCheckError to cpp (MacroFake)
Pull request description:
This allows to strip down the header file.
ACKs for top commit:
hebasto:
re-ACK 2222ec71fdf573a15bb593fc0dd42d2d28ca5449, only rebased and suggested changes since my recent [review](https://github.com/bitcoin/bitcoin/pull/25112#pullrequestreview-1182361605).
aureleoules:
ACK 2222ec71fdf573a15bb593fc0dd42d2d28ca5449
Tree-SHA512: 313b3c891bb000cf606df1793b068f93df99915a254fbd67a45f003d440cce7355cdcc6b196f35757cc02d3697970d30e9de0d675f2aa8eb74107c13d663927a
fa84df1f033a5d1a8342ea941eca0b5ef73d78e7 scripted-diff: wallet: rename AvailableCoinsParams members to snake_case (furszy)
61c2265629fdf11a2cc266fad54ceb0a1247bb5e wallet: group AvailableCoins filtering parameters in a single struct (furszy)
f0f6a3577bef2e9ebd084fe35850e4e9580128a9 RPC: listunspent, add "include immature coinbase" flag (furszy)
Pull request description:
Simple PR; adds a "include_immature_coinbase" flag to `listunspent` to include the immature coinbase UTXOs on the response. Requested by #25728.
ACKs for top commit:
danielabrozzoni:
reACK fa84df1f033a5d1a8342ea941eca0b5ef73d78e7
achow101:
ACK fa84df1f033a5d1a8342ea941eca0b5ef73d78e7
aureleoules:
reACK fa84df1f033a5d1a8342ea941eca0b5ef73d78e7
kouloumos:
reACK fa84df1f033a5d1a8342ea941eca0b5ef73d78e7
theStack:
Code-review ACK fa84df1f033a5d1a8342ea941eca0b5ef73d78e7
Tree-SHA512: 0f3544cb8cfd0378a5c74594480f78e9e919c6cfb73a83e0f3112f8a0132a9147cf846f999eab522cea9ef5bd3ffd60690ea2ca367dde457b0554d7f38aec792
db929893ef0bc86ea2708cdbcf41152240cd7c73 Faster -reindex by initially deserializing only headers (Larry Ruane)
c72de9990ae8f1744006d9c852023b882d5ed80c util: add CBufferedFile::SkipTo() to move ahead in the stream (Larry Ruane)
48a68908ba3d5e077cda7bd1e908b923fbead824 Add LoadExternalBlockFile() benchmark (Larry Ruane)
Pull request description:
### Background
During the first part of reindexing, `LoadExternalBlockFile()` sequentially reads raw blocks from the `blocks/blk00nnn.dat` files (rather than receiving them from peers, as with initial block download) and eventually adds all of them to the block index. When an individual block is initially read, it can't be immediately added unless all its ancestors have been added, which is rare (only about 8% of the time), because the blocks are not sorted by height. When the block can't be immediately added to the block index, its disk location is saved in a map so it can be added later. When its parent is later added to the block index, `LoadExternalBlockFile()` reads and deserializes the block from disk a second time and adds it to the block index. Most blocks (92%) get deserialized twice.
### This PR
During the initial read, it's rarely useful to deserialize the entire block; only the header is needed to determine if the block can be added to the block index immediately. This change to `LoadExternalBlockFile()` initially deserializes only a block's header, then deserializes the entire block only if it can be added immediately. This reduces reindex time on mainnet by 7 hours on a Raspberry Pi, which translates to around a 25% reduction in the first part of reindexing (adding blocks to the index), and about a 6% reduction in overall reindex time.
Summary: The performance gain is the result of deserializing each block only once, except its header which is deserialized twice, but the header is only 80 bytes.
ACKs for top commit:
andrewtoth:
ACK db929893ef0bc86ea2708cdbcf41152240cd7c73
achow101:
ACK db929893ef0bc86ea2708cdbcf41152240cd7c73
aureleoules:
ACK db929893ef0bc86ea2708cdbcf41152240cd7c73 - minor changes and new benchmark since last review
theStack:
re-ACK db929893ef0bc86ea2708cdbcf41152240cd7c73
stickies-v:
re-ACK db929893e
Tree-SHA512: 5a5377192c11edb5b662e18f511c9beb8f250bc88aeadf2f404c92c3232a7617bade50477ebf16c0602b9bd3b68306d3ee7615de58acfd8cae664d28bb7b0136
fa4ec1be513c80d6db644f1e3170e980035e7306 test: Split overly large util_tests.cpp file (MacroFake)
Pull request description:
The file has ~3kLOC and is slow to compile.
Fix both issues by splitting it. (On my machine the compilation goes from 25 seconds previously to 17+10 seconds for the two smaller files)
To review, `--color-moved=dimmed-zebra` can be used.
ACKs for top commit:
RandyMcMillan:
ACK fa4ec1be513c80d6db644f1e3170e980035e7306 for:
shaavan:
ACK fa4ec1be513c80d6db644f1e3170e980035e7306
aureleoules:
ACK fa4ec1be513c80d6db644f1e3170e980035e7306
Tree-SHA512: 4719439c7ee0c6c06b6f6ccf07b3a037c0cae58b1bd6e6e929ebfeab8403be3d1905581669ed733bff0cbf4e385c27ae58d519ce031e145e6889bd5bce1c1d03
MacOS 13 sends a window focus change notification after the main
window has been destroyed but before the QTApplication has been
destroyed. This results in the menu bar receiving a notification
despite it no longer existing. The solution is to pass the main
window as context when subscribing to the notifications. Qt
automatically unsubscribes to notifications if the sender OR
context is destroyed.
2dede9f67596787e18904d3d5961f48ec75f241e Adjust RPCTypeCheckObj error string (Leonardo Araujo)
Pull request description:
Unifies the JSON type error strings as mentioned in #26214. Also refer to #25737.
ACKs for top commit:
furszy:
ACK 2dede9f6
Tree-SHA512: c918889e347ba32cb6d0e33c0de5956c2077dd40c996151e16741b0c4983ff098c60258206ded76ad7bbec4876c780c6abb494a97e4f1e05717d28a59b9167a6
fa095257511e53d7a593c6714724aafb484e6b6f univalue: string_view test (MacroFake)
1111c7e3f1f5c550f62016d71ccc518aafd97acf univalue: Avoid std::string copies (MacroFake)
Pull request description:
This shouldn't matter too much, unless a really large string is pushed into a json struct, but I think it also clarifies the code.
ACKs for top commit:
martinus:
Code review ACK fa09525751
aureleoules:
reACK fa095257511e53d7a593c6714724aafb484e6b6f
ryanofsky:
Code review ACK fa095257511e53d7a593c6714724aafb484e6b6f
Tree-SHA512: 74c441912bd0b00cdb9ea7890121f71ae5d62a7594e7d29aa402c9e3f033710c5d3afb27a37c552e6513804b249aa37e375ce013a3db853a25d1fd7b6e6cd3a8
737c285f6955253b4bb6f5e2cb024cd79aca8ee4 test: Don't pass add_to_wallet option to walletcreatefundedpsbt (Ryan Ofsky)
Pull request description:
It's not a documented option. Noticed while working on #19762
ACKs for top commit:
achow101:
ACK 737c285f6955253b4bb6f5e2cb024cd79aca8ee4
Tree-SHA512: 1bf4186fae4390233b2f23389eb6c515c7f0209f12553592df5166e75c452ccd1fb125d9246047c08cff0b869fdda7793812d15da01441e2c4777514446f3ed6
887d85e43d136dbfc2428f873ced3de50076bbd0 test: add missing bech32m / BIP86 test-cases to wallet_descriptor.py (Sebastian Falbesoner)
Pull request description:
This small PR adds missing "bech32m" address type / BIP86 checks w.r.t. to the `getnewaddress`/`getrawchangeaddress` RPC and descriptor export functionality to the functional test `wallet_descriptor.py`.
ACKs for top commit:
shaavan:
ACK 887d85e43d136dbfc2428f873ced3de50076bbd0
kristapsk:
ACK 887d85e43d136dbfc2428f873ced3de50076bbd0
Tree-SHA512: 05b443ae14138769dc3c87a0178f21db2698fa5bcbeaa953c50ed0c9cf5dcd1effcf4afd09551ca9f4ce73898a7882caaf4c57078767beb6a6a65eb3a662726d
c8f91478c10affdfba8e52200c85379052f4e1b1 test: Avoid collision with valid path names in `getarg_tests/logargs` (Hennadii Stepanov)
Pull request description:
This PR prevents test failure when "private" is a part of a valid path.
For example, `/private/var` is a valid path on macOS for temporary files, which in turn causes test failure on CI for tests managed by the [CTest](https://github.com/bitcoin/bitcoin/pull/25797) framework.
ACKs for top commit:
MarcoFalke:
ACK c8f91478c10affdfba8e52200c85379052f4e1b1
Tree-SHA512: 09d257f8fa6be903ec8092b2ae92887a4bec2d05085c76c110637657f4a4bfe2714bf87e2e4727719b3624c8fa4c835ce2ca259c2c9c93033837f997b2057e4f
