42bbbba7c83d1e2baad18b4c6f05bad1358eb117 message-capture-parser: fix out of bounds error for empty vectors (Sebastian Falbesoner)
Pull request description:
The script [message-capture-parser.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/message-capture/message-capture-parser.py) currently throws an "out of bounds" error if a message containing an empty integer vector element is tried to converted to JSON (e.g. by the BIP157 message `cfcheckpt` with empty `FilterHeaders` vector):
```
Traceback (most recent call last):
File "/home/honey/bitcoin/./contrib/message-capture/message-capture-parser.py", line 217, in <module>
main()
File "/home/honey/bitcoin/./contrib/message-capture/message-capture-parser.py", line 202, in main
process_file(str(capture), messages, "recv" in capture.stem, progress_bar)
File "/home/honey/bitcoin/./contrib/message-capture/message-capture-parser.py", line 162, in process_file
msg_dict["body"] = to_jsonable(msg)
File "/home/honey/bitcoin/./contrib/message-capture/message-capture-parser.py", line 85, in to_jsonable
elif slot in HASH_INT_VECTORS and isinstance(val[0], int):
IndexError: list index out of range
```
Fix this by using the `all(...)` predicate rather to access the first element `val[0]` (which in the error case doesn't exist).
ACKs for top commit:
laanwj:
Code review ACK 42bbbba7c83d1e2baad18b4c6f05bad1358eb117
Tree-SHA512: 139ec6b90304a69f26ec731e6f12b216fa10e554f777505b61adfa1e569f6861a4a849159dd1eae7a1aa0427e8598af226b6f0c4015020dcac8ab109fbc35dba
ecff20db286e2f5d3afe32cfaae72de69d34d23c logging: use LogPrintfCategory rather than a manual category (Jon Atack)
eb8aab759fb15824a5dd3004e689d0eb5b884a32 logging: add LogPrintfCategory to log unconditionally with category (Jon Atack)
Pull request description:
These are the next two commits from #25203.
- Add `LogPrintfCategory` to log unconditionally while prefixing the output with the passed category name. Add documentation and a unit test, and update the `lint-logs.py` and `lint-format-strings.py` scripts.
- Replace the log messages that manually print a category, with `LogPrintfCategory`. In upcoming commits, it will likely be used in many other cases, such as to replace `LogPrintf` where it makes sense.
ACKs for top commit:
klementtan:
Code Review ACK ecff20db286e2f5d3afe32cfaae72de69d34d23c
laanwj:
Code review ACK ecff20db286e2f5d3afe32cfaae72de69d34d23c
brunoerg:
ACK ecff20db286e2f5d3afe32cfaae72de69d34d23c
Tree-SHA512: ad3a82835254f7606efcd14b88f3d9072f1eb9b25db1321ed38ef6a4ec60efd555d78f5e19d93736f2f8500251d06f8beee9d694a153f24bf5cce3590a2a45a5
1cb42aeda37f4979923cd7e1c85febe994480de6 util: modify Win32LockedPageAllocator to query windows for limit (Oskar Mendel)
Pull request description:
This PR resolves a todo within the Win32LockedPageAllocator: `// TODO is there a limit on Windows, how to get it?`.
The idea is to use the Windows API to get the limits like the posix based allocator does with `getrlimit`.
I use [GetProcessWorkingSetSize](https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-getprocessworkingsetsize) to perform this task and fallback to `return std::numeric_limits<size_t>::max();` just like the posix implementation does.
ACKs for top commit:
sipsorcery:
tACK 1cb42aeda37f4979923cd7e1c85febe994480de6.
Tree-SHA512: 7bdd8a57a4e64ee59d752417a519656e03526878462060753be4dce481eff4889fb5edc1bdbd575b707d9b2dfe255c87da9ef67baac97de9ac5e70a04c852081
14093d5d243f6eb9cfef721c80f92848d95032ee doc: add distcc to productivity notes (Sjors Provoost)
Pull request description:
If you have more than one computer at your disposal, you can use [distcc](https://www.distcc.org) to speed up compilation.
ACKs for top commit:
laanwj:
ACK 14093d5d243f6eb9cfef721c80f92848d95032ee
brunoerg:
ACK 14093d5d243f6eb9cfef721c80f92848d95032ee
w0xlt:
ACK 14093d5d24
Tree-SHA512: 2c436bdea5ab750330055778eb5817361d16b046f219d53692577439e2fd8403febf78ac8e8b20ed158c650c76252b50cfc91f4ec8375cdd522cc408068d547b
fa779de665227cf15a4056959e9753618cc013ac test: Remove MiniWallet mempool_valid option (MacroFake)
Pull request description:
It seems an unnecessary burden to force MiniWallet call-sites to figure out for each tx whether it is mempool valid or not. The result will only be used for internal sanity checks. So remove the option:
* Replace the vsize sanity check with a call to `get_vsize()`.
* Drop the fee check. Hopefully any future bug here will be caught by code-review or otherwise.
ACKs for top commit:
theStack:
Code-review ACK fa779de665227cf15a4056959e9753618cc013ac
Tree-SHA512: df44a0e116a0b6b15389c80038f9b45e17f186d0e3d7b0925e367fd2cbbcab9a7a6f7add41859ffb5603885f304233a5d28fbd57a4008ebdfe5edbe83bb1d7c3
b167e536d0f5ae4c86d0c3da4a63337f9f0448ba test: refactor: use `create_lots_of_big_transactions` to dedup where possible (Sebastian Falbesoner)
8973eeb4124112a8cb7bb3d486780ac668b3e7bd test: use MiniWallet for mining_prioritisetransaction.py (Sebastian Falbesoner)
Pull request description:
This PR enables one more of the non-wallet functional tests (mining_prioritisetransaction.py) to be run even with the Bitcoin Code wallet by using the MiniWallet instead, as proposed in #20078. Note that the adapted helper function `create_lots_of_big_transactions` is currently only used in this test, i.e. there was no need to change any others.
ACKs for top commit:
ayush933:
tACK b167e53
danielabrozzoni:
tACK b167e536d0f5ae4c86d0c3da4a63337f9f0448ba
kouloumos:
ACK b167e536d0f5ae4c86d0c3da4a63337f9f0448ba
furszy:
ACK b167e536
Tree-SHA512: ccae20d7d414a720efdeea9c2ae399aa53a3a0e7db72bff8d0cb75d90621a7ae7c019ba68d24f9d06f7b111f87ff33bb9d8e5aa08b763e606cf10268780e205c
5733ae51ceaa9a2c225c43596552b2c918e5334e test: Fix previous release binary download script for Apple ARM64 (Fabian Jahr)
Pull request description:
The Apple M1 chip binaries at https://bitcoincore.org/bin/bitcoin-core-23.0/ are use `arm64` and not `aarch64` in the file name. This means on my M1 Macbook the v23 binary could not be downloaded: "Binary tag was not found".
This changes the script to map the `aarch64` from the host detection to `arm64`.
ACKs for top commit:
fanquake:
ACK 5733ae51ceaa9a2c225c43596552b2c918e5334e
Tree-SHA512: bd70c5361c8b549363b11be770ad285f885787ec8781f51a69c1ebbfd08f762ac6ec4d5a92c0d83ce3c8bfb7fc72750c25d42092666d00cc372e62de0f052171
e47c6c76561807d30cff3c2e5372ea83c91a3677 Reset settings.json when GUI options are reset (Ryan Ofsky)
99ccc02b652cf67a3aec66371fcb6bbe737571a7 Add release notes about unified bitcoin-qt and bitcoind persistent settings (Ryan Ofsky)
504b06b1dec9d9329c83b13c7c36ca710ebcd349 Migrate -lang setting from QSettings to settings.json (Ryan Ofsky)
9a016a3c07d4becf0651ef58c7160180c5f25a0c Migrate -prune setting from QSettings to settings.json (Ryan Ofsky)
f067e1943361b7bfa78a423528759e9edffa7482 Migrate -proxy and -onion settings from QSettings to settings.json (Ryan Ofsky)
a09e3b7cf29c3b1fd320badbed32275e0aa83cda Migrate -listen and -server settings from QSettings to settings.json (Ryan Ofsky)
d2ada6e63583cad91e92b49c4dbf8c7ff086a758 Migrate -upnp and -natpmp settings from QSettings to settings.json (Ryan Ofsky)
1dc4fc29c1086420b7dc51b20c0b7a18fecb4462 Migrate -spendzeroconfchange and -signer settings from QSettings to settings.json (Ryan Ofsky)
a7ef6d5975a5f40b90b2709b32a00647bd2bd5a3 Migrate -par setting from QSettings to settings.json (Ryan Ofsky)
284f339de68905131331f7fdb4c0b945c9a1b8cd Migrate -dbcache setting from QSettings to settings.json (Ryan Ofsky)
Pull request description:
If a setting like pruning, port mapping, or a network proxy is enabled in the GUI, it will now be stored in the bitcoin persistent setting file in the datadir and shared with bitcoind, instead of being stored as Qt settings which end up in the the windows registry or platform specific config files and are ignored by bitcoind.
This PR has been split off from bitcoin/bitcoin#15936 so some review of these commits previously took place in that PR.
ACKs for top commit:
furszy:
Code review ACK e47c6c76
hebasto:
ACK e47c6c76561807d30cff3c2e5372ea83c91a3677
Tree-SHA512: 076ea7c7efe67805b4a357113bfe1643dce364d0032774106de59566a0ed5771d57a5923920085e03d686beb34b98114bd278555dfdf8bb7af0b778b0f35b7d2
1f653dc2623ee2be3e1eeeaf5ce0e97966cecd6a qt, wallet, refactor: Make `WalletModel::sendCoins()` return `void` (Hennadii Stepanov)
Pull request description:
Currently, the `WalletModel::sendCoins()` function always returns the same value.
Also dead and noop (calling `processSendCoinsReturn(OK)`) code has been removed.
The other `return` statements have been removed from the `WalletModel::sendCoins()` function in bitcoin/bitcoin#17154 and bitcoin/bitcoin#17165.
ACKs for top commit:
kristapsk:
cr ACK 1f653dc2623ee2be3e1eeeaf5ce0e97966cecd6a
furszy:
Code review ACK 1f653dc2
shaavan:
Code Review ACK 1f653dc2623ee2be3e1eeeaf5ce0e97966cecd6a
w0xlt:
Code Review ACK 1f653dc262
Tree-SHA512: 2b59495a7fc10b4de30fcc63fc3af92d50406e16031112eb72494736dce193ac1fbac0802623496cf81edcd16766e1647d9c4f3a607b3eb84cc50e273b999c04
44c2452fd Merge bitcoin-core/secp256k1#1105: Don't export symbols in static libraries
6f6cab998 abi: Don't export symbols in static Windows libraries
485f608fa Merge bitcoin-core/secp256k1#1104: Fix the false positive of `SECP_64BIT_ASM_CHECK`
8b013fce5 Merge bitcoin-core/secp256k1#1056: Save negations in var-time group addition
7efc9835a Fix the false positive of `SECP_64BIT_ASM_CHECK`
2f984ffc4 Save negations in var-time group addition
git-subtree-dir: src/secp256k1
git-subtree-split: 44c2452fd387f7ca604ab42d73746e7d3a44d8a2
Achieve this by adding a MAIN_FUNCTION macro, consolidating the docs, and
introducing the macro across our distributed binaries.
Also update the docs to explain that anyone using binutils < 2.36 is
effected by this issue. Release builds are not, because they use binutils
2.37. Currently LTS Linux distros, like Ubuntu Focal, ship with 2.34.
https://packages.ubuntu.com/focal/binutils
fa7a711a30b707cbdee4435dd0a956bffb7aaccb test: Fix out-of-range port collisions (MacroFake)
Pull request description:
Otherwise the test will fail if two tests running in parallel use the same port. See https://github.com/bitcoin/bitcoin/pull/25096#discussion_r892558783 and https://github.com/bitcoin/bitcoin/pull/25312
ACKs for top commit:
dergoegge:
ACK fa7a711a30b707cbdee4435dd0a956bffb7aaccb - This gets rid of some rather arbitrary choices for ports in some of our functional tests that can cause port collisions across test runs, resulting in intermittent failures.
Tree-SHA512: ac73da8a498230b992ab12e1ee3c4ff3d868cd63c00d2c71537d156cb7c8f8be8598ec574646b17c5a44ae3ac5bb54bf29d300f054a36cec6f6ce8054a0da0a4
ce893c0497fc9b8ab9752153dfcc77c9f427545e doc: Update developer notes (Anthony Towns)
d2852917eecad6ab422a7b2c9892d351a7f0cc96 sync.h: Imply negative assertions when calling LOCK (Anthony Towns)
bba87c0553780eacf0317fbfec7330ea27aa02f8 scripted-diff: Convert global Mutexes to GlobalMutexes (Anthony Towns)
a559509a0b8cade27199740212d7b589f71a0e3b sync.h: Add GlobalMutex type (Anthony Towns)
be6aa72f9f8d50b6b5b19b319a74abe7ab4099ff qt/clientmodel: thread safety annotation for m_cached_tip_mutex (Anthony Towns)
f24bd45b37e1b2d19e5a053dbfefa30306c1d41a net_processing: thread safety annotation for m_tx_relay_mutex (Anthony Towns)
Pull request description:
This changes `LOCK(mutex)` for non-global, non-recursive mutexes to be annotated with the negative capability for the mutex it refers to, to prevent . clang applies negative capabilities recursively, so this helps avoid forgetting to annotate functions.
This can't reasonably be used for globals, because clang would require every function to be annotated with `EXCLUSIVE_LOCKS_REQUIRED(!g_mutex)` for each global mutex; so this introduces a trivial `GlobalMutex` subclass of `Mutex`, and reduces the annotations for both `GlobalMutex` to `LOCKS_EXCLUDED` which only catches trivial errors (eg (`LOCK(x); LOCK(x);`).
ACKs for top commit:
MarcoFalke:
review ACK ce893c0497fc9b8ab9752153dfcc77c9f427545e 🐦
hebasto:
ACK ce893c0497fc9b8ab9752153dfcc77c9f427545e
Tree-SHA512: 5c35e8c7677ce3d994a7e3774f4344adad496223a51b3a1d1d3b5f20684b2e1d5cff688eb3fbc8d33e1b9940dfa76e515f9434e21de6f3ce3c935e29a319f529
Code introduced in #15649 added usage of `timingsafe_bcmp()`, if
available, otherwise falling back to our own implementation. However
the relevant build system check was never added, so currently, we'll
always just use our implementation, as HAVE_TIMINGSAFE_BCMP will never
be defined.
Add the check for timingsafe_bcmp. Note that as far as I'm aware, it's
only available on OpenBSD.
ea54ba2f42f6d0b23570c665c2369f977bf55cf6 [test] Fix port collisions caused by p2p_getaddr_caching.py (dergoegge)
f9682e75ac184a62c7e29287882df34c25303033 [test_framework] Set PortSeed.n directly after initialising params (dergoegge)
Pull request description:
This PR fixes the issue mentioned [here](https://github.com/bitcoin/bitcoin/pull/25096#discussion_r892558783), to avoid port collisions between nodes spun up by the test framework.
Top commit has no ACKs.
Tree-SHA512: ec9159f0af90db636f7889d664c24e1430cf2bcb3c02a9ab2dcfe531b2a4d18f6e3a0f8ba73071bdf2f7db518df9d5d86a9cd06695e67644d20fe4515fac32b7
d575413fb8f8569803ed0050c15be003ef50951c doc: add `desig` to ignore-words (brunoerg)
c06cc41ddbac3611d5c7826a2a3f2cbae9aeea12 doc: fix typo in kernel/context.h (brunoerg)
Pull request description:
This PR fixes a typo in `kernel/context.h` (libary => library) and add `desig` to ignore-words since it's a valid word, see:
b9416c3847/src/net.cpp (L1105-L1117)
ACKs for top commit:
fanquake:
ACK d575413fb8f8569803ed0050c15be003ef50951c
Tree-SHA512: 2d548c737b8184d0243445c7503f3f68256ecb0970bd834d52de099de3cd8c8b9c140e2b77d55e2542fbd45b1d21cbdee639f5b2ef8138c37b8b72e5211029c3
8888bd43c100f9f0ca1122fcc896fb7b999d61c6 Remove redundant nLastTry check (MarcoFalke)
00001e57fe74c061aa9cbc72b07252335cb566e0 Remove redundant nTime checks (MarcoFalke)
Pull request description:
Split out from https://github.com/bitcoin/bitcoin/pull/24697 because it makes sense on its own.
ACKs for top commit:
dergoegge:
re-ACK 8888bd43c100f9f0ca1122fcc896fb7b999d61c6
naumenkogs:
utACK 8888bd43c100f9f0ca1122fcc896fb7b999d61c6
Tree-SHA512: 32c6cde1c71e943c76b7991c2c24caf29ae467ab4ea2d758483a0cee64625190d1a833b468e8eab1f834beeb2c365af96552c14b05270f08cf63790e0707581d
fa74b63c01db412f6a4378cb669d89496a89d02e test: Fix wait_for_debug_log UnicodeDecodeError (MacroFake)
Pull request description:
Fix the intermittent `UnicodeDecodeError` when the debug log is truncated on an (multi-byte) unicode character by treating everything as bytes.
Also, remove the `ignore_case` option and the`re.search+re.escape` wrap. All of this is unused and doesn't exist on raw byte strings.
Fixes https://github.com/bitcoin/bitcoin/issues/24575
ACKs for top commit:
jonatack:
ACK fa74b63c01db412f6a4378cb669d89496a89d02e
brunoerg:
ACK fa74b63c01db412f6a4378cb669d89496a89d02e
Tree-SHA512: c67c9355073e784fa8d9d48b8e79ff0c98f5ae9cd4d704ad12a76d2604733946054bc74b8ab346aa2184db23d740b85c8c13eb892d76cba92e42ebfd73f2f1bf
06e18e0b53ed34933ecd7f3976a508be72f687aa build: use BOOST_MULTI_INDEX_ENABLE_SAFE_MODE when debugging (fanquake)
Pull request description:
Use of this macro enables precondition checks for iterators and functions of the library. It's use is recommended in debug builds. See https://www.boost.org/doc/libs/1_78_0/libs/multi_index/doc/tutorial/debug.html for more info.
There is also a `BOOST_MULTI_INDEX_ENABLE_INVARIANT_CHECKING` macro:
> When this mode is in effect, all public functions of Boost.MultiIndex will perform post-execution tests aimed at ensuring that the basic internal invariants of the data structures managed are preserved.
ACKs for top commit:
laanwj:
Concept and code review ACK 06e18e0b53ed34933ecd7f3976a508be72f687aa
Tree-SHA512: 7ee489eccda81c7dbca9210af6d3007d5b2c704b645139d2714c077af157789dd9478c29d0d212e210e96686ea83713aaf3d458e879122b3cde64f3e3e3789d2
Here we update only the log messages that manually print a category.
In upcoming commits, LogPrintCategory will likely be used in many
other cases, such as to replace `LogPrintf` where it makes sense.
292828cd7744ec7eadede4ad54aa2117087c5435 [test] Test addr cache for multiple onion binds (dergoegge)
3382905befd23364989d941038bf7b1530fea0dc [net] Seed addr cache randomizer with port from binding address (dergoegge)
f10e80b6e4fbc151abbf1c20fbdcc3581d3688f0 [net] Use ConnectedThroughNetwork() instead of GetNetwork() to seed addr cache randomizer (dergoegge)
Pull request description:
The addr cache id randomizer is currently supposed to be seeded with the network of the inbound connection and the local socket (only the address is used not the port): a8098f2cef/src/net.cpp (L2800-L2804)
For inbound onion connections `CNode::addr.GetNetwork()` returns `NET_UNROUTABLE` and `CNode::addrBind` is set to `127.0.0.1:<onion bind port>`. This results in the same addr cache for all inbound connections on 127.0.0.1 binds.
To avoid the same addr cache across all onion and other 127.0.0.1 binds, we should seed the addr cache randomizer with the correct network for inbound onion connections (using `CNode::ConnectedThroughNetwork()`) as well as the port of `CNode::addrBind`.
ACKs for top commit:
sipa:
utACK 292828cd7744ec7eadede4ad54aa2117087c5435
mzumsande:
Code Review ACK 292828cd7744ec7eadede4ad54aa2117087c5435
naumenkogs:
utACK 292828cd7744ec7eadede4ad54aa2117087c5435
Tree-SHA512: d0be13bab6bc121c2926d4b168687f6c2ed4ce0c9dd19be71eb4886adeba8afc3daacdc4e232a0ba3b03a89d69b618abc5595b69abd1ad0c476d825bc6ea1f9f
Now that we use GCC 10 for release builds, we no-longer need to
pass-Wl,-z,noexecstack to get a non-executable stack in RISC-V binaries.
This was originally removed in #21036, but then re-added in #21799, when
we reverted to using GCC 8.
