9086e0dd3c924b6c9a5ad05799d30d97b9ced3be build: set Unicode true for NSIS installer (fanquake)
Pull request description:
Now that we are using Focal for gitian builds, and have [NSIS 3.0+ available](https://packages.ubuntu.com/focal/nsis) (also in Guix), we can create installers that [support unicode](https://nsis.sourceforge.io/Docs/Chapter4.html#aunicodetarget).
Unicode is only becoming the NSIS default [beginning with the 3.07 release](https://nsis.sourceforge.io/Docs/AppendixF.html#v3.07-cl), so we need to set this attribute to get support.
Should close: #13817
Gitian builds:
```bash
b8553615b6b4be5e4459e03796e700b30b5d198a7f184f27be6983ff901b5592 bitcoin-9086e0dd3c92-win-unsigned.tar.gz
a6b024a5a68e0196e8e118168c918285e820f2d0ffe9c38db680580459da8bf3 bitcoin-9086e0dd3c92-win64-debug.zip
ff4003d4f61127c707e44b5235eaf924b30351f20cde27e775131982a1b4cf92 bitcoin-9086e0dd3c92-win64-setup-unsigned.exe
1876bee55fa9ea99b91203975c13d0ad8a046b4b58068bde41c977fd1d12de13 bitcoin-9086e0dd3c92-win64.zip
000f2778f8f166a89b4ab35f155156c1c34800be6e47d29b5308043c50128392 src/bitcoin-9086e0dd3c92.tar.gz
d650a9b8f2dd1df777bf42439dfcbcf6bc358e30ec148b9992a18b39f76b1ecf bitcoin-core-win-22-res.yml
```
ACKs for top commit:
laanwj:
ACK 9086e0dd3c924b6c9a5ad05799d30d97b9ced3be
hebasto:
ACK 9086e0dd3c924b6c9a5ad05799d30d97b9ced3be, tested on Windows 10 Pro (20H2, build 19042.804):
Tree-SHA512: cc7b7ca05877571d0a29a7d36e40279f54d886d8ab27facfa722c2ee95a1fc06c2bad8ef1eb1980d283ae981659d737021a46c8f4618e24d510b5ab384990e09
a701fcf01f3ea9a12e869bfa52321302cf68351c net: Do not skip the I2P network from GetNetworkNames() (Vasil Dimov)
0181e244394bd9e68e9f0d44704e7b0fd12a6b1f net: recognize I2P from ParseNetwork() so that -onlynet=i2p works (Vasil Dimov)
b905363fa8b0bb03fe34b53b5410880f42e0af39 net: accept incoming I2P connections from CConnman (Vasil Dimov)
0635233a1e7e8c303073430092afd3e0fb0d927b net: make outgoing I2P connections from CConnman (Vasil Dimov)
9559bd1404fbf74b0d09fe9019a9305cb4e151ce net: add I2P to the reachability map (Vasil Dimov)
76c35c60f338937071bcfad4211ef7254d3830ec init: introduce I2P connectivity options (Vasil Dimov)
c22daa2ecff1acd25426cd46f98f2587d1d324c3 net: implement the necessary parts of the I2P SAM protocol (Vasil Dimov)
5bac7e45e1d3a07115b5ff002d988438fcc92a53 net: extend Sock with a method to check whether connected (Vasil Dimov)
42c779f503eb8437b6232773a4a2472306cc9f3d net: extend Sock with methods for robust send & read until terminator (Vasil Dimov)
ea1845315a109eb105113cb5fbb6f869e1cf010c net: extend Sock::Wait() to report a timeout (Vasil Dimov)
78fdfbea666201b25919dd67454eb04d6a34326f net: dedup MSG_NOSIGNAL and MSG_DONTWAIT definitions (Vasil Dimov)
34bcfab562bac9887ca9c3831cf4fd0ee7f98149 net: move the constant maxWait out of InterruptibleRecv() (Vasil Dimov)
cff65c4a270887ec171293409ab84f5d0d0be7fc net: extend CNetAddr::SetSpecial() to support I2P (Vasil Dimov)
f6c267db3be2d7077fd2bdbd34860eba838dea99 net: avoid unnecessary GetBindAddress() call (Vasil Dimov)
7c224fdac4699a2c4953b33ab423f9cddbf95cf7 net: isolate the protocol-agnostic part of CConnman::AcceptConnection() (Vasil Dimov)
1f75a653dd3b24ba2e4383bf951a6e5a3d5ccbcf net: get the bind address earlier in CConnman::AcceptConnection() (Vasil Dimov)
25605895afe84b1765dd9da9240af22f99489df7 net: check for invalid socket earlier in CConnman::AcceptConnection() (Vasil Dimov)
545bc5f81d60fa6ff7c5cc43a2e9eef82f911466 util: fix WriteBinaryFile() claiming success even if error occurred (Vasil Dimov)
8b6e4b3b23027da263d257b342f5d9a53e4032d5 util: fix ReadBinaryFile() returning partial contents (Vasil Dimov)
4cba2fdafa483cbdb70f581174138ec253c80d48 util: extract {Read,Write}BinaryFile() to its own files (Vasil Dimov)
Pull request description:
Add I2P support by using the [I2P SAM](https://geti2p.net/en/docs/api/samv3) protocol. Unlike Tor, for incoming connections we get the I2P address of the peer (and they also receive ours when we are the connection initiator).
Two new options are added:
```
-i2psam=<ip:port>
I2P SAM proxy to reach I2P peers and accept I2P connections (default:
none)
-i2pacceptincoming
If set and -i2psam is also set then incoming I2P connections are
accepted via the SAM proxy. If this is not set but -i2psam is set
then only outgoing connections will be made to the I2P network.
Ignored if -i2psam is not set. Notice that listening for incoming
I2P connections is done through the SAM proxy, not by binding to
a local address and port (default: true)
```
# Overview of the changes
## Make `ReadBinary()` and `WriteBinary()` reusable
We would need to dump the I2P private key to a file and read it back later. Move those two functions out of `torcontrol.cpp`.
```
util: extract {Read,Write}BinaryFile() to its own files
util: fix ReadBinaryFile() returning partial contents
util: fix WriteBinaryFile() claiming success even if error occurred
```
## Split `CConnman::AcceptConnection()`
Most of `CConnman::AcceptConnection()` is agnostic of how the socket was accepted. The other part of it deals with the details of the `accept(2)` system call. Split those so that the protocol-agnostic part can be reused if we accept a socket by other means.
```
net: check for invalid socket earlier in CConnman::AcceptConnection()
net: get the bind address earlier in CConnman::AcceptConnection()
net: isolate the protocol-agnostic part of CConnman::AcceptConnection()
net: avoid unnecessary GetBindAddress() call
```
## Implement the I2P [SAM](https://geti2p.net/en/docs/api/samv3) protocol (not all of it)
Just the parts that would enable us to make outgoing and accept incoming I2P connections.
```
net: extend CNetAddr::SetSpecial() to support I2P
net: move the constant maxWait out of InterruptibleRecv()
net: dedup MSG_NOSIGNAL and MSG_DONTWAIT definitions
net: extend Sock::Wait() to report a timeout
net: extend Sock with methods for robust send & read until terminator
net: extend Sock with a method to check whether connected
net: implement the necessary parts of the I2P SAM protocol
```
## Use I2P SAM to connect to and accept connections from I2P peers
Profit from all of the preceding commits.
```
init: introduce I2P connectivity options
net: add I2P to the reachability map
net: make outgoing I2P connections from CConnman
net: accept incoming I2P connections from CConnman
net: recognize I2P from ParseNetwork() so that -onlynet=i2p works
net: Do not skip the I2P network from GetNetworkNames()
```
ACKs for top commit:
laanwj:
re-ACK a701fcf01f3ea9a12e869bfa52321302cf68351c
jonatack:
re-ACK a701fcf01f3ea9a12e869bfa52321302cf68351c reviewed diff per `git range-diff ad89812 2a7bb34 a701fcf`, debug built and launched bitcoind with i2pd v2.35 running a dual I2P+Torv3 service with the I2P config settings listed below (did not test `onlynet=i2p`); operation appears nominal (same as it has been these past weeks), and tested the bitcoind help outputs grepping for `-i i2p` and the rpc getpeerinfo and getnetworkinfo helps
Tree-SHA512: de42090c9c0bf23b43b5839f5b4fc4b3a2657bde1e45c796b5f3c7bf83cb8ec6ca4278f8a89e45108ece92f9b573cafea3b42a06bc09076b40a196c909b6610e
8a8c6383f6f9da10b931f00ca1220408fede8f35 zmq test: fix sync-up by matching notification to generated block (Sebastian Falbesoner)
Pull request description:
This is a follow-up PR for #21008, fixes#21216.
In the course of investigating the problem with jnewbery (analyzing the Cirrus log https://cirrus-ci.com/task/4660108304056320), it turned out that the "sync up" procedure of repeatedly generating a block and waiting for a notification with timeout is too brittle in its current form, as the following scenario could happen:
- generate block A
- receive notification, timeout happens => repeat procedure
- generate block B
- node publishes block A notification
- receive notification, we receive the one caused by block A (!!!) => sync-up procedure is completed
- node publishes block B notification
- the actual test starts
- on the first notification reception, the one caused by block B is received, rather than the one actually caused by test code => assertion failure
This change in the PR ensures that after each test block generation, we wait for the notification that is actually caused by that block and ignore others from possibly earlier blocks. The matching is kind of ugly, it assumes that one out of four components in the block is contained in the notification: the block hash, the tx id, the raw block data or the raw transaction data. (Unfortunately we have to support all publisher topics.)
I'm aware that this is quite a lot of code now only for establishing a robust test setup. OTOH I wouldn't know of a better method right now, suggestions are very welcome.
Note for potential reviewers: for both reproducing the issue on master branch and verifying on PR branch, one can simply generate two blocks in the sync-up procedure rather than one.
ACKs for top commit:
MarcoFalke:
Concept ACK 8a8c6383f6f9da10b931f00ca1220408fede8f35
Tree-SHA512: a2eb78ba06dfd0fda7b1c111b6bbfb5dab4ab08500cc19c7ea02c3239495d5c74cc7d45250a8b3ecc78ca42d97ee6719bf73db8a137839e5e09a3cfcf08ed29e
Now that we are using Focal for gitian builds, and have NSIS 3.0+
available (also in Guix), we can create installers that support unicode.
Unicode is only becoming the default beginning with the 3.07 release,
https://nsis.sourceforge.io/Docs/AppendixF.html#v3.07-cl,
so we need to set this attribute to get support.
Should close: #13817
Co-authored-by: Chun Kuan Lee <ken2812221@gmail.com>
fbbb2d4fc13971c98c83a51635166ac532e71a32 lint: Fix spelling errors in comments (fyquah)
Pull request description:
Found some spelling errors while running spelling linter https://github.com/bitcoin/bitcoin/pull/21245
This PR fixes them.
ACKs for top commit:
fanquake:
ACK fbbb2d4fc13971c98c83a51635166ac532e71a32 - I thought we just fixed all of these.
Tree-SHA512: 95525040001f94e899b778c616cb66ebafb679dff88835b66fccf6349d8eb942d6b7374c536a44e393f13156bce9a32ed57e6a82bb02074d2b3cddb2696addb2
c33b199456e57d83c21eacd36d3c56d0a123b0d0 guix: Bump glibc and linux-headers (Carl Dong)
65363a1bd8b886f5aef5fbc97ca88c9c9b243b21 guix: Rebase on 95aca2991b (1.2.0-12.dffc918) (Carl Dong)
Pull request description:
On bumping the time-machine:
```
A few changes which are useful for us:
1. 'gnu: cross-gcc-arguments: Enable 128 bit long double for POWER9.' is
now merged into master.
2. gnutls is bumped to 3.6.15 and the temporal test failure in
status-request-revoked is fixed. Note that this does not fix the case
where one has installed Guix v1.2.0 and is running a substitute-less
bootstrap build, since the `guix time-machine` command itself has a
dependency on gnutls v3.6.12 (the one with the broken test) and will
thus try to build it before attempting to jump forwards in time. This
does however, mean that those who build a version of Guix that also
contains this fix will not go backwards in time to build the broken
gnutls v3.6.12.
```
On bumping the rest:
```
Bump glibc and linux-headers to match those of our Gitian counterparts.
We also require a glibc >= 2.28 for the test-symbol-check scripts to
work properly.
The default BASE-GCC-FOR-LIBC also has to be bumped since glibc 2.31
requires a gcc >= 6.2
```
This is a prerequisite for #20980
ACKs for top commit:
fanquake:
ACK c33b199456e57d83c21eacd36d3c56d0a123b0d0 - I think going ahead with this now and to sycn back up to gitian is fine. It will also unblock #20980. Potential code signing related issues can be sorted out in #21239 and later PRs.
Tree-SHA512: 31f022aadb93ba44813b0da005b1f2e5d67d76e8cdcdb53368924d1ea6cb076a21218c26831a6b0dcdcfe33507f54934330489ba557371d740f5587b7d727b95
a0a7a4337d06553ab625bbd66ed4198c4bf7f18c guix, doc: Update default HOSTS value (Hennadii Stepanov)
Pull request description:
This is a #21089 follow up.
ACKs for top commit:
fanquake:
ACK a0a7a4337d06553ab625bbd66ed4198c4bf7f18c
Tree-SHA512: c1813cc2b9212a79fd34d4e25cd0816b58264e1890daf777cd59411bd20fcc9affe312871d06fab1308b8f55c1a78ac1101e631882c18360a4709ecef4529f05
e3e0a2432c587ee06e469c37ffae133b7ac55c77 Add benchmark to write JSON into a string (Martin Ankerl)
Pull request description:
The benchmark `BlockToJsonVerbose` only tests generating (and destroying)
the JSON data structure, but serializing into a string is also a
performance critical aspect of the RPC calls.
Extracts test setup into a `struct TestBlockAndIndex`, and uses it in
both `BlockToJsonVerbose` and `BlockToJsonVerboseWrite`.
Also, use `ankerl::nanobench::doNotOptimizeAway` to make sure the compiler
can't optimize the result of the calls away.
Here are benchmark results on my Intel i7-8700:
| ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
| 71,807,017.00 | 13.93 | 0.4% | 555,782,961.00 | 220,788,645.00 | 2.517 | 102,279,341.00 | 0.4% | 0.80 | `BlockToJsonVerbose`
| 27,916,835.00 | 35.82 | 0.1% | 235,084,034.00 | 89,033,525.00 | 2.640 | 42,911,139.00 | 0.3% | 0.32 | `BlockToJsonVerboseWrite`
ACKs for top commit:
laanwj:
Code review ACK e3e0a2432c587ee06e469c37ffae133b7ac55c77
Tree-SHA512: bc4d6d1588d47d4bd7af8e7908e44b8561bc391a2d73eccd7c0aa37fc40e8a9ce1fa1f3c29b416eef24a73c6bce3036839c0bbfe1b8dbd6d1bba3718b7ca5383
So that help texts include "i2p" in:
* `./bitcoind -help` (in `-onlynet` description)
* `getpeerinfo` RPC
* `getnetworkinfo` RPC
Co-authored-by: Jon Atack <jon@atack.com>
Introduce two new options to reach the I2P network:
* `-i2psam=<ip:port>` point to the I2P SAM proxy. If this is set then
the I2P network is considered reachable and we can make outgoing
connections to I2P peers via that proxy. We listen for and accept
incoming connections from I2P peers if the below is set in addition to
`-i2psam=<ip:port>`
* `-i2pacceptincoming` if this is set together with `-i2psam=<ip:port>`
then we accept incoming I2P connections via the I2P SAM proxy.
Implement the following commands from the I2P SAM protocol:
* HELLO: needed for all of the remaining ones
* DEST GENERATE: to generate our private key and destination
* NAMING LOOKUP: to convert .i2p addresses to destinations
* SESSION CREATE: needed for STREAM CONNECT and STREAM ACCEPT
* STREAM CONNECT: to make outgoing connections
* STREAM ACCEPT: to accept incoming connections
Introduce two high level, convenience methods in the `Sock` class:
* `SendComplete()`: keep trying to send the specified data until either
successfully sent all of it, timeout or interrupted.
* `RecvUntilTerminator()`: read until a terminator is encountered (never
after it), timeout or interrupted.
These will be convenient in the I2P SAM implementation.
`SendComplete()` can also be used in the SOCKS5 implementation instead
of calling `send()` directly.
Previously `Sock::Wait()` would not have signaled to the caller whether
a timeout or one of the requested events occurred since that was not
needed by any of the callers.
Such functionality will be needed in the I2P implementation, thus extend
the `Sock::Wait()` method.
Recognize also I2P addresses in the form `base32hashofpublickey.b32.i2p`
from `CNetAddr::SetSpecial()`.
This makes `Lookup()` support them, which in turn makes it possible to
manually connect to an I2P node by using
`-proxy=i2p_socks5_proxy:port -addnode=i2p_address.b32.i2p:port`
Co-authored-by: Lucas Ontivero <lucasontivero@gmail.com>
Our local (bind) address is already saved in `CNode::addrBind` and there
is no need to re-retrieve it again with `GetBindAddress()`.
Also, for I2P connections `CNode::addrBind` would contain our I2P
address, but `GetBindAddress()` would return something like
`127.0.0.1:RANDOM_PORT`.
Isolate the second half of `CConnman::AcceptConnection()` into a new
separate method, which could be reused if we accept incoming connections
by other means than `accept()` (first half of
`CConnman::AcceptConnection()`).
Call `GetBindAddress()` earlier in `CConnman::AcceptConnection()`. That
is specific to the TCP protocol and makes the code below it reusable for
other protocols, if the caller provides `addr_bind`, retrieved by other
means.
This check is related to an `accept()` failure. So do the check earlier,
closer to the `accept()` call.
This will allow to isolate the `accept()`-specific code at the beginning
of `CConnman::AcceptConnection()` and reuse the code that follows it.
`fclose()` is flushing any buffered data to disk, so if it fails then
that could mean that the data was not completely written to disk.
Thus, check if `fclose()` succeeds and only then claim success from
`WriteBinaryFile()`.
If an error occurs and `fread()` returns `0` (nothing was read) then the
code before this patch would have returned "success" with a partially
read contents of the file.
Extract `ReadBinaryFile()` and `WriteBinaryFile()` from `torcontrol.cpp`
to its own `readwritefile.{h,cpp}` files, so that it can be reused from
other modules.
It turned out that the "sync up" procedure of repeatedly generating a
block and waiting for a notification once with timeout is too naive in
its current form, as the following scenario could happen:
- generate block A
- receive notification, timeout happens -> repeat procedure
- generate block B
- node publishes block A notification
- receive notification, we receive the one caused by block A
-> sync-up procedure is completed
- node publishes block B
- the actual test starts
- on the first notification reception, one caused by block B is received,
rather than the one actually caused by test code, leading to failure
This change ensures that after each test block generation, we wait for
the notification that is actually caused by that block and ignore others
from possibly earlier blocks.
Co-authored-by: Jon Atack <jon@atack.com>
faa06ecc9c357428f3fa7a8f86b41b1220809951 build: Bump minimum Qt version to 5.9.5 (Hennadii Stepanov)
Pull request description:
Close#20104.
ACKs for top commit:
laanwj:
Code review ACK faa06ecc9c357428f3fa7a8f86b41b1220809951
jarolrod:
ACK faa06ecc9c357428f3fa7a8f86b41b1220809951
fanquake:
ACK faa06ecc9c357428f3fa7a8f86b41b1220809951 - this should be ok to do now.
Tree-SHA512: 7295472b5fd37ffb30f044e88c39d375a5a5187d3f2d44d4e73d0eb0c7fd923cf9949c2ddab6cddd8c5da7e375fff38112b6ea9779da4fecce6f024d05ba9c08
88c4b9b761964d0b3011126796f272cfca2a9fa2 test: remove unneeded node from feature_blockfilterindex_prune.py (Jon Atack)
ace3f4cbdf476ea8b954d98dfaa931076f87a080 test: improve assertions in feature_blockfilterindex_prune.py (Jon Atack)
Pull request description:
- improves the assertions
- removes an unneeded node, reducing from two to one, and some unneeded `extra_arg` code
ACKs for top commit:
MarcoFalke:
ACK 88c4b9b761964d0b3011126796f272cfca2a9fa2
brunoerg:
Tested ACK 88c4b9b761964d0b3011126796f272cfca2a9fa2
Tree-SHA512: 295700da3a5f583ee02ae2d184db93cce0e13aba69115d5db07f83e96a66b4b850adaff2c3725b6585799565b9ee654b1fee8a6245eaba8c21e1cb5ce524eb2b
13a9fd11a507fd3398bc2c0a0575bdc81579243f guix: Passthrough SDK_PATH into container (Carl Dong)
Pull request description:
This is a usability improvement for Guix builders so that they don't have to extract the Xcode tarball into `depends/SDKs` every time.
Inspiration: https://github.com/bitcoin/bitcoin/pull/21089#issuecomment-778639698
ACKs for top commit:
laanwj:
Tested ACK 13a9fd11a507fd3398bc2c0a0575bdc81579243f
Tree-SHA512: 63392d537e48a0da9f0ee04a929613b139bef1ac5643187871c9ea5376afd2a3d95df0f5e0950ae0eccd2813b166667be98401e5a248ae9c187fe4e84e54d427
70d3c5d0b987411873d557440607aa4f3f65b1ec gui: add "Last Block" (CNodeStats::nLastBlockTime) to peer details (Jon Atack)
a21be7c401e1b885bb338645248f40023af7a88f gui: add "Last Tx" (CNodeStats::nLastTXTime) to peer details (Jon Atack)
4dc2fd6c37f8d85c2e9f19d5f03c72c5a1707455 qt: add RPCConsole::TimeDurationField helper, call systime only once (Jon Atack)
Pull request description:
- add `RPCConsole::TimeDurationField` helper to replace repeated code and call system time only once in `RPCConsole::updateDetailWidget`
- add "Last Tx" (`CNodeStats::nLastTXTime`) field to peer details
- add "Last Block" (`CNodeStats::nLastBlockTime`) field to peer details
ACKs for top commit:
hebasto:
re-ACK 70d3c5d0b987411873d557440607aa4f3f65b1ec
Tree-SHA512: 2611b71fd358ba9ffb6a6206275c08ecb5e683b6f87d022faaaba9802a15030430113afdb434814a9ae2681d04429aa733164bc110b64337ceaae12a0420f4f1
fa730e915766b012ff145e3c2af782668defb21f test: Avoid connecting to real network when running tests (MarcoFalke)
fa1b713941f71b077da07e9dedb743c7e99c3ca8 test: Assume node is running in subtests (MarcoFalke)
Pull request description:
Introduced in #19884
ACKs for top commit:
Sjors:
ACK fa730e915766b012ff145e3c2af782668defb21f
Tree-SHA512: fe132a9ffe2fae1ab16857a3dec9839526fdf74d27a1ae794fbffca8356f639c4b916dc888b260281e9cc793916706c18d1687ebb5a076d4e1c481d218d308d3
fae216a73d71441432cf07212313d18771ce7deb scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext (MarcoFalke)
fa4fbec03e4395a6ab75ea98a696bee6d768e97e scripted-diff: Rename PROVIDE_MAIN_FUNCTION -> PROVIDE_FUZZ_MAIN_FUNCTION (MarcoFalke)
Pull request description:
Split out two renames from #21003:
* `PROVIDE_FUZZ_MAIN_FUNCTION`. *Reason*: This in only used by fuzzing, so the name should indicate that.
* `MakeNoLogFileContext`. *Reason*: Better reflects what the helper does. Also, prepares it to be used in non-fuzz tests in the future.
ACKs for top commit:
practicalswift:
cr ACK fae216a73d71441432cf07212313d18771ce7deb: scripted-diff looks correct
Tree-SHA512: e5d347746f5da72b0c86fd4f07ac2e4b3016e88e8c97a830c73bd79d0af6d0245fe7712487fc20344d6cc25958941716c1678124a123930407e3a437265b71df
fa560cc6c493764efc2f2f1702d45286060fe15f test: Intermittent issue in feature_blockfilterindex_prune (MarcoFalke)
Pull request description:
https://cirrus-ci.com/task/4962244553342976?command=ci#L5131
The index is built in a background thread, so we have to wait for it.
ACKs for top commit:
jonatack:
ACK fa560cc6c493764efc2f2f1702d45286060fe15f
Tree-SHA512: e7a246fe43a28511581fe34b1f5a85303b1874b2535330afc0405269cce7306984ecc6af389791321e3aa4b224819e89d9b89dd5bc080d60baa20bd007412787
