autotools and automake changes to support multiprocess execution.
This adds a new --enable-multiprocess flag, and build configuration code to
detect libraries needed for multiprocess support. The --enable-multiprocess
flag builds new bitcoin-node and bitcoin-gui executables, which are updated in
https://github.com/bitcoin/bitcoin/pull/10102 to communicate across processes.
But for now they are functionally equivalent to existing bitcoind and
bitcoin-qt executables.
0ed2d8e07d test: add BIP37 remote crash bug [CVE-2013-5700] test to p2p_filter.py (Sebastian Falbesoner)
Pull request description:
Integrates the missing message type `filteradd` to the test framework and checks that the BIP37 implementation is not vulnerable to the "remote crash bug" [CVE-2013-5700](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-5700) anymore. Prior to v.0.8.4, it was possible to trigger a division-by-zero error on the following line in the function `CBloomFilter::Hash()`:
f0d6487e29/src/bloom.cpp (L45)
By setting a zero-length filter via `filterload`, `vData.size()` is 0, so the modulo operation above, called on any .insert() or .contains() operation then crashed the node. The test uses the approach of just sending an arbitrary `filteradd` message after, which calls `CBloomFilter::insert()` (and in turn `CBloomFilter::Hash()`) on the node. The vulnerability was fixed by commit 37c6389c5a (an intentional covert fix, [according to gmaxwell](https://github.com/bitcoin/bitcoin/issues/18483#issuecomment-608224095)), which introduced flags `isEmpty`/`isFull` that wouldn't call the `Hash()` member function if `isFull` is true (set to true by default constructor).
To validate that the test fails if the implementation is vulnerable, one can simply set the flags to false in the member function `UpdateEmptyFull()` (that is called after a filter received via `filterload` is constructed), which activates the vulnerable code path calling `Hash` in any case on adding or testing for data in the filter:
```diff
diff --git a/src/bloom.cpp b/src/bloom.cpp
index bd6069b..ef294a3 100644
--- a/src/bloom.cpp
+++ b/src/bloom.cpp
@@ -199,8 +199,8 @@ void CBloomFilter::UpdateEmptyFull()
full &= vData[i] == 0xff;
empty &= vData[i] == 0;
}
- isFull = full;
- isEmpty = empty;
+ isFull = false;
+ isEmpty = false;
}
```
Resulting in:
```
$ ./p2p_filter.py
[...]
2020-04-03T14:38:59.593000Z TestFramework (INFO): Check that division-by-zero remote crash bug [CVE-2013-5700] is fixed
2020-04-03T14:38:59.695000Z TestFramework (ERROR): Assertion failed
[...]
[... some exceptions following ...]
```
ACKs for top commit:
naumenkogs:
utACK 0ed2d8e07d
Tree-SHA512: 02d0253d13eab70c4bd007b0750c56a5a92d05d419d53033523eeb3ed80318bc95196ab90f7745ea3ac9ebae7caee3adbf2a055a40a4124e0915226e49018fe8
691e2a7af7 build: create test_fuzz library from src/test/fuzz/fuzz.cpp (Harris)
Pull request description:
This PR creates a static library **libtest_fuzz.a** to speed up the compilation of fuzz tests. It is functionally similar to https://github.com/bitcoin/bitcoin/pull/17542
Fixes https://github.com/bitcoin/bitcoin/issues/18527
ACKs for top commit:
MarcoFalke:
ACK 691e2a7af7🦁
Tree-SHA512: 39d7d2731ca4370db518dbb969eb17ddbf9c030c3fe0dec0d04ff6578f24a128563fe5aced78300c92ce296623a7079fea5aea70619819a20c56fb34191f00ef
4670006762 test: remove redundant sync_with_ping after add_p2p_connection (Jon Atack)
Pull request description:
Now that #18247 is merged, these calls are redundant.
ACKs for top commit:
vasild:
utACK 4670006
Tree-SHA512: bdbfe8bcf9dbdde0a8115e3a62bfe359910798d7a3010d920ffca07049cb5f97bf8fb9b6f70079b0607105192b61a6d665774e59a2b678597b47ad6237595ad5
3dc8c012f0 test: remaining replacements of (send_message+sync_with_ping) with send_and_ping (Sebastian Falbesoner)
Pull request description:
This is a tiny follow-up PR to #18494, substituting the remaining occurences of `send_message(...)`/`sync_with_ping(...)` pairs with `send_and_ping(...)`, as suggested in the comment https://github.com/bitcoin/bitcoin/pull/18494#pullrequestreview-386418913. Thanks to jonatack and [MarcoFalke](https://github.com/bitcoin/bitcoin/pull/18494#issuecomment-608496342) for giving me the hint to do this follow-up.
ACKs for top commit:
practicalswift:
ACK 3dc8c012f0
Tree-SHA512: 44d64332933c23a7f59c0415e008ce1b2b2e07177f81cb9473b7c71558188f1c698e8973de5cc940280e4697f9553af852d9a42841304f82469673d1c8162852
facc332dc5 fuzz: Avoid running over all inputs after merging them (MarcoFalke)
Pull request description:
This cuts the time it takes to merge inputs by half
ACKs for top commit:
practicalswift:
ACK facc332dc5
Tree-SHA512: bb22992c463dd985d3b1e9b8908c591d0c8e620c38eba0a932d880f87133bfe4ca2036b166c4f79b92ddf7940f56c044e9cb8cc50309c74204df122b369c167d
fab32557f2 rpc: Make rpc documentation not depend on rpc args (MarcoFalke)
Pull request description:
This is required to host the documentation on a static resource (like a website or pdf)
ACKs for top commit:
emilengler:
utACK fab32557f2
promag:
ACK fab32557f2.
Tree-SHA512: 3ca2691c7fbd5f17c75df2887753da152f66521dcb7dee4c29af6339fdea011cecdd51f825b96bde9c6aaf82f4d915cbd5aacb52e4eae3898d9dbc216f627171
f32ab443a9 Bugfix: RPC: JSON null is not "None" (Luke Dashjr)
26dcf39581 Bugfix: RPC: Don't use a continuation elipsis after an elision elipsis (Luke Dashjr)
eca65caadc Bugfix: RPC: Add missing commas and correct indentation of explicit ELISION (Luke Dashjr)
Pull request description:
1. listsinceblock had a double ellipsis (elision + continuation); this looks ugly, just one is needed.
2. Elision ellipsis wasn't getting a comma, so was truncated to `".."` by comma-removal code.
3. Elision ellipsis was indented incorrectly (as if it was a subitem).
4. Similarly, type "none" would get truncated to `"Non"`, when it should really be `"null"` anyway.
ACKs for top commit:
MarcoFalke:
ACK f32ab443a9🐰
Tree-SHA512: 34e1c72673790ed11cdee838d64ea5e0ac498de19258df99d54b5322e003060123c65ad27ac2fd4729a1dfe52066a0629602a132b1ef85d4154affd99a065a3f
Update hardcoded seeds from seeds_emzy.txt seeds_lukejr.txt
seeds_sipa.txt seeds_sjors.txt, according to release process.
Output from makeseeds.py:
```
IPv4 IPv6 Onion Pass
1364173 244127 2454 Initial
1364173 244127 2454 Skip entries with invalid address
1129552 213117 2345 After removing duplicates
1129548 213117 2345 Skip entries from suspicious hosts
338216 191944 2249 Enforce minimal number of blocks
336851 188993 2189 Require service bit 1
6998 1520 150 Require minimum uptime
5682 1290 89 Require a known and recent user agent
5622 1279 89 Filter out hosts with multiple bitcoin ports
512 146 89 Look up ASNs and limit results per ASN and per net
```
fad2f68353 init: Replace URL_WEBSITE with PACKAGE_URL (MarcoFalke)
Pull request description:
This is needed for rebranding efforts such as #18489
ACKs for top commit:
hebasto:
ACK fad2f68353, tested on Linux Mint 19.3:
fanquake:
ACK fad2f68353 - clicked a link.
Tree-SHA512: c26e18cd328d3dd3fd7e25413e1bab780026687a148f126b8673e5f6cc13249f6c16689e45eba9da1545915c6001f96cd33f4e656c08cda3eae1c3fd88da23ea
error: unknown switch `a'
usage: git add [<options>] [--] <pathspec>...
-n, --dry-run dry run
-v, --verbose be verbose
-i, --interactive interactive picking
-p, --patch select hunks interactively
-e, --edit edit current diff and apply
-f, --force allow adding otherwise ignored files
-u, --update update tracked files
--renormalize renormalize EOL of tracked files (implies -u)
-N, --intent-to-add record only the fact that the path will be added later
-A, --all add changes from all tracked and untracked files
--ignore-removal ignore paths removed in the working tree (same as --no-all)
--refresh don't add, only refresh the index
--ignore-errors just skip files which cannot be added because of errors
--ignore-missing check if - even missing - files are ignored in dry run
--chmod (+|-)x override the executable bit of the listed files
7b2975ae8f doc: Update webchat URLs in README.md (Suriyaa Sundararuban)
Pull request description:
#### What happend?
Web links in `doc/README.md` redirected from `http://webchat.freenode.net?channels=bitcoin` to `https://webchat.freenode.net/#bitcoin`.
#### What did I changed?
* Remove URL redirection. (Update all webchat links.)
* Use HTTPS protocol instead of HTTP.
ACKs for top commit:
fanquake:
ACK 7b2975ae8f
Tree-SHA512: e55970ad368ce6dbb79295d12629d0f16318e7c43d8d194876e16e81c6e325bb136c9e8b361d61c5a04e2f3624350f81c99131b1e85d9ee5410ac22f524a4ef2
d695eb4c21 doc: note the costs of fetching all pull requests (Vasil Dimov)
Pull request description:
Also mention that it is possible to fetch just one pull request.
ACKs for top commit:
MarcoFalke:
ACK d695eb4c21
fanquake:
ACK d695eb4c21
Tree-SHA512: afe080fd018b2e773fb974956937e819085831bf0c1c5623f7f12c728639906b80666b785234058ee39fd98115a53a2fad431c54ee0840667e60bb317e4a828d
332f373a9d [scripts] previous_release: improve failed download error message (Sebastian Falbesoner)
Pull request description:
Currently, if the earlier release build/fetch script `previous_release.sh` is invoked with the option `-b` (intending to fetch a binary package from `https://bitcoin.org`) and the download fails, the user sees the following confusing output:
```
$ contrib/devtools/previous_release.sh -r -b v0.9.5
[...]
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
```
This implies that the download worked, but the archive is corrupted, when in reality the HTML document containing the delivery fail reason (most likely 404 Not Found) is saved and tried to get unpacked. In contrast to wget, curl is a bit stubborn and needs explicit instructions to react to server errors via the flag `-f` (outputs error message and returns error code, ideal for scripts): https://curl.haxx.se/docs/manpage.html#-f
On the PR branch, the output on failed download looks now the following:
```
$ contrib/devtools/previous_release.sh -r -b v0.9.5
[...]
curl: (22) The requested URL returned error: 404 Not Found
Download failed.
```
ACKs for top commit:
fanquake:
ACK 332f373a9d
Tree-SHA512: 046c931ad9e78aeb2d13faa4866d46122ed325aa142483547c2b04032d03223ed2411783b00106fcab0cd91b2f78691531ac526ed7bb3ed7547b6e2adbfb2e93
f65c9ad40f Check for overflow when calculating sum of outputs (Elichai Turkel)
Pull request description:
This was reported by practicalswift here #18046
The exact order of the if, is important, we first do `!MoneyRange(tx_out.nValue)` to make sure the amount is non-negative. and then `std::numeric_limits<CAmount>::max() - tx_out.nValue < nValueOut` checks that the addition cannot overflow (if we won't check that the amount is positive this check can also overflow! (by doing something like `max - -max`))
and only then we make sure that the some is also valid `!MoneyRange(nValueOut + tx_out.nValue)`
if any of these conditions fail we throw.
the overflowing logic:
```
a + b > max // we want to fail if a+b is more than the maximum -> will overflow
b > max - a
max - a < b
```
Closes: #18046
ACKs for top commit:
MarcoFalke:
ACK f65c9ad40f, checked that clang with O2 produces identical binaries 💕
practicalswift:
ACK f65c9ad40f
instagibbs:
utACK f65c9ad40f
vasild:
ACK f65c9ad40f modulo `s/assert.h/cassert/`
Tree-SHA512: 512d6cf4762f24c41cf9a38da486b17b19c634fa3f4efbdebfe6608779e96fc3014d5d2d29adb8001e113152c0217bbd5b3900ac4edc7b8abe77f82f36209e33
This bump will includes a couple of commits which improve the
reproducibility of the mingw-w64 toolchain. Most of which came from
debian. They will be upstreamed as upstream Guix release timeline
allows.
- Add "--no-insert-timestamp" LDFLAG for x86_64-w64-mingw32 builds
"The option --no-insert-timestamp can be used to insert a zero value for
the timestamp, this ensuring that binaries produced from identical
sources will compare identically." - ld(1)
- Set "SetDateSave off" in NSIS script
From https://nsis.sourceforge.io/Docs/Chapter4.html#flags
"This command sets the file date/time saving flag which is used by the
File command to determine whether or not to save the last write date and
time of the file, so that it can be restored on installation. Valid
flags are 'on' and 'off'. 'on' is the default."
- Add commented out NSIS options for reproducibility debugging in NSIS
script
- Make ZIPs deterministic by reseting file modification times to
SOURCE_DATE_EPOCH using touch(1) (Reference:
https://reproducible-builds.org/docs/archives/)
