This implements the new sighashing scheme from BIP341, with all relevant
whole-transaction values precomputed once and cached.
Includes changes to PrecomputedTransactionData by Pieter Wuille.
Whenever a transaction is added to the mempool or orphan pool, both
its txid and wtxid are considered AlreadyHave, and thus will eventually
be removed from m_txrequest.
The same is true for hashes added to the reject filter, but note that sometimes
only the wtxid is added (in which case only the wtxid can be removed from
m_txrequest).
Maintaining up to 100000 INVs per peer is excessive, as that is far more
than fits in a typical mempool.
Also disable the "overload" penalty for PF_RELAY peers.
This removes most transaction request logic from net_processing, and
replaces it with calls to a global TxRequestTracker object.
The major changes are:
* Announcements from outbound (and whitelisted) peers are now always
preferred over those from inbound peers. This used to be the case for the
first request (by delaying the first request from inbound peers), and
a bias afters. The 2s delay for requests from inbound peers still exists,
but after that, if viable outbound peers remain for any given transaction,
they will always be tried first.
* No more hard cap of 100 in flight transactions per peer, as there is less
need for it (memory usage is linear in the number of announcements, but
independent from the number in flight, and CPU usage isn't affected by it).
Furthermore, if only one peer announces a transaction, and it has over 100
in flight and requestable already, we still want to request it from them.
The cap is replaced with an additional 2s delay (possibly combined with the
existing 2s delays for inbound connections, and for txid peers when wtxid
peers are available).
Includes functional tests written by Marco Falke and Antoine Riard.
This adds a fuzz test that reimplements a naive reimplementation of
TxRequestTracker (with up to 16 fixed peers and 16 fixed txhashes),
and compares the real implementation against it.
This adds a new module (unused for now) which defines TxRequestTracker, a data
structure that maintains all information about transaction requests, and coordinates
requests.
3984b78cd7f49e409377f2175a56e8e4bd71d1d8 test: Add tests for CNode::ConnectedThroughNetwork (Hennadii Stepanov)
49fba9c1aa699d3aa47ea4dafe07b47c8d0aac6e net: Add CNode::ConnectedThroughNetwork member function (Hennadii Stepanov)
d4dde24034d7467883b290111da60527ab8048f8 net: Add CNode::m_inbound_onion data member (Hennadii Stepanov)
Pull request description:
This PR:
- adds `CNode::ConnectedThroughNetwork` member function
- is based on #19991, and only last two commits belong to it
- is required for https://github.com/bitcoin-core/gui/pull/86 and #20002
ACKs for top commit:
jonatack:
re-ACK 3984b78cd7f49e409377f2175a56e8e4bd71d1d8 per `git diff 3989fcf 3984b78c`
laanwj:
Code review ACK 3984b78cd7f49e409377f2175a56e8e4bd71d1d8
Tree-SHA512: 23a9c8bca8dca75113b5505fe443b294f2d42d03c98c7e34919da12d8396beb8d0ada3a58ae16e3da04b7044395f72cf9c216625afc078256cd6c897ac42bf3d
Add an option to `./configure` to suppress compilation warnings from
external headers. The option is off by default (no change in behavior,
show warnings from external headers).
This option is useful if e.g. Boost or Qt is installed outside of
`/usr/include` (warnings from headers in `/usr/include` are already
suppressed by default) and those warnings stand in the way of compiling
Bitcoin Core with `-Werror[=...]` or they just clutter the build output
too much and make our own warnings hard to spot.
A BIP-341 signature message may commit to the scriptPubKeys and amounts
of all spent outputs (including other ones than the input being signed
for spends), so keep them available to signature hashing code.
dcf0cb477699d11afd0ff37c8bfb2b1b4f7f1ee5 tor: make a TORv3 hidden service instead of TORv2 (Vasil Dimov)
353a3fdaad055eea42a0baf7326bdd591f541170 net: advertise support for ADDRv2 via new message (Vasil Dimov)
201a4596d92d640d5eb7e76cc8d959228fa09dbb net: CAddress & CAddrMan: (un)serialize as ADDRv2 (Vasil Dimov)
1d3ec2a1fda7446323786a52da1fd109c01aa6fb Support bypassing range check in ReadCompactSize (Pieter Wuille)
Pull request description:
This PR contains the two remaining commits from #19031 to complete the [BIP155](https://github.com/bitcoin/bips/blob/master/bip-0155.mediawiki) implementation:
`net: CAddress & CAddrMan: (un)serialize as ADDRv2`
`net: advertise support for ADDRv2 via new message`
plus one more commit:
`tor: make a TORv3 hidden service instead of TORv2`
ACKs for top commit:
jonatack:
re-ACK dcf0cb477699d11afd0ff37c8bfb2b1b4f7f1ee5 per `git diff 9b56a68 dcf0cb4` only change since last review is an update to the release notes which partially picked up the suggested text. Running a node on this branch and addnode-ing to 6 other Tor v3 nodes, I see "addrv2" and "sendaddrv2" messages in getpeerinfo in both the "bytesrecv_per_msg" and "bytessent_per_msg" JSON objects.
sipa:
ACK dcf0cb477699d11afd0ff37c8bfb2b1b4f7f1ee5
hebasto:
re-ACK dcf0cb477699d11afd0ff37c8bfb2b1b4f7f1ee5, the node works flawlessly in all of the modes: Tor-only, clearnet-only, mixed.
laanwj:
Edit: I have to retract this ACK for now, I'm having some problems with this PR on a FreeBSD node. It drops all outgoing connections with this dcf0cb477699d11afd0ff37c8bfb2b1b4f7f1ee5 merged on master (12a1c3ad1a43634d2a98717e49e3f02c4acea2fe).
ariard:
Code Review ACK dcf0cb4
Tree-SHA512: 28d4d0d817b8664d2f4b18c0e0f31579b2f0f2d23310ed213f1f436a4242afea14dfbf99e07e15889bc5c5c71ad50056797e9307ff8a90e96704f588a6171308
Introduce a new message `sendaddrv2` to signal support for ADDRv2.
Send the new message immediately after sending the `VERACK` message.
Add support for receiving and parsing ADDRv2 messages.
Send ADDRv2 messages (instead of ADDR) to a peer if he has
advertised support for it.
Co-authored-by: Carl Dong <contact@carldong.me>
Change the serialization of `CAddrMan` to serialize its addresses
in ADDRv2/BIP155 format by default. Introduce a new `CAddrMan` format
version (3).
Add support for ADDRv2 format in `CAddress` (un)serialization.
Co-authored-by: Carl Dong <contact@carldong.me>
This is needed when we want to encode an arbitrary number as CompactSize
like node service flags, which is a bitmask and could be bigger than the
usual size of an object.
1afcd41a906e6417925e80578c0d850d269dc008 [net] Remove CombinerAll (John Newbery)
Pull request description:
This was introduced in 9519a9a4 for use with boost signals. Boost signals
have not been used in net since 8ad663c1, so this code is unused.
ACKs for top commit:
MarcoFalke:
review ACK 1afcd41a906e6417925e80578c0d850d269dc008
laanwj:
code review ACK 1afcd41a906e6417925e80578c0d850d269dc008
Tree-SHA512: a4313142afb88bf12f15abc4e717b3b0d0b40d2d5db2638494af3181e1cd680d7b036087050fc0e0dfe606228849a2e20ae85135908a9ebe8ff2130f163920e1
907f142fc7e1d35f443be076367739faf11cc2cc rpc: change no wallet loaded message to be clearer (Andrew Chow)
Pull request description:
Changes the no wallet is loaded rpc error message to be clearer that no wallet is loaded and how the user can load or create a wallet. Also changes the error code from METHOD_NOT_FOUND to RPC_WALLET_NOT_FOUND as that makes more sense.
ACKs for top commit:
MarcoFalke:
review ACK 907f142fc7e1d35f443be076367739faf11cc2cc
kristapsk:
ACK 907f142fc7e1d35f443be076367739faf11cc2cc. In addition to standard tests, just in case tested that this doesn't break anything with JoinMarket.
meshcollider:
utACK 907f142fc7e1d35f443be076367739faf11cc2cc
Tree-SHA512: 4b413e6ab5430ec75a79de9db6583f2f3f38ccdf71aa373d8386a56e64f07f92200c8107c8c82c92c7c431d739615977c208b771a24c5960fa8676789b5497a2
fae7a1c18803675e70b9bf66575e1e0a6e01f6f6 fuzz: Configure check for main function (MarcoFalke)
Pull request description:
Instead of the PP jungle, use a proper configure check
Fixes https://github.com/google/honggfuzz/issues/336#issuecomment-702972138
ACKs for top commit:
practicalswift:
ACK fae7a1c18803675e70b9bf66575e1e0a6e01f6f6
Tree-SHA512: 2e55457d01f9ac598bb1e119d8b49dca55a28f88ec164cee6b5f071c29e9791f5a46cc8ee2b801b3a3faf906348da964ce32e7254da981c1104b9210a3508100
Changes the no wallet is loaded rpc error message to be clearer that no
wallet is loaded and how the user can load or create a wallet. Also
changes the error code from METHOD_NOT_FOUND to RPC_WALLET_NOT_FOUND as
that makes more sense.
d103484fe81a8a5bf1d692f3f7d1c0ef1be5f63c util: Do not use gArgs global in ArgsManager member functions (Hennadii Stepanov)
Pull request description:
ACKs for top commit:
practicalswift:
ACK d103484fe81a8a5bf1d692f3f7d1c0ef1be5f63c: patch looks correct
Tree-SHA512: dda7a5062363170c6995f2fd8fda48c0a919e5ca67be9faa8f0fa66f9d3b535f134eb6f4860a0859bc5457c02230b34a8d1264045f22bed8d30668158ac2271f
b048b275d9711f70847afaea5450f17a0f7e673a [validation] Remove absurdfee from accepttomempool (John Newbery)
932564b9cfda8446a957649c2316a52e868ad5d4 scripted-diff: update max-fee-exceeded error message to include RPC (gzhao408)
8f1290c60159a3171c27250bc95687548c5c1b84 [rpc/node] check for high fee before ATMP in clients (gzhao408)
Pull request description:
Picked up from #15810. Add separate fee-checking logic for clients that need to enforce max fee rates, then remove the `absurdFee` logic from ATMP.
ATMP's `nAbsurdFee` argument is used to enforce user-specific behavior (it is not policy since it isn't applied consistently: it is only ever used in RPC and wallet, and set to 0 everywhere else internally). It should be removed from `AcceptToMemoryPool` because (1) validation results/mempool behavior should not be user-specific and (2) enforcing a max fee rate should be the responsibility of the client instead of the mempool.
Note: this PR does not intend to _remove_ protection from high fees, just re-delegate the responsibility to clients.
ACKs for top commit:
jnewbery:
utACK b048b275d9711f70847afaea5450f17a0f7e673a
LarryRuane:
re-ACK b048b275d9711f70847afaea5450f17a0f7e673a
MarcoFalke:
re-ACK b048b275d9 , only change is squashing one commit 🏦
instagibbs:
utACK b048b275d9
Tree-SHA512: 57c17ba16d230a4cae2896dd6a64c924f307757824e35784bf96da7b10aff2d8ea910710edf35e981035623a155f8766209a92a0fdb856549fde78bc3eaae4d2
In preparation for adding Schnorr versions of `CheckSig`, `VerifySignature`, and
`ComputeEntry`, give them an ECDSA specific name.
-BEGIN VERIFY SCRIPT-
sed -i 's/CheckSig(/CheckECDSASignature(/g' $(git grep -l CheckSig ./src)
sed -i 's/VerifySignature(/VerifyECDSASignature(/g' $(git grep -l VerifySignature ./src)
sed -i 's/ComputeEntry(/ComputeEntryECDSA(/g' $(git grep -l ComputeEntry ./src)
-END VERIFY SCRIPT-
-BEGIN VERIFY SCRIPT-
sed -i 's/Fee exceeds maximum configured by \-\maxtxfee/Fee exceeds maximum configured by user (e.g. -maxtxfee, maxfeerate)/g' src/util/error.cpp test/functional/rpc_fundrawtransaction.py test/functional/rpc_psbt.py test/functional/rpc_rawtransaction.py test/functional/wallet_bumpfee.py test/functional/wallet_create_tx.py
-END VERIFY SCRIPT-
Check absurd fee in BroadcastTransaction and RPC,
return TransactionError::MAX_FEE_EXCEEDED instead
of TxValidationResult::TX_NOT_STANDARD because this
is client preference, not a node-wide policy.
1885ad35467f201f2a210057797aae8a450e7cdf RPC: remove duplicate line in getblock help (Fabian Jahr)
Pull request description:
Line simply seems duplicated in error.
Testing instructions:
Run `src/bitcoin-cli help getblock` on master branch to reproduce. Then build this PR and compare its results.
ACKs for top commit:
dhruv:
tACK `1885ad3`
kristapsk:
ACK 1885ad35467f201f2a210057797aae8a450e7cdf
Emzy:
tACK 1885ad35467f201f2a210057797aae8a450e7cdf
Tree-SHA512: 870c035cb553b0e1d5ef72e64231ef277e0392efe94bc6ecf47129023bd94a6d5a276f46529807f68a1db55c7baa94d9119c7264d9947bc4e5dd9dcefd1b13e7
675e55e01392971aa56bda56cb09498b466d0902 Ignore unknown messages before VERACK (Suhas Daftuar)
Pull request description:
This allows for feature negotiation to take place with messages between VERSION and VERACK in the future, without requiring additional software changes to specifically ignore messages for features that are unimplemented by our software.
ACKs for top commit:
sipa:
utACK 675e55e01392971aa56bda56cb09498b466d0902
practicalswift:
ACK 675e55e01392971aa56bda56cb09498b466d0902: patch looks correct
MarcoFalke:
ACK 675e55e01392971aa56bda56cb09498b466d0902
hebasto:
ACK 675e55e01392971aa56bda56cb09498b466d0902, the offender peer will be eventually disconnected due to the timeout.
Tree-SHA512: 8d2b1d8b9843f2ee26b2c30f7c5ff0bfcfbe3f46b32cd0369c48ece26624151091237e83ce3f18c6da004099026602cfab1642ac916db777f047d170b365c007
f471a3be00c2b6433b8c258b716982c0539da13f scripted diff: Improve invalid vout value rpc error message (Nima Yazdanmehr)
Pull request description:
Since the `vout` value can start at `0`, the error message for *negative* values can be improved to something like: `vout cannot be negative`.
ACKs for top commit:
fanquake:
ACK f471a3be00c2b6433b8c258b716982c0539da13f
promag:
Code review ACK f471a3be00c2b6433b8c258b716982c0539da13f.
Tree-SHA512: fbdee3d0ddd5b58eb93934a1217b44e125a9ad39e672b1f35c7609c6c5fcf45ae1b731d3d6135b7225d98792dbfc34a50907b8c41274a5b029d7b5c59f886560
d76925478efd35e6fd835370639f2139b28381e4 [doc] Clarify semantic of peer's m_protect w.r.t to outbound eviction logics (Antoine Riard)
ac71fe936da290adf5a3155fe8db5f78b485f1f1 [doc] Clarify scope of eviction protection of outbound block-relay peers (Antoine Riard)
Pull request description:
Block-relay-only peers were introduced by #15759. According to its
author, it was intented to make them only immune to outbound peer
rotation-based eviction and not from all eviction as modified comment
leans to think of.
Clearly indicate that outbound block-relay peers aren't protected
from eviction by the bad/lagging chain logic.
Fix#19863
ACKs for top commit:
naumenkogs:
ACK d76925478efd35e6fd835370639f2139b28381e4
jonatack:
ACK d76925478efd35e6fd835370639f2139b28381e4
Tree-SHA512: 597fbd62838a6e39276024165b11514cad20a2e9d33cf9202d261cbadcb62b2df427c858e0cb57e585840d4c1d4600104aa53916bb868541f2580e4eed9b4b52
