16842 Commits

Author SHA1 Message Date
fanquake
d82b2c6e65
Merge #19898: log: print unexpected version warning in validation log category
62dba9628d2532dca0c44934600f5aac3b21e275 log: print unexpected version warning in validation log category (nthumann)

Pull request description:

  Fixes #19603: As suggested by practicalswift, instead of always printing `<n> of the last 100 blocks have unexpected version` as a warning appended to UpdateTip, it is now printed in the validation log category and therefore only visible with `-debug=validation` enabled.

  Before:
  `2020-09-06T15:56:00Z UpdateTip: new best=00000000000000000001b2872e107a98b57913120e5c6c87ce2715a34c40adf8 height=646969 version=0x20400000 log2_work=92.261571 tx=565651941 date='2020-09-06T10:35:36Z' progress=0.999888 cache=32.2MiB(237417txo) warning='72 of last 100 blocks have unexpected version'`
  After:
  `2020-09-06T16:31:26Z UpdateTip: new best=0000000000000000000b3bd786dc42745dd7be4a8c695500a04518cb9e2f4dc1 height=646971 version=0x20000000 log2_work=92.261607 tx=565655901 date='2020-09-06T10:57:19Z' progress=0.999883 cache=3.8MiB(27550txo)`
  `2020-09-06T16:31:26Z 71 of last 100 blocks have unexpected version`

  Ran unit & functional tests, confirmed that the warning is now only printed when validation category is enabled.

ACKs for top commit:
  theStack:
    ACK 62dba9628d2532dca0c44934600f5aac3b21e275
  MarcoFalke:
    re-ACK 62dba96
  practicalswift:
    ACK 62dba9628d2532dca0c44934600f5aac3b21e275 -- only change since last ACK is `s/nUpgraded/num_unexpected_version/`
  hebasto:
    re-ACK 62dba9628d2532dca0c44934600f5aac3b21e275, https://github.com/bitcoin/bitcoin/pull/19898#pullrequestreview-483158708 is resolved now.

Tree-SHA512: 2100ca7d6d3fd67c92e81d75162d2506d6f1ecf1761d5180d76663fac06771b35e5c4235ebe1a00731b5f7db82db3cd19328627929c8f22912df592686ba51d3
2020-09-29 20:41:11 +08:00
fanquake
ec9b4492eb
Merge #19630: Cleanup fee estimation code
a3abeec33a6ae903e514c7a7b6f587b7c17288a0 policy/fees: remove a floating-point division by zero (Antoine Poinsot)
c36869bbf6a38626833b4aea53be024c48ede475 policy/fees: unify some duplicated for loops (Antoine Poinsot)
569d92a4d2924a1f6d50775980b591552f6372e7 policy/fees: small readability improvements (Antoine Poinsot)
5b8cb35621891b681f9b49a9de5f6d8da4ccdecc policy/fee: remove requireGreater parameter in EstimateMedianVal() (Antoine Poinsot)
dba8196b447b6a85be66890db70928100e867d8b policy/fees: correct decay explanation comments (Antoine Poinsot)

Pull request description:

  This (*does not* change behaviour and) cleans up a bit of unused code in `CBlockPolicyEstimator` and friends, and slightly improves readability of the rest (comment correction etc.). The last commit is a small reformatting one which I could not resist but am happy to remove at will.

ACKs for top commit:
  jnewbery:
    utACK a3abeec33a6ae903e514c7a7b6f587b7c17288a0
  MarcoFalke:
    ACK a3abeec33a6ae903e514c7a7b6f587b7c17288a0 💹
  ariard:
    Code Review ACK a3abeec.

Tree-SHA512: b7620bcd23a2ffa8f7ed859467868fc0f6488279e3ee634f6d408872cb866ad086a037e8ace76599a05b7e9c07768adf5016b0ae782d153196b9c030db4c34a5
2020-09-29 16:35:01 +08:00
fanquake
6af9b31bfc
Merge #19107: p2p: Move all header verification into the network layer, extend logging
deb52711a17236d0fca302701b5af585341ab42a Remove header checks out of net_processing (Troy Giorshev)
52d4ae46ab822d0f54e246a6f2364415cda149bd Give V1TransportDeserializer CChainParams& member (Troy Giorshev)
5bceef6b12fa16d20287693be377dace3dfec3e5 Change CMessageHeader Constructor (Troy Giorshev)
1ca20c1af8f08f07c407c3183c37b467ddf0f413 Add doxygen comment for ReceiveMsgBytes (Troy Giorshev)
890b1d7c2b8312d41d048d2db124586c5dbc8a49 Move checksum check from net_processing to net (Troy Giorshev)
2716647ebf60cea05fc9edce6a18dcce4e7727ad Give V1TransportDeserializer an m_node_id member (Troy Giorshev)

Pull request description:

  Inspired by #15206 and #15197, this PR moves all message header verification from the message processing layer and into the network/transport layer.

  In the previous PRs there is a change in behavior, where we would disconnect from peers upon a single failed checksum check.  In various discussions there was concern over whether this was the right choice, and some expressed a desire to see how this would look if it was made to be a pure refactor.

  For more context, see https://bitcoincore.reviews/15206.html#l-81.

  This PR improves the separation between the p2p layers, helping improvements like [BIP324](https://github.com/bitcoin/bitcoin/pull/18242) and #18989.

ACKs for top commit:
  ryanofsky:
    Code review ACK deb52711a17236d0fca302701b5af585341ab42a just rebase due to conflict on adjacent line
  jnewbery:
    Code review ACK deb52711a17236d0fca302701b5af585341ab42a.

Tree-SHA512: 1a3b7ae883b020cfee1bef968813e04df651ffdad9dd961a826bd80654f2c98676ce7f4721038a1b78d8790e4cebe8060419e3d8affc97ce2b9b4e4b72e6fa9f
2020-09-29 16:14:40 +08:00
fanquake
e36aa351a3
Merge #19969: Send RPC bug fix and touch-ups
f7b331ea85d45c7337e527b6e77a45da7a689b7d rpc: add brackets to ConstructTransaction (Sjors Provoost)
d813d26f06248aaa7be3c698c87939cc777fafd0 [rpc] send: various touch-ups (Sjors Provoost)
0fc1c685e1ca68ca8ed2b35f623bbe6a9fc36d66 [rpc] send: fix parsing replaceable option (Sjors Provoost)
efc9b85e6f4aa431d308089874a18f0bbdcdd0fd Mark send RPC experimental (Sjors Provoost)

Pull request description:

  Followup based on #16378 nits. It also fixes an argument parsing error (uncaught because the test wasn't sufficiently thorough).

  I marked the RPC as experimental so we can tweak it a bit over the next release cycle.

ACKs for top commit:
  meshcollider:
    utACK f7b331ea85d45c7337e527b6e77a45da7a689b7d
  fjahr:
    utACK f7b331ea85d45c7337e527b6e77a45da7a689b7d
  kallewoof:
    ACK f7b331ea85d45c7337e527b6e77a45da7a689b7d

Tree-SHA512: 82dd8ac76a6558872db3f5249d4d6440469400aaa339153bc627d1ee673a91ecfadecb486bc1939ba87ebbd80e26ff29698e93e358599f3d26fde0e526892afe
2020-09-29 15:14:08 +08:00
fanquake
5db44c740e
Merge #20014: doc: Mention signet in -help output
b3972bca9f74b3e75918676ef029f1964b0622a1 doc: Mention signet in -help output (Hennadii Stepanov)

Pull request description:

  ```
  $ src/bitcoind -help | grep -A 4 -e '-chain=' | head -8
    -chain=<chain>
         Use the chain <chain> (default: main). Allowed values: main, test,
         signet, regtest

    -signet
         Use the signet chain. Equivalent to -chain=signet. Note that the network
         is defined by the -signetchallenge parameter

  ```

  ```
  $ src/bitcoind -help | grep -A 3 -e '-port='
    -port=<port>
         Listen for connections on <port> (default: 8333, testnet: 18333 signet:
         38333, regtest: 18444)

  ```

  ```
  $ src/bitcoind -help | grep -A 3 -e '-rpcport='
    -rpcport=<port>
         Listen for JSON-RPC connections on <port> (default: 8332, testnet:
         18332, signet: 38332, regtest: 18443)

  ```

ACKs for top commit:
  jonatack:
    ACK b3972bca9f74b3e75918676ef029f1964b0622a1
  kallewoof:
    ACK b3972bca9f74b3e75918676ef029f1964b0622a1
  ajtowns:
    ACK b3972bca9f74b3e75918676ef029f1964b0622a1 - skimmed code only, looks fine to me

Tree-SHA512: 66c59cdc3c19e8f8a02d3f3f992ff1db06769df63244d4af62629e18aaf4a12b3b7e75e4a0b9f616033cdc4415da046053ba36fede8be145b2dc695b2aa69a02
2020-09-29 14:04:25 +08:00
Pieter Wuille
655937ebcb
Merge #19845: net: CNetAddr: add support to (un)serialize as ADDRv2
7be6ff61875a8d5d2335bff5d1f16ba40557adb0 net: recognize TORv3/I2P/CJDNS networks (Vasil Dimov)
e0d73573a37bf4b519f6f61e5678572d48a64517 net: CNetAddr: add support to (un)serialize as ADDRv2 (Vasil Dimov)
fe42411b4b07b99c591855f5f00ad45dfeec8e30 test: move HasReason so it can be reused (Vasil Dimov)
d2bb681f96fb327b4c4d5b2b113692ca22fdffbf util: move HasPrefix() so it can be reused (Vasil Dimov)

Pull request description:

  (chopped off from #19031 to ease review)

  Add an optional support to serialize/unserialize `CNetAddr` in ADDRv2 format (BIP155). The new serialization is engaged by ORing a flag into the stream version.

  So far this is only used in tests to ensure the new code works as expected.

ACKs for top commit:
  Sjors:
    re-tACK 7be6ff61875a8d5d2335bff5d1f16ba40557adb0
  sipa:
    re-utACK 7be6ff61875a8d5d2335bff5d1f16ba40557adb0
  eriknylund:
    ACK 7be6ff61875a8d5d2335bff5d1f16ba40557adb0 I built the PR on macOS Catalina 10.15.6, ran both tests and functional tests. I've reviewed the code and think the changes look good and according to BIP155. I verified that the added Base32 encoding test looks as proposed and working. I've run a node for a week only with Onion addresses `-onlynet=onion` without issues and I can connect to other peer reviewers running TorV3 on their nodes and I can connect both of my test nodes to each other.
  jonatack:
    re-ACK 7be6ff61875a8d5d2335bff5d1f16ba40557adb0 per `git diff b9c46e0 7be6ff6`, debug build, ran/running bitcoind with this change and observed the log and `-netinfo` peer connections while connected as a tor v2 service to both tor v2 peers and also five tor v3 peers.
  hebasto:
    ACK 7be6ff61875a8d5d2335bff5d1f16ba40557adb0, tested on Linux Mint 20 (x86_64): on top of this pull and #19031 I'm able to connect to onion v3 addresses, and jonatack is able to connect to my created onion v3 address.

Tree-SHA512: dc621411ac4393993aa3ccad10991717ec5f9f2643cae46a24a89802df0a33d6042994fc8ff2f0f397a3dbcd1c0e58fe4724305a2f9eb64d9342c3bdf784d9be
2020-09-28 12:27:08 -07:00
MarcoFalke
2552702000
Merge #15367: feature: Added ability for users to add a startup command
090530cc24054d6b4658752bb88f75a3b73eab5d feature: Added ability for users to add a startup command (Ben Carman)

Pull request description:

  Thoughts for adding the feature is for users to be able to add things like electrum-personal-server or lnd to run whenever Bitcoin Core is running.  Open to feedback about the feature.

ACKs for top commit:
  MarcoFalke:
    re-ACK 090530cc24
  dongcarl:
    tACK 090530c

Tree-SHA512: ba514d2fc8b4fb12b781c1a9c89845a25fce0b80ba7c907761cde4abb81edd03fa643682edc895986dc20b273ac3b95769508806db7fbd99ec28623f85c41e67
2020-09-28 20:44:33 +02:00
Ben Carman
090530cc24
feature: Added ability for users to add a startup command 2020-09-28 10:38:36 -05:00
MarcoFalke
c95784e3d3
Merge #20016: uint256: 1 is a constant
4cc7171c9887e88027642927c979e507d7b78dda wallet: no need for duplicate storage for ABANDON_HASH constant (Anthony Towns)
82cf4641f4a161834d07ce83c18982d9b143c040 scripted-diff: Replace UINT256_ONE() with uint256::ONE (Anthony Towns)
183f308fff4caad3e3ada654b6fdf597d262c4c1 uint256: Update constructors to c++11, make ONE static (Anthony Towns)

Pull request description:

  `UINT256_ONE()` returns a reference to a global; mark it as const to be sure someone doesn't accidently modify it.

ACKs for top commit:
  promag:
    ACK 4cc7171c9887e88027642927c979e507d7b78dda
  MarcoFalke:
    re ACK 4cc7171c98, only change is some constexpr shuffling 🛁
  kallewoof:
    ACK 4cc7171c9887e88027642927c979e507d7b78dda

Tree-SHA512: 7f399658bfd9ffa4075bc2349049476d842b9579a67518fb7151f56eab36907ef24b1474ee1e89bdc69fe181abe7295dfe19e33b3623d43cec71fc00e356e347
2020-09-28 09:16:56 +02:00
Anthony Towns
4cc7171c98 wallet: no need for duplicate storage for ABANDON_HASH constant 2020-09-28 12:14:19 +10:00
Anthony Towns
82cf4641f4 scripted-diff: Replace UINT256_ONE() with uint256::ONE
-BEGIN VERIFY SCRIPT-
sed -i '/inline.* UINT256_ONE() {/,+1d' src/uint256.h
sed -i 's/UINT256_ONE()/uint256::ONE/' $(git grep -l UINT256_ONE)
-END VERIFY SCRIPT-
2020-09-28 12:14:19 +10:00
Anthony Towns
183f308fff uint256: Update constructors to c++11, make ONE static
Replace the memset with C++11 value/aggregate initialisation of
the m_data array, which still ensures the unspecified values end
up as zero-initialised.

This then allows changing UINT256_ONE() from dynamically allocating an
object, to a simpler referencing a static allocation.
2020-09-28 12:14:19 +10:00
MarcoFalke
4f45ea1f73
Merge #19725: [RPC] Add connection type to getpeerinfo, improve logs
a512925e19a70d7f6b80ac530a169f45ffaafa1c [doc] Release notes (Amiti Uttarwar)
50f94b34a33c954f6e207f509c93d33267a5c3e2 [rpc] Deprecate getpeerinfo addnode field (Amiti Uttarwar)
df091b9b509f0b10e4315c0bfa2da0cc0c31c22f [refactor] Rename test file to allow any getpeerinfo deprecations. (Amiti Uttarwar)
395acfa83a5436790c1a722a5609ac9d48df235f [rpc] Add connection type to getpeerinfo RPC, update tests (Amiti Uttarwar)
49c10a9ca40967d28ae16dfea9cccc6f3a6624a1 [log] Add connection type to log statement (Amiti Uttarwar)

Pull request description:

  After #19316, we can more directly expose information about the connection type on the `getpeerinfo` RPC. Doing so also makes the existing addnode field redundant, so this PR begins the process of deprecating this field.

  This PR also includes one commit that improves a log message, as both use a shared function to return the connection type as a string.

  Suggested by sdaftuar- https://github.com/bitcoin/bitcoin/pull/19316#discussion_r468001604 & https://github.com/bitcoin/bitcoin/pull/19316#discussion_r468018093

ACKs for top commit:
  jnewbery:
    Code review ACK a512925e19a70d7f6b80ac530a169f45ffaafa1c.
  sipa:
    utACK a512925e19a70d7f6b80ac530a169f45ffaafa1c
  guggero:
    Tested and code review ACK a512925e.
  MarcoFalke:
    cr ACK a512925e19a70d7f6b80ac530a169f45ffaafa1c 🌇
  promag:
    Code review ACK a512925e19a70d7f6b80ac530a169f45ffaafa1c.

Tree-SHA512: 601a7a38aee235ee59aca690784f886dc2ae4e418b2e6422c4b58cd597376c00f74910f66920b08a08a0bec28bf8022e71a1435785ff6ba8a188954261aba78e
2020-09-26 17:24:54 +02:00
Hennadii Stepanov
b3972bca9f
doc: Mention signet in -help output 2020-09-25 15:23:01 +03:00
MarcoFalke
1b313cacc9
Merge #19927: validation: Reduce direct g_chainman usage
72a1d5c6f3834e206719ee5121df7727aed5b786 validation: Remove review-only comments + assertions (Carl Dong)
3756853b15902d63f4b5a3129e8b5d82e84e125b docs: Move FindFilesToPrune{,Manual} doxygen comment (Carl Dong)
485899a93c6f5fff62090907efb0ac938992e1fb style: Make FindFilesToPrune{,Manual} match style guide (Carl Dong)
3f5b5f3f6db0e5716911b3fba1460ce327e8a845 validation: Move FindFilesToPrune{,Manual} to BlockManager (Carl Dong)
f8d4975ab3fcd3553843cf0862251289c88c106b validation: Move PruneOneBlockFile to BlockManager (Carl Dong)
74f73c783d46b012f375d819e2cd09c792820cd5 validation: Pass in chainman to UnloadBlockIndex (Carl Dong)
4668ded6d6ea4299d998abbb57543f37519812e2 validation: Move ~CMainCleanup logic to ~BlockManager (Carl Dong)

Pull request description:

  This PR paves the way for de-globalizing `g_chainman` entirely by removing the usage of `g_chainman` in the following functions/methods:
  - `~CMainCleanup`
  - `CChainState::FlushStateToDisk`
  - `UnloadBlockIndex`

  The remaining direct uses of `g_chainman` are as follows:
  1. In initialization codepaths:
  	- `AppTests`
  	- `AppInitMain`
  	- `TestingSetup::TestingSetup`
  2. `::ChainstateActive`
  3. `LookupBlockIndex`
  	- Note: `LookupBlockIndex` is used extensively throughout the codebase and require a much larger set of changes, therefore I've left it out of this initial PR

ACKs for top commit:
  MarcoFalke:
    re-ACK 72a1d5c6f3 👚
  jnewbery:
    utACK 72a1d5c6f3834e206719ee5121df7727aed5b786

Tree-SHA512: 944a4fa8405eecf39706ff944375d6824373aaeea849d11473f08181eff26b12f70043a8348a5b08e6e9021b243b481842fbdfbc7c3140ca795fce3688b7f5c3
2020-09-23 20:35:54 +02:00
MarcoFalke
5e14fafb31
Merge #19994: Assert that RPCArg names are equal to CRPCCommand ones (net, rpcwallet)
fa14f57fbc3c1fa2b9eea5df687f0fb36d452bd5 Assert that RPCArg names are equal to CRPCCommand ones (net, rpcwallet) (MarcoFalke)

Pull request description:

  This is the last part split out from #18531 to just touch some RPC methods. Description from the main pr:

  ### Motivation

  RPCArg names in the rpc help are currently only used for documentation. However, in the future they could be used to teach the server the named arguments. Named arguments are currently registered by the `CRPCCommand`s and duplicate the RPCArg names from the documentation. This redundancy is fragile, and has lead to errors in the past (despite having linters to catch those kind of errors). See section "bugs found" for a list of bugs that have been found as a result of the changes here.

  ### Changes

  The changes here add an assert in the `CRPCCommand` constructor that the RPCArg names are identical to the ones in the `CRPCCommand`.

  ### Future work

  > Here or follow up, makes sense to also assert type of returned UniValue?

  Sure, but let's not get ahead of ourselves. I am going to submit any further works as follow-ups, including:

  * Removing the CRPCCommand arguments, now that they are asserted to be equal and thus redundant
  * Removing all python regex linters on the args, now that RPCMan can be used to generate any output, including the cli.cpp table
  * Auto-formatting and sanity checking the RPCExamples with RPCMan
  * Checking passed-in json in self-check. Removing redundant checks
  * Checking returned json against documentation to avoid regressions or false documentation
  * Compile the RPC documentation at compile-time to ensure it doesn't change at runtime and is completely static

  ### Bugs found

  * The assert identified issue #18607
  * The changes itself fixed bug #19250

ACKs for top commit:
  fjahr:
    tACK fa14f57fbc3c1fa2b9eea5df687f0fb36d452bd5
  ryanofsky:
    Code review ACK fa14f57fbc3c1fa2b9eea5df687f0fb36d452bd5. Just straightforward replacements except code moved in `addnode`, and displatching updated in `bumpfee_helper`

Tree-SHA512: e07af150f1d95a88e558256ce197a6b7dc6cd722a6d6c13c75d944c49c2e2441f8b8237e9f94b03db69fa18f9bda627b0781d5e1da70bf5415e09b38728a8cb1
2020-09-23 20:11:08 +02:00
MarcoFalke
8235dca621
Merge #19979: Replace LockAssertion with AssertLockHeld, remove LockAssertion
0bd1184adf6610c0bd14f4e9a25c0a200e65ae25 Remove unused LockAssertion struct (Hennadii Stepanov)
ab2a44297fd0796bf5797ae2a477e8e56d9c3c12 Replace LockAssertion with a proper thread safety annotations (Hennadii Stepanov)
73f71e19965e07534eb47701f2b23c9ed59ef475 refactor: Use explicit function type instead of template (Hennadii Stepanov)

Pull request description:

  This PR replaces `LockAssertion` with `AssertLockHeld`, and removes `LockAssertion`.

  This PR is compared with alternatives in https://github.com/bitcoin-core/bitcoin-devwiki/wiki/AssertLockHeld-PRs

ACKs for top commit:
  MarcoFalke:
    ACK 0bd1184adf6610c0bd14f4e9a25c0a200e65ae25
  ajtowns:
    ACK 0bd1184adf6610c0bd14f4e9a25c0a200e65ae25
  vasild:
    ACK 0bd1184ad

Tree-SHA512: ef7780dd689faf0bb479fdb97c49bc652e2dd10c148234bb95502dfbb676442d8565ee37864d923ca21a25f9dc2a335bf46ee82c095e387b59a664ab05c0ae41
2020-09-23 16:37:07 +02:00
Wladimir J. van der Laan
9e217f5a6f
Merge #19572: ZMQ: Create "sequence" notifier, enabling client-side mempool tracking
759d94e70f6844443106404882c7b105f3a4dba7 Update zmq notification documentation and sample consumer (Gregory Sanders)
68c3c7e1bdd00bbe7d70592a8eb39520fa3f87f1 Add functional tests for zmq sequence topic and mempool sequence logic (Gregory Sanders)
e76fc2b84d065c9d06010d0a10b316f1f9d36fb9 Add 'sequence' zmq publisher to track all block (dis)connects, mempool deltas (Gregory Sanders)
1b615e61bfc464f215a1b48e6e27d1e8fc16b2d1 zmq test: Actually make reorg occur (Gregory Sanders)

Pull request description:

  This PR creates a new ZMQ notifier that gives a "total hash history" of block (dis)connection, mempool addition/substraction, all in one pipeline. It also exposes a "mempool sequence number" to both this notifier and `getrawmempool` results, which allows the consumer to use the results together without confusion about ordering of results and without excessive `getrawmempool` polling.

  See the functional test `interfaces_zmq.py::test_mempool_sync` which shows the proposed user flow for the client-side tracking of mempool contents and confirmations.

  Inspired by https://github.com/bitcoin/bitcoin/pull/19462#issuecomment-656140421
  Alternative to https://github.com/bitcoin/bitcoin/pull/19462 due to noted deficiencies in current zmq notification streams.

  Also fixes a legacy zmq test that didn't actually trigger a reorg because of identical blocks being generated on each side of the split(oops)

ACKs for top commit:
  laanwj:
    Code review ACK 759d94e70f6844443106404882c7b105f3a4dba7

Tree-SHA512: 9daf0d7d996190f3a68ff40340a687519323d7a6c51dcb26be457fbc013217ea7b62fbd0700b74b654433d2e370704feb61e5584399290692464fcfcb72ce3b7
2020-09-23 13:55:24 +02:00
MarcoFalke
8219893825
Merge #19993: refactor: Signet fixups
facaf9e61f4b9ea91fab554d495ebea1043d08fb doc: Document signet BIP (MarcoFalke)
faf0a26711eed9264113463e56b988cf9fe549fd doc: Update comments for new chain settings (-signet and -chain=signet) (MarcoFalke)
fae0548686cb3d095086d3f0fef38dcfcd31d8ca fuzz: Remove needless guard (MarcoFalke)
77771a03df6c5d940b340d15eb88f2ac9a29c13a refactor: Remove SignetTxs::m_valid and use optional instead (MarcoFalke)
fa2ad5dae17b237641b8ece0e68ffcdd79d543bf test: Run signet test even when wallet was not compiled (MarcoFalke)

Pull request description:

  Some doc and test fixups for #18267

ACKs for top commit:
  ajtowns:
    ACK facaf9e61f4b9ea91fab554d495ebea1043d08fb -- code review only
  dr-orlovsky:
    Reviewed & ACK facaf9e61f
  kallewoof:
    Code review ACK facaf9e61f4b9ea91fab554d495ebea1043d08fb

Tree-SHA512: 8085027c488d84bb4bddccba78bd2d4c5af0d8e2644ee72265f1f30fa8c83f61a961d9da2c796f2940e69682291cbee7b1028b6a6ce123ad9134c0ebbf4723b0
2020-09-23 09:02:00 +02:00
Troy Giorshev
deb52711a1 Remove header checks out of net_processing
This moves header size and netmagic checking out of net_processing and
into net.  This check now runs in ReadHeader, so that net can exit early
out of receiving bytes from the peer.  IsValid is now slimmed down, so
it no longer needs a MessageStartChars& parameter.

Additionally this removes the rest of the m_valid_* members from
CNetMessage.
2020-09-22 22:05:18 -04:00
Troy Giorshev
52d4ae46ab Give V1TransportDeserializer CChainParams& member
This adds a CChainParams& member to V1TransportDeserializer member, and
use it in place of many Params() calls.  In addition to reducing the
number of calls to a global, this removes a parameter from GetMessage
(and will later allow us to remove one from CMessageHeader::IsValid())
2020-09-22 22:01:14 -04:00
Troy Giorshev
5bceef6b12 Change CMessageHeader Constructor
This commit removes the single-parameter contructor of CMessageHeader
and replaces it with a default constructor.

The single parameter contructor isn't used anywhere except for tests.
There is no reason to initialize a CMessageHeader with a particular
messagestart.  This messagestart should always be replaced when
deserializing an actual message header so that we can run checks on it.

The default constructor initializes it to zero, just like the command
and checksum.

This also removes a parameter of a V1TransportDeserializer constructor,
as it was only used for this purpose.
2020-09-22 22:01:14 -04:00
Troy Giorshev
1ca20c1af8 Add doxygen comment for ReceiveMsgBytes 2020-09-22 22:01:14 -04:00
Troy Giorshev
890b1d7c2b Move checksum check from net_processing to net
This removes the m_valid_checksum member from CNetMessage.  Instead,
GetMessage() returns an Optional.

Additionally, GetMessage() has been given an out parameter to be used to
hold error information.  For now it is specifically a uint32_t used to
hold the raw size of the corrupt message.

The checksum check is now done in GetMessage.
2020-09-22 22:01:14 -04:00
Troy Giorshev
2716647ebf Give V1TransportDeserializer an m_node_id member
This is intended to only be used for logging.

This will allow log messages in the following commits to keep recording
the peer's ID, even when logging is moved into V1TransportDeserializer.
2020-09-22 22:01:14 -04:00
MarcoFalke
faf0a26711
doc: Update comments for new chain settings (-signet and -chain=signet) 2020-09-22 22:32:25 +02:00
MarcoFalke
fae0548686
fuzz: Remove needless guard 2020-09-22 22:32:18 +02:00
MarcoFalke
77771a03df
refactor: Remove SignetTxs::m_valid and use optional instead
m_valid implies the block solution has been checked, which is not the
case. It only means the txs could be parsed. C++17 comes with
std::optional, so just use that instead.
2020-09-22 22:31:31 +02:00
MarcoFalke
fa14f57fbc
Assert that RPCArg names are equal to CRPCCommand ones (net, rpcwallet) 2020-09-22 20:49:30 +02:00
Gregory Sanders
e76fc2b84d Add 'sequence' zmq publisher to track all block (dis)connects, mempool deltas
Using the zmq notifications to avoid excessive mempool polling can be difficult
given the current notifications available. It announces all transactions
being added to mempool or included in blocks, but announces no evictions
and gives no indication if the transaction is in the mempool or a block.

Block notifications for zmq are also substandard, in that it only announces
block tips, while all block transactions are still announced.

This commit adds a unified stream which can be used to closely track mempool:

1) getrawmempool to fill out mempool knowledge
2) if txhash is announced, add or remove from set
based on add/remove flag
3) if blockhash is announced, get block txn list,
remove from those transactions local view of mempool
4) if we drop a sequence number, go to (1)

The mempool sequence number starts at the value 1, and
increments each time a transaction enters the mempool,
or is evicted from the mempool for any reason, including
block inclusion. The mempool sequence number is published
via ZMQ for any transaction-related notification.

These features allow for ZMQ/RPC consumer to track mempool
state in a more exacting way, without unnecesarily polling
getrawmempool. See interface_zmq.py::test_mempool_sync for
example usage.
2020-09-22 11:34:30 -04:00
MarcoFalke
d692d192cd
Merge #19849: Assert that RPCArg names are equal to CRPCCommand ones (blockchain,rawtransaction)
fa6bb0ce5dba33970e2c1e47ea4d0d2c0718eccb Assert that RPCArg names are equal to CRPCCommand ones (rawtransaction) (MarcoFalke)
fa80c814874a2893e4323ba5148fba21d7f421cd Assert that RPCArg names are equal to CRPCCommand ones (blockchain) (MarcoFalke)

Pull request description:

  This is split out from #18531 to just touch some RPC methods. Description from the main pr:

  ### Motivation

  RPCArg names in the rpc help are currently only used for documentation. However, in the future they could be used to teach the server the named arguments. Named arguments are currently registered by the `CRPCCommand`s and duplicate the RPCArg names from the documentation. This redundancy is fragile, and has lead to errors in the past (despite having linters to catch those kind of errors). See section "bugs found" for a list of bugs that have been found as a result of the changes here.

  ### Changes

  The changes here add an assert in the `CRPCCommand` constructor that the RPCArg names are identical to the ones in the `CRPCCommand`.

  ### Future work

  > Here or follow up, makes sense to also assert type of returned UniValue?

  Sure, but let's not get ahead of ourselves. I am going to submit any further works as follow-ups, including:

  * Removing the CRPCCommand arguments, now that they are asserted to be equal and thus redundant
  * Removing all python regex linters on the args, now that RPCMan can be used to generate any output, including the cli.cpp table
  * Auto-formatting and sanity checking the RPCExamples with RPCMan
  * Checking passed-in json in self-check. Removing redundant checks
  * Checking returned json against documentation to avoid regressions or false documentation
  * Compile the RPC documentation at compile-time to ensure it doesn't change at runtime and is completely static

  ### Bugs found

  * The assert identified issue #18607
  * The changes itself fixed bug #19250

ACKs for top commit:
  fjahr:
    utACK fa6bb0ce5dba33970e2c1e47ea4d0d2c0718eccb
  tryphe:
    utACK fa6bb0ce5dba33970e2c1e47ea4d0d2c0718eccb. Reducing data duplication is nice. Code changes are minimal and concise.

Tree-SHA512: deb0edc3f999baf055526eaa199b98c500635e12502dece7aa3cad5319db330eb5ee7459a5c8f040a83671a7f20c560c19a2026fb76c8416f138aa332727cbce
2020-09-22 17:08:08 +02:00
Amiti Uttarwar
50f94b34a3 [rpc] Deprecate getpeerinfo addnode field
This field is now redundant since the connection type field will indicate
MANUAL for addnode connections.
2020-09-21 19:01:29 -07:00
Amiti Uttarwar
395acfa83a [rpc] Add connection type to getpeerinfo RPC, update tests 2020-09-21 19:01:29 -07:00
Amiti Uttarwar
49c10a9ca4 [log] Add connection type to log statement
In addition to adding more specificity to the log statement about the type of
connection, this change also consolidates two statements into one. Previously,
the second one should have never been hit, since block-relay connections would
match the "!IsInboundConn()" condition and return early.
2020-09-21 19:01:29 -07:00
Wladimir J. van der Laan
77376034d4
Merge #17785: p2p: Unify Send and Receive protocol versions
ddefb5c0b759950942ac03f28c43b548af7b4033 p2p: Use the greatest common version in peer logic (Hennadii Stepanov)
e084d45562b94827b3a7873895882fcaae9f4d48 p2p: Remove SetCommonVersion() from VERACK handler (Hennadii Stepanov)
8d2026796a6f7add0c2cda9806e759817d1eae6f refactor: Rename local variable nSendVersion (Hennadii Stepanov)
e9a6d8b13b0558b17cdafbd32fd2663b4138ff11 p2p: Unify Send and Receive protocol versions (Hennadii Stepanov)

Pull request description:

  On master (6fef85bfa3cd7f76e83b8b57f9e4acd63eb664ec) `CNode` has two members to keep protocol version:
  - `nRecvVersion` for received messages
  - `nSendVersion` for messages to send

  After exchanging with `VERSION` and `VERACK` messages via protocol version `INIT_PROTO_VERSION`, both nodes set `nRecvVersion` _and_ `nSendVersion` to _the same_ value which is the greatest common protocol version.

  This PR:
  - replaces two `CNode` members, `nRecvVersion` `nSendVersion`, with `m_greatest_common_version`
  - removes duplicated getter and setter

  There is no change in behavior on the P2P network.

ACKs for top commit:
  jnewbery:
    ACK ddefb5c0b759950942ac03f28c43b548af7b4033
  naumenkogs:
    ACK ddefb5c0b759950942ac03f28c43b548af7b4033
  fjahr:
    Code review ACK ddefb5c0b759950942ac03f28c43b548af7b4033
  amitiuttarwar:
    code review but untested ACK ddefb5c0b7
  benthecarman:
    utACK `ddefb5c`

Tree-SHA512: 5305538dbaa5426b923b0afd20bdef4f248d310855d1d78427210c00716c67b7cb691515c421716b6157913e453076e293b10ff5fd2cd26a8e5375d42da7809d
2020-09-22 00:14:32 +02:00
Wladimir J. van der Laan
8c5f68118c
Merge #18267: BIP-325: Signet [consensus]
8258c4c0076bb5f27efdc117a04b27fcd6dd00b2 test: some sanity checks for consensus logic (Anthony Towns)
e47ad375bf17557f805bd206e789b8db78c6338a test: basic signet tests (Karl-Johan Alm)
4c189abdc452f08dfa758564b5381bc78c42d481 test: add small signet fuzzer (practicalswift)
ec9b25d046793be50da1c11ba61d1b4b13b295b0 test: signet network selection tests (Karl-Johan Alm)
3efe298dccb248f25d6b01ab6a80b1cd6c9e1a1e signet: hard-coded parameters for Signet Global Network VI (2020-09-07) (Karl-Johan Alm)
c7898bca4e1ccbc6edafd3b72eaf80df38e3af32 qt: update QT to support signet network (Karl-Johan Alm)
a8de47a1c9033fac3355590f1fe2158a95011bb3 consensus: add signet validation (Karl-Johan Alm)
e8990f121405af8cd539b904ef082439261e6c93 add signet chain and accompanying parameters (Karl-Johan Alm)
404682b7cdb54494e7c98f0ba0cac8b51f379750 add signet basic support (signet.cpp) (Karl-Johan Alm)
a2147d7dadec1febcd9c2b8ebbbf78dce6d0556b validation: move GetWitnessCommitmentIndex to consensus/validation (Karl-Johan Alm)

Pull request description:

  This PR is a part of BIP-325 (https://github.com/bitcoin/bips/blob/master/bip-0325.mediawiki), and is a sub-PR of #16411.

  * Signet consensus (this)
  * Signet RPC tools (pending)
  * Signet utility scripts (contrib/signet) (pending)

ACKs for top commit:
  jonatack:
    re-ACK 8258c4c0076bb5f27efdc117a04b27fcd6dd00b per `git diff dbeea65 8258c4c`, only change since last review is updated `-signet*` config option naming.
  fjahr:
    re-ACK 8258c4c
  laanwj:
    ACK 8258c4c0076bb5f27efdc117a04b27fcd6dd00b2
  MarcoFalke:
    Approach ACK 8258c4c007 🌵

Tree-SHA512: 5d158add96755910837feafa8214e13695b769a6aec3a2da753cf672618bef377fac43b0f4b772a87b25dd9f0c1c9b29f2789785d7a7d47a155cdcf48f7c975d
2020-09-21 22:33:00 +02:00
Wladimir J. van der Laan
c0c409dcd3
Merge #19697: Improvements on ADDR caching
0d04784af151de249bbbcbad51e6e8ad9af8f5a3 Refactor the functional test (Gleb Naumenko)
83ad65f31b5c9441ae1618614082e584854a14e1 Address nits in ADDR caching (Gleb Naumenko)
81b00f87800f40cb14f2131ff27668bd2bb9e551 Add indexing ADDR cache by local socket addr (Gleb Naumenko)
42ec5585424ceb91bed07826dde15697c020661a Justify the choice of ADDR cache lifetime (Gleb Naumenko)

Pull request description:

  This is a follow-up on #18991 which does 3 things:
  - improves privacy of a node listening to multiple addresses via adding cache index by local socket addr (suggested [here](https://github.com/bitcoin/bitcoin/pull/18991#issuecomment-668219345))
  - documents on the choice of 24h cache lifetime
  - addresses nits from #18991

ACKs for top commit:
  jnewbery:
    utACK 0d04784af151de249bbbcbad51e6e8ad9af8f5a3
  vasild:
    ACK 0d04784
  jonatack:
    Code review ACK 0d04784

Tree-SHA512: bb65a34dd1ce2811186d3e4469bc33e8399cebaaa494ce13041c7cff23275870e4176a719f7a72f8d779c49f8b2344bf4fa1aeb3ea4e2626d5ae76514f00a750
2020-09-21 19:36:57 +02:00
Carl Dong
72a1d5c6f3
validation: Remove review-only comments + assertions
[META] This is a followup to "validation: Move FindFilesToPrune{,Manual}
       to BlockManager" removing comments and assertions meant only to
       show that the change is correct.
2020-09-21 13:30:27 -04:00
Carl Dong
3756853b15
docs: Move FindFilesToPrune{,Manual} doxygen comment
[META] This is a pure comment commit.

They belong in the member declarations in the header file.
2020-09-21 13:30:21 -04:00
Carl Dong
485899a93c
style: Make FindFilesToPrune{,Manual} match style guide
[META] This is a pure style commit.
2020-09-21 13:28:08 -04:00
Carl Dong
3f5b5f3f6d
validation: Move FindFilesToPrune{,Manual} to BlockManager
[META] No behaviour change is intended in this commit.
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.

Also stop FindFilesToPrune{,Manual} from unnecessary reaching for
::ChainActive() by passing in the necessary information.
2020-09-21 13:27:44 -04:00
Vasil Dimov
7be6ff6187
net: recognize TORv3/I2P/CJDNS networks
Recognizing addresses from those networks allows us to accept and gossip
them, even though we don't know how to connect to them (yet).

Co-authored-by: eriknylund <erik@daychanged.com>
2020-09-21 10:13:34 +02:00
Hennadii Stepanov
0bd1184adf
Remove unused LockAssertion struct 2020-09-19 18:02:42 +03:00
Hennadii Stepanov
ab2a44297f
Replace LockAssertion with a proper thread safety annotations 2020-09-19 18:02:02 +03:00
Hennadii Stepanov
73f71e1996
refactor: Use explicit function type instead of template 2020-09-19 17:50:58 +03:00
fanquake
831b0ecea9
Merge #13686: ZMQ: Small cleanups in the ZMQ code
6fe2ef2acb00b1df7f6a0c0dea1a81a1924be0e1 scripted-diff: Rename SendMessage to SendZmqMessage. (Daniel Kraft)
a3ffb6ebebd753cec294c91cef7c603a30cf217e Replace zmqconfig.h by a simple zmqutil. (Daniel Kraft)
7f2ad1b9acef4ccc1b3e1a9f551416235d95cbfd Use std::unique_ptr for CZMQNotifierFactory. (Daniel Kraft)
b93b9d54569145bfcec6cee10968284fe05fe254 Simplify and fix notifier removal on error. (Daniel Kraft)
e15b1cfc310df739b92bd281112dbeb31d3bb30a Various cleanups in zmqnotificationinterface. (Daniel Kraft)

Pull request description:

  This contains various small code cleanups that make the ZMQ code easier to read and maintain (at least in my opinion).  The only functional change is that a potential memory leak is fixed that would have occured when a notifier is removed from the `notifiers` list after its callback function returned `false` (which is likely not relevant in practice but still a bug).

ACKs for top commit:
  instagibbs:
    utACK 6fe2ef2acb00b1df7f6a0c0dea1a81a1924be0e1
  hebasto:
    re-ACK 6fe2ef2acb00b1df7f6a0c0dea1a81a1924be0e1, only the latest commit got a scripted-diff since my [previous](https://github.com/bitcoin/bitcoin/pull/13686#pullrequestreview-487649808) review.

Tree-SHA512: 8206f8713bf3698d7cd4cb235f6657dc1c4dd920f50a8c5f371a559dd17ce5ab6d94d6281165eef860a22fc844a6bb25489ada12c83ebc780efd7ccdc0860f70
2020-09-19 17:13:28 +08:00
fanquake
83b23848f7
Merge #18790: gui: Improve thread naming
ead771bf6fc7a4b96a03d4938796c88657c69ba6 qt: Rename qt-init thread before logging start (Hennadii Stepanov)
ad5f614bf326d739424e8b403066f2d4275e4c1b qt: Name ClientModel timer QThread (Hennadii Stepanov)
2c7f5d8c2e6dae099a73fe748f6194da3c961a48 qt: Name WalletController worker QThread (Hennadii Stepanov)
27dcc37d429626c75c540331340c62723529f37e qt: Name RPCConsole executor QThread (Hennadii Stepanov)

Pull request description:

  On **master** (eef90c14ed0f559e3f6e187341009270b84f45cb):
  - thread list from OS:
  ![Screenshot from 2020-04-28 00-25-41](https://user-images.githubusercontent.com/32963518/80425413-3de07100-88ec-11ea-8d7a-79bd9e152395.png)
  - log excerpt:
  ```
  2020-04-27T21:25:26Z [] GUI: initialize : Running initialization in thread
  ...
  2020-04-27T21:26:04Z [] Using BerkeleyDB version Berkeley DB 4.8.30: (April  9, 2010)
  2020-04-27T21:26:04Z [] Using wallet /home/hebasto/.bitcoin/testnet3/wallets/test2
  2020-04-27T21:26:04Z [] BerkeleyEnvironment::Open: LogDir=/home/hebasto/.bitcoin/testnet3/wallets/test2/database ErrorFile=/home/hebasto/.bitcoin/testnet3/wallets/test2/db.log
  2020-04-27T21:26:04Z [] init message: Loading wallet...
  2020-04-27T21:26:04Z [] BerkeleyEnvironment::Open: LogDir=/home/hebasto/.bitcoin/testnet3/wallets/test2/database ErrorFile=/home/hebasto/.bitcoin/testnet3/wallets/test2/db.log
  2020-04-27T21:26:04Z [] [test2] Wallet File Version = 169900
  2020-04-27T21:26:04Z [] [test2] Keys: 2001 plaintext, 0 encrypted, 2001 w/ metadata, 2001 total. Unknown wallet records: 0
  2020-04-27T21:26:04Z [] [test2] Wallet completed loading in              26ms
  2020-04-27T21:26:04Z [] GUI: TransactionTablePriv::refreshWallet
  2020-04-27T21:26:04Z [] [test2] setKeyPool.size() = 2000
  2020-04-27T21:26:04Z [] [test2] mapWallet.size() = 0
  2020-04-27T21:26:04Z [] [test2] m_address_book.size() = 0
  ```

  With **this PR**:
  - thread list from OS:
  ![Screenshot from 2020-04-28 00-21-40](https://user-images.githubusercontent.com/32963518/80425527-7a13d180-88ec-11ea-8a34-dfc774bb1c75.png)
  - log excerpt:
  ```
  2020-04-27T21:21:25Z [qt-init] GUI: initialize : Running initialization in thread
  ...
  2020-04-27T21:23:08Z [qt-walletctrl] Using BerkeleyDB version Berkeley DB 4.8.30: (April  9, 2010)
  2020-04-27T21:23:08Z [qt-walletctrl] Using wallet /home/hebasto/.bitcoin/testnet3/wallets/test2
  2020-04-27T21:23:08Z [qt-walletctrl] BerkeleyEnvironment::Open: LogDir=/home/hebasto/.bitcoin/testnet3/wallets/test2/database ErrorFile=/home/hebasto/.bitcoin/testnet3/wallets/test2/db.log
  2020-04-27T21:23:08Z [qt-walletctrl] init message: Loading wallet...
  2020-04-27T21:23:08Z [qt-walletctrl] BerkeleyEnvironment::Open: LogDir=/home/hebasto/.bitcoin/testnet3/wallets/test2/database ErrorFile=/home/hebasto/.bitcoin/testnet3/wallets/test2/db.log
  2020-04-27T21:23:08Z [qt-walletctrl] [test2] Wallet File Version = 169900
  2020-04-27T21:23:08Z [qt-walletctrl] [test2] Keys: 2001 plaintext, 0 encrypted, 2001 w/ metadata, 2001 total. Unknown wallet records: 0
  2020-04-27T21:23:08Z [qt-walletctrl] [test2] Wallet completed loading in              37ms
  2020-04-27T21:23:08Z [qt-walletctrl] init message: Rescanning...
  2020-04-27T21:23:08Z [qt-walletctrl] [test2] Rescanning last 112924 blocks (from block 1609206)...
  2020-04-27T21:23:08Z [qt-walletctrl] [test2] Rescan started from block 000000000000003761c81f7efbd8cebf217f39d353ec1ac59c624ac2dddfc2a8...
  2020-04-27T21:23:22Z [qt-walletctrl] [test2] Rescan completed in           14157ms
  2020-04-27T21:23:22Z [qt-walletctrl] GUI: TransactionTablePriv::refreshWallet
  2020-04-27T21:23:22Z [qt-walletctrl] [test2] setKeyPool.size() = 2000
  2020-04-27T21:23:22Z [qt-walletctrl] [test2] mapWallet.size() = 0
  2020-04-27T21:23:22Z [qt-walletctrl] [test2] m_address_book.size() = 0
  ```

ACKs for top commit:
  Sjors:
    tACK ead771bf6fc7a4b96a03d4938796c88657c69ba6

Tree-SHA512: a3b2789990414ab23b69236ca36b656a3f026e11e88fb5940ef4fecfc2053df5ed886615afb37f98584f6e19b953209d3884baab057740b2e9eed68661880dd3
2020-09-19 16:28:04 +08:00
fanquake
c30f79d418
Merge #19940: rpc: Return fee and vsize from testmempoolaccept
23c35bf0059bd6270218e0b732959e9c754f9812 [test] add get_vsize util for more programmatic testing (gzhao408)
2233a93a109b10b6fe0f5f26c2bb529c8de3dde7 [rpc] Return fee and vsize from testmempoolaccept (codeShark149)

Pull request description:

  From #19093 and resolves #19057.

  Difference from #19093: return `vsize` and `fees` object (similar to `getmempoolentry`) when the test accept is successful. Updates release-notes.md.

ACKs for top commit:
  jnewbery:
    utACK 23c35bf0059bd6270218e0b732959e9c754f9812
  fjahr:
    utACK 23c35bf
  instagibbs:
    reACK 23c35bf005

Tree-SHA512: dcb81b7b817a4684e9076bc5d427a6f2d549d2edc66544e718260c4b5f8f1d5ae1d47b754175e9f0c8a3bd8371ce116c2dca0583588d513a7d733d5d614f2b04
2020-09-19 15:04:03 +08:00
Anthony Towns
8258c4c007
test: some sanity checks for consensus logic 2020-09-18 10:19:43 +09:00
practicalswift
4c189abdc4
test: add small signet fuzzer 2020-09-18 10:19:42 +09:00