26969 Commits

Author SHA1 Message Date
Pieter Wuille
37fe80e626 Only consider addrv2 peers for relay of non-addrv1 addresses 2020-12-15 12:45:41 -08:00
Pieter Wuille
83f8821a6f refactor: add IsAddrCompatible() to CNode 2020-12-15 12:45:32 -08:00
Wladimir J. van der Laan
d9a4738c9d
Merge #20660: Move signet onion seed from v2 to v3
3e6657a14d501c6315ab46ffe7d204684491c710 Move signet onion seed from v2 to v3 (Sjors Provoost)

Pull request description:

  Since v0.21 hidden services use the longer v3 address format.

  It may make sense to backport this to the v0.21 branch, although onion nodes can always use the non-onion seeds.

ACKs for top commit:
  MarcoFalke:
    Concept ACK 3e6657a14d501c6315ab46ffe7d204684491c710
  laanwj:
    ACK 3e6657a14d501c6315ab46ffe7d204684491c710
  Saibato:
    tACK 3e6657a14d  👍

Tree-SHA512: 9bb4d82345ab25d6ea971f8f106c30778ade2ba11a292e8d7449ea39581b91e51c4b34851b5c06f5dfbe07e7b04e5dc92c48c98c47a1c7ef3d5f350c3a3ad4f7
2020-12-15 21:38:33 +01:00
Dhruv Mehta
739d39022d ci: Move linter task to cirrus 2020-12-15 10:13:55 -08:00
MarcoFalke
8bb40d5f56
Merge #20560: fuzz: Link all targets once
fa13e1b0c52738492310b6b421d8e38cb04da5b1 build: Add option --enable-danger-fuzz-link-all (MarcoFalke)
44444ba759480237172d83f42374c5c29c76eda0 fuzz: Link all targets once (MarcoFalke)

Pull request description:

  Currently the linker is invoked more than 150 times when compiling with `--enable-fuzz`. This is problematic for several reasons:

  * It wastes disk space north of 20 GB, as all libraries and sanitizers are linked more than 150 times
  * It wastes CPU time, as the link step can practically not be cached (similar to ccache for object files)
  * It makes it a blocker to compile the fuzz tests by default for non-fuzz builds #19388, for the aforementioned reasons
  * The build file is several thousand lines of code, without doing anything meaningful except listing each fuzz target in a highly verbose manner
  * It makes writing new fuzz tests unnecessarily hard, as build system knowledge is required; Compare that to boost unit tests, which can be added by simply editing an existing cpp file
  * It encourages fuzz tests that re-use the `buffer` or assume the `buffer` to be concatenations of seeds, which increases complexity of seeds and complexity for the fuzz engine to explore; Thus reducing the effectiveness of the affected fuzz targets

  Fixes #20088

ACKs for top commit:
  practicalswift:
    Tested ACK fa13e1b0c52738492310b6b421d8e38cb04da5b1
  sipa:
    ACK fa13e1b0c52738492310b6b421d8e38cb04da5b1. Reviewed the code changes, and tested the 3 different test_runner.py modes (run once, merge, generate). I also tested building with the new --enable-danger-fuzz-link-all

Tree-SHA512: 962ab33269ebd51810924c51266ecc62edd6ddf2fcd9a8c359ed906766f58c3f73c223f8d3cc49f2c60f0053f65e8bdd86ce9c19e673f8c2b3cd676e913f2642
2020-12-15 19:00:36 +01:00
MarcoFalke
a35a3466ef
Merge #20653: doc: Move addr relay comment in net to correct place
fa86217e97234aac6f815a6768afc1b87b8b2ae8 doc: Move add relay comment in net to correct place (MarcoFalke)

Pull request description:

  The comment was previously attached to `m_addr_known`, but now it is attached to `id`, which is wrong.

  Fix that by moving the comment to `RelayAddrsWithConn`.

ACKs for top commit:
  practicalswift:
    cr ACK fa86217e97234aac6f815a6768afc1b87b8b2ae8: patch looks correct
  jnewbery:
    ACK fa86217e97
  theStack:
    Code review ACK fa86217e97234aac6f815a6768afc1b87b8b2ae8 🌳

Tree-SHA512: ec3d5f1996aded38947d2a5fd0bb63539e88f83964cd3254984002edfd51abb4dde813c7c81619a8a3a5c55b7e9ae83c8c5be8ad6c84b4593ed3bbf463fe8979
2020-12-15 17:56:11 +01:00
Wladimir J. van der Laan
ec6149c01e
Merge #20616: Check CJDNS address is valid
f7264fff0a098f8b6354c7373b8790791c25dd07 Check if Cjdns address is valid (Lucas Ontivero)

Pull request description:

  CJDNS addresses start with 0xFC and for that reason if a netaddr was unserialized with network type cjdns but its address prefix is not 0xFC then that netaddr should be considered invalid.

ACKs for top commit:
  jonatack:
    ACK f7264fff0a098f8b6354c7373b8790791c25dd07
  practicalswift:
    cr ACK f7264fff0a098f8b6354c7373b8790791c25dd07: patch looks correct
  theStack:
    ACK f7264fff0a098f8b6354c7373b8790791c25dd07 ✔️

Tree-SHA512: 5300df2ffbbd69c40271b6d8df96cca98eb3e1ee76aba62c9c76025d083788ab1f1332775890c63b06e02ca593863a867cd53956bce5962383e8450487898669
2020-12-15 17:51:41 +01:00
Wladimir J. van der Laan
cd03513dc2 init: Signal-safe instant shutdown
Replace the 200ms polling loop with a faster and more efficient waiting
operation.

This was tried a few times before, but given up every time because
solutions use a condition variable which is not safe for use in signals
as they need to be reentrant.

On UNIX-ish OSes, use a safe way: a pipe. When shutdown is requested
write a dummy byte to the pipe. Waiting for shutdown is a matter of a
blocking read from the pipe.

On Windows, there are no signals so using a condition variable is safe.
2020-12-15 17:21:06 +01:00
MarcoFalke
70150824dc
Merge #20437: fuzz: Avoid time-based "non-determinism" in fuzzing harnesses by using mocked GetTime()
8c09c0c1d18885ef94f79b3f2d073f43269bc95d fuzz: Avoid time-based "non-determinism" in fuzzing harnesses by using mocked GetTime() (practicalswift)

Pull request description:

  Avoid time-based "non-determinism" in fuzzing harnesses by using mocked `GetTime()`.

  Prior to this commit the fuzzing harnesses `banman`, `connman`, `net` and `rbf` had time-based "non-determinism". `addrman` is fixed in #20425. `process_message` and `process_messages` are left to fix: simply using mock time is not enough for them due to interaction with `IsInitialBlockDownload()`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  MarcoFalke:
    review ACK 8c09c0c1d18885ef94f79b3f2d073f43269bc95d
  practicalswift:
    > review ACK [8c09c0c](8c09c0c1d1)

Tree-SHA512: 32dfbead3dfd18cf4ff56dc2ea341aa977441b4e19a54879cf54fa5820c7e2b14b92c7e238d32fd785654f3b28cc82826ae66c03e94c292633c63c41196ba9a8
2020-12-15 17:11:59 +01:00
Sjors Provoost
3e6657a14d
Move signet onion seed from v2 to v3 2020-12-15 14:12:01 +01:00
fanquake
f1f2418433
Merge #20253: net: use std::chrono throughout maxOutbound logic
0475c8ba4d10dce79092361bc4c23c11dadba39a net: use std::chrono throughout maxOutbound logic (fanquake)
f805933e70d4320e62739f4cecfc08bdbad8afaa init: set nMaxOutboundLimit connection option directly (fanquake)
173d0d35f1bb271175047c9eb7dd58cc46ed35bf net: remove nMaxOutboundTimeframe from connection options (fanquake)
b117eb148678b0fc5be02346cef29d87d4f81af9 net: remove SetMaxOutboundTimeframe (fanquake)
2f3f1aec1f8aadd4a6fb08ca5da7eeda31eb388f net: remove SetMaxOutboundTarget (fanquake)

Pull request description:

  Switch to using `std::chrono` types for the max outbound related logic.
  Removes some unnecessary code from init.

ACKs for top commit:
  jnewbery:
    utACK 0475c8ba4d10dce79092361bc4c23c11dadba39a
  MarcoFalke:
    review ACK 0475c8ba4d10dce79092361bc4c23c11dadba39a 🎭

Tree-SHA512: 5a6d5b61e0d4c08a235cfc0257dae65d09a5df019d8d230b1a58a3e2483ddf4a31efdefc885c4a02e4715e4180b0ed92ebc0a1c08b2bf476a391945114593514
2020-12-15 20:48:21 +08:00
Jonas Schnelli
33d6337269
Merge bitcoin-core/gui#115: Replace "Hide tray icon" option with positive "Show tray icon" one
03edb52eee5a87af16161c23bdc6cde91a2e5b8b qt: Remove redundant BitcoinGUI::setTrayIconVisible (Hennadii Stepanov)
17174f8328c44ae84479e8365c122ad8502bd7da gui: Replace "Hide tray icon" option with positive "Show tray icon" one (Hennadii Stepanov)

Pull request description:

  This change makes easier both (1) using this option, and (2) reasoning about the code.

ACKs for top commit:
  jonasschnelli:
    utACK 03edb52eee5a87af16161c23bdc6cde91a2e5b8b

Tree-SHA512: 38e317492210d4fb13302dea383bd1f4f0ae1219d7ff2fdcb78607f15ac61a51969acaadb59b72c3f075b6356ef54368eb46fb49e6e1bd42db6d5804b97e232b
2020-12-15 09:28:50 +01:00
Wladimir J. van der Laan
94a9cd25fd
Merge #20594: Fix getauxval calls in randomenv.cpp
836a3dc02c72f917db5be386b9b4787a59d48610 Avoid weak-linked getauxval support on non-linux platforms (like macOS) (Jonas Schnelli)
41a413b31746cc749f3c64ed8070cea9cc6cfdbe Define correct symbols for getauxval (Jonas Schnelli)

Pull request description:

  PR #20358 made use of the two preprocessor symbols `HAVE_STRONG_GETAUXVAL` as well as `HAVE_WEAK_GETAUXVAL`.

  These symbols have not been defined in configure.ac. They where only passed selective as CRC32 CPPFLAGS in https://github.com/bitcoin/bitcoin/blob/master/src/Makefile.crc32c.include#L16.

  PR #20358 would have broken the macOS build since `getauxval` is not supported on macOS (but weak-linking does pass).

  This PR defines the two symbols correctly and reduces calls to `getauxval` to linux.

ACKs for top commit:
  laanwj:
    Code review ACK 836a3dc02c72f917db5be386b9b4787a59d48610
  jonatack:
    utACK 836a3dc02c72f917db5be386b9b4787a59d48610

Tree-SHA512: 6527f4a617b937f4c368a3cb1c162f1ac38a6f5e6341295554961eaf322906e9b27398a6f7b00819854ceebb5c828d3e6ce0a779edd769adc4053ce8beda3739
2020-12-14 21:03:40 +01:00
Lucas Ontivero
f7264fff0a Check if Cjdns address is valid 2020-12-14 14:48:52 -03:00
MarcoFalke
fa86217e97
doc: Move add relay comment in net to correct place
Can be reviewed with
--ignore-all-space --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2020-12-14 18:15:51 +01:00
MarcoFalke
fff7d056a9
Merge #20599: net processing: Tolerate sendheaders and sendcmpct messages before verack
b316dcb758e3dbd302fbb5941a1b5b0ef5f1f207 [net processing] Tolerate sendheaders and sendcmpct messages before verack (John Newbery)

Pull request description:

  BIP 130 (sendheaders) and BIP 152 (compact blocks) do not specify at
  which stage the `sendheaders` or `sendcmpct` messages should be sent.
  Therefore we should tolerate them being sent before the version-verack
  handshake is complete.

ACKs for top commit:
  MarcoFalke:
    review ACK b316dcb758e3dbd302fbb5941a1b5b0ef5f1f207 📒
  jonatack:
    Code review re-ACK b316dcb758e3dbd302fbb5941a1b5b0ef5f1f207 per `git range-diff b103fdcb 82d0a43 b316dcb`, rebase only
  luke-jr:
    utACK b316dcb758e3dbd302fbb5941a1b5b0ef5f1f207, only code movement from after verack check to before

Tree-SHA512: a8da3990a786a53c195a33cd278eb9d1b6b09e9a33717674d5bc6ef5629811189216f3eda1a3dd11f6972b9680c7b7a532bf7343b6e7440009dd831bef128e1d
2020-12-14 17:10:55 +01:00
MarcoFalke
fa13e1b0c5
build: Add option --enable-danger-fuzz-link-all 2020-12-14 16:55:56 +01:00
John Newbery
b316dcb758 [net processing] Tolerate sendheaders and sendcmpct messages before verack
BIP 130 (sendheaders) and BIP 152 (compact blocks) do not specify at
which stage the `sendheaders` or `sendcmpct` messages should be sent.
Therefore we should tolerate them being sent before the version-verack
handshake is complete.
2020-12-14 13:23:43 +00:00
John Newbery
ea36a453e3 [net] Make p2p recv buffer timeout 20 minutes for all peers
The timeout interval for the send and recv buffers was changed from 90
minutes to 20 minutes in commit f1920e86 in 2013, except for peers that
did not support the pong message (where the recv buffer timeout remained
at 90 minutes). A few observations:

- for peers that support BIP 31 (pong messages), this recv buffer
  timeout is almost redundant with the ping timeout. We send a ping
  message every two minutes, and set a timeout of twenty minutes to
  receive the pong response. If the recv buffer was really timing out,
  then the pong response would also time out.
- BIP 31 is supported by all nodes of p2p version 60000 and higher, and
  has been in widespread use since 2013. I'd be very surprised if there
  are many nodes on the network that don't support pong messages.
- The recv buffer timeout is not specified in any p2p BIP. We're free to
  set it at any value we want.
- A peer that doesn't support BIP 31 and hasn't sent any message to us
  at all in 90 minutes is unlikely to be useful for us, and is more likely
  to be evicted AttemptToEvictConnection() since it'll have the worst
  possible ping time and isn't providing blocks/transactions.

Therefore, we remove this check, and sent the recv buffer timeout to 20
minutes for all peers. This removes the final p2p version dependent
logic from the net layer, so all p2p version data can move into the
net_processing layer.

Alternative approaches:

- Set the recv buffer timeout to 90 minutes for all peers. This almost
  wouldn't be a behaviour change at all (pre-BIP 31 peers would still
  have the same recv buffer timeout, and we can't ever reach a recv buffer
  timeout higher than 21 minutes for post-BIP31 peers, because the pong
  timeout would be hit first).
- Stop supporting peers that don't support BIP 31. BIP 31 has been in
  use since 2012, and implementing it is trivial.
2020-12-14 12:59:30 +00:00
MarcoFalke
b103fdcb3b
Merge #19763: net: don't try to relay to the address' originator
7fabe0f359ae16ed36ce4ca2c33631d038c21448 net: don't relay to the address' originator (Vasil Dimov)

Pull request description:

  For each address to be relayed we "randomly" pick 2 nodes to send the
  address to (in `RelayAddress()`). However we do not take into
  consideration that it does not make sense to relay the address back to
  its originator (`CNode::PushAddress()` will do nothing in that case).

  This means that if the originator is among the "randomly" picked nodes,
  then we will relay to one node less than intended.

  Fix this by skipping the originating node when choosing candidates to
  relay to.

ACKs for top commit:
  sdaftuar:
    ACK 7fabe0f359ae16ed36ce4ca2c33631d038c21448 (this time I looked at the test, and verified the test breaks in expected ways if I break the code).
  jnewbery:
    utACK 7fabe0f359ae16ed36ce4ca2c33631d038c21448 (only net_processing changes. I haven't reviewed the test changes)
  jonatack:
    re-ACK 7fabe0f359ae16ed36ce4ca2c33631d038c21448 per `git range-diff b76abae fd897f8 7fabe0f`, change since last review is rebase and more readable Doxygen documentation

Tree-SHA512: c6a9d11c7afc97ab4e8960513f6416648d4a8c0c64b713c145a7482a7b9e54946f81386a3351e3ec0011e5594ba5ccff4d10c6f656bb80680d9f0d0a63366165
2020-12-14 13:37:21 +01:00
Hennadii Stepanov
267f259c0d
depends: Drop workaround for a fixed bug in Qt build system
The bug reports:
 - https://bugreports.qt.io/browse/QTBUG-35444
 - https://bugreports.qt.io/browse/QTBUG-32519
Fixed in Qt 5.3.0 (the workaround was introduced for Qt 5.2.1).
2020-12-14 14:07:49 +02:00
MarcoFalke
eec9366f7d
Merge #20624: net processing: Remove nStartingHeight check from block relay
f6360088de8ca02f9d198da2f8cca4ea8d64c992 [net processing] Clarify UpdatedBlockTip() (John Newbery)
94d2cc35be98d3b20db88b2a3745322e5b0aa9d4 [net processing] Remove unnecesary nNewHeight variable in UpdatedBlockTip() (John Newbery)
8b57013473c44fc87c9f1c4611224a89187c289a [net processing] Remove nStartingHeight check from block relay (John Newbery)

Pull request description:

  nStartingHeight was introduced in commit 7a47324c7 (Bitcoin version
  0.2.9, P2P version 209) with the comment "better prevention of inventory
  relaying during initial download". At that time, there was no function
  to determine whether the node was still in Initial Block Download, so to
  prevent syncing nodes from relaying old blocks to their peers, a check
  was added to never relay a block to a peer where the height was lower
  than 2000 less than the peer's best block. That check was updated
  several times in later commits to ensure that we weren't relaying blocks
  before the latest checkpoint if the peer didn't provide a
  startingheight. The checkpoint comparison was changed to compare with an
  estimate of the highest block in commit eae82d8e.

  In commit 202e0194, all block relay was gated on being out of Initial
  Block Download. In commit 0278fb5f, the comparison to nBlockEstimate was
  removed since "we already checked IsIBD()".

  We can remove the check against nStartingHeight entirely. If the node is
  out of Initial Block Download, then its tip height must have been within
  24 hours of current time, so should not be more than ~144 blocks behind
  the most work tip.

  This simplifies moving block inventory state into the `Peer` object (#19829).

ACKs for top commit:
  Sjors:
    utACK f636008
  jonatack:
    ACK f6360088de8ca02f9d198da2f8cca4ea8d64c992
  MarcoFalke:
    ACK f6360088de8ca02f9d198da2f8cca4ea8d64c992 💽
  ariard:
    Code Review ACK f636008

Tree-SHA512: 4959cf35f1dcde46f34bffec1375729a157e1b2a1fd8a8ca33da9771c3c89a6c43e7050cdeeab8d90bb507b0795703db8c8bc304a1a5065ef00aae7a6992ca4f
2020-12-14 11:01:01 +01:00
MarcoFalke
096bd374da
Merge #20592: doc: update wtxidrelay documentation per BIP339
4b7b58b3fef5b9566a475871c441ed6ae73af89e Update net_processing WTXID documentation per BIP339 (Jon Atack)

Pull request description:

  BIP339 currently states:

  *The wtxidrelay message MUST be sent in response to a version message from a peer whose protocol version is >= 70016 and prior to sending a verack. A wtxidrelay message received after a verack message MUST be ignored or treated as invalid.*

ACKs for top commit:
  MarcoFalke:
    ACK 4b7b58b3fef5b9566a475871c441ed6ae73af89e
  practicalswift:
    ACK 4b7b58b3fef5b9566a475871c441ed6ae73af89e
  RiccardoMasutti:
    ACK 4b7b58b

Tree-SHA512: 58ca6b197618cc73c70aa5de0a2d9d89a68b4cad9d5a708278ef17a9d6854d4362bcc384b6d29696642924977204a8fc120b31e91e2d97b6072b7b0d41c9f2dc
2020-12-14 10:52:28 +01:00
fanquake
25bc840e03
Merge #20617: p2p: Remove m_is_manual_connection from CNodeState
a33442fdc73eabd1c5596ab92954344edc9517e6 Remove m_is_manual_connection from CNodeState (Antoine Riard)

Pull request description:

  Currently, this member is only used to exclude MANUAL peers from discouragement
  in MaybePunishNodeForBlock(). Manual connections are already protected in
  MaybeDiscourageAndDisconnect(), independently from their network
  processing behaviors.

ACKs for top commit:
  MarcoFalke:
    cr ACK a33442fdc73eabd1c5596ab92954344edc9517e6
  promag:
    Code review ACK a33442fdc73eabd1c5596ab92954344edc9517e6.
  jnewbery:
    utACK a33442fdc73eabd1c5596ab92954344edc9517e6
  amitiuttarwar:
    code review ACK a33442fdc73eabd1c5596ab92954344edc9517e6

Tree-SHA512: cfe3f3dfa131373e3299002d34ae9e22ca6e1a966831bab32fcf06ff1d08f06095b4ab020cc4d267f3ec05ae23fbdc22373382ab828b999c0db11b8c842a4f0c
2020-12-14 10:37:19 +08:00
Pieter Wuille
a4118c6e20 Add patch to make codesign_allocate compatible with Apple's 2020-12-13 12:20:39 -08:00
fanquake
0475c8ba4d
net: use std::chrono throughout maxOutbound logic 2020-12-13 11:41:33 +08:00
fanquake
f805933e70
init: set nMaxOutboundLimit connection option directly
DEFAULT_MAX_UPLOAD_TARGET is a compile time constant.
2020-12-13 11:12:05 +08:00
fanquake
173d0d35f1
net: remove nMaxOutboundTimeframe from connection options
It's not actually possible to change this value, so remove the
indirection of it being a conn option.

DEFAULT_MAX_UPLOAD_TIMEFRAME is a compile time constant.
2020-12-13 11:10:40 +08:00
fanquake
b117eb1486
net: remove SetMaxOutboundTimeframe
This was introduced in 872fee3fccc8b33b9af0a401b5f85ac5504b57eb and it's unclear
if it's ever been used.
2020-12-13 10:38:24 +08:00
fanquake
2f3f1aec1f
net: remove SetMaxOutboundTarget
This has been unused since f3552da81393a8e78ce3e2afed0b9c9d1ff5cee0.
2020-12-13 10:38:24 +08:00
fanquake
ade38b6ee8
Merge #20588: Remove unused and confusing CTransaction constructor
fac39c198324715565897f4240709340477af0bf wallet: document that tx in CreateTransaction is purely an out-param (MarcoFalke)
faac31521bb7ecbf999541cf918d3750ff589de4 Remove unused and confusing CTransaction constructor (MarcoFalke)

Pull request description:

  The constructor is confusing and dangerous (as explained in the TODO), fix that by removing it.

ACKs for top commit:
  laanwj:
    Code review ACK fac39c198324715565897f4240709340477af0bf
  promag:
    Code review ACK fac39c198324715565897f4240709340477af0bf.
  theStack:
    Code review ACK fac39c198324715565897f4240709340477af0bf

Tree-SHA512: e0c8cffce8d8ee0166b8e1cbfe85ed0657611e26e2af0d69fde70eceaa5d75cbde3eb489af0428fe4fc431360b4c791fb1cc21b8dee7d4c7a4f17df00836229d
2020-12-13 10:36:22 +08:00
MarcoFalke
b18978066d
Merge #20079: p2p: Treat handshake misbehavior like unknown message
faaad1bbac46cfeb22654b4c59f0aac7a680c03a p2p: Ignore version msgs after initial version msg (MarcoFalke)
fad68afcff731153d1c83f7f56c91ecbb264b59a p2p: Ignore non-version msgs before version msg (MarcoFalke)

Pull request description:

  Handshake misbehaviour doesn't cost us more than any other unknown message, so it seems odd to treat it differently

ACKs for top commit:
  jnewbery:
    utACK faaad1bbac46cfeb22654b4c59f0aac7a680c03a
  practicalswift:
    ACK faaad1bbac46cfeb22654b4c59f0aac7a680c03a: patch looks correct

Tree-SHA512: 9f30c3b5c1f6604fd02cff878f10999956152419a3dd9825f8267cbdeff7d06787418b41c7fde8a00a5e557fe89204546e05d5689042dbf7b07fbb7eb95cddff
2020-12-12 12:32:46 +01:00
MarcoFalke
ffc4d04990
Merge #20275: wallet: List all wallets in non-SQLite and non-BDB builds
f3d870fc2271bf45e0269e5ae135bced1a26f620 wallet: List all wallets in non-SQLite or non-BDB builds (Russell Yanofsky)
d70dc89e78ee6355e0bc37cc36cfc04ef7a86885 refactor: Consolidate redundant wallet database path and exists functions (Russell Yanofsky)
6a7a63644cd2fc56538d323cc0d5c1d7945247fd refactor: Drop call to GetWalletEnv in wallet salvage code (Russell Yanofsky)
6ee9cbdd18a70894f89dd268c276d5eb47a34827 refactor: Replace ListWalletDir() function with ListDatabases() (Russell Yanofsky)
5aaeb6cf877055c47fa2bbd2ea5e8d3d2033933b MOVEONLY: Move IsBDBFile, IsSQLiteFile, and ListWalletDir (Russell Yanofsky)

Pull request description:

  This PR does not change behavior when bitcoin is built normally with both the SQLite and BDB libraries. It just makes non-SQLite and non-BDB builds more similar to the normal build. Specifically:

  - It makes wallet directory lists always include all wallets so wallets don't appear missing depending on the build.

  - It now triggers specific "Build does not support SQLite database format" and "Build does not support Berkeley DB database format" errors if a wallet can't be loaded instead of the more ambiguous and scary "Data is not in recognized format" error.

  Both changes are implemented in the last commit. The previous commits are just refactoring cleanups that make the last commit possible and consolidate and reduce code.

ACKs for top commit:
  achow101:
    ACK f3d870fc2271bf45e0269e5ae135bced1a26f620
  promag:
    Tested ACK f3d870fc2271bf45e0269e5ae135bced1a26f620. Tested a --without-sqlite build with sqlite wallets.

Tree-SHA512: 029ad21559dbc338b5f351d05113c51bc25bce830f4f4e18bcd82287bc528275347a60249da65b91d252632aeb70b25d057bd59c704bfcaafb9f790bc5b59762
2020-12-12 09:20:11 +01:00
pox
cc3044ccdb fix misleading comment about call to non-existing function 2020-12-12 07:01:38 +02:00
Carl Dong
22437fc72e build: Run libdmg-hfsplus's DMG tool in make deploy
Previously, the compression of the .iso file to a .dmg file was done
outside of `make deploy' in order to use the faketime-wrapped version of
libdmg-hfsplus's DMG tool.

Specifying the faketime-wrapped version of the DMG tool to ./configure
fixes this and simplifies build scripts.
2020-12-11 11:43:11 -05:00
John Newbery
f6360088de [net processing] Clarify UpdatedBlockTip() 2020-12-11 10:40:19 +00:00
John Newbery
94d2cc35be [net processing] Remove unnecesary nNewHeight variable in UpdatedBlockTip() 2020-12-11 10:35:32 +00:00
John Newbery
8b57013473 [net processing] Remove nStartingHeight check from block relay
nStartingHeight was introduced in commit 7a47324c7 (Bitcoin version
0.2.9, P2P version 209) with the comment "better prevention of inventory
relaying during initial download". At that time, there was no function
to determine whether the node was still in Initial Block Download, so to
prevent syncing nodes from relaying old blocks to their peers, a check
was added to never relay a block to a peer where the height was lower
than 2000 less than the peer's best block. That check was updated
several times in later commits to ensure that we weren't relaying blocks
before the latest checkpoint if the peer didn't provide a
startingheight. The checkpoint comparison was changed to compare with an
estimate of the highest block in commit eae82d8e.

In commit 202e0194, all block relay was gated on being out of Initial
Block Download. In commit 0278fb5f, the comparison to nBlockEstimate was
removed since "we already checked IsIBD()".

We can remove the check against nStartingHeight entirely. If the node is
out of Initial Block Download, then its tip height must have been within
24 hours of current time, so should not be more than ~144 blocks behind
the most work tip.
2020-12-11 10:08:37 +00:00
MarcoFalke
6a48063671
Merge #19858: Periodically make block-relay connections and sync headers
b3a515c0bec97633a76bec101af47c3c90c0b749 Clarify comments around outbound peer eviction (Suhas Daftuar)
daffaf03fbede6c01287779e464379ee3acb005a Periodically make block-relay connections and sync headers (Suhas Daftuar)
3cc8a7a0f5fa183cd7f0cf5e56f16f9a9d1f2441 Use conn_type to identify block-relay peers, rather than m_tx_relay == nullptr (Suhas Daftuar)
91d61952a82af3e8887e8ae532ecc19d87fe9073 Simplify and clarify extra outbound peer counting (Suhas Daftuar)

Pull request description:

  To make eclipse attacks more difficult, regularly initiate outbound connections
  and stay connected long enough to sync headers and potentially learn of new
  blocks. If we learn a new block, rotate out an existing block-relay peer in
  favor of the new peer.

  This augments the existing outbound peer rotation that exists -- currently we
  make new full-relay connections when our tip is stale, which we disconnect
  after waiting a small time to see if we learn a new block.  As block-relay
  connections use minimal bandwidth, we can make these connections regularly and
  not just when our tip is stale.

  Like feeler connections, these connections are not aggressive; whenever our
  timer fires (once every 5 minutes on average), we'll try to initiate a new
  block-relay connection as described, but if we fail to connect we just wait for
  our timer to fire again before repeating with a new peer.

ACKs for top commit:
  ariard:
    Code Review ACK b3a515c, only change since last time is dropping a useless `cs_main` taking. I manually tested a previous version of the PR, and not substantial change has been introduced since then which would alter behavior IMO.
  jonatack:
    Tested ACK b3a515c0bec97633a76bec101af47c3c90c0b749 over several weeks, though this change and behavior could benefit from test coverage and other follow-ups (refactoring, etc.) described in the review feedback. I did not verify the behavior of `m_start_extra_block_relay_peers` only being enabled after initial chain sync. Since my last review, one unneeded `cs_main` lock was removed.

Tree-SHA512: 75fc6f8e8003e88e93f86b845caf2d30b8b9c0dbb0a6b8aabe4e24ea4f6327351f736a068a3b2720a8a581b789942a3a47f921e2afdb47e88bc50d078aa37b6f
2020-12-11 10:20:01 +01:00
fanquake
736eb4d808
Merge #19982: test: Fix inconsistent lock order in wallet_tests/CreateWallet
e1e68b6305beb47ebf7ee48f14e12fdebdfea1ef test: Fix inconsistent lock order in wallet_tests/CreateWallet (Hennadii Stepanov)
cb23fe01c125e1820f3c37348e06d98c93e6aec2 [skip ci] sync: Check precondition in LEAVE_CRITICAL_SECTION() macro (Hennadii Stepanov)
c5e3e74f70c29ac8852903ef425f5f327d5da969 sync: Improve CheckLastCritical() (Hennadii Stepanov)

Pull request description:

  This PR:
  - fixes #19049 that was caused by #16426
  - removes `wallet_tests::CreateWallet` suppression from the `test/sanitizer_suppressions/tsan`

  The example of the improved `CheckLastCritical()`/`LEAVE_CRITICAL_SECTION()` log (could be got when compiled without the last commit):
  ```
  2020-09-20T08:34:28.429485Z [test] INCONSISTENT LOCK ORDER DETECTED
  2020-09-20T08:34:28.429493Z [test] Current lock order (least recent first) is:
  2020-09-20T08:34:28.429501Z [test]  'walletInstance->cs_wallet' in wallet/wallet.cpp:4007 (in thread 'test')
  2020-09-20T08:34:28.429508Z [test]  'cs_wallets' in wallet/wallet.cpp:4089 (in thread 'test')
  ```

  Currently, there are other "naked" `LEAVE_CRITICAL_SECTION()` in the code base:
  b99a1633b2/src/rpc/mining.cpp (L698)

  b99a1633b2/src/checkqueue.h (L208)

ACKs for top commit:
  MarcoFalke:
    review ACK e1e68b6305beb47ebf7ee48f14e12fdebdfea1ef 💂
  ryanofsky:
    Code review ACK e1e68b6305beb47ebf7ee48f14e12fdebdfea1ef. Just trivial rebase and suggested switch to BOOST_CHECK_EXCEPTION since last review
  vasild:
    ACK e1e68b630

Tree-SHA512: a627680eac3af4b4c02772473d68322ce8d3811bf6b035d3485ccc97d35755bef933cffabd3f20b126f89e3301eccecec3f769df34415fb7c426c967b6ce36e6
2020-12-11 09:44:56 +08:00
Jarol Rodriguez
c175690561
doc: Update for FreeBSD 12.2, add GUI Build Instructions
The current FreeBSD Build documentation is a little outdated and underwhelming. This PR updates the doc to
be more informative. It also adds new instructions for building the GUI and adding support for  descriptor
wallets.

on vasild's recommendation: it is ok to point the user to download db5 which has an active port package
instead of building db4 from the provided script.

Co-authored-by: Vasil Dimov <vd@freebsd.org>
2020-12-10 15:38:20 -05:00
Hennadii Stepanov
e1e68b6305
test: Fix inconsistent lock order in wallet_tests/CreateWallet 2020-12-10 20:49:06 +02:00
Hennadii Stepanov
cb23fe01c1
[skip ci] sync: Check precondition in LEAVE_CRITICAL_SECTION() macro
This change reveals a bug in the wallet_tests/CreateWalletFromFile test,
that will be fixed in the following commit.
2020-12-10 20:46:39 +02:00
Hennadii Stepanov
c5e3e74f70
sync: Improve CheckLastCritical()
This commit adds actual lock stack logging if check fails.
2020-12-10 20:46:29 +02:00
Antoine Riard
a33442fdc7 Remove m_is_manual_connection from CNodeState
Currently, this member is only used to exclude MANUAL peers from discouragement
in MaybePunishNodeForBlock(). Manual connections are already protected in
MaybeDiscourageAndDisconnect(), independently from their network
processing behaviors.
2020-12-10 12:53:02 -05:00
MarcoFalke
da957cd62e
Merge #20613: test: Use Popen.wait instead of RPC in assert_start_raises_init_error
fa918dd537fea775c19a590e5f9161bf51a5839b test: Use Popen.wait instead of RPC in assert_start_raises_init_error (MarcoFalke)

Pull request description:

  Using RPC (`wait_for_rpc_connection`) has several issue:

  * It polls in a loop, which might be slow
  * It tries to read the RPC cookie file, which might not be present, thus leading to intermittent issues

  Fix both by using `Popen.wait`

ACKs for top commit:
  laanwj:
    Code review ACK ~~faf7b05be9c86ee61c39e5314511fe2410128a6b~~ fa918dd537fea775c19a590e5f9161bf51a5839b
  darosior:
    ACK fa918dd537fea775c19a590e5f9161bf51a5839b

Tree-SHA512: 5368ad0d0ea2deb0af9582a42667c9290efe8f2705f37a236afc2c7908b04265ab342e2dd356a57156e99389f4a27ab6da9fa7bf9161fb7568240aa005e693b9
2020-12-10 18:38:18 +01:00
Michael Dietz
3b064fcb9d
test: run mempool_expiry.py even with wallet disabled
Test coverage is also extended in this commit to:
Ensure that another transaction in the mempool is not evicted
when other transactions expire. Note: this other transaction does
not have any ancestors in the mempool that expired. Otherwise it
would be evicted from the mempool, and we already test this case.
2020-12-10 10:14:35 -06:00
MarcoFalke
faf2c6e32e
cirrus: Schedule one task with paid credits for faster CI feedback 2020-12-10 15:04:54 +01:00
Suhas Daftuar
b3a515c0be Clarify comments around outbound peer eviction 2020-12-10 08:46:51 -05:00
Suhas Daftuar
daffaf03fb Periodically make block-relay connections and sync headers
To make eclipse attacks more difficult, regularly initiate outbound connections
and stay connected long enough to sync headers and potentially learn of new
blocks. If we learn a new block, rotate out an existing block-relay peer in
favor of the new peer.

This augments the existing outbound peer rotation that exists -- currently we
make new full-relay connections when our tip is stale, which we disconnect
after waiting a small time to see if we learn a new block.  As block-relay
connections use minimal bandwidth, we can make these connections regularly and
not just when our tip is stale.

Like feeler connections, these connections are not aggressive; whenever our
timer fires (once every 5 minutes on average), we'll try to initiate a new
block-relay connection as described, but if we fail to connect we just wait for
our timer to fire again before repeating with a new peer.
2020-12-10 08:46:39 -05:00