Commit Graph

91 Commits

Author SHA1 Message Date
willcl-ark
42e99ad773 net: skip non-route netlink responses
This shouldn't usually be hit, but is a good belt-and-braces.
2025-04-02 09:47:27 +01:00
willcl-ark
57ce645f05 net: filter for default routes in netlink responses
Filter netlink responses to only consider default routes by checking the
destination prefix length (rtm_dst_len == 0).

Previously, we selected the first route with an RTA_GATEWAY attribute,
which for IPv6 often resulted in choosing a non-default route instead of
the actual default.

This caused occasional PCP port mapping failures because a gateway for a
non-default route was selected.
2025-04-02 09:46:56 +01:00
MarcoFalke
fa942332b4 scripted-diff: Bump copyright headers after std::span changes
Historically, the headers have been bumped some time after a file has
been touched. Do it now to avoid having to touch them again in the
future for that reason.

-BEGIN VERIFY SCRIPT-
 sed -i --regexp-extended 's;( 20[0-2][0-9])(-20[0-2][0-9])? The Bitcoin Core developers;\1-present The Bitcoin Core developers;g' $( git show --pretty="" --name-only HEAD~1 )
-END VERIFY SCRIPT-
2025-03-12 19:46:54 +01:00
MarcoFalke
fade0b5e5e scripted-diff: Use std::span over Span
-BEGIN VERIFY SCRIPT-

 ren() { sed -i "s!\<$1\>!$2!g" $( git grep -l "$1" -- "./src" ":(exclude)src/span.h" ":(exclude)src/leveldb/db/log_test.cc" ) ; }

 ren Span            std::span
 ren AsBytes         std::as_bytes
 ren AsWritableBytes std::as_writable_bytes

 sed -i 's!SpanPopBack(Span!SpanPopBack(std::span!g' ./src/span.h

-END VERIFY SCRIPT-
2025-03-12 19:45:37 +01:00
Antoine Poinsot
6fe1c35c05 pcp: make NAT-PMP error codes uint16_t
They are defined as being 16 bits in the RFC and correctly parsed in the code
which may result in an implicit conversion from uint16_t to uint8_t.
2025-02-12 10:31:43 -05:00
Antoine Poinsot
01906ce912 pcp: make the ToString method const 2025-02-12 10:31:43 -05:00
laanwj
caf9521033 net: Use mockable steady clock in PCP implementation
This will be needed for the test harness.
2025-01-13 21:53:56 +01:00
laanwj
ab1d3ece02 net: Add optional length checking to CService::SetSockAddr
In almost all cases (the only exception is `getifaddrs`), we know the
size of the data passed into SetSockAddr, so we can check this to be
what is expected.
2025-01-13 21:53:56 +01:00
Hodlinator
e85abe92c7 args: Catch directories in place of config files
Previously passing a directory path as -conf would lead to an ifstream being opened for it, and would not trigger any errors.
2024-12-03 11:04:10 +01:00
Hodlinator
483f0dacc4 args: Properly support -noconf
-noconf would previously lead to an ifstream "successfully" being opened to the ".bitcoin"-directory (not a file). (Guards against the general case of directories as configs are added in grandchild commit to this one).

Other users of AbsPathForConfigVal() in combination with negated args have been updated earlier in this PR ("args: Support -nopid" and "args: Support -norpccookiefile...").
2024-12-03 11:04:10 +01:00
Ava Chow
4a31f8ccc9 Merge bitcoin/bitcoin#31156: test: Don't enforce BIP94 on regtest unless specified by arg
e60cecc811 doc: add release note for 31156 (Martin Zumsande)
fc7dfb3df5 test: Don't enforce BIP94 on regtest unless specified by arg (Martin Zumsande)

Pull request description:

  The added arg `-test=bip94` is only used in a functional test for BIP94. This is done because the default regtest consensus rules should follow mainnet, not testnet.

  Fixes #31137.

ACKs for top commit:
  achow101:
    ACK e60cecc811
  tdb3:
    cr and light test ACK e60cecc811
  rkrux:
    tACK e60cecc811
  BrandonOdiwuor:
    utACK e60cecc811
  laanwj:
    Code review ACK e60cecc811

Tree-SHA512: ca2f322f89d8808dfc3565fe020d2615cfcc110e188a02128ad7108fef51c735b33d55b5e6a70c505d78f7291f3c635dc7dfbcd78be1348d4d6e483883be4216
2024-10-30 17:00:14 -04:00
Martin Zumsande
fc7dfb3df5 test: Don't enforce BIP94 on regtest unless specified by arg
The added regtest option -test=bip94 is only used in the functional
test for BIP94.
This is done because the default regtest consensus rules
should aim to follow to mainnet, not testnet.
2024-10-28 11:38:38 -04:00
Hennadii Stepanov
70713303b6 scripted-diff: Rename PACKAGE_* variables to CLIENT_*
This change ensures consistent use of the `CLIENT_` namespace everywhere
in the repository.

-BEGIN VERIFY SCRIPT-

ren() { sed -i "s/\<$1\>/$2/g" $( git grep -l "$1" ./cmake ./src :\(exclude\)./src/secp256k1 ./test ) ; }

ren PACKAGE_NAME      CLIENT_NAME
ren PACKAGE_VERSION   CLIENT_VERSION_STRING
ren PACKAGE_URL       CLIENT_URL
ren PACKAGE_BUGREPORT CLIENT_BUGREPORT

-END VERIFY SCRIPT-
2024-10-28 12:36:19 +00:00
Ava Chow
7640cfdd62 Merge bitcoin/bitcoin#31118: doc: replace -? with -h and -help
33a28e252a Change default help arg to `-help` and mention `-h` and `-?` as alternatives (Lőrinc)
f0130ab1a1 doc: replace `-?` with `-h` for bench_bitcoin help (Lőrinc)

Pull request description:

  The question mark is interpreted as a wildcard for any single character in Zsh (see https://www.techrepublic.com/article/globbing-wildcard-characters-with-zsh), so `bench_bitcoin -?` will not show the help message on systems using Zsh, such as macOS.

  Since `-h` provides equivalent help functionality (as defined in https://github.com/bitcoin/bitcoin/blob/master/src/common/args.cpp#L684-L693), the `benchmarking.md` documentation has been updated to ensure compatibility with macOS.

  ----

  ### -?
  > % cmake -B build -DBUILD_BENCH=ON && cmake --build build && build/src/bench/bench_bitcoin -?
  zsh: no matches found: -?

  ### -h
  > % cmake -B build -DBUILD_BENCH=ON && cmake --build build && build/src/bench/bench_bitcoin -h
  Usage:  bench_bitcoin [options]
  Options:
  ...

  ----

  Based on the comments the args help default was also changed to `-help`, mentioning `-h` and `-?` (instead of `-?` being the default)

ACKs for top commit:
  edilmedeiros:
    tACK 33a28e252a
  maflcko:
    lgtm ACK 33a28e252a
  achow101:
    ACK 33a28e252a
  rkrux:
    tACK 33a28e252a
  laanwj:
    Code review ACK 33a28e252a

Tree-SHA512: 8c6e27488462be9ba9186b34abe6249c1d93026b3963acc0f42c75496f39407563766ae518cf1839156039cc0047e29d91f70d191cfb97e0fbde85665e88c71e
2024-10-24 18:01:41 -04:00
Lőrinc
6c6b2442ed build: Replace MAC_OSX macro with existing __APPLE__
Adopting `__APPLE__` aligns our project with broader industry practices, including those in prominent projects such as the Linux kernel (and even our own code).

See: https://sourceforge.net/p/predef/wiki/OperatingSystems/#macos
2024-10-24 12:29:26 +02:00
Lőrinc
33a28e252a Change default help arg to -help and mention -h and -? as alternatives
% build/src/bench/bench_bitcoin -h
[...]
  -help
       Print this help message and exit (also -h or -?)
2024-10-21 11:08:51 +02:00
Sebastian Falbesoner
1786be7b4a scripted-diff: drop config/ subdir for bitcoin-config.h, rename to bitcoin-build-config.h
Follow-up for PR #30856, commit 0dd66251.

-BEGIN VERIFY SCRIPT-
sed -i "s|config/bitcoin-config\.h|bitcoin-build-config.h|g" $(git grep -l config/bitcoin-config\.h)
sed -i "s|bitcoin-config\.h|bitcoin-build-config.h|g" $(git grep -l "bitcoin-config\.h" ./src ./test ./cmake)
git mv ./cmake/bitcoin-config.h.in ./cmake/bitcoin-build-config.h.in
-END VERIFY SCRIPT-
2024-10-10 12:22:12 +02:00
Ryan Ofsky
d51edecddc common: move pcp.cpp and netif.cpp files from util to common library since they depend on netaddress.cpp
Prevents check-deps.sh errors reported by fanquake
https://github.com/bitcoin/bitcoin/pull/30415#issuecomment-2385475097
2024-10-01 09:28:31 -04:00
Russell Yanofsky
30073e6b3a multiprocess: Add -ipcbind option to bitcoin-node
Add `-ipcbind` option to `bitcoin-node` to listen on an IPC socket and accept
connections from other processes. In the future, there will be an `-ipcconnect`
option added to `bitcoin-wallet` and `bitcoin-node` to allow wallet and gui
processes to connect to the node and access it.

Example usage:

    src/bitcoin-node -regtest -debug -ipcbind=unix
    src/bitcoin-wallet -regtest -ipcconnect=unix info
    src/bitcoin-gui -regtest -ipcconnect=unix
    src/bitcoin-mine -regtest -ipcconnect=unix
2024-09-06 09:08:10 -04:00
Ava Chow
81276540d3 Merge bitcoin/bitcoin#30148: cli: restrict multiple exclusive argument usage in bitcoin-cli
c8e6771af0 test: restrict multiple CLI arguments (naiyoma)
8838c4f171 common/args.h: automate check for multiple cli commands (naiyoma)

Pull request description:

  This PR is a continuation of the validation suggested [here](https://github.com/bitcoin/bitcoin/pull/27815) to ensure that only one Request Handler can be specified at a time.

ACKs for top commit:
  stratospher:
    reACK c8e6771.
  achow101:
    ACK c8e6771af0
  tdb3:
    cr re ACK c8e6771af0

Tree-SHA512: f4fe036fee342a54f1a7ac702ac35c40bf3d420fde6ab16313a75327292d5ee5c8ece1be9f852a13fcf73da1148b143b37b4894e294052abdde6eefb2e8c6f3f
2024-09-04 14:47:38 -04:00
Ava Chow
da083d4bbd Merge bitcoin/bitcoin#29775: Testnet4 including PoW difficulty adjustment fix
6bfa26048d testnet: Add timewarp attack prevention for Testnet4 (Fabian Jahr)
0100907ca1 testnet: Add Testnet4 difficulty adjustment rules fix (Fabian Jahr)
74a04f9e7a testnet: Introduce Testnet4 (Fabian Jahr)

Pull request description:

  To supplement the [ongoing conceptual discussion about a testnet reset](https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/9yCPo3uUBwAJ) I have drafted a move to v4 including a fix to the difficulty adjustment mechanism, which was part of the motivation that started the discussion.

  Conceptual considerations:
  - The conceptual discussion about doing a testnet4 or softforking the fix into testnet3 is outside of the scope of this PR and I would ask reviewers to contribute their opinions on this on the ML instead. However, I am happy to adapt this PR to a softfork change on testnet3 if there is consensus for that instead.
  - The difficulty adjustment fix suggested here touches the `CalculateNextWorkRequired` function and uses the same logic used in `GetNextWorkRequired` to find the last previous block that was not mined with difficulty 1 under the exceptionf. An alternative fix briefly mentioned on the mailing list by Jameson Lopp would be to "restrict the special testnet minimum difficulty rule so that it can't be triggered on the block right before a difficulty retarget". That would also fix the issue but I find my suggestion here a bit more elegant.

ACKs for top commit:
  jsarenik:
    tACK 6bfa26048d
  achow101:
    ACK 6bfa26048d
  murchandamus:
    tACK 6bfa26048d

Tree-SHA512: 0b8b69a621406a944da5be551b863d065358ba94d85dd3b80d83c412660e230ee93b27316081fbee9b4851cc4ff8585db64c7dfa26cb5148ac835663f2712c3d
2024-08-07 13:05:04 -04:00
Ava Chow
2987ba6637 Merge bitcoin/bitcoin#30525: doc, rpc : #30275 followups
fa2f26960e [rpc, fees]: add more detail on the fee estimation modes (ismaelsadeeq)
6e7e620864 [doc]: add `30275` release notes (ismaelsadeeq)

Pull request description:

  This PR:
  1. Adds release notes for #30275
  2. Describe fee estimation modes in RPC help texts

ACKs for top commit:
  achow101:
    ACK fa2f26960e
  glozow:
    ACK fa2f26960e
  willcl-ark:
    ACK fa2f26960e
  tdb3:
    re ACK fa2f26960e

Tree-SHA512: b8ea000b599297b954dc770137c29b47153e68644c58550a73e34b74ecb8b65e78417875481efdfdf6aab0018a9cd1d90d8baa5a015e70aca0975f6e1dc9598c
2024-08-07 01:27:42 -04:00
naiyoma
8838c4f171 common/args.h: automate check for multiple cli commands
Co-authored-by: Anthony Towns <aj@erisian.com.au>
2024-08-06 14:55:20 +03:00
Fabian Jahr
74a04f9e7a testnet: Introduce Testnet4 2024-08-06 01:38:10 +02:00
willcl-ark
87b1880525 rpc: clarify ALREADY_IN_CHAIN rpc errors
When using `sendrawtransaction` the ALREADY_IN_CHAIN error help string
may be confusing.

Rename TransactionError::ALREADY_IN_CHAIN to
TransactionError::ALREADY_IN_UTXO_SET and update the rpc help string.

Remove backwards compatibility alias as no longer required.
2024-08-05 15:45:58 +01:00
ismaelsadeeq
fa2f26960e [rpc, fees]: add more detail on the fee estimation modes
- Add description that indicates the fee estimation modes behaviour.
- This description will be returned in the RPC's help texts.
2024-08-02 15:40:43 +01:00
Pieter Wuille
ddc184d999 random: get rid of GetRand by inlining 2024-07-01 12:39:53 -04:00
Ava Chow
011a895a82 Merge bitcoin/bitcoin#29015: kernel: Streamline util library
c7376babd1 doc: Clarify distinction between util and common libraries in libraries.md (Ryan Ofsky)
4f74c59334 util: Move util/string.h functions to util namespace (Ryan Ofsky)
4d05d3f3b4 util: add TransactionError includes and namespace declarations (Ryan Ofsky)
680eafdc74 util: move fees.h and error.h to common/messages.h (Ryan Ofsky)
02e62c6c9a common: Add PSBTError enum (Ryan Ofsky)
0d44c44ae3 util: move error.h TransactionError enum to node/types.h (Ryan Ofsky)
9bcce2608d util: move spanparsing.h to script/parsing.h (Ryan Ofsky)
6dd2ad4792 util: move spanparsing.h Split functions to string.h (Ryan Ofsky)
23cc8ddff4 util: move HexStr and HexDigit from util to crypto (TheCharlatan)
6861f954f8 util: move util/message to common/signmessage (Ryan Ofsky)
cc5f29fbea build: move memory_cleanse from util to crypto (Ryan Ofsky)
5b9309420c build: move chainparamsbase from util to common (Ryan Ofsky)
ffa27af24d test: Add check-deps.sh script to check for unexpected library dependencies (Ryan Ofsky)

Pull request description:

  Remove `fees.h`, `errors.h`, and `spanparsing.h` from the util library. Specifically:

  - Move `Split` functions from `util/spanparsing.h` to `util/string.h`, using `util` namespace for clarity.
  - Move remaining spanparsing functions to `script/parsing.h` since they are used for descriptor and miniscript parsing.
  - Combine `util/fees.h` and `util/errors.h` into `common/messages.h` so there is a place for simple functions that generate user messages to live, and these functions are not part of the util library.

  Motivation for this change is that the util library is a dependency of the kernel, and we should remove functionality from util that shouldn't be called by kernel code or kernel applications. These changes should also improve code organization and make functions easier to discover. Some of these same moves are (or were) part of #28690, but did not help with code organization, or made it worse, so it is better to move them and clean them up in the same PR so code only has to change one time.

ACKs for top commit:
  achow101:
    ACK c7376babd1
  TheCharlatan:
    Re-ACK c7376babd1
  hebasto:
    re-ACK c7376babd1.

Tree-SHA512: 5bcef16c1255463b1b69270548711e7ff78ca0dd34e300b95e3ca1ce52ceb34f83d9ddb2839e83800ba36b200de30396e504bbb04fa02c6d0c24a16d06ae523d
2024-06-12 17:12:54 -04:00
Ava Chow
915d7276e4 Merge bitcoin/bitcoin#27064: system: use %LOCALAPPDATA% as default datadir on windows
84900ac34f doc: add release-notes-27064.md (Matthew Zipkin)
855dd8d592 system: use %LOCALAPPDATA% as default datadir on windows (Matthew Zipkin)

Pull request description:

  Closes https://github.com/bitcoin/bitcoin/issues/2391

  This PR changes the default datadir location on Windows from `C:\Users\Username\AppData\Roaming\Bitcoin` to `C:\Users\Username\AppData\Local\Bitcoin`. This change only applies to fresh installs. To preserve backwards compatibility, on startup we check for the existence of `C:\Users\Username\AppData\Roaming\Bitcoin\chainstate` and if it is there, we continue using the "Roaming" directory as the default datadir location.

  [Note that in Windows 11 this change may be moot:](https://learn.microsoft.com/en-us/uwp/api/windows.storage.applicationdata.roamingfolder?view=winrt-22621)

  > Roaming data and settings is no longer supported as of Windows 11. The recommended replacement is [Azure App Service](https://learn.microsoft.com/en-us/azure/app-service/). Azure App Service is widely supported, well documented, reliable, and supports cross-platform/cross-ecosystem scenarios such as iOS, Android and web. Settings stored here no longer roam (as of Windows 11), but the settings store is still available.

ACKs for top commit:
  achow101:
    ACK 84900ac34f
  BenWestgate:
    crACK 84900ac34f
  hebasto:
    re-ACK 84900ac34f, only addressed feedback since my recent [review](https://github.com/bitcoin/bitcoin/pull/27064#pullrequestreview-2028718273).

Tree-SHA512: 807c6e89571287e2c8f4934229aec91ef28e7d0a675234acf1b7d085c24c7b73a08b6e345fbfc9038e6239187b6b69c08490ddaa1c057de5ea975c4a000bba42
2024-05-23 12:11:55 -04:00
Ryan Ofsky
4f74c59334 util: Move util/string.h functions to util namespace
There are no changes to behavior. Changes in this commit are all additions, and
are easiest to review using "git diff -U0 --word-diff-regex=." options.

Motivation for this change is to keep util functions with really generic names
like "Split" and "Join" out of the global namespace so it is easier to see
where these functions are defined, and so they don't interfere with function
overloading, especially since the util library is a dependency of the kernel
library and intended to be used with external code.
2024-05-16 10:16:08 -05:00
Ryan Ofsky
4d05d3f3b4 util: add TransactionError includes and namespace declarations
Add TransactionError to node namespace and include it directly instead of
relying on indirect include through common/messages.h

This is a followup to a previous commit which moved the TransactionError enum.
These changes were done in a separate followup just to keep the previous commit
more minimal and easy to review.
2024-05-16 10:16:08 -05:00
Ryan Ofsky
680eafdc74 util: move fees.h and error.h to common/messages.h
Move enum and message formatting functions to a common/messages header where
they should be more discoverable, and also out of the util library, so they
will not be a dependency of the kernel

The are no changes in behavior and no changes to the moved code.
2024-05-16 10:16:08 -05:00
Ryan Ofsky
02e62c6c9a common: Add PSBTError enum
Add separate PSBTError enum instead of reusing TransactionError enum for PSBT
operations, and drop unused error codes. The error codes returned by PSBT
operations and transaction broadcast functions mostly do not overlap, so using
an unified enum makes it harder to call any of these functions and know which
errors actually need to be handled.

Define PSBTError in the common library because PSBT functionality is
implemented in the common library and used by both the node (for rawtransaction
RPCs) and the wallet.
2024-05-16 10:16:08 -05:00
Ryan Ofsky
6861f954f8 util: move util/message to common/signmessage
Move util/message to common/signmessage so it is named more clearly, and
because the util library is not supposed to depend on other libraries besides
the crypto library. The signmessage functions use CKey, CPubKey, PKHash, and
DecodeDestination functions in the consensus and common libraries.
2024-05-16 11:16:08 -04:00
MarcoFalke
dddd40ba82 scripted-diff: Add IWYU pragma keep to bitcoin-config.h includes
-BEGIN VERIFY SCRIPT-
 perl -0777 -pi -e 's/#if defined\(HAVE_CONFIG_H\)\n#include <config\/bitcoin-config.h>.*\n#endif.*\n/#include <config\/bitcoin-config.h> \/\/ IWYU pragma: keep\n/g' $( git grep -l '#include <config/bitcoin-config.h>' )
-END VERIFY SCRIPT-
2024-05-01 08:33:04 +02:00
Matthew Zipkin
855dd8d592 system: use %LOCALAPPDATA% as default datadir on windows 2024-04-30 11:03:04 -04:00
Ryan Ofsky
16a6174613 Merge bitcoin/bitcoin#29904: refactor: Use our own implementation of urlDecode
992c714451 common: Don't terminate on null character in UrlDecode (Fabian Jahr)
099fa57151 scripted-diff: Modernize name of urlDecode function and param (Fabian Jahr)
8f39aaae41 refactor: Remove hooking code for urlDecode (Fabian Jahr)
650d43ec15 refactor: Replace libevent use in urlDecode with our own code (Fabian Jahr)
46bc6c2aaa test: Add unit tests for urlDecode (Fabian Jahr)

Pull request description:

  Fixes #29654 (as a side-effect)

  Removing dependencies is a general goal of the project and the xz backdoor has been an additional wake up call recently. Libevent shows many of the same symptoms, few maintainers and slow releases. While libevent can not be removed completely over night we should start removing it’s usage where it's possible, ideally with the end goal to removing it completely.

  This is a pretty easy win in that direction. The [`evhttp_uridecode` function from libevent](e0a4574ba2/http.c (L3542)) we were using in `urlDecode` could be easily emulated in fewer LOC. This also ports the [applicable test vectors over from libevent](https://github.com/libevent/libevent/blob/master/test/regress_http.c#L3430).

ACKs for top commit:
  achow101:
    ACK 992c714451
  theStack:
    Code-review ACK 992c714451
  maflcko:
    ACK 992c714451 👈
  stickies-v:
    ACK 992c714451

Tree-SHA512: 78f76ae7ab3b6710eab2aaac20f55eb0da7803e057eaa6220e865f328666a5399ef1a479702aaf630b2f974ad3aa15e2b6adac9c11bc8c3d4be21e8af1667fea
2024-04-25 13:02:43 -04:00
Fabian Jahr
992c714451 common: Don't terminate on null character in UrlDecode
The previous behavior was the result of casting the result returned from the libevent function evhttp_uridecode to std:string but this was probably not intended.
2024-04-24 23:27:50 +02:00
Fabian Jahr
099fa57151 scripted-diff: Modernize name of urlDecode function and param
-BEGIN VERIFY SCRIPT-
sed -i 's/urlDecode/UrlDecode/g' $(git grep -l 'urlDecode' ./src)
sed -i 's/urlEncoded/url_encoded/g' $(git grep -l 'urlEncoded' ./src)
-END VERIFY SCRIPT-
2024-04-24 23:26:24 +02:00
Fabian Jahr
8f39aaae41 refactor: Remove hooking code for urlDecode
The point of this was to be able to build bitcoin-tx and bitcoin-wallet without libevent, see #18504.

Now that we use our own implementation of urlDecode this is not needed anymore.

Co-authored-by: stickies-v <stickies-v@protonmail.com>
2024-04-24 23:23:38 +02:00
Fabian Jahr
650d43ec15 refactor: Replace libevent use in urlDecode with our own code 2024-04-24 23:23:34 +02:00
Hennadii Stepanov
d8e4ba4d05 refactor: Rename subprocess.hpp to follow our header name conventions 2024-04-23 18:22:58 +01:00
Sebastian Falbesoner
70434b1c44 external_signer: replace boost::process with cpp-subprocess
This primarily affects the `RunCommandParseJSON` utility function.
2024-03-27 14:16:37 +00:00
Ava Chow
a945f09fa6 Merge bitcoin/bitcoin#29007: test: create deterministic addrman in the functional tests
2cc8ca19f4 [test] Use deterministic addrman in addrman info tests (stratospher)
a897866109 [test] Restart a node with empty addrman (stratospher)
71c19915c0 [test] Use deterministic addrman in addpeeraddress test (stratospher)
7b868e6b67 Revert "test: avoid non-determinism in asmap-addrman test" (stratospher)
69e091f3e1 [init] Create deterministic addrman in tests using -test=addrman (stratospher)
be25ac3092 [init] Remove -addrmantest command line arg (stratospher)
802e6e128b [init] Add new command line arg for use only in functional tests (stratospher)

Pull request description:

  An address is placed in a `[bucket,position]` in the addrman table (new table or tried table) using the `addpeeraddress` RPC. This `[bucket,position]` is calculated using `nKey`(and other metrics) for the addrman which is chosen randomly during every run.

  Supposing there are 2 addresses to be placed in an addrman table. During every test run, a different `[bucket,position]` would be calculated for each address.These calculated `[bucket,position]` could even be the same for the 2 addresses in some test runs and result in collisions in the addrman. We wouldn't be able to predict when the collisions are going to happen because we can't predict the `nKey` value which is chosen at random. This can cause flaky tests.

  Because of these non deterministic collisions, we are limited in what we can do to test addrman functionality. Currently in our tests don't add a second address to prevent these collisions from happening - we only keep 1 address in the new table and 1 address in the tried table. See https://github.com/bitcoin/bitcoin/pull/26988#discussion_r1091145647, https://github.com/bitcoin/bitcoin/pull/23084, [#22831(comment)](https://github.com/bitcoin/bitcoin/pull/22831/files#r708302639).

  This PR lets us create a deterministic addrman with fixed `nKey` so that we can know the `[bucket,position]` collisions beforehand, safely add more addresses in an addrman table and write more extensive tests.

ACKs for top commit:
  amitiuttarwar:
    ACK 2cc8ca19f4
  achow101:
    ACK 2cc8ca19f4
  0xB10C:
    ACK 2cc8ca19f4
  mzumsande:
    Code Review ACK 2cc8ca19f4

Tree-SHA512: 8acd9bdfe7de1eb44d22373bf13533d8ecf602df966fdd5b8b78afcd8cc35a286c95d2712f67a89473a0d68dded7d38f5599f6e4bf95a6589475444545bfb189
2024-03-11 10:29:31 -04:00
TheCharlatan
9d1dbbd4ce scripted-diff: Fix bitcoin_config_h includes
-BEGIN VERIFY SCRIPT-

regex_string='^(?!//).*(AC_APPLE_UNIVERSAL_BUILD|BOOST_PROCESS_USE_STD_FS|CHAR_EQUALS_INT8|CLIENT_VERSION_BUILD|CLIENT_VERSION_IS_RELEASE|CLIENT_VERSION_MAJOR|CLIENT_VERSION_MINOR|COPYRIGHT_HOLDERS|COPYRIGHT_HOLDERS_FINAL|COPYRIGHT_HOLDERS_SUBSTITUTION|COPYRIGHT_YEAR|ENABLE_ARM_SHANI|ENABLE_AVX2|ENABLE_EXTERNAL_SIGNER|ENABLE_SSE41|ENABLE_TRACING|ENABLE_WALLET|ENABLE_X86_SHANI|ENABLE_ZMQ|HAVE_BOOST|HAVE_BUILTIN_CLZL|HAVE_BUILTIN_CLZLL|HAVE_BYTESWAP_H|HAVE_CLMUL|HAVE_CONSENSUS_LIB|HAVE_CXX20|HAVE_DECL_BE16TOH|HAVE_DECL_BE32TOH|HAVE_DECL_BE64TOH|HAVE_DECL_BSWAP_16|HAVE_DECL_BSWAP_32|HAVE_DECL_BSWAP_64|HAVE_DECL_FORK|HAVE_DECL_FREEIFADDRS|HAVE_DECL_GETIFADDRS|HAVE_DECL_HTOBE16|HAVE_DECL_HTOBE32|HAVE_DECL_HTOBE64|HAVE_DECL_HTOLE16|HAVE_DECL_HTOLE32|HAVE_DECL_HTOLE64|HAVE_DECL_LE16TOH|HAVE_DECL_LE32TOH|HAVE_DECL_LE64TOH|HAVE_DECL_PIPE2|HAVE_DECL_SETSID|HAVE_DECL_STRERROR_R|HAVE_DEFAULT_VISIBILITY_ATTRIBUTE|HAVE_DLFCN_H|HAVE_DLLEXPORT_ATTRIBUTE|HAVE_ENDIAN_H|HAVE_EVHTTP_CONNECTION_GET_PEER_CONST_CHAR|HAVE_FDATASYNC|HAVE_GETENTROPY_RAND|HAVE_GETRANDOM|HAVE_GMTIME_R|HAVE_INTTYPES_H|HAVE_LIBADVAPI32|HAVE_LIBCOMCTL32|HAVE_LIBCOMDLG32|HAVE_LIBGDI32|HAVE_LIBIPHLPAPI|HAVE_LIBKERNEL32|HAVE_LIBOLE32|HAVE_LIBOLEAUT32|HAVE_LIBSHELL32|HAVE_LIBSHLWAPI|HAVE_LIBUSER32|HAVE_LIBUUID|HAVE_LIBWINMM|HAVE_LIBWS2_32|HAVE_MALLOC_INFO|HAVE_MALLOPT_ARENA_MAX|HAVE_MINIUPNPC_MINIUPNPC_H|HAVE_MINIUPNPC_UPNPCOMMANDS_H|HAVE_MINIUPNPC_UPNPERRORS_H|HAVE_NATPMP_H|HAVE_O_CLOEXEC|HAVE_POSIX_FALLOCATE|HAVE_PTHREAD|HAVE_PTHREAD_PRIO_INHERIT|HAVE_STDINT_H|HAVE_STDIO_H|HAVE_STDLIB_H|HAVE_STRERROR_R|HAVE_STRINGS_H|HAVE_STRING_H|HAVE_STRONG_GETAUXVAL|HAVE_SYSCTL|HAVE_SYSCTL_ARND|HAVE_SYSTEM|HAVE_SYS_ENDIAN_H|HAVE_SYS_PRCTL_H|HAVE_SYS_RESOURCES_H|HAVE_SYS_SELECT_H|HAVE_SYS_STAT_H|HAVE_SYS_SYSCTL_H|HAVE_SYS_TYPES_H|HAVE_SYS_VMMETER_H|HAVE_THREAD_LOCAL|HAVE_TIMINGSAFE_BCMP|HAVE_UNISTD_H|HAVE_VM_VM_PARAM_H|LT_OBJDIR|PACKAGE_BUGREPORT|PACKAGE_NAME|PACKAGE_STRING|PACKAGE_TARNAME|PACKAGE_URL|PACKAGE_VERSION|PTHREAD_CREATE_JOINABLE|QT_QPA_PLATFORM_ANDROID|QT_QPA_PLATFORM_COCOA|QT_QPA_PLATFORM_MINIMAL|QT_QPA_PLATFORM_WINDOWS|QT_QPA_PLATFORM_XCB|QT_STATICPLUGIN|STDC_HEADERS|STRERROR_R_CHAR_P|USE_ASM|USE_BDB|USE_DBUS|USE_NATPMP|USE_QRCODE|USE_SQLITE|USE_UPNP|_FILE_OFFSET_BITS|_LARGE_FILES)'

exclusion_files=":(exclude)src/minisketch :(exclude)src/crc32c :(exclude)src/secp256k1 :(exclude)src/crypto/sha256_arm_shani.cpp :(exclude)src/crypto/sha256_avx2.cpp :(exclude)src/crypto/sha256_sse41.cpp :(exclude)src/crypto/sha256_x86_shani.cpp"

git grep --perl-regexp --files-with-matches "$regex_string" -- '*.cpp' $exclusion_files | xargs git grep -L "bitcoin-config.h" | while read -r file; do line_number=$(awk -v my_file="$file" '/\/\/ file COPYING or https?:\/\/www.opensource.org\/licenses\/mit-license.php\./ {line = NR} /^\/\// && NR == line + 1 {while(getline && /^\/\//) line = NR} END {print line+1}' "$file"); sed -i "${line_number}i\\\\n\#if defined(HAVE_CONFIG_H)\\n#include <config/bitcoin-config.h>\\n\#endif" "$file"; done;

git grep --perl-regexp --files-with-matches "$regex_string" -- '*.h' $exclusion_files | xargs git grep -L "bitcoin-config.h" | while read -r file; do sed -i "/#define.*_H/a \\\\n\#if defined(HAVE_CONFIG_H)\\n#include <config/bitcoin-config.h>\\n\#endif" "$file"; done;

for file in $(git grep --files-with-matches 'bitcoin-config.h' -- '*.cpp' '*.h' $exclusion_files); do if ! grep -q --perl-regexp "$regex_string" $file; then sed -i '/HAVE_CONFIG_H/{N;N;N;d;}' $file; fi; done;

-END VERIFY SCRIPT-

The first command creates a regular expression for matching all bitcoin-config.h symbols in the following form: ^(?!//).*(AC_APPLE_UNIVERSAL_BUILD|BOOST_PROCESS_USE_STD_FS|...|_LARGE_FILES). It was generated with:
./autogen.sh && printf '^(?!//).*(%s)' $(awk '/^#undef/ {print $2}' src/config/bitcoin-config.h.in | paste -sd "|" -)

The second command holds a list of files and directories that should not be processed. These include subtree directories as well as some crypto files that already get their symbols through the makefile.

The third command checks for missing bitcoin-config headers in .cpp files and adds the header if it is missing.

The fourth command checks for missing bitcoin-config headers in .h files and adds the header if it is missing.

The fifth command checks for unneeded bitcoin-config headers in sources files and removes the header if it is unneeded.
2024-02-13 20:10:44 +00:00
furszy
987a1b51ee init: settings, do not load auto-generated warning msg
The settings warning message is meant to be used only to discourage
users from modifying the file manually. Therefore, there is no need
to keep it in memory.
2024-01-23 21:01:32 -03:00
furszy
e9014042a6 settings: add auto-generated warning msg for editing the file manually
Hopefully, refraining users from modifying the file unless they are
certain about the potential consequences.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-01-22 10:50:03 -03:00
furszy
966f5de99a init: improve corrupted/empty settings file error msg
The preceding "Unable to parse settings file" message lacked
the necessary detail and guidance for users on what steps to
take next in order to resolve the startup error.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-01-22 10:50:03 -03:00
stratospher
69e091f3e1 [init] Create deterministic addrman in tests using -test=addrman
Supposing there are 2 different addresses to be placed in an addrman
table. During every test run, a different [bucket,position] would be
calculated for each address. These calculated [bucket,position] could
end up being the same for the 2 different addresses in some test runs
and result in collisions in the addrman. We wouldn't be able to
predict when the collisions are going to happen because we can't
predict the nKey value which is chosen at random. This can cause
flaky tests.

Improve this by allowing deterministic addrman creation in the
functional tests. This creates an addrman with fixed `nKey` = 1 and
we can know the [bucket,position] collisions beforehand, safely add
more addresses in an addrman table and write more extensive tests.
2024-01-08 21:54:56 +05:30
stratospher
802e6e128b [init] Add new command line arg for use only in functional tests
some of the existing command line args are to be only used in
functional tests. ex: addrmantest, fastprune etc.. make a separate
category -test=<option> for these so that code is cleaner and
user's debug-help output is straightforward.
2024-01-08 21:54:55 +05:30