31278 Commits

Author SHA1 Message Date
MarcoFalke
c86f546f98
Merge bitcoin/bitcoin#23458: ci: Do not print git log for empty COMMIT_RANGE
095f07744cf500adc1f1587eb5b7a61df6e6b05f ci: Do not print `git log` for empty COMMIT_RANGE (Hennadii Stepanov)

Pull request description:

  On master (77a2f5d30c5ecb764b8a7c098492e1f5cdec90f0) a CI lint task [log](https://api.cirrus-ci.com/v1/task/4817858858319872/logs/lint.log) exceeds 20K lines.

  This PR fixes this issue.

ACKs for top commit:
  MarcoFalke:
    cr ACK 095f07744cf500adc1f1587eb5b7a61df6e6b05f

Tree-SHA512: 89180018aeccf1599cdf218924cbab12dcbae0f6674bb90e13b64e342cdd908a880b885039c23f0d1d03493e55a94fe04abf39481616ae6550c6a759f5ca9a35
2021-11-08 09:32:11 +01:00
MarcoFalke
f075e83b81
Merge bitcoin/bitcoin#23439: test: Open streams_test_tmp file in temporary folder
a04350b86c14a8038117aa013952b4e2b5f91cb0 Open streams_test_tmp file in temporary folder (Martin Leitner-Ankerl)

Pull request description:

  The tests `streams_tests/streams_buffered_file` and `streams_tests/streams_buffered_file_rand`
  did not use a the temporary directory provided by  `BasicTestingSetup`, so it was not possible
  to execute multiple of them in parallel. This fixes that.

  To reproduce, run

  ```sh
  parallel --halt now,fail=1 './src/test/test_bitcoin --run_test=streams_tests/streams_buffered_file_rand' -- ::: {1..1000}
  ```

  This executes the test 1000 times, one job per CPU. It works on that commit but mergebase fails quickly.

ACKs for top commit:
  theStack:
    Tested ACK a04350b86c14a8038117aa013952b4e2b5f91cb0

Tree-SHA512: 705b127e99a0fdbf33362283cc94036ac3aeff364a5e75b0f0a2490114bbaa44cf310ac13d43c254a4b51b72d4e856e9ee584904ce56b2f496b92d995ac7dc24
2021-11-08 09:28:10 +01:00
fanquake
ed40f37e00
Merge bitcoin/bitcoin#22076: build: Fix make apk if ccache enabled, and enable it on CI
15fb57556ee9a5f7332c9fff61edc0cc67634053 ci: Enable ccache for "ARM64 Android APK" job (Hennadii Stepanov)
7a777ec98c7136c1cf1f785140b9aa41765bf631 build: Fix `make apk` if ccache enabled (Hennadii Stepanov)

Pull request description:

  On master (456c8d6cd80fc3461957a3553a0483756396b988) `make apk` is broken for Android targets if the build system was configured with `--enable-ccache` explicitly or by default.

  This PR fixes this bug, and enables `ccache` for "ARM64 Android APK" Cirrus CI job.

ACKs for top commit:
  fanquake:
    ACK 15fb57556ee9a5f7332c9fff61edc0cc67634053

Tree-SHA512: e3c94dccc9f07a712e0e01983462ee607122623b213173c74d17e91024243239de212a35e5db94036bfd0a347e727b133865c26c67d6bd279604416ea971ab94
2021-11-08 15:58:48 +08:00
fanquake
ddc90293c1
Merge bitcoin/bitcoin#23464: doc: remove mention of system univalue from build-unix.md
78e36700a0b42f558af2be567eab9fbf6c9ef0b1 doc: remove mention of system univalue (fanquake)

Pull request description:

  Should have been part of #22646.

ACKs for top commit:
  hebasto:
    ACK 78e36700a0b42f558af2be567eab9fbf6c9ef0b1

Tree-SHA512: a5d54d73526033825ce4467cc3c57c26064739eef546556975a4c6f1f5bea84004640acd426734f90f98bc7a76ec837d716aa31167f2bdce7ee3887ad92e3152
2021-11-08 15:43:47 +08:00
fanquake
170e2687ba
Merge bitcoin/bitcoin#23446: doc: Mention that BerkeleyDB is for legacy wallet in build-unix
7eb5b25e6ec790577f84f48942c9e8791dec8697 doc: Mention that BerkeleyDB is for legacy wallet in build-unix (W. J. van der Laan)

Pull request description:

  This updates build-unix for the descriptor wallet, and prepares for eventual legacy wallet deprecation.

  - Move 'descriptor wallet' dependencies above legacy wallet deps both for Debian and Fedora.
  - Explicitly mention 'legacy wallet' where referring to the BerkeleyDB wallet. Shorten BerkeleyDB instruction to a single paragraph.

ACKs for top commit:
  lsilva01:
    ACK 7eb5b25
  hebasto:
    ACK 7eb5b25e6ec790577f84f48942c9e8791dec8697, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 8a0859bfaf170a1c5a8b3e8f873d6b788f3b1cebd8d006569348c5b415d5ebbdfd56a5a294c509114975addb599da8a50c4be618b0545ff402065e948f692ba1
2021-11-08 15:42:23 +08:00
Hennadii Stepanov
15fb57556e
ci: Enable ccache for "ARM64 Android APK" job 2021-11-08 07:19:21 +02:00
Hennadii Stepanov
7a777ec98c
build: Fix make apk if ccache enabled 2021-11-08 07:19:10 +02:00
fanquake
78e36700a0
doc: remove mention of system univalue
Should have been part of #22646.
2021-11-08 10:06:40 +08:00
fanquake
f21ce0bbbe
Merge bitcoin/bitcoin#23450: doc: update SECURITY.md inline with recent changes to bitcoincore.org
e7c0d506d50d359c47b2d2b2d19c9c0b3350dea5 doc: update SECURITY.md inline with recent changes to bitcoincore.org (fanquake)

Pull request description:

  See https://github.com/bitcoin-core/bitcoincore.org/pull/809 and https://github.com/bitcoin-core/bitcoincore.org/pull/819 and https://bitcoincore.org/en/contact/.

ACKs for top commit:
  laanwj:
    ACK e7c0d506d50d359c47b2d2b2d19c9c0b3350dea5, also checked that fingerprint matches
  kristapsk:
    ACK e7c0d506d50d359c47b2d2b2d19c9c0b3350dea5, checked that fingerprint matches.
  Zero-1729:
    ACK e7c0d506d50d359c47b2d2b2d19c9c0b3350dea5, fingerprint matches.

Tree-SHA512: 24aeb5c107db55a006b848bfd2c0f644219975acbd21f30749aff6a506832471d43de388100bbb195d48bca6f799e95e6fd2c54e06fc26c1a051bdcfb3db0982
2021-11-08 09:47:33 +08:00
fanquake
12f2b6ac01
Merge bitcoin/bitcoin#23453: Remove the build_msvc/testconsensus project
bb1c84082c8a8709fec201f5768deab22c938241 Remove the build_msvc/testconsensus project (Aaron Clauson)

Pull request description:

  The testconsensus project is not integral to the Bitcoin Core code. It was originally added as a quick and dirty demo of how to do a consensus check with the msvc build. There are better examples.

  PR #23438 made a change that caused a compiler error with the buildmsvc/testconsensus code. Rather than leave it hanging around to incur potential bitrot, or further PR build failures, it should be removed.

ACKs for top commit:
  hebasto:
    ACK bb1c84082c8a8709fec201f5768deab22c938241, tested on Windows 10 Pro (20H2).

Tree-SHA512: d81b99eb09171b66c179961b15f0b2e2e97e5ee7f011f18667e890c90e3d169593ad9aedd05a8616e962212952048827b7159d3c2a2ecb7ac378136b80bf6b23
2021-11-08 09:43:48 +08:00
Hennadii Stepanov
095f07744c
ci: Do not print git log for empty COMMIT_RANGE 2021-11-07 17:04:06 +02:00
Aaron Clauson
bb1c84082c
Remove the build_msvc/testconsensus project
The testconsensus project is not integral to the Bitcoin Core code. It was originally added as a quick and dirty demo of how to do a consensus check with the msvc build. There are better examples.

PR #23438 made a change that caused a compiler error with the buildmsvc/testconsensus code. Rather than leave it hanging around to incur potential bitrot, or furhter PR build failures, it should be removed.
2021-11-06 21:32:34 +00:00
fanquake
e7c0d506d5
doc: update SECURITY.md inline with recent changes to bitcoincore.org
See https://github.com/bitcoin-core/bitcoincore.org/pull/809 and
https://github.com/bitcoin-core/bitcoincore.org/pull/819 and
https://bitcoincore.org/en/contact/.
2021-11-06 18:05:07 +08:00
W. J. van der Laan
7eb5b25e6e doc: Mention that BerkeleyDB is for legacy wallet in build-unix
This updates build-unix for the descriptor wallet, and prepares for
eventual legacy wallet deprecation.

- Move 'descriptor wallet' dependencies above legacy wallet deps both
for Debian and Fedora.
- Explicitly mention 'legacy wallet' where referring to the BerkeleyDB
wallet. Shorten BerkeleyDB instruction to a single paragraph.
2021-11-05 19:27:17 +01:00
MarcoFalke
77a2f5d30c
Merge bitcoin/bitcoin#23334: fuzz: Descriptor wallet
11115169a14d0d0be5b7b1c3f6fdc9673a9098d9 ci: Build fuzz with libsqlite3-dev (MarcoFalke)
fa7c6efca66627e4c76adecc824f96da220af69c fuzz: Add wallet fuzz test (MarcoFalke)
fa59d2ce5b8d6fe8c610f170a13675c756aef58f refactor: Use local args instead of global gArgs in CWallet::Create (MarcoFalke)
fadb44606f26a80daf4320eee046c9572e85fe3e build: Inline FUZZ_SUITE_LDFLAGS_COMMON (MarcoFalke)

Pull request description:

  Initial sketch to fuzz descriptor wallets. Can be improved in the future.

ACKs for top commit:
  mjdietzx:
    Code review ACK 1111516

Tree-SHA512: b1d2f24504d1ed5f3c6a031210f04c27c13d4e15576c4acbf50ded37ac45f7b7a5c7553e91d81d4a06e9ea73b3d745a552218d3ef3b2932fa5325a8331b0d3fd
2021-11-05 10:29:21 +01:00
Martin Leitner-Ankerl
a04350b86c
Open streams_test_tmp file in temporary folder
The tests `streams_tests/streams_buffered_file` and `streams_tests/streams_buffered_file_rand`
did not use a the temporary directory provided by  `BasicTestingSetup`, so it was not possible
to execute multiple of them in parallel. This fixes that.

To reproduce, run

```sh
parallel --halt now,fail=1 './src/test/test_bitcoin --run_test=streams_tests/streams_buffered_file_rand' -- ::: {1..1000}
```

This executes the test 1000 times, one job per CPU. It works on that commit but mergebase fails quickly.
2021-11-04 18:47:14 +01:00
Samuel Dobson
24abd8312e
Merge bitcoin/bitcoin#22949: fee: Round up fee calculation to avoid a lower than expected feerate
80dc829be7f8c3914074b85bb4c125baba18cb2c tests: Calculate fees more similarly to CFeeRate::GetFee (Andrew Chow)
ce2cc44afd51f3df4ee7f14ea05b8da229183923 tests: Test for assertion when feerate is rounded down (Andrew Chow)
0fbaef9676a1dcb84bcf95afd8d994831ab327b6 fees: Always round up fee calculated from a feerate (Andrew Chow)

Pull request description:

  When calculating the fee for a feerate, it is possible that the final calculation will have fractional satoshis. Currently those are ignored via truncation which results in the absolute fee being rounded down. Rounding down is problematic because it results in a feerate that is slightly lower than the feerate represented by the `CFeeRate` object. A slightly lower feerate particularly causes issues for coin selection as it can trigger an assertion error. To avoid potentially underpaying the feerate (and the assertion), always round up the calculated fee.

  A test is added for the assertion, along with a comment explaining what happens.

  It is unlikely that a user can trigger this as it requires a very specific set of rounding errors to occur as well as the transaction not needing any change and being right on the lower bound of the exact match window. However I was able to trigger the assertion while running coin selection simulations, albeit after thousands of transactions and with some weird feerates.

ACKs for top commit:
  ryanofsky:
    Code review ACK 80dc829be7f8c3914074b85bb4c125baba18cb2c
  promag:
    Tested ACK 80dc829be7f8c3914074b85bb4c125baba18cb2c.
  lsilva01:
    tACK 80dc829
  meshcollider:
    utACK 80dc829be7f8c3914074b85bb4c125baba18cb2c

Tree-SHA512: fe26684c60f236cab48ea6a4600c141ce766dbe59504ec77595dcbd7fd0b34559acc617007f4f499c9155d8fda0a336954413410ba862b19c765c0cfac79d642
2021-11-05 00:08:00 +13:00
MarcoFalke
23ae7931be
Merge bitcoin/bitcoin#23154: doc: add assumeutxo notes
9ab440199d5c888363a42c957433d0e46cd0d2ff doc: add assumeutxo notes (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11) (parent PR: #15606)

  ---

  Adds some notes on assumeutxo design.

  Related: https://github.com/bitcoin/bitcoin/pull/21526#discussion_r715558994

ACKs for top commit:
  ariard:
    ACK 9ab4401
  naumenkogs:
    ACK 9ab4401
  michaelfolkson:
    ACK 9ab440199d5c888363a42c957433d0e46cd0d2ff
  fjahr:
    ACK 9ab440199d5c888363a42c957433d0e46cd0d2ff

Tree-SHA512: 2fca8373b78701754957d12bc43ce18aa6928507965448741cb4e8c56589ad61d261f8542e348094fc9631d46ee6a7afee75c965c0db993fc816758569137b74
2021-11-03 13:01:53 +01:00
MarcoFalke
e2b5192d1c
Merge bitcoin/bitcoin#23211: refactor: move update_* structs from txmempool.h to .cpp file
65aaf9495d19ea3fb875228a7e14aab6c1f2986d refactor: move `update_*` structs from txmempool.h to .cpp file (Sebastian Falbesoner)
9947ce62626c05bd186ae8a4864aa382f673ec1a refactor: use const reference for parents in `CTxMemPool::UpdateAncestorsOf` (Sebastian Falbesoner)

Pull request description:

  These helpers are exclusively used in txmempool.cpp, hence they should also be moved there. The PR also contains a commit which fixes const-correctness for parents in `CTxMemPool::UpdateAncestorsOf` and declares them as reference to avoid a copy.

ACKs for top commit:
  promag:
    Code review ACK 65aaf9495d19ea3fb875228a7e14aab6c1f2986d. Verified move-only commit locally.

Tree-SHA512: 7ce29f3ba0e68b5355001f27725b00f6d54cc993015356eb40b61b8cdd17db49b980f4c3d798c8e0c940d245dc3a72c474bb9ff3c0ee971ead450786076812c2
2021-11-03 08:59:03 +01:00
MarcoFalke
3c4729a515
Merge bitcoin/bitcoin#23223: Disable lock contention logging in checkqueue_tests
6ae9f1cf9604227e9dfda1f6c91fc711d154362e Disable lock contention logging in checkqueue_tests (Jon Atack)

Pull request description:

  This patch disables lock contention logging in the checkqueue_tests as some of these tests are designed to be heavily contested to trigger race conditions or other issues. This created very large log files when run with DEBUG_LOCKCONTENTION defined (up to v22) or with lock logging enabled by default in current master.

  Examples running the following command:

  ```
  $ ./src/test/test_bitcoin -t checkqueue_tests/test_CheckQueue_Correct_Random -- DEBUG_LOG_OUT > testlog.txt

  -rw-r--r--   87042178 Oct  8 12:41 testlog-with-DEBUG_LOCKCONTENTION-at-v22-run1.txt
  -rw-r--r--   73879896 Oct  8 12:42 testlog-with-DEBUG_LOCKCONTENTION-at-v22-run2.txt
  -rw-r--r--   65150518 Oct  8 12:51 testlog-with-DEBUG_LOCKCONTENTION-at-bb9f76a-run1.txt
  -rw-r--r--   65774554 Oct  8 12:52 testlog-with-DEBUG_LOCKCONTENTION-at-bb9f76a-run2.txt
  -rw-r--r--   73493309 Oct  8 13:00 testlog-current-master-at-991753e-run1.txt
  -rw-r--r--   65616977 Oct  8 13:01 testlog-current-master-at-991753e-run2.txt
  -rw-r--r--       5093 Oct  8 13:04 testlog-with-this-commit-run1.txt
  -rw-r--r--       5093 Oct  8 13:05 testlog-with-this-commit-run2.txt
  ```

  Resolves #23167.

ACKs for top commit:
  vasild:
    ACK 6ae9f1cf9604227e9dfda1f6c91fc711d154362e

Tree-SHA512: b16812ed60c58a1cf40c04ebeca9197ac076b2415f71673ac7bb5b7960a1ff80ba2c909345ad221c7689b0562d17f63a32a629f5d6dbcf0e57130bf5760388c1
2021-11-02 20:54:19 +01:00
MarcoFalke
9e3f7dcaa2
Merge bitcoin/bitcoin#22735: [net] Don't return an optional from TransportDeserializer::GetMessage()
f3e451bebfe2e2d8de901d8ac29c064a51d3b746 [net] Replace GetID() with id in TransportDeserializer constructor (Troy Giorshev)
8c96008ab18075abca03bff6b3675643825a21ca [net] Don't return an optional from TransportDeserializer::GetMessage() (Troy Giorshev)

Pull request description:

  Also, access mapRecvBytesPerMsgCmd with `at()` not `find()`. This
  throws an error if COMMAND_OTHER doesn't exist, which should never
  happen. `find()` instead just accessed the last element, which could make
  debugging more difficult.

  Resolves review comments from PR19107:

  - https://github.com/bitcoin/bitcoin/pull/19107#discussion_r478718436
  - https://github.com/bitcoin/bitcoin/pull/19107#discussion_r478714497

ACKs for top commit:
  theStack:
    Code-review ACK f3e451bebfe2e2d8de901d8ac29c064a51d3b746
  ryanofsky:
    Code review ACK f3e451bebfe2e2d8de901d8ac29c064a51d3b746. Changes since last review in https://github.com/bitcoin/bitcoin/pull/20364#pullrequestreview-534369904 were simplifying by dropping the third commit, rebasing, and cleaning up some style & comments in the first commit.

Tree-SHA512: 37de4b25646116e45eba50206e82ed215b0d9942d4847a172c104da4ed76ea4cee29a6fb119f3c34106a9b384263c576cb8671d452965a468f358d4a3fa3c003
2021-11-02 13:40:09 +01:00
MarcoFalke
76886ce11e
Merge bitcoin/bitcoin#23410: doc: Add output script descriptors BIPs 380..386
c02a674e97b61f675c94341fca1bf232070e6991 doc: Add output script descriptors BIPs 380..386 (Hennadii Stepanov)

Pull request description:

  BIPs 380..385 are implemented as of v0.17.0.
  BIP 386 is implemented as of v22.0.

ACKs for top commit:
  sipa:
    ACK c02a674e97b61f675c94341fca1bf232070e6991
  jarolrod:
    ACK c02a674e97b61f675c94341fca1bf232070e6991
  shaavan:
    ACK c02a674e97b61f675c94341fca1bf232070e6991

Tree-SHA512: 40f0252d3aad08c61a8e1476d26a590dbcf3f9d66c1f8315d15d13feb17288cc25b9c75df5b938f77695eafaba847dacc0020a880ba6034a511e7c9b7f40fd8f
2021-11-02 09:41:46 +01:00
Hennadii Stepanov
c02a674e97
doc: Add output script descriptors BIPs 380..386 2021-11-01 16:20:35 +02:00
MarcoFalke
5adc5c0280
Merge bitcoin/bitcoin#23403: test: Fix segfault in the psbt_wallet_tests/psbt_updater_test
68018e4c3e76f7e5bebf5f90ffd972c7bf01e0a0 test: Avoid excessive locking of `cs_wallet` (Hennadii Stepanov)
7986faf2e09ea85b1d4564ce910f07a4c4de8685 test: Fix segfault in the psbt_wallet_tests/psbt_updater_test (Hennadii Stepanov)

Pull request description:

  The dcd6eeb64adb2b532f5003cbb86ba65b3c08a87b commit (bitcoin/bitcoin#23288) introduced an intermittent failure in the `psbt_wallet_tests/psbt_updater_test` unit test. See bitcoin/bitcoin#23368.

  The test failure can be easily made reproducible with the following patch:
  ```diff
  --- a/src/scheduler.cpp
  +++ b/src/scheduler.cpp
  @@ -57,6 +57,8 @@ void CScheduler::serviceQueue()
               Function f = taskQueue.begin()->second;
               taskQueue.erase(taskQueue.begin());

  +            UninterruptibleSleep(100ms);
  +
               {
                   // Unlock before calling f, so it can reschedule itself or another task
                   // without deadlocking:
  ```

  This PR implements an idea which was mentioned in the [comment](https://github.com/bitcoin/bitcoin/issues/23368#issuecomment-953796339):
  > Yes, as I said before this looks like a race where the wallet is deleted before stopping the scheduler: [#23368 (comment)](https://github.com/bitcoin/bitcoin/issues/23368#issuecomment-952808824)
  >
  > IIRC, the order should be:
  >
  >    * stop scheduler
  >
  >    * delete wallet
  >
  >    * delete scheduler

  The second commit introduces a refactoring with no behavior change.

  Fixes bitcoin/bitcoin#23368.

ACKs for top commit:
  mjdietzx:
    Code review ACK 68018e4c3e76f7e5bebf5f90ffd972c7bf01e0a0

Tree-SHA512: d9103f6252aab807453628159dec243bc543a2595eecaa04ec761dca3c2370085592c55d6f50967d69a4ac6e8b5827eec30dd9b025132c99b0bb9aa5911ad915
2021-11-01 14:24:22 +01:00
fanquake
3fc3641043
Merge bitcoin/bitcoin#22766: refactor: Clarify and disable unused ArgsManager flags
c5d7e34bd9a4ad752c5ec88032420e2e90ab17ab scripted-diff: disable unimplemented ArgsManager BOOL/INT/STRING flags (Russell Yanofsky)
b8c069b7a952e326d2d974cc671889d1a3b38aa4 refactor: Add explicit DISALLOW_NEGATION ArgsManager flag to clarify flag usage (Russell Yanofsky)
26a50ab322614bceb5bc62e2c282f83e5987bad8 refactor: Split InterpretOption into Interpret{Key,Value} functions (Russell Yanofsky)

Pull request description:

  This is preparation for #16545 or another PR implementing type validation for ArgsManager settings. It fixes misleading usages of existing flags, prevents flags from being similarly misused in the future, and allows validation logic to be added without breaking backwards compatibility.

  ---

  Currently, ALLOW_{INT|BOOL|STRING} flags don't do any real validation, so current uses of these flags are misleading and will also break backwards compatibility whenever these flags are implemented in a future PR (draft PR is #16545).

  An additional complication is that while these flags don't do any real settings validation, they do affect whether setting negation syntax is allowed.

  Fix this mess by disabling ALLOW_{INT|BOOL|STRING} flags until they are implemented, and adding an unambiguous DISALLOW_NEGATION flag. This is done in three commits, with the first commit cleaning up some code, the second commit adding the DISALLOW_NEGATION flag, and the next commit disabling the ALLOW_{INT|BOOL|STRING} flags.

  None of the changes affect behavior in any way.

ACKs for top commit:
  ajtowns:
    utACK c5d7e34bd9a4ad752c5ec88032420e2e90ab17ab
  promag:
    Code review ACK c5d7e34bd9a4ad752c5ec88032420e2e90ab17ab, which as the new argument `-legacy`.

Tree-SHA512: cad0e06361e8cc584eb07b0a1f8b469e3beea18abb458c4e43d9d16e9f301b12ebf1d1d426a407fbd96f99724ad6c0eae5be05c713881da7c55e0e08044674eb
2021-11-01 11:25:42 +08:00
fanquake
994aaaa88d
Merge bitcoin/bitcoin#23380: addrman: Fix AddrMan::Add() return semantics and logging
61ec0539b26a902a41a2602187a71f9dba3c6935 [MOVEONLY] reorder functions in addrman_impl.h and addrman.cpp (John Newbery)
2095df7b7bfcb9ab0c5710a93112f7f341e753c9 [addrman] Add Add_() inner function, fix Add() return semantics (John Newbery)
2658eb6d68460272deefb3fcc653b03f6ec6e7cf [addrman] Rename Add_() to AddSingle() (John Newbery)
e58598e833d5737900fe3c4369e26f2a08166892 [addrman] Add doxygen comment to AddrMan::Add() (John Newbery)

Pull request description:

  Previously, Add() would return true if the function created a new
  AddressInfo object, even if that object could not be successfully
  entered into the new table and was deleted. That would happen if the new
  table position was already taken and the existing entry could not be
  removed.

  Instead, return true if the new AddressInfo object is successfully
  entered into the new table. This fixes a bug in the "Added %i addresses"
  log, which would not always accurately log how many addresses had been
  added.

ACKs for top commit:
  naumenkogs:
    ACK 61ec0539b26a902a41a2602187a71f9dba3c6935
  mzumsande:
    ACK 61ec0539b26a902a41a2602187a71f9dba3c6935
  shaavan:
    ACK 61ec0539b26a902a41a2602187a71f9dba3c6935

Tree-SHA512: 276f1e8297d4b6d411d05d06ffc7c176f6290a784da039926ab6c471a8ed8e9159ab4f56c893b1285737ae292954930f0d28012d89dfb3f2f825d7df41016feb
2021-11-01 10:58:27 +08:00
Hennadii Stepanov
68018e4c3e
test: Avoid excessive locking of cs_wallet 2021-10-31 13:48:00 +02:00
Hennadii Stepanov
7986faf2e0
test: Fix segfault in the psbt_wallet_tests/psbt_updater_test
The bug was introduced in dcd6eeb64adb2b532f5003cbb86ba65b3c08a87b.
2021-10-31 13:48:00 +02:00
fanquake
7efc628539
Merge bitcoin/bitcoin#23385: refactor: get wallet path relative to wallet_dir
9ba7c44265a47880585e39d0167d057ba935ff16 refactor: get wallet path relative to wallet_dir (Michael Dietz)

Pull request description:

  Now that boost has been updated > 1.60 (see #22320), we can simplify how we get
  wallet path relative to wallet_dir by using:
  `boost::filesystem::lexically_relative`, removing a TODO.

  Test coverage comes from `test/functional/wallet_multiwallet.py`

  I first tried this in #20265 which was my first attempted PR, and funny enough exactly 1 year later I'm opening this one to hopefully finally close this.

ACKs for top commit:
  ryanofsky:
    Code review ACK 9ba7c44265a47880585e39d0167d057ba935ff16. Basically this same code change is made in #20744 commit b70c84348ac7a8e427a1183f894c73e52c734529, so this PR helps simplify that one
  lsilva01:
    Code Review ACK 9ba7c44

Tree-SHA512: 6ccb91a18bcb52c3ae0c789a94a18fb5be7db7769fd1121552d63f259fbd32b50c3dcf169cec0b02f978321db3bc60eb4b881b8327e9764f32e700236e0d8a35
2021-10-30 15:20:27 +08:00
Michael Dietz
9ba7c44265
refactor: get wallet path relative to wallet_dir
Now that boost has been updated > 1.60, we can simplify how we get
wallet path relative to wallet_dir by using:
`boost::filesystem::lexically_relative`
2021-10-29 09:36:32 -05:00
MarcoFalke
5574881ce3
Merge bitcoin/bitcoin#23354: Introduce new V4 format addrman
d891ae768185b464cae476c16c74c365372d4a3c Introduce new V4 format addrman (Pieter Wuille)

Pull request description:

  #23306 effectively changed the on-disk format in an incompatible way: old deserializers cannot deal with multiple entries for the same IP.

  Introduce a `V4_MULTIPORT` format, and increment the compatibility base, so that old versions correctly recognize it as an incompatible future version, rather than corruption.

ACKs for top commit:
  naumenkogs:
    ACK d891ae768185b464cae476c16c74c365372d4a3c
  ajtowns:
    utACK d891ae768185b464cae476c16c74c365372d4a3c
  vasild:
    ACK d891ae768185b464cae476c16c74c365372d4a3c

Tree-SHA512: de2153beb59152504ee0656dd0cc0b879b09136eb07e3ce0426d2fea778adfabacebbce5cf1a9a65dc99ad4e99cda42ab26743fe672fb82a9fbfec49c4cccb4d
2021-10-29 13:13:03 +02:00
MarcoFalke
8bac3b1096
Merge bitcoin/bitcoin#23375: test: MiniWallet: more deterministic coin selection for coinbase UTXOs (oldest first)
d2c4904ef707e2023ceb8dfbfe61a92c4060e100 test: MiniWallet: more deterministic coin selection for coinbase UTXOs (oldest first) (Sebastian Falbesoner)

Pull request description:

  The coin selection strategy for MiniWallet is quite straight-forward: simply pick a single UTXO with the largest value:

  ab25ef8c7f/test/functional/test_framework/wallet.py (L173-L174)

  If there are several candidates with the same value, however, it is not clear which one is taken.  This can be particularly problematic for coinbase outputs with fixed block subsidy, since spending could lead to a `bad-txns-premature-spend-of-coinbase` reject if an UTXO from a too-recent block is picked.  Introduce block height as second criteria (saved in `self._utxos` in the methods `generate(...)` and `rescan_utxos(...)`), in order to avoid potential issues with coinbases that are not matured yet. If there is a tie between coinbase UTXOs and non-coinbase UTXOs (the latter are added via `scan_tx(...)`), prefer the non-coinbase UTXOs, since those don't need to mature.

  The issue came up while refactoring the test rpc_blockchain.py, see https://github.com/bitcoin/bitcoin/pull/23371#discussion_r737401936 (PR #23371).

ACKs for top commit:
  MarcoFalke:
    review ACK d2c4904ef707e2023ceb8dfbfe61a92c4060e100
  shaavan:
    ACK d2c4904ef707e2023ceb8dfbfe61a92c4060e100

Tree-SHA512: 15d67b42fb8b77fd53022ea2ab8a6ed2b615567f3ce73bab16c06bfcb687c1a04dcb0360d0c2287c526b604cd3ac5eef7b14ce46fc31e23047ce1a3290027306
2021-10-29 12:55:51 +02:00
MarcoFalke
c426e0dc6f
Merge bitcoin/bitcoin#22972: test: fix misleading fee unit in mempool_limit.py
2600db6c36c11bf49a0a113ee2e2274406ade61c test: fix misleading fee unit in mempool_limit.py (Sebastian Falbesoner)

Pull request description:

  The PR is a follow-up to #22543. The helper `send_large_txs` in its current interface has a fee_rate parameter, implying that it would create a transaction with exactly that rate. Unfortunately, this fee rate is only passed to MiniWallet's `create_self_transfer` method, which can't know that we append several tx outputs after, increasing the tx's vsize and decreasing it's fee rate accordingly.

  In our case, the fee rate is off by several orders of magnitude, as the tx's vsize changes changes from 96 to 67552 vbytes (>700x), i.e. the value passed to this function is neither really a fee rate nor an absolute fee, but something in-between, which is very confusing. It was suggested to simply in-line this helper as it's currently only used in this single test (https://github.com/bitcoin/bitcoin/pull/22543#discussion_r701685136, https://github.com/bitcoin/bitcoin/pull/22543#issuecomment-918986896), but I could imagine that this helper may also become useful for other tests and may be moved to a library (e.g. wallet.py) in the future.

  Clarify the interface by passing an absolute fee that is deducted in the end (and also verified, via testmempoolaccept) and also describe how we come up with the value passed. On master, the comment says that the fee rate needs to increased "massively"; this word is also removed because the fee rate only needs to be higher for the test to succeed.

ACKs for top commit:
  stratospher:
    ACK 2600db6.

Tree-SHA512: 0bfacc3fa87603970d86c1d0186e51511f6c20c64b0559e19e7e12a68647f79dcb4f436000dee718fd832ce6a68e3bbacacb29145e0287811f1cb03d2f316843
2021-10-29 12:33:04 +02:00
MarcoFalke
baa9fc941c
Merge bitcoin/bitcoin#22787: refactor: actual immutable pointing
54011e7aa274bdc1b921440cc8b4623aa1e0d89e refactor: use CWallet const shared pointers when possible (Karl-Johan Alm)
96461989a2de737151bc4fb216221bf49cb53ce6 refactor: const shared_ptrs (Karl-Johan Alm)

Pull request description:

  ```C++
  const std::shared_ptr<CWallet> wallet = x;
  ```
  means we can not do `wallet = y`, but we can totally do `wallet->DestructiveOperation()`, contrary to what that line looks like.

  This PR

  * introduces a new convention: always use const shared pointers to `CWallet`s (even when we mutate the pointed-to thing)
  * uses `const shared_ptr<const CWallet>` everywhere where wallets are not modified

  In the future, this should preferably apply to all shared pointers, not limited to just `CWallet`s.

  Both of these serve the same purpose: to dispell the misconception that `const shared_ptr<X>` immutates `X`. It doesn't, and it's dangerous to leave this misconception as is, for obvious reasons.

ACKs for top commit:
  theStack:
    re-ACK 54011e7aa274bdc1b921440cc8b4623aa1e0d89e

Tree-SHA512: 3bf4062fc821751be30770c6b4ead10a016847970f155a0a5156f304347d221b9830840030c2fbfba8cd1e282f4eda45f5b4107fe6df8138afdcb6c2e95a2836
2021-10-29 10:52:37 +02:00
John Newbery
61ec0539b2 [MOVEONLY] reorder functions in addrman_impl.h and addrman.cpp
Keep the internal {Function}_() functions grouped together.

Review with `git diff --color-moved=dimmed-zebra`
2021-10-28 14:01:40 +01:00
John Newbery
2095df7b7b [addrman] Add Add_() inner function, fix Add() return semantics
Previously, Add() would return true if the function created a new
AddressInfo object, even if that object could not be successfully
entered into the new table and was deleted. That would happen if the new
table position was already taken and the existing entry could not be
removed.

Instead, return true if the new AddressInfo object is successfully
entered into the new table. This fixes a bug in the "Added %i addresses"
log, which would not always accurately log how many addresses had been
added.

p2p_addrv2_relay.py and p2p_addr_relay.py need to be updated since they
were incorrectly asserting on the buggy log (assuming that addresses are
added to addrman, when there could in fact be new table position
collisions that prevent some of those address records from being added).
2021-10-28 14:00:21 +01:00
MarcoFalke
11115169a1
ci: Build fuzz with libsqlite3-dev 2021-10-28 13:32:47 +01:00
John Newbery
2658eb6d68 [addrman] Rename Add_() to AddSingle() 2021-10-28 12:52:27 +01:00
John Newbery
e58598e833 [addrman] Add doxygen comment to AddrMan::Add()
Does not document the return value since we're going to fix the
semantics in a future commit.
2021-10-28 12:51:19 +01:00
Sebastian Falbesoner
d2c4904ef7 test: MiniWallet: more deterministic coin selection for coinbase UTXOs (oldest first)
The coin selection strategy for MiniWallet is quite straight-forward: simply
pick a single UTXO with the largest value:

self._utxos = sorted(self._utxos, key=lambda k: k['value'])
utxo_to_spend = utxo_to_spend or self._utxos.pop()

If there are several candidates with the same value, however, it is not clear
which one is taken.  This can be particularly problematic for coinbase outputs
with fixed block subsidy, since spending could lead to a
'bad-txns-premature-spend-of-coinbase' reject if an UTXO from a too-recent
block is picked.  Introduce block height as second criteria (saved in
self._utxos in the methods generate(...) and rescan_utxos(...)), in order to
avoid potential issues with coinbases that are not matured yet.
2021-10-27 17:49:53 +02:00
MarcoFalke
ab25ef8c7f
Merge bitcoin/bitcoin#23305: test: refactor: add script_util helper for creating bare multisig scripts
4718897ce3a7c728ff7aebbadabcc8ed7a0b8d6e test: add script_util helper for creating bare multisig scripts (Sebastian Falbesoner)

Pull request description:

  This PR is a follow-up to #22363 and #23118 and introduces a helper `keys_to_multisig_script` for creating bare multisig outputs in the form of
  ```
  OP_K PubKey1 PubKey2 ... PubKeyN OP_N OP_CHECKMULTISIG
  ```
  The function takes a list of pubkeys (both hex- and byte-strings are accepted due to the `script_util.check_key` helper being used internally) and optionally a threshold _k_. If no threshold is passed, a n-of-n multisig output is created, with _n_ being the number of passed pubkeys.

ACKs for top commit:
  shaavan:
    utACK 4718897ce3a7c728ff7aebbadabcc8ed7a0b8d6e
  rajarshimaitra:
    tACK 4718897ce3

Tree-SHA512: b452d8a75b0d17316b66ac4ed4c6893fe59c7c417719931d4cd3955161f59afca43503cd09b83a35b5a252a122eb3f0fbb9da9f0e7c944cf8da572a02219ed9d
2021-10-27 11:49:01 +01:00
MarcoFalke
e77d9679fd
Merge bitcoin/bitcoin#23006: multiprocess: Add new bitcoin-gui, bitcoin-qt, bitcoin-wallet init implementations
d5f985e51f2863fda0c0d632e836eba42a5c3e15 multiprocess: Add new bitcoin-gui, bitcoin-qt, bitcoin-wallet init implementations (Russell Yanofsky)

Pull request description:

  Add separate `interfaces::Init` subclasses for `bitcoin-wallet`,  `bitcoin-gui`, and `bitcoin-qt` binaries instead of sharing `bitcoind` and `bitcoin-node` init subclasses in different binaries. After this, the new init subclasses can be customized in #10102, so node and wallet code is dropped from the `bitcoin-gui` binary and wallet code is dropped from into the `bitcoin-node` binary.

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

ACKs for top commit:
  lsilva01:
    reACK d5f985e
  hebasto:
    re-ACK d5f985e51f2863fda0c0d632e836eba42a5c3e15, only suggested changes since my [previous](https://github.com/bitcoin/bitcoin/pull/23006#pullrequestreview-787537444) review.

Tree-SHA512: 6784210bd9ce3a6fbc66852680d0e9bc513c072dc538aeb7f48cb6a41580d3f567ccef04f975ee767a714a4b05d4d87273e94a79abda1b9c25d5ac4bbe752006
2021-10-26 15:54:52 +01:00
Sebastian Falbesoner
4718897ce3 test: add script_util helper for creating bare multisig scripts 2021-10-26 14:51:18 +02:00
MarcoFalke
af4275e8db
Merge bitcoin/bitcoin#23332: doc: Fix CWalletTx::Confirmation doc
fa8fef6ef2287cd36ae14fbf10e852ddef7e62ac doc: Fix CWalletTx::Confirmation doc (MarcoFalke)

Pull request description:

  Follow-up to:
  * commit 700c42b85d20e624bef4228eef062c93084efab5, which replaced pIndex
    with block_hash in AddToWalletIfInvolvingMe.

  * commit 9700fcb47feca9d78e005b8d18b41148c8f6b25f, which replaced
    posInBlock with confirm.nIndex.

ACKs for top commit:
  laanwj:
    Code review ACK fa8fef6ef2287cd36ae14fbf10e852ddef7e62ac
  ryanofsky:
    Code review ACK fa8fef6ef2287cd36ae14fbf10e852ddef7e62ac

Tree-SHA512: e7643b8e9200b8ebab3eda30435ad77006568a03476006013c9ac42c826c84e42e29bcdefc6f443fe4d55e76e903bfed5aa7a51f831665001c204634b6f06508
2021-10-26 13:50:15 +01:00
Pieter Wuille
d891ae7681 Introduce new V4 format addrman
92617b7a758c0425330fba4b886296730567927c effectively changed the
on-disk format in an incompatible way: old deserializers cannot
deal with multiple entries for the same IP.

Introduce a V4_MULTIPORT format, and increment the compatibility base,
so that old versions correctly recognize it as an incompatible future
version.
2021-10-25 13:48:21 -04:00
Russell Yanofsky
c5d7e34bd9 scripted-diff: disable unimplemented ArgsManager BOOL/INT/STRING flags
This commit does not change behavior in any way. See previous commit for
complete rationale, but these flags are being disabled because they
aren't implemented and will otherwise break backwards compatibility when
they are implemented.

-BEGIN VERIFY SCRIPT-
sed -i 's:\(ALLOW_.*\)   \(//!< unimplemented\):// \1\2:' src/util/system.h
sed -i '/DISALLOW_NEGATION.*scripted-diff/d' src/util/system.cpp
git grep -l 'ArgsManager::ALLOW_\(INT\|STRING\)'  | xargs sed -i 's/ArgsManager::ALLOW_\(INT\|STRING\)/ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION/g'
git grep -l 'ALLOW_BOOL' -- ':!src/util/system.h' | xargs sed -i 's/ALLOW_BOOL/ALLOW_ANY/g'
-END VERIFY SCRIPT-
2021-10-25 10:44:17 -04:00
MarcoFalke
22a9018649
Merge bitcoin/bitcoin#23306: Make AddrMan support multiple ports per IP
92617b7a758c0425330fba4b886296730567927c Make AddrMan support multiple ports per IP (Pieter Wuille)

Pull request description:

  For a long part of Bitcoin's history, this codebase has aggressively avoided making automatic connections to anything but nodes running on port 8333. I'd like to propose changing that, and this is a first PR necessary for that.

  The folklore justification (eventually actually added as a comment to the codebase in #20668) is that this is to prevent the Bitcoin P2P network from being leveraged to perform a DoS attack on other services, if their IP/port would get rumoured. It appears, at least the current network scale - and probably significantly larger - that the impact is very low at best (see calculations by vasild in https://github.com/bitcoin/bitcoin/issues/5150#issuecomment-853888909 e.g.). Another possible justification would be a risk that treating different IP:port combinations separately would help perform Eclipse attacks (by an attacker rumouring their own IP with many ports). This concern is (a) no different than what is possible with IPv6 (where large ranges of IP addresses are very cheaply available), and (b) already hopefully sufficiently addressed by addrman's design (which limits access through based selected based on network groups).

  And this policy has downsides too; in particular, a fixed port is easy to detect, and a very obvious sign a Bitcoin node is running there.

  One obstacle in moving away from a default port that is the fact that addrman is currently restricted to a single entry per IP address. If ports are no longer expected to be generally always the default one, we need to deal with the case where conflicting information is relayed. It turns out there is a very natural solution to this: treat (IP,port) combination exactly as we're treating IPs now; this automatically means that the same IP may appear with multiple ports, simply because those would be distinct entries. Given that indexing into addrman's bucket _already_ uses the port number, the only change required is making all addrman lookup be (IP,port) (aka `CService`) based, rather than IP (aka `CNetAddr`) based.

  This PR doesn't include any change to the actual outbound connection preference logic, as perhaps that's something that we want to phase in more gradually.

ACKs for top commit:
  jnewbery:
    Code review ACK 92617b7a758c0425330fba4b886296730567927c
  naumenkogs:
    ACK 92617b7a758c0425330fba4b886296730567927c
  ajtowns:
    ACK 92617b7a758c0425330fba4b886296730567927c
  vasild:
    ACK 92617b7a758c0425330fba4b886296730567927c

Tree-SHA512: 9eef06ce97a8b54a3f05fb8acf6941f253a9a5e0be8ce383dd05c44bb567cea243b74ee5667178e7497f6df2db93adab97ac66edbc37c883fd8ec840ee69a33f
2021-10-25 16:44:17 +02:00
Russell Yanofsky
b8c069b7a9 refactor: Add explicit DISALLOW_NEGATION ArgsManager flag to clarify flag usage
Currently, ALLOW_{INT|BOOL|STRING} flags don't do any real validation,
so current uses of these flags are misleading and will also break
backwards compatibility whenever these flags are implemented in a future
PR (draft PR is #16545).

An additional complication is that while these flags don't do any real
settings validation, they do affect whether setting negation syntax is
allowed.

Fix this mess by disabling ALLOW_{INT|BOOL|STRING} flags until they are
implemented, and adding an unambiguous DISALLOW_NEGATION flag. This is
done in two commits, with this commit adding the DISALLOW_NEGATION flag,
and the next commit disabling the ALLOW_{INT|BOOL|STRING} flags.
2021-10-25 10:44:17 -04:00
Russell Yanofsky
26a50ab322 refactor: Split InterpretOption into Interpret{Key,Value} functions
Co-authored-by: Anthony Towns <aj@erisian.com.au>
2021-10-25 10:44:17 -04:00
MarcoFalke
d565d9b56d
Merge bitcoin/bitcoin#23312: tests: reduce feature_segwit.py usage of the legacy wallet
e9ade032f3283971025943d750b1305bc8da56fc tests: Add feature_segwit.py --descriptors to test_runner.py (Andrew Chow)
ae6cbcc90926b65099ed8747e7a13a4aefba787a tests: restrict feature_segwit legacy wallet import tests (Andrew Chow)
1d13c44a4cb02b0acef6c6a108410cfc7eb20f71 tests: Use descriptors for feature_segwit multisig setup (Andrew Chow)

Pull request description:

  `feature_segwit.py` has tests for some legacy wallet behavior, but otherwise does not really need the legacy wallet. Those parts now require `--legacy-wallet` to be provided (as with other legacy wallet tests). Other parts of the test are changed to work with descriptor wallets as well.

ACKs for top commit:
  lsilva01:
    tACK e9ade03

Tree-SHA512: 4ae76a4d2a8d318e7f8ad4c984748e3cdd67ed00359fcd798a08492ed9399b1d01be88c9ebea3d6c996fbe190cf41708a15c98f088f0cb5c47d6d00fff258944
2021-10-25 15:32:27 +02:00