38326 Commits

Author SHA1 Message Date
Andrew Chow
cd43a8444b
Merge bitcoin/bitcoin#27460: rpc: Add importmempool RPC
fa776e61cd64a5ffd9a4be589ab8efeb5421861a Add importmempool RPC (MarcoFalke)
fa20d734a29ba50cd19b78cb4fe39a2d826131b7 refactor: Add and use kernel::ImportMempoolOptions (MarcoFalke)
fa8866990dba7817427977bfe834efdb17114d37 doc: Clarify the getmempoolinfo.loaded RPC field documentation (MarcoFalke)
6888886cecf6665da70b3dc3772b3c12ef06ad76 Remove Chainstate::LoadMempool (MarcoFalke)

Pull request description:

  Currently it is possible to import a mempool by placing it in the datadir and starting the node. However this has many issues:

  * Users aren't expected to fiddle with the datadir, possibly corrupting it
  * An existing mempool file in the datadir may be overwritten
  * The node needs to be restarted
  * Importing an untrusted file this way is dangerous, because it can corrupt the mempool

  Fix all issues by adding a new RPC.

ACKs for top commit:
  ajtowns:
    utACK fa776e61cd64a5ffd9a4be589ab8efeb5421861a
  achow101:
    ACK fa776e61cd64a5ffd9a4be589ab8efeb5421861a
  glozow:
    reACK fa776e61cd64a5ffd9a4be589ab8efeb5421861a

Tree-SHA512: fcb1a92d6460839283c546c47a2d930c363ac1013c4c50dc5215ddf9fe5e51921d23fe0abfae0a5a7631983cfc7e2fff3788b70f95937d0a989a203be4d67546
2023-08-15 10:15:22 -04:00
fanquake
80d70cb6b0
Merge bitcoin/bitcoin#28185: ci: Use hard-coded root path for CI containers (bugfix)
fafa17c00b87c685200198979ed13018e55e474a ci: Use hard-coded root path for CI containers (MarcoFalke)
fa084f5ba5dad58a55b5a0a0e9d839c54e9a9238 ci: Only create folders when needed (MarcoFalke)
fab27127f4794b57a00328bdf8e779d9db07a2f5 ci: Drop BASE_SCRATCH_DIR from LIBCXX_DIR (MarcoFalke)

Pull request description:

  Currently the CI system will fail if the git folder that holds the Bitcoin Core source is moved from one location to another.

  Fix this by using a single hard-coded root path *inside* the CI system containers.

  Steps to test:

  * Run the CI system: `MAKEJOBS="-j$(nproc)" FILE_ENV="./ci/test/00_setup_env_win64.sh" ./ci/test_run_all.sh`
  * Move the git folder: `pwd && cd .. && mv bitcoin_core_folder_1 bitcoin_core_folder_2 && cd ./bitcoin_core_folder_2 && pwd`
  * Run the CI system again: (same cmd as above)

  On master (error):

  ```
  STRIPPROG="x86_64-w64-mingw32-strip" /bin/bash /bitcoin_core_folder_2/ci/scratch/build/bitcoin-x86_64-w64-mingw32/build-aux/install-sh -c -s ./src/qt/bitcoin-qt.exe ./release
  /bitcoin_core_folder_2/ci/scratch/build/bitcoin-x86_64-w64-mingw32/build-aux/install-sh: ./src/qt/bitcoin-qt.exe does not exist.
  make: *** [Makefile:1258: bitcoin-25.99.0-win64-setup.exe] Error 1
  ```

  On this pull: (pass).

ACKs for top commit:
  fanquake:
    ACK fafa17c00b - somewhat tested. MSAN changes are the same as what we did for tidy.

Tree-SHA512: 2ce693a3773c70fcfca062c2a6f0e5a16b94960b34a6145d10cee1a28f79154829d59d014465ccbb80e1cb9dcd5aa043729cee9afd2c4175b05e9bc945364b79
2023-08-15 11:17:39 +01:00
fanquake
85e672ab3d
Merge bitcoin/bitcoin#28269: ci: Drop no longer needed macos_sdk_cache
c2a87bd302585c242d0d66db72c6ce6c60f81b78 ci: Drop no longer needed `macos_sdk_cache` (Hennadii Stepanov)

Pull request description:

  It has been cached in the Docker image since https://github.com/bitcoin/bitcoin/pull/27028.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK c2a87bd302585c242d0d66db72c6ce6c60f81b78

Tree-SHA512: fba888d132910f9600db0acccf633400e699f7d5ca802ef109a792546f60f3a04791c503b43d156c8debc3bd0bce2ad911f6209eabc47cd63fce7996a6ae3cfc
2023-08-15 11:13:44 +01:00
fanquake
5606d7f5a8
Merge bitcoin/bitcoin#28267: crypto: BIP324 ciphersuite follow-up
93cb8f03807dcd3297b7fe18b6c901a8d2a01596 refactor: add missing headers for BIP324 ciphersuite (stratospher)
d22d5d925c000bf25ad2410ca66c4c21eea75004 crypto: BIP324 ciphersuite follow-up (stratospher)

Pull request description:

  follow-up to #28008.
  * move `dummy_tag` variable in FSChaCha20Poly1305 crypto_tests outside of the loop to be reused every time
  * use easy to read `cipher.last()` in `AEADChaCha20Poly1305::Decrypt()`
  * comment for initiator in `BIP324Cipher::Initialize()`
  * systematically damage ciphertext with bit positions in bip324_tests
  * use 4095 max bytes for `aad` in bip324 fuzz test

ACKs for top commit:
  fanquake:
    ACK 93cb8f03807dcd3297b7fe18b6c901a8d2a01596 - thanks for following up here.

Tree-SHA512: 361f3e226d3168fdef69a2eebe6092cfc04ba14ce009420222e762698001eaf8be69a1138dab0be237964509c2b96a41a0b4db5c1df43ef75062f143c5aa741a
2023-08-15 11:11:55 +01:00
fanquake
e38c225261
Merge bitcoin/bitcoin#28215: fuzz: fix a couple incorrect assertions in the coins_view target
e417c988f61bf9d3948d5c8e169626922fe6e24c fuzz: coins_view: remove an incorrect assertion (Antoine Poinsot)
c5f6b1db56f67f529377bfb61f58c0a8c17b0127 fuzz: coins_view: correct an incorrect assertion (Antoine Poinsot)

Pull request description:

  The `coins_view` fuzz target would assert in two places that the cache is consistent with the backend. But it's never the case (that's the whole point of using a cache).

  The only reason this didn't result in a crash was that we would never actually hit these assertions. I ran into this while introducing a new target with an in-memory `CCoinsViewDB` as the backend view (see https://github.com/bitcoin/bitcoin/pull/28216) which made the code paths with those assertions actually reachable.

ACKs for top commit:
  dergoegge:
    Code review ACK e417c988f61bf9d3948d5c8e169626922fe6e24c

Tree-SHA512: 5847bb2744a2f2831dace62d32b79cc491bf54e2af4ce425411d245d566622d9aff816d9be5ec8e830d10851c13f2500bf4f0c004d88b4d7cca1d483ef8960a6
2023-08-15 11:05:42 +01:00
stratospher
93cb8f0380 refactor: add missing headers for BIP324 ciphersuite 2023-08-15 07:30:48 +05:30
Hennadii Stepanov
c2a87bd302
ci: Drop no longer needed macos_sdk_cache
It has been cached in the Docker image since https://github.com/bitcoin/bitcoin/pull/27028.
2023-08-14 17:02:06 +01:00
fanquake
aadaa5625e
Merge bitcoin/bitcoin#28232: test: locked_wallet, skip default fee estimation
5364dd8666ae1afa35536f9b4fc0170eeaf37888 test: locked_wallet, skip default fee estimation (furszy)

Pull request description:

  Coming from https://github.com/bitcoin/bitcoin/pull/28139#discussion_r1284563239.

  No test case in this file is meant to exercise fee estimation. All default wallets have a
  custom tx fee set [here](b7138252ac/test/functional/wallet_fundrawtransaction.py (L100)). The only one missing is the one created for `locked_wallet`.

ACKs for top commit:
  theStack:
    ACK 5364dd8666ae1afa35536f9b4fc0170eeaf37888

Tree-SHA512: 514c02708081d18330d759d10e306cee16c6350de243c68f0973777d2582f5d81968a237393c1f59aba245297e03f3f98d3ae5249a042469d0d016255f568719
2023-08-14 16:18:10 +01:00
fanquake
6c508ac3ff
Merge bitcoin/bitcoin#28258: bitcoin-tidy: fix macOS build
bb3263d3e3d9f9d4db86dde679f469e7278bf737 bitcoin-tidy: fix macOS build (Cory Fields)

Pull request description:

  [LLVM uses these options](https://github.com/llvm/llvm-project/blob/main/llvm/cmake/modules/HandleLLVMOptions.cmake#L178) for building as well, so there's precedent.

  Also fix the shared library extension which was incorrectly being set to dylib.

  Thanks to jonatack for reporting and debugging.

ACKs for top commit:
  jonatack:
    ACK bb3263d3e3d9f9d4db86dde679f469e7278bf737 tested with arm64 macos 13.5, llvm 16.0.6 and cmake 3.27.2

Tree-SHA512: de7bfd497f38f1565a14d217d0b057cbfa788bdda702b5942b7f0b55947ae5e1c05af13e7d6a073ed036bc4db57035868f180034508b6e084ab9b901a5baaf2f
2023-08-14 13:10:13 +01:00
stratospher
d22d5d925c crypto: BIP324 ciphersuite follow-up
follow-up to #28008.
* move `dummy_tag` variable in FSChaCha20Poly1305 crypto_tests
outside of the loop to be reused every time
* use easy to read `cipher.last()` in `AEADChaCha20Poly1305::Decrypt()`
* comment for initiator in `BIP324Cipher::Initialize()`
* systematically damage ciphertext with bit positions in bip324_tests
* use 4095 max bytes for aad in bip324 fuzz test
2023-08-14 09:03:21 +05:30
Antoine Poinsot
e417c988f6
fuzz: coins_view: remove an incorrect assertion
Again, this was not hit because the default implementation of
`CCoinsView` return `false` for `GetCoin`.
2023-08-11 18:11:07 +02:00
fanquake
3654d84c6f
Merge bitcoin/bitcoin#28245: doc: use llvm-config for bitcoin-tidy example
d82bb90a5b9dc1fd48b10514bdcd8f425aced256 doc: use llvm-config for bitcoin-tidy example (fanquake)

Pull request description:

  An LLVM installation will have `llvm-config` available to query for info. Ask it for the `--cmakedir`, and use that in our bitcoin-tidy example, rather than listing multiple different (potential) paths per distro/OS etc.

ACKs for top commit:
  theuni:
    ACK d82bb90a5b9dc1fd48b10514bdcd8f425aced256.
  jonatack:
    ACK d82bb90a5b9dc1fd48b10514bdcd8f425aced256
  TheCharlatan:
    Nice, Re-ACK d82bb90a5b9dc1fd48b10514bdcd8f425aced256

Tree-SHA512: e07e979231f8f000deafce0751bed4b73ff0eff995bec49e90f579c9051cf5859dac5e49554b8219d33b00c81192db979eed98fee1c643a9205ea8babfce2c5d
2023-08-11 11:36:38 +02:00
Cory Fields
bb3263d3e3 bitcoin-tidy: fix macOS build
LLVM uses these options for building as well, so there's precedent.

Also fix the shared library extension which was incorrectly being set to dylib
2023-08-10 21:14:50 +00:00
furszy
5364dd8666
test: locked_wallet, skip default fee estimation
Same as we do with the nodes default wallets.
No test case on this file is meant to exercise fee estimation.
2023-08-10 09:50:53 -03:00
fanquake
d82bb90a5b
doc: use llvm-config for bitcoin-tidy example
An LLVM installation will have `llvm-config` available to query for
info. Ask it for the `--cmakedir`, and use that in our bitcoin-tidy
example, rather than listing multiple different (potential) paths per
distro/OS etc.
2023-08-10 12:39:35 +02:00
fanquake
b2ec0326fd
Merge bitcoin/bitcoin#28008: BIP324 ciphersuite
1c7582ead6e1119899922041c1af2b4169b0bc74 tests: add decryption test to bip324_tests (Pieter Wuille)
990f0f8da92a2d11828a7c05ca93bf0520b2a95e Add BIP324Cipher, encapsulating key agreement, derivation, and stream/AEAD ciphers (Pieter Wuille)
c91cedf281e5207fb5fd2ca81feec9760f7c2ed0 crypto: support split plaintext in ChaCha20Poly1305 Encrypt/Decrypt (Pieter Wuille)
af2b44c76e5de8ce880381e5535ead37ab0b3ba9 bench: add benchmark for FSChaCha20Poly1305 (Pieter Wuille)
aa8cee93342ee857931afec9af3ff5dbd8ce7749 crypto: add FSChaCha20Poly1305, rekeying wrapper around ChaCha20Poly1305 (Pieter Wuille)
0fee267792eb8cbdd48ad78f1712420b5d8d905b crypto: add FSChaCha20, a rekeying wrapper around ChaCha20 (Pieter Wuille)
9ff0768bdcca06836ccc673eacfa648e801930cb crypto: add the ChaCha20Poly1305 AEAD as specified in RFC8439 (Pieter Wuille)
9fd085a1a49d317abcaf1492b71c48bf1a1b3007 crypto: remove outdated variant of ChaCha20Poly1305 AEAD (Pieter Wuille)

Pull request description:

  Depends on #27985 and #27993, based on and partially replaces #25361, part of #27634. Draft while dependencies are not merged.

  This adds implementations of:
  * The ChaCha20Poly1305 AEAD from [RFC8439 section 2.8](https://datatracker.ietf.org/doc/html/rfc8439#section-2.8), including test vectors.
  * The FSChaCha20 stream cipher as specified in [BIP324](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#rekeying-wrappers-fschacha20poly1305-and-fschacha20), a rekeying wrapper around ChaCha20.
  * The FSChaCha20Poly1305 AEAD as specified in [BIP324](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#rekeying-wrappers-fschacha20poly1305-and-fschacha20), a rekeying wrapper around ChaCha20Poly1305.
  * A BIP324Cipher class that encapsulates key agreement, key derivation, and stream ciphers and AEADs for [BIP324 packet encoding](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#overall-packet-encryption-and-decryption-pseudocode).

  The ChaCha20Poly1305 and FSChaCha20Poly1305 implementations are new, taking advance of the improvements in #27993.

ACKs for top commit:
  jamesob:
    reACK 1c7582e
  theStack:
    ACK 1c7582ead6e1119899922041c1af2b4169b0bc74
  stratospher:
    tested ACK 1c7582e.

Tree-SHA512: 06728b4b95b21c5b732ed08faf40e94d0583f9d86ff4db3b92dd519dcd9fbfa0f310bc66ef1e59c9e49dd844ba8c5ac06e2001762a804fb5aa97027816045a46
2023-08-10 11:58:59 +02:00
fanquake
ef3f9f389f
Merge bitcoin/bitcoin#28189: doc: diversify network outbounds release note
7463d259e14676846f0b43e2b5f2af3faf688cb4 doc: Add release note (Amiti Uttarwar)

Pull request description:

  release notes for #27213

ACKs for top commit:
  mzumsande:
    ACK 7463d259e14676846f0b43e2b5f2af3faf688cb4

Tree-SHA512: 16c479774ed9242d8d044d08cc919550ccd07020423a3dcd99f07dad36e4dafd8243dc47f9f7f0c8eedcb53efd85ec65afedba56422452f637d313ec7c901520
2023-08-09 19:11:51 +02:00
glozow
0d9a13ddd8
Merge bitcoin/bitcoin#28149: net processing: clamp PeerManager::Options user input
547fa52443cbb5e8ccfee993486f5ced8cdbb33b net processing: clamp -blockreconstructionextratxn to uint32_t bounds (stickies-v)
e451d1e3c66350017da195335f428a96fdc7d840 net processing: clamp -maxorphantx to uint32_t bounds (stickies-v)
aa89e04e07ca9ff51b1d7d310a11821c6ad963cf doc: document PeerManager::Options members (stickies-v)

Pull request description:

  Avoid out-of-bounds user input for `PeerManager::Options` by safely clamping `-maxorphantx` and `-blockreconstructionextratxn`, and avoid platform-specific behaviour by changing `PeerManager::Options::max_extra_txs` from `size_t` to a `uint32_t`. Addresses https://github.com/bitcoin/bitcoin/pull/27499#pullrequestreview-1544114932.

  Also documents all `PeerManager::Options` members, addressing https://github.com/bitcoin/bitcoin/pull/27499#discussion_r1272302469.

ACKs for top commit:
  dergoegge:
    Code review ACK 547fa52443cbb5e8ccfee993486f5ced8cdbb33b
  glozow:
    reACK 547fa52443cbb5e8ccfee993486f5ced8cdbb33b

Tree-SHA512: 042d47b35bb8a7b29ef3dadd4c0c5d26f13a8f174f33687855d603c19f8de0fcbbda94418453331e149885412d4edd5f402d640d938f6d94b4dcf54e2fdbbcc9
2023-08-09 14:26:03 +02:00
MarcoFalke
fafa17c00b
ci: Use hard-coded root path for CI containers 2023-08-09 12:32:44 +02:00
MarcoFalke
fa084f5ba5
ci: Only create folders when needed
Now that container volumes are used, the folders are no longer mounted.
They are only needed when running without a container engine (docker,
podman).
2023-08-09 12:32:30 +02:00
MarcoFalke
fab27127f4
ci: Drop BASE_SCRATCH_DIR from LIBCXX_DIR
Using a hard-coded path avoids non-determinism issues and improves CI
UX.
2023-08-09 12:32:15 +02:00
fanquake
492257019d
Merge bitcoin/bitcoin#28087: ci: Use qemu-user through container engine
fad0b67c212dcb8a16fcbda5a74acc959ed4e284 ci: Use qemu-user through container engine (MarcoFalke)

Pull request description:

  Currently the CI containers always run on the host architecture, and only wrap `bitcoind` into `qemu-user` when needed. This has many issues:

  * The `i386` tasks can not be run on non-x86 hosts.
  * `config.guess` isn't present when building the CI image, which is fine. But it prints a warning, see https://github.com/bitcoin/bitcoin/pull/27739#pullrequestreview-1446580353
  * The python tests are run on the host architecture, making it harder to find architecture specific bugs. See for example https://github.com/bitcoin/bitcoin/pull/27529#issuecomment-1521387810
  * All modern container engines support automatic dispatch to qemu-user, so it seems redundant to re-invent the wheel.

  Fix all issues by:
  * removing `HOST` from `ci/test/00_setup_env.sh`.
  * removing `QEMU_USER_CMD` and `ci/test/wrap-qemu.sh`.
  * removing `DPKG_ADD_ARCH` where possible, and pruning `PACKAGES` where possible.
  * specifying the architecture in `CI_IMAGE_NAME_TAG` to be used by the container engine.

ACKs for top commit:
  fanquake:
    ACK fad0b67c212dcb8a16fcbda5a74acc959ed4e284 - this seems ok to me, and removes complexity from our CI system.

Tree-SHA512: 85e79f9f570e292d70a629d112fd4a6e6217d96226a1b665ed13485f616d84720ad2126b7d4b22fc603049f72fa7f2163b56a6bc276319fcd8b0496304ea4157
2023-08-09 12:09:00 +02:00
fanquake
b565485c24
Merge bitcoin/bitcoin#28186: kernel: Prune leveldb headers
d8f1222ac50f089a0af29eaf8ce0555bad8366ef refactor: Correct dbwrapper key naming (TheCharlatan)
be8f159ac59b9e700cbd3314ed71ebf39bd5b67a build: Remove leveldb from BITCOIN_INCLUDES (TheCharlatan)
c95b37d641b1eed4a62d55ca5342a6ed8c7a1ce7 refactor: Move CDBWrapper leveldb members to their own context struct (TheCharlatan)
c534a615e93452a5f509aaf5f68c600391a98d6a refactor: Split dbwrapper CDBWrapper::EstimateSize implementation (TheCharlatan)
586448888b72f7c87db4dcd30fc4e4044afae13b refactor: Move HandleError to dbwrapper implementation (TheCharlatan)
dede0eef7adb7413f62f5abd68cac8e01635ba4a refactor: Split dbwrapper CDBWrapper::Exists implementation (TheCharlatan)
a5c2eb57484314b04ec94523d14e0ef0c6c46d4f refactor: Fix logging.h includes (TheCharlatan)
84058e0eed9c05bc30984b39131e88ad1425628f refactor: Split dbwrapper CDBWrapper::Read implementation (TheCharlatan)
e4af2408f2ac59788567b6fc8cb3a68fc43da9fe refactor: Pimpl leveldb::Iterator for CDBIterator (TheCharlatan)
ef941ff1281e76308c3e746e592375bec023e9e4 refactor: Split dbwrapper CDBIterator::GetValue implementation (TheCharlatan)
b7a1ab5cb4e60230f62c94efb3a10d07c9af4883 refactor: Split dbwrapper CDBIterator::GetKey implementation (TheCharlatan)
d7437908cdf242626263ba9d5541addcddadc594 refactor: Split dbwrapper CDBIterator::Seek implementation (TheCharlatan)
ea8135de7e617259cda3fc7b1c8e7569d454fd57 refactor: Pimpl leveldb::batch for CDBBatch (TheCharlatan)
b9870c920dc475ec759eaf7339ea42aecba92138 refactor: Split dbwrapper CDBatch::Erase implementation (TheCharlatan)
532ee812a499e13b123af6b8415d8de1f3804f0f refactor: Split dbwrapper CDBBatch::Write implementation (TheCharlatan)
afc534df9adbf5599b286b5dc3531a4b9ac2d056 refactor: Wrap DestroyDB in dbwrapper helper (TheCharlatan)

Pull request description:

  Leveldb headers are currently included in the `dbwrapper.h` file and thus available to many of Bitcoin Core's source files. However, leveldb-specific functionality should be abstracted by the `dbwrapper` and does not need to be available to the rest of the code. Having leveldb included in a widely-used header such as `dbwrapper.h` bloats the entire project's header tree.

  The `dbwrapper` is a key component of the libbitcoinkernel library. Future users of this library would not want to contend with  having the leveldb headers exposed and potentially polluting their project's namespace.

  For these reasons, the leveldb headers are removed from the `dbwrapper` by moving leveldb-specific code to the implementation file and creating a [pimpl](https://en.cppreference.com/w/cpp/language/pimpl) where leveldb member variables are indispensable. As a final step, the leveldb include flags are removed from the `BITCOIN_INCLUDES` and moved to places where the dbwrapper is compiled.

  ---

  This pull request is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587), and more specifically its stage 1 step 3 "Decouple most non-consensus headers from libbitcoinkernel".

ACKs for top commit:
  stickies-v:
    re-ACK d8f1222ac5
  MarcoFalke:
    ACK d8f1222ac50f089a0af29eaf8ce0555bad8366ef  🔠

Tree-SHA512: 0f58309be165af0162e648233451cd80fda88726fc10c0da7bfe4ec2ffa9afe63fbf7ffae9493698d3f39653b4ad870c372eee652ecc90ab1c29d86c387070f3
2023-08-07 22:31:46 +02:00
fanquake
064919e00d
Merge bitcoin/bitcoin#28231: doc: remove Fedora libdb4-*-devel install docs
11a499eb4d57f2c5eabd7955bcc7e419364b3194 doc: remove Fedora libdb4-*-devel install docs (fanquake)

Pull request description:

  These are no-longer installable on any recent Fedora (last working version was 32).
  Remove the install instructions, and consolidate this section to be the same as the
  Ubuntu & Debian BDB install instructions.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 11a499eb4d57f2c5eabd7955bcc7e419364b3194

Tree-SHA512: 11e3c92d6dcf475a6f5529a2e41dc9f79eeae8f8d3600087ce5ae083264f999782a2c04a4c4c70073e96d4053daa23037a344224197ee5f15a3d635172c201e2
2023-08-07 18:55:43 +02:00
MarcoFalke
fad0b67c21
ci: Use qemu-user through container engine 2023-08-07 17:36:14 +02:00
fanquake
624333455a
Merge bitcoin/bitcoin#26296: ci: Integrate bitcoin-tidy clang-tidy plugin
1c976c691cc4b20f43071aabf36c7afed1571057 tidy: Integrate bicoin-tidy clang-tidy plugin (fanquake)
7de23cceb8ac13fcc709453ef0fa14fb93c460b0 refactor: fix unterminated LogPrintf()s (fanquake)
0a1029aa2946787bfe91a84de882c2dd0de076e9 lint: remove  /* Continued */ markers from codebase (fanquake)
910007995d8603ffc466878856227153a638caff lint: remove lint-logs.py (fanquake)
d86a83d6b8587b0971e66c6910af23dd8c042969 lint: drop DIR_IWYU global (fanquake)

Pull request description:

  Demo of integrating the [bitcoin-tidy](https://github.com/theuni/bitcoin-tidy-plugin), [clang-tidy plugin](https://clang.llvm.org/extra/clang-tidy/) written by theuni into our tidy CI job.

  The plugin currently has a single check, `bitcoin-unterminated-logprintf`. This would replace our current Python driven, `git-grep`-based, `.cpp` file only, lint-logs linter.

ACKs for top commit:
  TheCharlatan:
    ACK 1c976c691cc4b20f43071aabf36c7afed1571057
  theuni:
    ACK 1c976c691cc4b20f43071aabf36c7afed1571057
  MarcoFalke:
    re-ACK 1c976c691cc4b20f43071aabf36c7afed1571057  👠

Tree-SHA512: 725b45c70e431d48e6f276671e05c694e10b6047cae1a31906ac3ee9093bc8105fb226b36a5bac6709557526ca6007222112d66aecec05a574434edc4897e4b8
2023-08-07 17:14:07 +02:00
fanquake
97ba72117c
Merge bitcoin/bitcoin#27401: tracepoints: Disables -Wgnu-zero-variadic-macro-arguments to compile without warnings
5197660e947435e510ef3ef72be8be8dee3ffa41 tracepoints: Disables `-Wgnu-zero-variadic-macro-arguments` to compile without warnings (Martin Leitner-Ankerl)

Pull request description:

  Fixes #26916 by disabling the warning `-Wgnu-zero-variadic-macro-arguments` when clang is used as the compiler.

  Also see the comments
  * Proposed changes in the bug  https://github.com/bitcoin/bitcoin/issues/26916#issuecomment-1480997053
  * Proposed changes when moving to a variadic maro: https://github.com/bitcoin/bitcoin/pull/26593#discussion_r1155488768

ACKs for top commit:
  hebasto:
    ACK 5197660e947435e510ef3ef72be8be8dee3ffa41, I've reconsidered my [comment](https://github.com/bitcoin/bitcoin/pull/27401#issuecomment-1507142439) and I think the current localized approach is optimal.
  fanquake:
    ACK 5197660e947435e510ef3ef72be8be8dee3ffa41 - checked that this fixes the warnings under Clang.

Tree-SHA512: c3dda3bcbb2540af6283ffff65885a9937bfdaaef3b00dc7d60b9f9740031d5c36ac9cb3d3d8756dbadce4812201a9754f5b8770df0d5e0d5ee690ba8a7135d2
2023-08-07 16:03:55 +02:00
fanquake
11a499eb4d
doc: remove Fedora libdb4-*-devel install docs
These are no-longer installable on any recent Fedora (33+).
Remove the install instructions.
Fix the typo in the Ubuntu/Debian instructions.
2023-08-07 14:48:35 +02:00
MarcoFalke
fa776e61cd
Add importmempool RPC
test_importmempool_union contributed by glozow

Co-authored-by: glozow <gloriajzhao@gmail.com>
2023-08-07 11:33:34 +02:00
MarcoFalke
fa20d734a2
refactor: Add and use kernel::ImportMempoolOptions
This allows optional named arguments with default values.
2023-08-07 11:32:34 +02:00
MarcoFalke
fa8866990d
doc: Clarify the getmempoolinfo.loaded RPC field documentation
Also, clarify the LoadMempool doxygen.
2023-08-07 11:32:29 +02:00
MarcoFalke
6888886cec
Remove Chainstate::LoadMempool
The 3-line function is only called once outside of tests, so it is
clearer to inline it.
2023-08-07 10:59:15 +02:00
fanquake
be44332803
Merge bitcoin/bitcoin#28191: refactor: Remove unused MessageStartChars parameters from BlockManager methods
fa69e3a95c452c2ba3221b17c19fba5993b5d073 Remove unused MessageStartChars parameters from BlockManager methods (MarcoFalke)

Pull request description:

  Seems odd to expose these for mocking, when it is not needed.

  Fix this by removing the the unused parameters and use the already existing member field instead.

ACKs for top commit:
  Empact:
    utACK fa69e3a95c452c2ba3221b17c19fba5993b5d073
  dergoegge:
    utACK fa69e3a95c452c2ba3221b17c19fba5993b5d073

Tree-SHA512: 7814e9560abba8d9c0926bcffc70f92e502d22f543af43671248f6fcd1433f35238553c0f05123fde6d8e0f80261af0ab0500927548115153bd68d57fe2da746
2023-08-07 10:57:39 +02:00
fanquake
b7138252ac
Merge bitcoin/bitcoin#27213: p2p: Diversify automatic outbound connections with respect to networks
1b52d16d07be3b5d968157913f04d9cd1e2d3678 p2p: network-specific management of outbound connections (Martin Zumsande)
65cff00ceea48ac8a887ffea79aedb4251aa097f test: Add test for outbound protection by network (Martin Zumsande)
034f61f83b9348664d868933dbbfd8f9f8882168 p2p: Protect extra full outbound peers by network (Martin Zumsande)
654d9bc27647fb3797001472e2464dededb45d3f p2p: Introduce data struct to track connection counts by network (Amiti Uttarwar)

Pull request description:

  This is joint work with mzumsande.

  This is a proposal to diversify outbound connections with respect to reachable networks. The existing logic evaluates peers for connection based purely on the frequency of available addresses in `AddrMan`. This PR adds logic to automatically connect to alternate reachable networks and adds eviction logic that protects one existing connection to each network.

  For instance, if `AddrMan` is populated primarily with IPv4 and IPv6 addresses and only a handful of onion addresses, it is likely that we won't establish any automatic outbound connections to Tor, even if we're capable of doing so. For smaller networks like CJDNS, this is even more of an issue and often requires adding manual peers to ensure regularly being connected to the network.

  Connecting to multiple networks improves resistance to eclipse attacks for individual nodes. It also benefits the entire p2p network by increasing partition resistance and privacy in general.

  The automatic connections to alternate networks is done defensively, by first filling all outbound slots with random addresses (as in the status quo) and then adding additional peers from reachable networks the node is currently not connected to. This approach ensures that outbound slots are not left unfilled while attempting to connect to a network that may be unavailable due to a technical issue or misconfiguration that bitcoind cannot detect.

  Once an additional peer is added and we have one more outbound connection than we want, outbound eviction ensures that peers are protected if they are the only ones for their network.

  Manual connections are also taken into account: If a user already establishes manual connections to a trusted peer from a network, there is no longer a need to make extra efforts to ensure we also have an automatic connection to it (although this may of course happen by random selection).

ACKs for top commit:
  naumenkogs:
    ACK 1b52d16d07be3b5d968157913f04d9cd1e2d3678
  vasild:
    ACK 1b52d16d07be3b5d968157913f04d9cd1e2d3678

Tree-SHA512: 5616c038a5fbb868d4c46c5963cfd53e4599feee25db04b0e18da426d77d22e0994dc4e1da0b810f5b457f424ebbed3db1704f371aa6cad002b3565b20170ec0
2023-08-06 18:44:42 +02:00
TheCharlatan
d8f1222ac5
refactor: Correct dbwrapper key naming
The ss- prefix should connotate a DataStream variable. Now that these
variables are byte spans, drop the prefix.
2023-08-05 10:45:19 +02:00
TheCharlatan
be8f159ac5
build: Remove leveldb from BITCOIN_INCLUDES
Since leveldb is no longer in our header tree, move its include flags to
whereever dbwrapper.cpp is built.
2023-08-05 10:45:17 +02:00
TheCharlatan
c95b37d641
refactor: Move CDBWrapper leveldb members to their own context struct
The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:45:12 +02:00
TheCharlatan
c534a615e9
refactor: Split dbwrapper CDBWrapper::EstimateSize implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

Since CharCast is no longer needed in the header, move it to the
implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:43:01 +02:00
TheCharlatan
586448888b
refactor: Move HandleError to dbwrapper implementation
Make it a static function in dbwrapper.cpp, since it is not used
elsewhere and when left in the header, would expose a leveldb type.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:59 +02:00
TheCharlatan
dede0eef7a
refactor: Split dbwrapper CDBWrapper::Exists implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:58 +02:00
TheCharlatan
a5c2eb5748
refactor: Fix logging.h includes
These were uncovered as missing by the next commit.
2023-08-05 10:42:56 +02:00
TheCharlatan
84058e0eed
refactor: Split dbwrapper CDBWrapper::Read implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:55 +02:00
TheCharlatan
e4af2408f2
refactor: Pimpl leveldb::Iterator for CDBIterator
Hide the leveldb::Iterator member variable with a pimpl in order not to
expose it directly in the header.

Also, move CDBWrapper::NewIterator to the dbwrapper implementation to
use the pimpl for CDBIterator initialziation.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:53 +02:00
TheCharlatan
ef941ff128
refactor: Split dbwrapper CDBIterator::GetValue implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:51 +02:00
TheCharlatan
b7a1ab5cb4
refactor: Split dbwrapper CDBIterator::GetKey implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:48 +02:00
TheCharlatan
d7437908cd
refactor: Split dbwrapper CDBIterator::Seek implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:45 +02:00
TheCharlatan
ea8135de7e
refactor: Pimpl leveldb::batch for CDBBatch
Hide the leveldb::WriteBatch member variable with a pimpl in order not
to expose it directly in the header.

Also move CDBBatch::Clear to the dbwrapper implementation to use the new
impl_batch.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:38 +02:00
TheCharlatan
b9870c920d
refactor: Split dbwrapper CDBatch::Erase implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:27:53 +02:00
TheCharlatan
532ee812a4
refactor: Split dbwrapper CDBBatch::Write implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:27:47 +02:00
fanquake
d096743150
Merge bitcoin/bitcoin#28213: scripted-diff: Specify Python major version explicitly on Windows
6a7686b44618eabd2f8ee9f1d357cfeb1bce6d91 scripted-diff: Specify Python major version explicitly on Windows (Hennadii Stepanov)

Pull request description:

  On Windows, it is the accepted practice to use `py.exe` launcher:
  - https://learn.microsoft.com/en-us/windows/python/faqs#what-is-py-exe-
  - https://docs.python.org/3/using/windows.html#python-launcher-for-windows

  One of its features is the correct handling of shebang lines like the one we use: `#!/usr/bin/env python3`.

  However, Windows OS app execution aliases might [interfere](https://learn.microsoft.com/en-us/windows/python/faqs#why-does-running-python-exe-open-the-microsoft-store-) with the launcher's behaviour. Such aliases are enabled on Windows 11 by default:

  ![image](https://github.com/bitcoin/bitcoin/assets/32963518/407837ec-e89a-4bc1-98b1-db983002065a)

  For example, on a fresh Windows 11 Pro installation with the Python installed from the [Chocolatey](https://community.chocolatey.org/packages/python/3.11.4) package manager, one will get the following error:
  ```
  >py -3 test\functional\rpc_signer.py
  2023-08-03T19:41:13.353000Z TestFramework (INFO): PRNG seed is: 2694758731106548661
  2023-08-03T19:41:13.353000Z TestFramework (INFO): Initializing test directory C:\Users\hebasto\AppData\Local\Temp\bitcoin_func_test_mldbzzw3
  2023-08-03T19:41:14.538000Z TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
    File "C:\Users\hebasto\bitcoin\test\functional\test_framework\util.py", line 140, in try_rpc
      fun(*args, **kwds)
    File "C:\Users\hebasto\bitcoin\test\functional\test_framework\coverage.py", line 50, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\hebasto\bitcoin\test\functional\test_framework\authproxy.py", line 129, in __call__
      raise JSONRPCException(response['error'], status)
  test_framework.authproxy.JSONRPCException: RunCommandParseJSON error: process(py C:\Users\hebasto\bitcoin\test\functional\mocks\signer.py enumerate) returned 9009: Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.
   (-1)

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "C:\Users\hebasto\bitcoin\test\functional\test_framework\test_framework.py", line 131, in main
      self.run_test()
    File "C:\Users\hebasto\bitcoin\test\functional\rpc_signer.py", line 72, in run_test
      assert_raises_rpc_error(-1, 'fingerprint not found',
    File "C:\Users\hebasto\bitcoin\test\functional\test_framework\util.py", line 131, in assert_raises_rpc_error
      assert try_rpc(code, message, fun, *args, **kwds), "No exception raised"
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\hebasto\bitcoin\test\functional\test_framework\util.py", line 146, in try_rpc
      raise AssertionError(
  AssertionError: Expected substring not found in error message:
  substring: 'fingerprint not found'
  error message: 'RunCommandParseJSON error: process(py C:\Users\hebasto\bitcoin\test\functional\mocks\signer.py enumerate) returned 9009: Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.
  '.
  2023-08-03T19:41:14.592000Z TestFramework (INFO): Stopping nodes
  2023-08-03T19:41:14.799000Z TestFramework (WARNING): Not cleaning up dir C:\Users\hebasto\AppData\Local\Temp\bitcoin_func_test_mldbzzw3
  2023-08-03T19:41:14.799000Z TestFramework (ERROR): Test failed. Test logging available at C:\Users\hebasto\AppData\Local\Temp\bitcoin_func_test_mldbzzw3/test_framework.log
  2023-08-03T19:41:14.799000Z TestFramework (ERROR):
  2023-08-03T19:41:14.799000Z TestFramework (ERROR): Hint: Call C:\Users\hebasto\bitcoin\test\functional\combine_logs.py 'C:\Users\hebasto\AppData\Local\Temp\bitcoin_func_test_mldbzzw3' to consolidate all logs
  2023-08-03T19:41:14.799000Z TestFramework (ERROR):
  2023-08-03T19:41:14.799000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
  2023-08-03T19:41:14.799000Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
  2023-08-03T19:41:14.799000Z TestFramework (ERROR):

  ```

  This PR resolves this issue by explicitly specifying the Python major version and makes testing of self-compiled binaries more straightforward.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 6a7686b44618eabd2f8ee9f1d357cfeb1bce6d91
  stickies-v:
    utACK 6a7686b44618eabd2f8ee9f1d357cfeb1bce6d91

Tree-SHA512: 5681141e222bc833c6250cb79fe3a1c8e02255eb2c86010bc0f8239afcdfed784ed7788c8579209d931bd357f58d5655cf33ffeb2f46b1879f37cdc30e7a7c91
2023-08-04 15:09:34 +02:00