43967 Commits

Author SHA1 Message Date
Matthew Zipkin
e56afcccbf
refactor: split http_request_cb into libevent callback and dispatch
The original function is passed to libevent as a callback when HTTP
requests are received and processed. It wrapped the libevent request
object in a http_libevent::HTTPRequest and then handed that off to
bitcoin for basic checks and finally dispatch to worker threads.

In this commit we split the function after the
http_libevent::HTTPRequest is created, and pass that object to a new
function that maintains the logic of checking and dispatching.

This will be the merge point for http_libevent and http_bitcoin,
where HTTPRequest objects from either namespace have the same
downstream lifecycle.
2025-03-13 14:33:09 -04:00
Matthew Zipkin
58fe646d3c
Add helper methods to HTTPRequest to match original API
These methods are called by http_request_cb() and are present in the
original http_libevent::HTTPRequest.
2025-03-13 14:33:09 -04:00
Matthew Zipkin
5b61d65034
define HTTP request methods at module level outside of class
This is a refactor to prepare for matching the API of HTTPRequest
definitions in both namespaces http_bitcoin and http_libevent. In
particular, to provide a consistent return type for GetRequestMethod()
in both classes.
2025-03-13 14:33:09 -04:00
Matthew Zipkin
0765a0c351
Allow http workers to send data optimistically as an optimization 2025-03-13 14:33:02 -04:00
Matthew Zipkin
d771b650f3
http: disconnect clients 2025-03-13 14:32:04 -04:00
Matthew Zipkin
057d648866
http: compose and send replies to connected clients 2025-03-13 13:33:08 -04:00
Matthew Zipkin
5d3678a235
http: support "chunked" Transfer-Encoding 2025-03-13 13:33:07 -04:00
Matthew Zipkin
5a2d625e08
http: read requests from connected clients 2025-03-13 13:33:07 -04:00
Matthew Zipkin
2658144186
http: Begin implementation of HTTPClient and HTTPServer 2025-03-13 13:33:07 -04:00
Matthew Zipkin
34e03406cf
http: Implement HTTPRequest class
HTTP Request message:
https://datatracker.ietf.org/doc/html/rfc1945#section-5

Request Line aka Control Line aka first line:
https://datatracker.ietf.org/doc/html/rfc1945#section-5.1

See message_read_status() in libevent http.c for how
`MORE_DATA_EXPECTED` is handled there
2025-03-13 13:33:07 -04:00
Matthew Zipkin
12dbb0d4ca
http: Implement HTTPResponse class
HTTP Response message:
https://datatracker.ietf.org/doc/html/rfc1945#section-6

Status line (first line of response):
https://datatracker.ietf.org/doc/html/rfc1945#section-6.1

Status code definitions:
https://datatracker.ietf.org/doc/html/rfc1945#section-9
2025-03-13 13:33:07 -04:00
Matthew Zipkin
47cbb2c153
http: Implement HTTPHeaders class
see:
https://www.rfc-editor.org/rfc/rfc2616#section-4.2
https://www.rfc-editor.org/rfc/rfc7231#section-5
https://www.rfc-editor.org/rfc/rfc7231#section-7
https://httpwg.org/specs/rfc9111.html#header.field.definitions
2025-03-13 13:33:07 -04:00
Matthew Zipkin
a31b62f926
http: enclose libevent-dependent code in a namespace
This commit is a no-op to isolate HTTP methods and objects that
depend on libevent. Following commits will add replacement objects
and methods in a new namespace for testing and review before
switching over the server.
2025-03-13 13:33:07 -04:00
Matthew Zipkin
a7db47d592
string: add LineReader
This is a helper struct to parse HTTP messages from data in buffers
from sockets. HTTP messages begin with headers which are
CRLF-terminated lines (\n or \r\n) followed by an arbitrary amount of
body data. Whitespace is trimmed from the field lines but not the body.

https://httpwg.org/specs/rfc9110.html#rfc.section.5
2025-03-13 13:33:06 -04:00
Matthew Zipkin
908e75cd0d
time: implement and test RFC7231 timestamp string
HTTP 1.1 responses require a timestamp header with a
specific format, specified in:
https://www.rfc-editor.org/rfc/rfc7231#section-7.1.1.1
2025-03-13 13:33:06 -04:00
Matthew Zipkin
c7a6f23e8a
string: add CaseInsensitiveComparator
https://httpwg.org/specs/rfc9110.html#rfc.section.5.1
Field names in HTTP headers are case-insensitive. This
comparator will be used in the headers map to search by key.
In libevent these are compared in lowercase:
  evhttp_find_header()
  evutil_ascii_strcasecmp()
  EVUTIL_TOLOWER_()
2025-03-13 13:33:06 -04:00
Matthew Zipkin
d6ea26c2fe
string: implement ParseUInt64Hex 2025-03-13 13:33:06 -04:00
Matthew Zipkin
3fa1408958
test: cover "chunked" Transfer-Encoding 2025-03-13 13:33:06 -04:00
Matthew Zipkin
4ccc28f8c3
test: cover -rpcservertimeout 2025-03-13 13:33:01 -04:00
Vasil Dimov
741f17e51d
net: move-only: improve encapsulation of SockMan
`SockMan` members

`AcceptConnection()`
`NewSockAccepted()`
`GetNewId()`
`m_i2p_sam_session`
`m_listen`

are now used only by `SockMan`, thus make them private.
2025-02-17 09:48:33 +01:00
Vasil Dimov
08dc1ee704
net: move sockets from CNode to SockMan
Move `CNode::m_sock` and `CNode::m_i2p_sam_session` to `SockMan::m_connected`.
Also move all the code that handles sockets to `SockMan`.

`CNode::CloseSocketDisconnect()` becomes
`CConnman::MarkAsDisconnectAndCloseConnection()`.

`CConnman::SocketSendData()` is renamed to
`CConnman::SendMessagesAsBytes()` and its sockets-touching bits are moved to
`SockMan::SendBytes()`.

`CConnman::GenerateWaitSockets()` goes to
`SockMan::GenerateWaitSockets()`.

`CConnman::ThreadSocketHandler()` and
`CConnman::SocketHandler()` are combined into
`SockMan::ThreadSocketHandler()`.

`CConnman::SocketHandlerConnected()` goes to
`SockMan::SocketHandlerConnected()`.

`CConnman::SocketHandlerListening()` goes to
`SockMan::SocketHandlerListening()`.
2025-02-17 09:48:33 +01:00
Vasil Dimov
52106d0136
net: tweak EventNewConnectionAccepted()
Move `MaybeFlipIPv6toCJDNS()`, which is Bitcoin P2P specific from the
callers of `CConnman::EventNewConnectionAccepted()` to inside that
method.

Move the IsSelectable check, the `TCP_NODELAY` option set and the
generation of new connection id out of
`CConnman::EventNewConnectionAccepted()` because those are protocol
agnostic. Move those to a new method `SockMan::NewSockAccepted()` which
is called instead of `CConnman::EventNewConnectionAccepted()`.
2025-02-17 09:48:32 +01:00
Vasil Dimov
455185b665
net: split CConnman::ConnectNode()
Move the protocol agnostic parts of `CConnman::ConnectNode()` into
`SockMan::ConnectAndMakeId()` and leave the Bitcoin-P2P specific
stuff in `CConnman::ConnectNode()`.

Move the protocol agnostic `CConnman::m_unused_i2p_sessions`, its mutex
and `MAX_UNUSED_I2P_SESSIONS_SIZE` to `SockMan`.

Move `GetBindAddress()` from `net.cpp` to `sockman.cpp`.
2025-02-17 09:48:31 +01:00
Vasil Dimov
ab9de0f226
net: isolate all remaining P2P specifics from SocketHandlerConnected()
Introduce 4 new methods for the interaction between `CConnman` and
`SockMan`:

* `EventReadyToSend()`:
  called when there is readiness to send and do the actual sending of data.

* `EventGotData()`, `EventGotEOF()`, `EventGotPermanentReadError()`:
  called when the corresponing recv events occur.

These methods contain logic that is specific to the Bitcoin-P2P protocol
and move it away from `CConnman::SocketHandlerConnected()` which will
become a protocol agnostic method of `SockMan`.

Also, move the counting of sent bytes to `CConnman::SocketSendData()` -
both callers of that method called `RecordBytesSent()` just after the
call, so move it from the callers to inside
`CConnman::SocketSendData()`.
2025-02-17 09:48:31 +01:00
Vasil Dimov
f0b40d1c06
net: isolate P2P specifics from SocketHandlerConnected() and ThreadSocketHandler()
Move some parts of `CConnman::SocketHandlerConnected()` and
`CConnman::ThreadSocketHandler()` that are specific to the Bitcoin-P2P
protocol to dedicated methods:
`EventIOLoopCompletedForOne(id)` and
`EventIOLoopCompletedForAll()`.

This brings us one step closer to moving `SocketHandlerConnected()` and
`ThreadSocketHandler()` to the protocol agnostic `SockMan` (which would
call `EventIOLoopCompleted...()` from `CConnman`).
2025-02-17 09:48:30 +01:00
Vasil Dimov
1cea2e4b84
net: isolate P2P specifics from GenerateWaitSockets()
Move the parts of `CConnman::GenerateWaitSockets()` that are specific to
the Bitcoin-P2P protocol to dedicated methods:
`ShouldTryToSend()` and `ShouldTryToRecv()`.

This brings us one step closer to moving `GenerateWaitSockets()` to the
protocol agnostic `SockMan` (which would call `ShouldTry...()` from
`CConnman`).
2025-02-17 09:48:30 +01:00
Vasil Dimov
93df2db96d
net: index nodes in CConnman by id
Change `CConnman::m_nodes` from `std::vector<CNode*>` to
`std::unordered_map<NodeId, CNode*>` because interaction
between `CConnman` and `SockMan` is going to be based on
`NodeId` and finding a node by its id would better be fast.

Change `PeerManagerImpl::EvictExtraOutboundPeers()` to account for nodes
no longer always being in order of id. The old code would have failed to
update `next_youngest_peer` correctly if `CConnman::m_nodes` hadn't
always had nodes in ascending order of id.

During fuzzing make sure that we don't generate duplicate `CNode` ids.
The easiest way to do that is to use sequential ids.

As a nice side effect the existent search-by-id operations in
`CConnman::AttemptToEvictConnection()`,
`CConnman::DisconnectNode()` and
`CConnman::ForNode()` now become `O(1)` (were `O(number of nodes)`),
as well as the erase in `CConnman::DisconnectNodes()`.
2025-02-17 09:48:29 +01:00
Vasil Dimov
be1d7418c1
net: move I2P-accept-incoming code from CConnman to SockMan 2025-02-17 09:48:28 +01:00
Vasil Dimov
a367d556fd
net: move CConnman-specific parts away from ThreadI2PAcceptIncoming()
CConnman-specific or in other words, Bitcoin P2P specific. Now
the `ThreadI2PAcceptIncoming()` method is protocol agnostic and
can be moved to `SockMan`.
2025-02-17 09:48:28 +01:00
Vasil Dimov
25203720a1
net: move the generation of ids for new nodes from CConnman to SockMan
Move `CConnman::GetNewNodeId()` to `SockMan::GetNewId()`. Avoid using
the word "node" because that is too specific for `CConnman`.
2025-02-17 09:48:27 +01:00
Vasil Dimov
221c9224b2
style: modernize the style of SockMan::AcceptConnection() 2025-02-17 09:48:27 +01:00
Vasil Dimov
c31fc1a993
net: split CConnman::AcceptConnection() off CConnman
Move the `CConnman::AcceptConnection()` method to `SockMan` and split
parts of it:
* the flip-to-CJDNS part: to just after the `AcceptConnection()` call
* the permissions part: at the start of `CreateNodeFromAcceptedSocket()`
2025-02-17 09:48:26 +01:00
Vasil Dimov
9d4e7e3bd7
style: modernize the style of SockMan::BindListenPort()
It was copied verbatim from `CConnman::BindListenPort()` in the previous
commit. Modernize its variables and style and log the error messages
from the caller. Also categorize the informative messages to the "net"
category because they are quite specific to the networking layer.
2025-02-17 09:48:25 +01:00
Vasil Dimov
98ba5c7965
net: split CConnman::BindListenPort() off CConnman
Introduce a new low-level socket managing class `SockMan`
and move the `CConnman::BindListenPort()` method to it.
2025-02-17 09:48:25 +01:00
Vasil Dimov
69ac6802ba
net: drop CConnman::ListenSocket
Now that `CConnman::ListenSocket` is a `struct` that contains only one
member variable of type `std::shared_ptr<Sock>`, drop `ListenSocket` and
use `shared_ptr` directly.

Replace the vector of `ListenSocket` with a vector of `shared_ptr`.
2025-02-17 09:48:24 +01:00
Vasil Dimov
7745ea523c
net: separate the listening socket from the permissions
They were coupled in `struct ListenSocket`, but the socket belongs to
the lower level transport protocol, whereas the permissions are specific
to the higher Bitcoin P2P protocol.
2025-02-17 09:48:24 +01:00
merge-script
db36a92c02
Merge bitcoin/bitcoin#31879: doc: add release note for #27432 (utxo-to-sqlite tool)
95722d048a85a313b35c69459680f734feb67695 doc: add release note for #27432 (utxo-to-sqlite tool) (Sebastian Falbesoner)

Pull request description:

  This PR adds a missing release note for https://github.com/bitcoin/bitcoin/pull/27432.

ACKs for top commit:
  kevkevinpal:
    ACK [95722d0](95722d048a)

Tree-SHA512: 47714182e814fb825dbaeede46a1b496c4b87f3c5bfa61ada00138926a6bf9eb9a3f99636eb698a7bcda6642f73d0a8c5bf531a726750d594decca8ba6a56e3a
2025-02-16 12:20:06 +01:00
Sebastian Falbesoner
95722d048a doc: add release note for #27432 (utxo-to-sqlite tool) 2025-02-16 02:24:04 +01:00
Ava Chow
43e71f7498
Merge bitcoin/bitcoin#27432: contrib: add tool to convert compact-serialized UTXO set to SQLite database
4080b66cbec2b6fc2fcfd7356941236f65d508e3 test: add test for utxo-to-sqlite conversion script (Sebastian Falbesoner)
ec99ed738083fc1ad4c9d85095e26e7e58372217 contrib: add tool to convert compact-serialized UTXO set to SQLite database (Sebastian Falbesoner)

Pull request description:

  ## Problem description

  There is demand from users to get the UTXO set in form of a SQLite database (#24628). Bitcoin Core currently only supports dumping the UTXO set in a binary _compact-serialized_ format, which was crafted specifically for AssumeUTXO snapshots (see PR #16899), with the primary goal of being as compact as possible. Previous PRs tried to extend the `dumptxoutset` RPC with new formats, either in human-readable form (e.g. #18689, #24202), or most recently, directly as SQLite database (#24952). Both are not optimal: due to the huge size of the ever-growing UTXO set with already more than 80 million entries on mainnet, human-readable formats are practically useless, and very likely one of the first steps would be to put them in some form of database anyway. Directly adding SQLite3 dumping support on the other hand introduces an additional dependency to the non-wallet part of bitcoind and the risk of increased maintenance burden (see e.g. https://github.com/bitcoin/bitcoin/pull/24952#issuecomment-1163551060, https://github.com/bitcoin/bitcoin/issues/24628#issuecomment-1108469715).

  ## Proposed solution

  This PR follows the "external tooling" route by adding a simple Python script for achieving the same goal in a two-step process (first create compact-serialized UTXO set via `dumptxoutset`, then convert it to SQLite via the new script). Executive summary:
  - single file, no extra dependencies (sqlite3 is included in Python's standard library [1])
  - ~150 LOC, mostly deserialization/decompression routines ported from the Core codebase and (probably the most difficult part) a little elliptic curve / finite field math to decompress pubkeys (essentialy solving the secp256k1 curve equation y^2 = x^3 + 7 for y given x, respecting the proper polarity as indicated by the compression tag)
  - creates a database with only one table `utxos` with the following schema:
    ```(txid TEXT, vout INT, value INT, coinbase INT, height INT, scriptpubkey TEXT)```
  - the resulting file has roughly 2x the size of the compact-serialized UTXO set (this is mostly due to encoding txids and scriptpubkeys as hex-strings rather than bytes)

  [1] note that there are some rare cases of operating systems like FreeBSD though, where the sqlite3 module has to installed explicitly (see #26819)

  A functional test is also added that creates UTXO set entries with various output script types (standard and also non-standard, for e.g. large scripts) and verifies that the UTXO sets of both formats match by comparing corresponding MuHashes. One MuHash is supplied by the bitcoind instance via `gettxoutsetinfo muhash`, the other is calculated in the test by reading back the created SQLite database entries and hashing them with the test framework's `MuHash3072` module.

  ## Manual test instructions
  I'd suggest to do manual tests also by comparing MuHashes. For that, I've written a go tool some time ago which would calculate the MuHash of a sqlite database in the created format (I've tried to do a similar tool in Python, but it's painfully slow).
  ```
  $ [run bitcoind instance with -coinstatsindex]
  $ ./src/bitcoin-cli dumptxoutset ~/utxos.dat
  $ ./src/bitcoin-cli gettxoutsetinfo muhash <block height returned in previous call>
  (outputs MuHash calculated from node)

  $ ./contrib/utxo-tools/utxo_to_sqlite.py ~/utxos.dat ~/utxos.sqlite
  $ git clone https://github.com/theStack/utxo_dump_tools
  $ cd utxo_dump_tools/calc_utxo_hash
  $ go run calc_utxo_hash.go ~/utxos.sqlite
  (outputs MuHash calculated from the SQLite UTXO set)

  => verify that both MuHashes are equal
  ```
  For a demonstration what can be done with the resulting database, see https://github.com/bitcoin/bitcoin/pull/24952#pullrequestreview-956290477 for some example queries. Thanks go to LarryRuane who gave me to the idea of rewriting this script in Python and adding it to `contrib`.

ACKs for top commit:
  ajtowns:
    ACK 4080b66cbec2b6fc2fcfd7356941236f65d508e3 - light review
  achow101:
    ACK 4080b66cbec2b6fc2fcfd7356941236f65d508e3
  romanz:
    tACK 4080b66cbe on signet (using [calc_utxo_hash](8981aa3e85/calc_utxo_hash/calc_utxo_hash.go)):
  tdb3:
    ACK 4080b66cbec2b6fc2fcfd7356941236f65d508e3

Tree-SHA512: be8aa0369a28c8421a3ccdf1402e106563dd07c082269707311ca584d1c4c8c7b97d48c4fcd344696a36e7ab8cdb64a1d0ef9a192a15cff6d470baf21e46ee7b
2025-02-14 15:22:10 -08:00
Ava Chow
e53310c47a
Merge bitcoin/bitcoin#30529: Fix -norpcwhitelist, -norpcallowip, and similar corner case behavior
a85e8c0e6158fad2408bda5cb1e36da707eb081b doc: Add some general documentation about negated options (Ryan Ofsky)
490c8fa17829c3f8ae4da739f526531c91f3ed87 doc: Add release notes summarizing negated option behavior changes. (Ryan Ofsky)
458ef0a11b57cb5af0e8903b50927723fbb3fcd6 refactor: Avoid using IsArgSet() on -connect list option (Ryan Ofsky)
752ab9c3c65e47fc05545d9b9c919be945851d51 test: Add test to make sure -noconnect disables -dnsseed and -listen by default (Ryan Ofsky)
3c2920ec98fc7d9f77abfd08fea17211b9ca7099 refactor: Avoid using IsArgSet() on -signetseednode and -signetchallenge list options (Ryan Ofsky)
d05668922a28e4e2c78dab2d4737433cd52d6302 refactor: Avoid using IsArgSet() on -debug, -loglevel, and -vbparams list options (Ryan Ofsky)
3d1e8ca53a05e7d4735a2207d1b200e1dcddc534 Normalize inconsistent -noexternalip behavior (Ryan Ofsky)
ecd590d4c1e7f310c6ba3b58373bc30679b491df Normalize inconsistent -noonlynet behavior (Ryan Ofsky)
5544a19f863737518944950fc73f97d9c1399a46 Fix nonsensical bitcoin-cli -norpcwallet behavior (Ryan Ofsky)
6e8e7f433fc3f753a20833aebe54692cdfe5ed75 Fix nonsensical -noasmap behavior (Ryan Ofsky)
b6ab3508064cd3135e1a356c884ae1269cda5250 Fix nonsensical -notest behavior (Ryan Ofsky)
6768389917a8d744f1b1ada4556d3d4fe63c310e Fix nonsensical -norpcwhitelist behavior (Ryan Ofsky)
e03409c70f7472d39e45d189df6c0cf6b676b761 Fix nonsensical -norpcbind and -norpcallowip behavior (Ryan Ofsky)
40c4899bc209921fb4bde02840359c3253663766 Fix nonsensical -nobind and -nowhitebind behavior (Ryan Ofsky)
5453e66fd91c303d04004d861ecad183ff177823 Fix nonsensical -noseednode behavior (Ryan Ofsky)

Pull request description:

  The PR changes behavior of negated `-noseednode`, `-nobind`, `-nowhitebind`, `-norpcbind`, `-norpcallowip`, `-norpcwhitelist`, `-notest`, `-noasmap`, `-norpcwallet`, `-noonlynet`, and `-noexternalip` options, so negating these options just clears previously specified values doesn't have other side effects.

  Negating options on the command line can be a useful way of resetting options that may have been set earlier in the command line or config file. But before this change, negating these options wouldn't fully reset them, and would have confusing and undocumented side effects (see commit descriptions for details). Now, negating these options just resets them and behaves the same as not specifying them.

  Motivation for this PR is to fix confusing behaviors and also to remove incorrect usages of the `IsArgSet()` function. Using `IsArgSet()` tends to lead to negated option bugs in general, but it especially causes bugs when used with list settings returned by `GetArgs()`, because when these settings are negated, `IsArgSet()` will return true but `GetArgs()` will return an empty list. This PR eliminates all uses of `IsArgSet()` and `GetArgs()` together, and followup PR #17783 makes it an error to use `IsArgSet()` on list settings, since calling `IsArgSet()` is never actually necessary. Most of the changes here were originally made in #17783 and then moved here to be easier to review and avoid a dependency on #16545.

ACKs for top commit:
  achow101:
    ACK a85e8c0e6158fad2408bda5cb1e36da707eb081b
  danielabrozzoni:
    re-ACK a85e8c0e6158fad2408bda5cb1e36da707eb081b
  hodlinator:
    re-ACK a85e8c0e6158fad2408bda5cb1e36da707eb081b

Tree-SHA512: dd4b19faac923aeaa647b1c241d929609ce8242b43e3b7bc32523cc48ec92a83ac0dc5aee79f1eba8794535e0314b96cb151fd04ac973671a1ebb9b52dd16697
2025-02-14 15:10:09 -08:00
Ava Chow
254fd89d39
Merge bitcoin/bitcoin#31863: random: Initialize variables in hardware RNG functions
99755e04ffadd5daad53c686f4f0feee2232eeb2 random: Initialize variables in hardware RNG functions (Eval EXEC)

Pull request description:

  See: https://github.com/bitcoin/bitcoin/pull/31826#discussion_r1955045279 , So this PR want to prevent potential uninitialized value issues and improve code clarity.

ACKs for top commit:
  sipa:
    utACK 99755e04ffadd5daad53c686f4f0feee2232eeb2
  achow101:
    ACK 99755e04ffadd5daad53c686f4f0feee2232eeb2

Tree-SHA512: 4cf9c214617769cf051b4f36453275b407e37d96315b6a206102d17019375b3834ba07e2ccb28c7650c90ff8e1f1034522fccafaa33e136dfe63cc68396a1f6e
2025-02-14 15:03:32 -08:00
Ava Chow
75f8396c90
Merge bitcoin/bitcoin#30746: test: cover base[32|58|64] with symmetric roundtrip fuzz (and padding) tests
f919d919eb8425ef2bb25aa0ebf61c90ab9b07fa fuzz: Add fuzzing for max_ret_len in DecodeBase58/DecodeBase58Check (Lőrinc)
635bc58f46b158cd6f77fda80001c2bccd5f83b0 test: Fuzz Base32/Base58/Base64 roundtrip conversions (Lőrinc)
5dd3a0d8a899e4c7263d5b999135f4d7584e1244 test: Extend base58_encode_decode.json with edge cases (Lőrinc)
ae40cf1a8e16462a8b9dfd076d440bc8ec796c2b test: Add padding tests for Base32/Base64 (Lőrinc)

Pull request description:

  Added fuzzed roundtrips for `base[32|58|64]` encoding to make sure encoding/decoding are symmetric.
  Note that if we omit the padding in `EncodeBase32` we won't be able to decode it with `DecodeBase32`.
  Added dedicated padding tests to cover failure behavior
  Also moved over the Base58 json test edge cases from https://github.com/bitcoin/bitcoin/pull/30035

ACKs for top commit:
  hodlinator:
    re-ACK f919d919eb8425ef2bb25aa0ebf61c90ab9b07fa
  achow101:
    ACK f919d919eb8425ef2bb25aa0ebf61c90ab9b07fa

Tree-SHA512: 6a6c63d0a659b70d42aad7a8f37ce6e372756e2c88c84e7be5c1ff1f2a7c58860ed7113acbe1a9658a7d19deb91f0abe2ec527ed660335845cd1e0a9380b4295
2025-02-14 14:48:01 -08:00
Ava Chow
c4b46b4589
Merge bitcoin/bitcoin#31629: wallet: fix rescanning inconsistency
4818da809f0e300016390dd41e02c6067ffa008f wallet: fix rescanning inconsistency (Martin Zumsande)

Pull request description:

  If the chain advances during a rescan, ScanForWalletTransactions would previously process the new blocks without adjusting `m_last_processed_block`, which would leave the wallet in an inconsistent state temporarily, and could lead to crashes in the GUI reported in #31474.
  Fix this by not rescanning blocks beyond `m_last_processed_block` - for all blocks beyond that height, there will be pending BlockConnected notifications that will process them after the rescan is finished.

  This means that if rescanning was triggered with `cs_wallet` permanently held (`AttachChain`), additional blocks that were connected during the rescan will only be processed with the pending `blockConnected` notifications after the lock is released.
  If rescanning without a permanent `cs_wallet` lock (`RescanFromTime`), additional blocks that were connected during the rescan can be re-processed here because `m_last_processed_block` was already updated by `blockConnected`.

  Fixes #31474

ACKs for top commit:
  psgreco:
    Not that it matters much, but UTACK 4818da809f0e300016390dd41e02c6067ffa008f
  achow101:
    ACK 4818da809f0e300016390dd41e02c6067ffa008f
  furszy:
    utACK 4818da809f0e300016390dd41e02c6067ffa008f

Tree-SHA512: 8e7dbc9e00019aef4f80a11776f3089cd671e0eadd3c548cc6267b5c722433f80339a9b2b338ff9b611863de75ed0a817a845e1668e729b71af70c9038b075af
2025-02-14 14:42:12 -08:00
Ava Chow
d0dfd6d3f6
Merge bitcoin/bitcoin#31865: build: move rpc/external_signer to node library
e501246e77cc36cff222fb07aed2fd1316e11e19 build: move rpc/external_signer to node library (fanquake)

Pull request description:

  Move `rpc/external_signer` from `bitcoin_common` to `bitcoin_node`.
  Remove the check-deps suppression.

ACKs for top commit:
  maflcko:
    lgtm ACK e501246e77cc36cff222fb07aed2fd1316e11e19
  achow101:
    ACK e501246e77cc36cff222fb07aed2fd1316e11e19
  TheCharlatan:
    ACK e501246e77cc36cff222fb07aed2fd1316e11e19

Tree-SHA512: d535da9038a6b37bd83e852721b42c0806b9ddf060a9b96544027a34d11c1728b3b97385768fca1acc483c6632c28050e2194e2d2ac831d944f332431bfd6792
2025-02-14 14:32:28 -08:00
Ava Chow
ce4dbfc359
Merge bitcoin/bitcoin#31851: doc: build: Fix instructions for msvc gui builds
c3fa043ae560289759b6f836ac62736d97ba91bf doc: build: Fix instructions for msvc gui builds (David Gumberg)

Pull request description:

  If the instructions in `doc/build-windows-msvc.md` are followed as-is, and "Developer (PowerShell|Command Prompt) for VS 2022" is used to execute the suggested build commands, the root directory of vcpkg (e.g. in VS 2022 Community edition: `C:\Program Files\Microsoft Visual Studio\2022\Community\VC\vcpkg`), is too long, and when vcpkg attempts to build any of the QT packages, it will fail because of build steps that require path lengths greater than Windows' `MAX_PATH` 260 character limit. This can be avoided without needing to move the vcpkg root dir by setting [`--x-buildtrees-root`](https://learn.microsoft.com/en-us/vcpkg/commands/common-options#buildtrees-root) to a short path, like `C:\vcpkg`.

  See e.g. https://github.com/microsoft/vcpkg/issues/28451, https://github.com/microsoft/vcpkg/issues/28083, https://github.com/microsoft/vcpkg/issues/24751.

ACKs for top commit:
  achow101:
    ACK c3fa043ae560289759b6f836ac62736d97ba91bf
  hebasto:
    ACK c3fa043ae560289759b6f836ac62736d97ba91bf.
  TheCharlatan:
    ACK c3fa043ae560289759b6f836ac62736d97ba91bf

Tree-SHA512: 7de11d38b9125de63e72415f79d82f18818123a1b37f679f2229c4c82f5628dd7d1039dbc5dcdf1bc1c8c382e3e29de74a31d256e73872cbf1fa2687c52185ca
2025-02-14 14:11:54 -08:00
Ava Chow
504d0c21e2
Merge bitcoin/bitcoin#31439: validation: In case of a continued reindex, only activate chain in the end
c9136ca90605bbe29f005f538b92ff96ca360a13 validation: fix issue with an interrupted -reindex (Martin Zumsande)
a2675897e2a499aacbd0183fdccf1401953e8de5 validation: Don't loop over all chainstates in LoadExternalBlock (Martin Zumsande)

Pull request description:

  If a user interrupts a reindex while it is iterating over the block files, it will continue to reindex with the next node start (if the `-reindex` arg is dropped, otherwise it will start reindexing from scratch).
  However, due to an early call to `ActivateBestChainState()` that only exists to connect the genesis block during
  the original `-reindex`, it wil start connecting blocks immediately before having iterated through all block files.
  Because later headers above the minchainwork threshold won't be loaded in this case, `-assumevalid` will not
  be applied and the process is much slower due to script validation being done.

  Fix this by only calling `ActivateBestChainState()` here if Genesis is not connected yet (equivalent to `ActiveHeight() == -1`).
  Also simplify this spot by only doing this for the active chainstate instead of looping over all chainstates (first commit).

  This issue was discussed in the thread below https://github.com/bitcoin/bitcoin/pull/31346#discussion_r1856824817, the impact on assumevalid was found by l0rinc.

  The fix can be tested by manually aborting a `-reindex` e.g. on signet and observing in the debug log the order in which blockfiles are indexed / blocks are connected with this branch vs master.

ACKs for top commit:
  achow101:
    ACK c9136ca90605bbe29f005f538b92ff96ca360a13
  ryanofsky:
    Code review ACK c9136ca90605bbe29f005f538b92ff96ca360a13. Only comments changed since last review. Appreciate the new comments, I think they make a little clearer what things code is trying to do and what things are just side-effects.
  TheCharlatan:
    Re-ACK c9136ca90605bbe29f005f538b92ff96ca360a13

Tree-SHA512: 6f34abc317ad7e605ccc0c2f4615e4ea6978223d207f80f768f39cc135a9ac0adf31681fadfa2aed45324a5d27a4f68c5e118ee7eec18ca5c40ef177caa9cc47
2025-02-14 13:59:34 -08:00
Ava Chow
0b48f77e10
Merge bitcoin/bitcoin#31413: rpc: Remove deprecated dummy alias for listtransactions::label
fa8e0956c23789fb819ad1b31377711df091ec1b rpc: Remove deprecated dummy alias for listtransactions::label (MarcoFalke)

Pull request description:

  The RPC arg is not a dummy, but a label, so offering an undocumented alias is inconsistent with all other label interfaces and confusing at best, if not entirely unused.

  Fix it by removing the deprecated alias.

  This pull is a breaking change, but it should be limited, because it only affects someone using the deprecated named arg on this RPC. I can't imagine anyone doing this, because in all other places where label args are accepted, they are called `label`. If someone really didn't use `label` here as named arg, it would be trivial and less confusing for them to fix it up.

ACKs for top commit:
  achow101:
    ACK fa8e0956c23789fb819ad1b31377711df091ec1b
  rkrux:
    tACK fa8e0956c23789fb819ad1b31377711df091ec1b
  ryanofsky:
    Code review ACK fa8e0956c23789fb819ad1b31377711df091ec1b

Tree-SHA512: 0d0f3f53237ff9fac8c065b7d0a4245f5ff86efa427dbeeca711765494b7315a9d72b44751d346c76422847daf3d7ff90dbccb5ba200b089fb96128bd95da9f0
2025-02-14 13:29:17 -08:00
Ava Chow
21a0efaf8c
Merge bitcoin/bitcoin#29858: test: Add test for rpcwhitelistdefault
f0e5e4cdbec4df190c952d5a61d7a882a7e3b59e test: Add test for rpcwhitelistdefault (naiyoma)

Pull request description:

  This PR adds tests for `rpcwhitelistdefault.` The implementation is a continuation of this [PR](https://github.com/bitcoin/bitcoin/pull/17805).

  Applied suggestions  to include the tests in` rpc_whitelist.py` and to use a single node.

  PR covers three test cases:
  - rpcwhitelistdefault = 0, no permissions
  - rpcwhitelistdefault = 1, no permissions
  - rpcwhitelistdefault = 1, with user permissions

  I didn't add tests for rpcwhitelistdefault = 0 with user permissions since that is already tested here: [rpc_whitelist.py#L77](https://github.com/bitcoin/bitcoin/blob/master/test/functional/rpc_whitelist.py#L77).

ACKs for top commit:
  achow101:
    ACK f0e5e4cdbec4df190c952d5a61d7a882a7e3b59e
  ryanofsky:
    Code review ACK f0e5e4cdbec4df190c952d5a61d7a882a7e3b59e. PR seems very clear and simple, moving 1 test and adding 3 new tests.
  ismaelsadeeq:
    Tested and Code review ACK f0e5e4cdbec4df190c952d5a61d7a882a7e3b59e

Tree-SHA512: c3652940d2f23746e769ebe834e43dee47b7af8f258cbb133e38663aa8a05a1a8d0194d3008c3a10b0c54d11b5b95420c9cad0aa761c0fc1b9559277443b0696
2025-02-14 11:27:01 -08:00
Ava Chow
8a00b755e9
Merge bitcoin/bitcoin#31634: doc: Improve dependencies documentation
a759ea3e920dcba52798755ba9f3891f914afbb0 doc: Improve dependencies documentation (Nicola Leonardo Susca)

Pull request description:

  Initially there was a distinction between the compiler dependencies and
  other required dependencies (refs https://github.com/bitcoin/bitcoin/pull/23565) but the distinction was
  removed (refs https://github.com/bitcoin/bitcoin/pull/24585) which is why having two distinct tables could lead
  to confusion now.

ACKs for top commit:
  achow101:
    ACK a759ea3e920dcba52798755ba9f3891f914afbb0
  hodlinator:
    re-ACK a759ea3e920dcba52798755ba9f3891f914afbb0
  rkrux:
    ACK a759ea3e920dcba52798755ba9f3891f914afbb0

Tree-SHA512: 14aaf9356d65bd150c9993dcbc51b1b98c835a760b95e6d91e69460c97c18f1dd10eb52b9f1d70129e6aa5e361af3a55619fd35787ed4e1ec48909568adbb604
2025-02-14 11:04:24 -08:00
Ava Chow
e58605e04f
Merge bitcoin/bitcoin#31854: net: reduce CAddress usage to CService or CNetAddr
cd4bfaee103591f212b88afd17969c16c1056eb6 net: reduce CAddress usage to CService or CNetAddr (Vasil Dimov)

Pull request description:

  Using `CAddress` when only `CService` or `CNetAddr` is needed is excessive and confusing. Fix those occurrences to use the class they need:

  * `CConnman::CalculateKeyedNetGroup()` needs `CNetAddr`, not `CAddress`, thus change its argument.

  * Both callers of `CConnman::CreateNodeFromAcceptedSocket()` create a dummy `CAddress` from `CService`, so use `CService` instead.

  * `GetBindAddress()` only needs to return `CService`.

  * `CNode::addrBind` only needs to be `CService`.

ACKs for top commit:
  Sjors:
    ACK cd4bfaee10
  achow101:
    ACK cd4bfaee103591f212b88afd17969c16c1056eb6
  hodlinator:
    ACK cd4bfaee103591f212b88afd17969c16c1056eb6
  laanwj:
    Code review ACK cd4bfaee103591f212b88afd17969c16c1056eb6

Tree-SHA512: 0b41c1519784eeeaf9926c6a4d24f583b90c3376741f37a3199a3808b0dd6d143d3f929bd7c06f87b031f4fc1c2bd7a6dfc7d715ec1f79bf36b862c00fd67085
2025-02-14 10:56:14 -08:00