Commit Graph

55 Commits

Author SHA1 Message Date
Cory Fields
0626b90f50 multiprocess: align our logging with libmultiprocess's
Without this change, logging (even if unused) may account for a
substantial portion of bitcoin-node's and/or client's runtime cpu usage, due
to libmultiprocess's expensive message serialization.

This (along with some recent upstream changes) avoids the overhead by opting
out of log handling for messages that we're not interested in.

Info, Warning, and Error are logged unconditionally to match our behavior
elsewhere. See BCLog::Logger::GetCategoryLogLevel .
2025-10-10 21:20:50 +00:00
Cory Fields
9d068225ee multiprocess: update multiprocess EventLoop construction to use options
This uses the constructors recently added upstream.
2025-10-10 21:20:50 +00:00
Ryan Ofsky
eda91b07fd Merge commit '0f01e1577f7c6734eb345139a12aba329ef22a5f' into pr/subtree-6 2025-10-07 10:12:08 -04:00
Ryan Ofsky
c49a43591f Merge commit '535fa0ad0d2637f845beae92ea9dbbbbbe377c74' into pr/subtree-5 2025-09-17 05:30:43 -04:00
Ava Chow
0b0bd74c3e Merge bitcoin/bitcoin#33312: clang-tidy: Disable UndefinedBinaryOperatorResult check in src/ipc
589b65f06c clang-tidy: Disable `UndefinedBinaryOperatorResult` check in `src/ipc` (Hennadii Stepanov)

Pull request description:

  The warnings are false positive and have been fixed upstream. See: https://github.com/capnproto/capnproto/pull/2334.

  This PR:

  1. Disables the `UndefinedBinaryOperatorResult` clang-tidy check for source files generated by the `mpgen` tool.

  2. Is an alternative to the draft https://github.com/bitcoin/bitcoin/pull/33281.

  3. Fixes https://github.com/bitcoin/bitcoin/issues/33256.

ACKs for top commit:
  Sjors:
    ACK 589b65f06c
  fjahr:
    ACK 589b65f06c
  achow101:
    ACK 589b65f06c
  ryanofsky:
    Code review ACK 589b65f06c. Thanks for the fix!

Tree-SHA512: 6d376a82641a5b85d4dd1fa164fdcbd8e15f1262e7d4f582f4d9959031d35852e28ff1b8268336e39ba6779fdd10ecdb986af42407d0545f4217f41d64556272
2025-09-08 13:28:01 -07:00
Ryan Ofsky
a4ee70e5b6 Merge commit 'a334bbe9b79ddf1999003c792bc8945639b7e9c1' into pr/subtree-4 2025-09-05 15:43:16 -04:00
Hennadii Stepanov
589b65f06c clang-tidy: Disable UndefinedBinaryOperatorResult check in src/ipc
The warnings are false positive and have been fixed upstream.
See: https://github.com/capnproto/capnproto/pull/2334.

This change disables the `UndefinedBinaryOperatorResult` clang-tidy
check for source files generated by the `mpgen` tool.
2025-09-04 23:07:40 +01:00
Ryan Ofsky
323b3fd272 Merge commit 'dd68d0f40b614474f24469fbe1ba02f8f9146b31' into pr/subtree-3 2025-08-22 17:15:44 -04:00
Ryan Ofsky
2160995916 ipc: Add Ctrl-C handler for spawned subprocesses
This fixes an error reported by Antoine Poinsot <darosior@protonmail.com> in
https://github.com/bitcoin-core/libmultiprocess/issues/123 that does not happen
in master, but does happen with https://github.com/bitcoin/bitcoin/pull/10102
applied, where if Ctrl-C is pressed when `bitcoin-node` is started, it is
handled by both `bitcoin-node` and `bitcoin-wallet` processes, causing the
wallet to shutdown abruptly instead of waiting for the node and shutting down
cleanly.

This change fixes the problem by having the wallet process print to stdout when
it receives a Ctrl-C signal but not otherwise react, letting the node shut
everything down cleanly.
2025-08-04 13:38:26 -04:00
Ryan Ofsky
7f65aac78b ipc: Avoid waiting for clients to disconnect when shutting down
This fixes behavior reported by Antoine Poinsot <darosior@protonmail.com>
https://github.com/bitcoin/bitcoin/pull/29409#issuecomment-2546088852 where if
an IPC client is connected, the node will wait forever for it to disconnect
before exiting.
2025-08-04 13:38:26 -04:00
Ryan Ofsky
9a9fb19536 ipc: Use EventLoopRef instead of addClient/removeClient
Use EventLoopRef to avoid reference counting bugs and be more exception safe
and deal with removal of addClient/removeClient methods in
https://github.com/bitcoin-core/libmultiprocess/pull/160

A test update is also required due to
https://github.com/bitcoin-core/libmultiprocess/pull/160 to deal with changed
reference count semantics. In IpcPipeTest(), it is now necessary to destroy
the client Proxy object instead of just the client Connection object to
decrease the event loop reference count and allow the loop to exit so the test
does not hang on shutdown.
2025-08-04 12:38:26 -05:00
Ryan Ofsky
5c45bc989b Merge commit 'e886c65b6b37aaaf5d22ca68bc14e55d8ec78212' into pr/ipc-stop-base 2025-08-04 13:38:26 -04:00
Sjors Provoost
74b7e9c7db refactor: modernize deprecated ipc headers
Co-authored-by: fanquake <fanquake@gmail.com>
2025-06-19 16:29:55 +02:00
Sjors Provoost
94959b8dee Add checkBlock to Mining interface
Use it in miner_tests.

The getblocktemplate and generateblock RPC calls don't use this,
because it would make the code more verbose.
2025-06-14 14:32:45 +02:00
Sjors Provoost
6077157531 ipc: drop BlockValidationState special handling
The Mining interface avoids using BlockValidationState.
2025-06-14 14:32:45 +02:00
Ryan Ofsky
9f6565488f Merge commit '154af1eea1170f5626aa1c5f19cc77d1434bcc9d' into HEAD 2025-05-29 13:57:08 -04:00
MarcoFalke
fa123afa0e Reject + sign when checking -ipcfd 2025-05-15 22:12:15 +02:00
Ryan Ofsky
3f6fb40114 Merge commit 'a2f28e4be96e92079a219567cf20214996aefc53' as 'src/ipc/libmultiprocess' 2025-04-02 21:41:16 +08:00
Sjors Provoost
d4020f502a Add waitNext() to BlockTemplate interface 2025-02-19 17:20:57 +01:00
glozow
6b165f5906 Merge bitcoin/bitcoin#31384: mining: bugfix: Fix duplicate coinbase tx weight reservation
386eecff5f doc: add release notes (ismaelsadeeq)
3eaa0a3b66 miner: init: add `-blockreservedweight` startup option (ismaelsadeeq)
777434a2cd doc: rpc: improve `getmininginfo` help text (ismaelsadeeq)
c8acd4032d init: fail to start when `-blockmaxweight` exceeds `MAX_BLOCK_WEIGHT` (ismaelsadeeq)
5bb31633cc test: add `-blockmaxweight` startup option functional test (ismaelsadeeq)
2c7d90a6d6 miner: bugfix: fix duplicate weight reservation in block assembler (ismaelsadeeq)

Pull request description:

  * This PR attempts to fix the duplicate coinbase weight reservation issue we currently have.
  * Fixes #21950

  We reserve 4000 weight units for coinbase transaction in `DEFAULT_BLOCK_MAX_WEIGHT`

  7590e93bc7/src/policy/policy.h (L23)

  And also reserve additional `4000` weight units in the default `BlockCreationOptions` struct.

  7590e93bc7/src/node/types.h (L36-L40)

  **Motivation**

  - This issue was first noticed during a review here https://github.com/bitcoin/bitcoin/pull/11100#discussion_r136157411)
  - It was later reported in issue #21950.
  - I also came across the bug while writing a test for building the block template. I could not create a block template above `3,992,000` in the block assembler, and this was not documented anywhere. It took me a while to realize that we were reserving space for the coinbase transaction weight twice.

  ---
  This PR fixes this by consolidating the reservation to be in a single location in the codebase.

  This PR then adds a new startup option `-blockreservedweight` whose default is `8000` that can be used to lower or increase the block reserved weight for block header, txs count, coinbase tx.

ACKs for top commit:
  Sjors:
    ACK 386eecff5f
  fjahr:
    Code review ACK 386eecff5f
  glozow:
    utACK 386eecff5f, nonblocking nits. I do think the release notes should be clarified more
  pinheadmz:
    ACK 386eecff5f

Tree-SHA512: f27efa1da57947b7f4d42b9322b83d13afe73dd749dd9cac49360002824dd41c99a876a610554ac2d67bad7485020b9dcc423a8e6748fc79d6a10de6d4357d4c
2025-02-10 08:26:01 -05:00
ismaelsadeeq
2c7d90a6d6 miner: bugfix: fix duplicate weight reservation in block assembler
- This commit renamed coinbase_max_additional_weight to block_reserved_weight.

- Also clarify that the reservation is for block header, transaction count
  and coinbase transaction.
2025-02-04 11:53:03 -05:00
Ryan Ofsky
2221c8814d depends: Update libmultiprocess library before converting to subtree
This should be the final update to the libmultiprocess package via the depends
system. It brings in the libmultiprocess cmake changes from
https://github.com/chaincodelabs/libmultiprocess/pull/136 needed to support
building as subtree. After this, a followup PR will add libmultiprocess as a
git subtree and depends will just use the git subtree instead of hardcoding its
own version hash.

Since there have been libmultiprocess API changes since the last update, this
commit also updates bitcoin code to be compatible with them.

This update brings in the following changes:

https://github.com/chaincodelabs/libmultiprocess/pull/121 ProxyClientBase: avoid static_cast to partially constructed object
https://github.com/chaincodelabs/libmultiprocess/pull/120 proxy-types.h: add static_assert to detect int/enum size mismatch
https://github.com/chaincodelabs/libmultiprocess/pull/127 ProxyClientBase: avoid static_cast to partially destructed object
https://github.com/chaincodelabs/libmultiprocess/pull/129 Fix "disconnected: write(m_post_fd, &buffer, 1): Broken pipe" EventLoop shutdown races.
https://github.com/chaincodelabs/libmultiprocess/pull/130 refactor: Add CleanupRun function to dedup clean list code
https://github.com/chaincodelabs/libmultiprocess/pull/131 doc: fix startAsyncThread comment
https://github.com/chaincodelabs/libmultiprocess/pull/133 Fix debian "libatomic not found" error in downstream builds
https://github.com/chaincodelabs/libmultiprocess/pull/94 c++ 20 cleanups
https://github.com/chaincodelabs/libmultiprocess/pull/135 refactor: proxy-types.h API cleanup
https://github.com/chaincodelabs/libmultiprocess/pull/136 cmake: Support being included with add_subdirectory
https://github.com/chaincodelabs/libmultiprocess/pull/137 doc: Fix broken markdown links
2025-01-26 14:13:47 -05:00
Sjors Provoost
c991cea1a0 Remove processNewBlock() from mining interface
processNewBlock was added in 7b4d3249ce, but became unnecessary with the introduction of interfaces::BlockTemplate::submitSolution in 7b4d3249ce.

getTransactionsUpdated() is only needed by the implementation of waitFeesChanged() (not yet part of the interface).
2024-12-18 09:20:26 +07:00
Sjors Provoost
9a47852d88 Remove getTransactionsUpdated() from mining interface
It's unnecessary to expose it via this interface.
2024-12-18 09:19:12 +07:00
Sjors Provoost
bfc4e029d4 Remove testBlockValidity() from mining interface
It's very low level and not used by the proposed Template Provider.

This method was introduced in d8a3496b5a
and a74b0f93ef.
2024-12-18 09:18:21 +07:00
Ryan Ofsky
cd3d9fa5ea Merge bitcoin/bitcoin#31318: Drop script_pub_key arg from createNewBlock
52fd1511a7 test: drop scriptPubKeyIn arg from CreateNewBlock (Sjors Provoost)
ff41b9e296 Drop script_pub_key arg from createNewBlock (Sjors Provoost)
7ab733ede4 rpc: rename coinbase_script to coinbase_output_script (Sjors Provoost)

Pull request description:

  Providing a script for the coinbase transaction is only done in test code and for (unoptimized) CPU solo mining.

  Production miners use the `getblocktemplate` RPC which omits the coinbase transaction entirely from its block template, leaving it to external (pool) software to construct it.

  This commit removes the `script_pub_key argument` from `createNewBlock()` in the Mining interface.

  A coinbase script can still be passed via `BlockCreateOptions` instead. Tests are modified to do so.

ACKs for top commit:
  ryanofsky:
    Code review ACK 52fd1511a7. No change since last review other than rebase
  TheCharlatan:
    Re-ACK 52fd1511a7
  vasild:
    ACK 52fd1511a7

Tree-SHA512: c4b3a53774d9a5dc90950e77f47a64dbb68f971baffbb9a0d8f59332ef8e52d0c039130c925bde73135b3d0e79e65d91d1df30dc4cff13f32d8a72e5c56669d8
2024-12-17 11:50:44 -05:00
Hennadii Stepanov
df27ee9f02 refactor: Fix "modernize-use-starts-ends-with" clang-tidy warning 2024-12-12 11:51:40 +00:00
Sjors Provoost
ff41b9e296 Drop script_pub_key arg from createNewBlock
Providing a script for the coinbase transaction is only done in test code
and for CPU solo mining.

Production miners use the getblocktemplate RPC which omits the coinbase
transaction entirely from its block template, leaving it to external (pool)
software to construct it.

A coinbase script can still be passed via BlockCreateOptions instead.

A temporary overload is added so that the test can be modified in the
next commit.
2024-12-04 12:44:57 +07:00
Sjors Provoost
9aa50152c1 Add destroy to BlockTemplate schema
This ensures that if a client no longer needs a block template,
the node can clear its memory as soon as possible.

A block template may hold on to transactions that are no longer
in the mempool, so this can be significant.
2024-11-15 19:14:54 +01:00
fanquake
726cbee955 doc: correct typos 2024-11-11 14:14:39 +00:00
Sjors Provoost
525e9dcba0 Add submitSolution to BlockTemplate interface 2024-09-26 10:04:45 +02:00
Sjors Provoost
47b4875ef0 Add getCoinbaseMerklePath() to Mining interface 2024-09-26 10:04:45 +02:00
Ryan Ofsky
d043950ba2 multiprocess: Add serialization code for BlockValidationState
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
2024-09-23 16:03:04 -04:00
Ryan Ofsky
33c2eee285 multiprocess: Add IPC wrapper for Mining interface 2024-09-23 16:03:04 -04:00
Russell Yanofsky
06882f8401 multiprocess: Add serialization code for vector<char> 2024-09-23 15:03:04 -05:00
Russell Yanofsky
095286f790 multiprocess: Add serialization code for CTransaction
Add support for passing CTransaction and CTransactionRef types to IPC
functions.

These types can't be passed currently because IPC serialization code currently
only supports deserializing types that have an Unserialize() method, which
CTransaction does not, because it is supposed to represent immutable
transactions. Work around this by adding a CustomReadField overload that will
call CTransaction's deserialize_type constructor.

These types also can't be passed currently because serializing transactions
requires TransactionSerParams to be set. Fix this by setting TX_WITH_WITNESS as
default serialization parameters for IPC code.
2024-09-23 15:03:04 -05:00
Ryan Ofsky
69dfeb1876 multiprocess: update common-types.h to use C++20 concepts
Idea came from review comment by ion-
https://github.com/bitcoin/bitcoin/pull/30510#discussion_r1757372497
2024-09-23 16:03:04 -04:00
Ryan Ofsky
73fe7d7230 multiprocess: Add unit tests for connect, serve, and listen functions 2024-09-06 09:08:10 -04:00
Russell Yanofsky
955d4077aa multiprocess: Add IPC connectAddress and listenAddress methods
Allow listening on and connecting to unix sockets.
2024-09-06 09:08:10 -04:00
merge-script
d4cc0c6845 Merge bitcoin/bitcoin#30750: scripted-diff: LogPrint -> LogDebug
fa09cb41f5 refactor: Remove unused LogPrint (MarcoFalke)
3333415890 scripted-diff: LogPrint -> LogDebug (MarcoFalke)

Pull request description:

  `LogPrint` has many issues:

  * It seems to indicate that something is being "printed", however config options such as `-printtoconsole` actually control what and where something is logged.
  * It does not mention the log severity (debug).
  * It is a deprecated alias for `LogDebug`, according to the dev notes.
  * It wastes review cycles, because reviewers sometimes point out that it is deprecated.
  * It makes the code inconsistent, when both are used, possibly even in lines right next to each other (like in `InitHTTPServer`)

  Fix all issues by removing the deprecated alias.

  I checked all conflicting pull requests and at the time of writing there are no conflicts, except in pull requests that are marked as draft, are yet unreviewed, or are blocked on feedback for other reasons. So I think it is fine to do now.

ACKs for top commit:
  stickies-v:
    ACK fa09cb41f5
  danielabrozzoni:
    utACK fa09cb41f5
  TheCharlatan:
    ACK fa09cb41f5

Tree-SHA512: 14270f4cfa3906025a0b994cbb5b2e3c8c2427c0beb19c717a505a2ccbfb1fd1ecf2fd03f6c52d22cde69a8d057e50d2207119fab2c2bc8228db3f10d4288d0f
2024-09-02 11:59:56 +01:00
Hennadii Stepanov
d71ac76842 build: Remove Autotools-based build system 2024-08-30 21:31:39 +01:00
MarcoFalke
3333415890 scripted-diff: LogPrint -> LogDebug
-BEGIN VERIFY SCRIPT-
 sed -i 's/\<LogPrint\>/LogDebug/g' $( git grep -l '\<LogPrint\>'  -- ./contrib/ ./src/ ./test/ ':(exclude)src/logging.h' )
-END VERIFY SCRIPT-
2024-08-29 13:49:57 +02:00
Hennadii Stepanov
90cec4d251 cmake: Add MULTIPROCESS option 2024-08-16 21:19:10 +01:00
Ryan Ofsky
6acec6b9ff multiprocess: Add type conversion code for UniValue types
Extend IPC unit test to cover this and verify the serialization happens
correctly.
2023-11-28 12:35:50 -05:00
Ryan Ofsky
0cc74fce72 multiprocess: Add type conversion code for serializable types
Allow any C++ object that has Serialize and Unserialize methods and can be
serialized to a bitcoin CDataStream to be converted to a capnproto Data field
and passed as arguments or return values to capnproto methods using the Data
type.

Extend IPC unit test to cover this and verify the serialization happens
correctly.
2023-11-28 12:35:50 -05:00
TheCharlatan
7d3b35004b refactor: Move system from util to common library
Since the kernel library no longer depends on the system file, move it
to the common library instead in accordance to the diagram in
doc/design/libraries.md.
2023-05-20 12:08:13 +02:00
TheCharlatan
00e9b97f37 refactor: Move fs.* to util/fs.*
The fs.* files are already part of the libbitcoin_util library. With the
introduction of the fs_helpers.* it makes sense to move fs.* into the
util/ directory as well.
2023-03-23 12:55:18 +01:00
TheCharlatan
106b46d9d2 Add missing fs.h includes
The inclusion of this header should not depend on the inclusion of other
headers that include fs.h themselves.
2023-03-23 12:54:27 +01:00
Hennadii Stepanov
306ccd4927 scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-

Commits of previous years:
- 2021: f47dda2c58
- 2020: fa0074e2d8
- 2019: aaaaad6ac9
2022-12-24 23:49:50 +00:00
fanquake
a63d4cb26a refactor: use <cstdlib> over stdlib.h
We currently use both. Consolidate on the former.
2022-09-23 10:48:47 +01:00