3e3c3d66de doc: update manual pages for v30.3 (fanquake)
0a1d238b25 doc: update release notes for v30.3 (fanquake)
ad5b171a7b build: bump version to v30.3 (fanquake)
Pull request description:
The [`v30.3rc1` bins](https://bitcoincore.org/bin/bitcoin-core-30.3/test.rc1/) have been up for nearly 2 weeks. There doesn't seem to be anything else that needs backporting, and no issues have been reported. Finalise `v30.3`.
ACKs for top commit:
hebasto:
ACK 3e3c3d66de.
sedited:
ACK 3e3c3d66de
Tree-SHA512: 03677d5cdd3c910a8e84e91484358a3377bfc11db48dea43cd386808c8674359872e7c90ae91960a9e1f23a7d57affe74d1c96d55d3263c71be2582aadc7b31c
These are unused and removing them avoids clang warnings like:
src/test/fuzz/deserialize.cpp:42:26: error: variable g_setup set but not used [-Werror,-Wunused-but-set-variable]
Github-Pull: #34918
Rebased-From: fabbfec3b0
Full chainstate compaction can take minutes on large databases.
Move `CCoinsViewDB::CompactFull()` to a named `utxocompact` one-shot background thread so validation only schedules the work.
When validation selects compaction after a full flush, the chainstate was just written and another write is less likely to be needed immediately.
The coins view destructor waits for completion, and a mutex prevents compaction from using `m_db` while `ResizeCache()` replaces it.
Co-authored-by: Andrew Toth <andrewstoth@gmail.com>
Github-Pull: #35465
Rebased-From: 394e473d42
Full chainstate flushes are convenient maintenance points for long-term LevelDB cleanup because the chainstate was just written.
Randomize the trigger so nodes that flush near the same height do not compact together.
Add blocking chainstate compaction through `CCoinsViewDB::CompactFull()` and give each post-IBD full flush on the normal chainstate a 1/320 chance to start compaction.
With hourly flushes this averages roughly every two weeks and makes a six-month miss about one in a million.
This keeps the schedule stateless and leaves last-compaction height or timestamp bookkeeping out of chainstate metadata.
Co-authored-by: Andrew Toth <andrewstoth@gmail.com>
Github-Pull: #35465
Rebased-From: aa021b26f3
Add `CDBWrapper::GetProperty()` and expose it through `CCoinsViewDB::GetDBProperty()` so coins tests can inspect LevelDB runtime properties through the coins view.
Use it in a coins DB flush baseline that records the LevelDB layout after flushing while keeping readback coverage for the flushed coin and best block.
Co-authored-by: Andrew Toth <andrewstoth@gmail.com>
Github-Pull: #35465
Rebased-From: b10889d107
Seek compaction is causing a cascade effect in the chainstate DB, causing large parts of the database to be rewritten every ~hour.
Every periodic flush writes around 2 MiB. Since this is roughly the `write_buffer_size`, these writes regularly cause the memtable to rotate into a small L0 file. This file has a small seek budget, and with the random UTXO reads done during validation, it can get scheduled for seek compaction quickly.
That seek compaction pushes the small file down to L1. Since most UTXOs are already lower down in L4/L5, many reads that consult this file do not find the key there and continue downward. The bloom filter makes those misses cheap, but LevelDB still decrements the file's seek budget. The file then gets scheduled for another seek compaction, and the same pattern pushes it down through L2 and L3.
The expensive part happens around L3/L4. L4 has many ~32 MiB files holding the bulk of the UTXO set. When LevelDB compacts into L3, it may split the output into many smaller L3 files to limit how much L4 "grandparent" data any one output overlaps. Each of these small L3 files then gets its own small seek budget. Because chainstate keys are hash-random, each small L3 file can still have a broad key range, so many random reads consult it and quickly drain its budget. Once seek-compacted into L4, each tiny L3 file can overlap many L4 files, so compacting a few hundred KiB from L3 can require rewriting hundreds of MiB from L4. Repeating that across many small L3 files can rewrite most of the chainstate.
This is a poor fit for chainstate because UTXO keys are hash-random, the DB is large enough to have many levels, writes are relatively small and periodic, and reads are frequent. The result is that read misses trigger compactions much earlier than size pressure would, and those compactions have very high write amplification.
Disabling seek compaction may leave more files in upper levels for longer, so reads could theoretically consult more files. But Bitcoin Core enables bloom filters for all its LevelDB instances, so these misses are usually cheap in-memory filter checks rather than disk reads.
For the other DBs, the risk is much smaller. They also use bloom filters, and most are smaller and less read-heavy. With fewer levels and less random read pressure, disabling seek compaction should have little effect there.
Co-authored-by: l0rinc <pap.lorinc@gmail.com>
Github-Pull: #35313
Rebased-From: 6bfdb6093bba4710d0f8313ed0113967a8b5176f
`CalculateMaximumSignedInputSize()` is passed the outpoint being sized, but that context was not used when estimating the signed input size.
Pass the outpoint through so externally selected inputs are not underestimated.
Co-authored-by: Antoine Poinsot <darosior@protonmail.com>
Github-Pull: #35228
Rebased-From: cd8d3bd937
The FreeBSD, NetBSD, and OpenBSD build guides state that ZMQ support is compiled in when the package is installed. Since WITH_ZMQ defaults to OFF, update the wording to mention the required CMake option.
Github-Pull: #35283
Rebased-From: ca93ab808c
17de6d2680 validation: correct lifetime of precomputed tx data (Antoine Poinsot)
Pull request description:
This backports #35209 to the version 30 branch.
ACKs for top commit:
achow101:
ACK 17de6d2680
Tree-SHA512: cdb1f9cb8166a5eb0c0ca0fb88583077a050e07032be8665ef1b0c446b2405ce7ce0138688507a63df4096d52e5dc57a163c239927706043d6431388ce5c16f5
This makes sure `txsdata` always outlives the Script check queue (since local
objects are destructed in reverse order of construction).
This is the root cause for a security vulnerability reported by Cory Fields in
2024 that could be exploited by crafting an invalid block to cause nodes to
read freed memory. The vulnerability was covertly fixed in commit
`492e1f09943fcb6145c21d470299305a19e17d8b`.
See security advisory for CVE-2024-52911 for more details.
Github-Pull: #35209
Rebased-From: 1ed799fb21
This effectively reverts a3cb309e7c from PR #30194.
That PR reduced the multi_index type signatures as recommended upstream, but
this is no longer supported as of boost 1.91 because it is no longer necessary.
1.91 drops support for the pre-c++11 work-arounds that bloated the type
signatures to begin with.
The upstream `BOOST_MULTI_INDEX_ENABLE_MPL_SUPPORT` define is meant to provide
compatibility with removed features, but it does not work for this case. Using
`indexed_by` directly when defining the `multi_index` (as opposed to inheriting
from it) works with all versions, and avoids the use of the back-compat define.
This is a slight regression when building against boost < 1.91 because the
bloated type signatures are reintroduced in that case, but it's not significant
enough to go to the trouble of introducing version detection and ifdefs.
Github-Pull: #35175
Rebased-From: 0bc9d354df
Apply the timeout factor inside the add_block function.
Also, force named args for the two expected strings.
Also, add trailing comma for style.
Github-Pull: #35080
Rebased-From: fa02eb87df
When a large cluster of unconfirmed transactions exceeds the limit,
calculateCombinedBumpFee() returns std::nullopt.
Previously, we continued executing and the optional value was
accessed unconditionally, leading to a std::bad_optional_access
exception.
Fix this by returning early when the returned bumped fee is null.
Note:
This is a crash for the GUI, and an uncaught exception for the RPC
bumpfee and psbtbumpfee.
Github-Pull: #34870
Rebased-From: 6072a2a6a1
The comparison is evaluated before the assignment, so total_amount
ends up holding a boolean instead of the actual amount:
total_amount = (a - b < c)
which is not what we want here. This has been working by accident.
Github-Pull: #34888
Rebased-From: 0026b330c4
CMake version 4.3 deprecated the imported target `Sqlite::Sqlite3`.
Use the preferred name `Sqlite3::Sqlite3` instead and provide an
alias for older versions of CMake. Also define the same alias when
using vcpkg.
Github-Pull: #34848
Rebased-From: 498b6eb6b5
Rather than looking for /translations, which might not exist (it doesn't
in a recent brew installed qt on macOS). i.e:
```bash
ls /opt/homebrew/opt/qtbase/share/qt
doc
libexec
metatypes
mkspecs
modules
plugins
sbom
```
Github-Pull: #34787
Rebased-From: 66d80d57b4
Calling shutil.make_archive(), does not preserve symlinks when using the
zip format, see https://github.com/python/cpython/issues/139679.
Call `zip` using subprocess instead. This code is only run when using a
macos machine, and I think it's safe to assume that zip is available, same
as codesign, and all other tools we call in this script.
Github-Pull: #34787
Rebased-From: ab137cbfe2
The `fee` field in the `getblock` RPC result (verbosity 2 and 3) may be
omitted when block undo data is not available. Marking it optional in
the `RPCResult` aligns the documented schema with the runtime behavior.
Github-Pull: #34702
Rebased-From: f580cc7e9f
Add validation for pubkeys in MuSig2 pubnonce and partial signature deserialization to prevent crashes with invalid curve points.
- Validate aggregate and participant pubkeys in PSBT MuSig2 fields
- Add comprehensive test coverage for invalid pubkey rejection
- Ensure proper error handling during PSBT deserialization
Github-Pull: #34219
Rebased-From: f51665bee7
Improve dependencies.md to document IPC dependencies better:
- Link to native_capnp.mk file not capnp.mk file so it's possible to see what
version of Cap'n Proto is being used in release binaries. This is important
since #31895 dropped the "Version Used" column and the capnp.mk file does not
include version number.
- Indicate Capn"Proto is used for IPC and link to multiprocess.md documenting
the feature.
- Link to correct PR requiring Cap'n Proto 0.7.1. Previous link was
pointing at PR which required 0.7.0.
- Mention libmultiprocess as a dependency even though it is included as a git
subtree and can be built as a cmake subproject. Libmultiprocess still needs
to be built separately when cross compiling, and is useful to build separately
when developing, and is still a depends package.
Based on 2cf352fd8e from #33623 by willcl-ark
which made similar changes in the 29.x branch.
Github-Pull: #34706
Rebased-From: b87a1c27c9
