11d079d8b8 doc: update manual pages for v29.4 (fanquake)
184639fb1c doc: update release notes for v29.4 (fanquake)
873e354b6b build: bump version to v29.4 (fanquake)
Pull request description:
The [`v29.4rc1` bins](https://bitcoincore.org/bin/bitcoin-core-29.4/test.rc1/) have been up for nearly 2 weeks. There doesn't seem to be anything else that needs backporting, and no issues have been reported. Finalise `v29.4`.
ACKs for top commit:
sedited:
ACK 11d079d8b8
willcl-ark:
ACK 11d079d8b8
Tree-SHA512: 7beac3b0c2d7954146262c8be7027bbf42179ef5f8ebb66392aaab0e1255a06890770d27671f9a96b7f3e60e61c3f809cf0b83f2994bf41b947c6f1d5571eb59
These are unused and removing them avoids clang warnings like:
src/test/fuzz/deserialize.cpp:42:26: error: variable g_setup set but not used [-Werror,-Wunused-but-set-variable]
Full chainstate compaction can take minutes on large databases.
Move `CCoinsViewDB::CompactFull()` to a named `utxocompact` one-shot background thread so validation only schedules the work.
When validation selects compaction after a full flush, the chainstate was just written and another write is less likely to be needed immediately.
The coins view destructor waits for completion, and a mutex prevents compaction from using `m_db` while `ResizeCache()` replaces it.
Co-authored-by: Andrew Toth <andrewstoth@gmail.com>
Github-Pull: #35465
Rebased-From: 394e473d42
Full chainstate flushes are convenient maintenance points for long-term LevelDB cleanup because the chainstate was just written.
Randomize the trigger so nodes that flush near the same height do not compact together.
Add blocking chainstate compaction through `CCoinsViewDB::CompactFull()` and give each post-IBD full flush on the normal chainstate a 1/320 chance to start compaction.
With hourly flushes this averages roughly every two weeks and makes a six-month miss about one in a million.
This keeps the schedule stateless and leaves last-compaction height or timestamp bookkeeping out of chainstate metadata.
Co-authored-by: Andrew Toth <andrewstoth@gmail.com>
Github-Pull: #35465
Rebased-From: aa021b26f3
Add `CDBWrapper::GetProperty()` and expose it through `CCoinsViewDB::GetDBProperty()` so coins tests can inspect LevelDB runtime properties through the coins view.
Use it in a coins DB flush baseline that records the LevelDB layout after flushing while keeping readback coverage for the flushed coin and best block.
Co-authored-by: Andrew Toth <andrewstoth@gmail.com>
Github-Pull: #35465
Rebased-From: b10889d107
Seek compaction is causing a cascade effect in the chainstate DB, causing large parts of the database to be rewritten every ~hour.
Every periodic flush writes around 2 MiB. Since this is roughly the `write_buffer_size`, these writes regularly cause the memtable to rotate into a small L0 file. This file has a small seek budget, and with the random UTXO reads done during validation, it can get scheduled for seek compaction quickly.
That seek compaction pushes the small file down to L1. Since most UTXOs are already lower down in L4/L5, many reads that consult this file do not find the key there and continue downward. The bloom filter makes those misses cheap, but LevelDB still decrements the file's seek budget. The file then gets scheduled for another seek compaction, and the same pattern pushes it down through L2 and L3.
The expensive part happens around L3/L4. L4 has many ~32 MiB files holding the bulk of the UTXO set. When LevelDB compacts into L3, it may split the output into many smaller L3 files to limit how much L4 "grandparent" data any one output overlaps. Each of these small L3 files then gets its own small seek budget. Because chainstate keys are hash-random, each small L3 file can still have a broad key range, so many random reads consult it and quickly drain its budget. Once seek-compacted into L4, each tiny L3 file can overlap many L4 files, so compacting a few hundred KiB from L3 can require rewriting hundreds of MiB from L4. Repeating that across many small L3 files can rewrite most of the chainstate.
This is a poor fit for chainstate because UTXO keys are hash-random, the DB is large enough to have many levels, writes are relatively small and periodic, and reads are frequent. The result is that read misses trigger compactions much earlier than size pressure would, and those compactions have very high write amplification.
Disabling seek compaction may leave more files in upper levels for longer, so reads could theoretically consult more files. But Bitcoin Core enables bloom filters for all its LevelDB instances, so these misses are usually cheap in-memory filter checks rather than disk reads.
For the other DBs, the risk is much smaller. They also use bloom filters, and most are smaller and less read-heavy. With fewer levels and less random read pressure, disabling seek compaction should have little effect there.
Co-authored-by: l0rinc <pap.lorinc@gmail.com>
Github-Pull: #35313
Rebased-From: 6bfdb6093bba4710d0f8313ed0113967a8b5176f
`CalculateMaximumSignedInputSize()` is passed the outpoint being sized, but that context was not used when estimating the signed input size.
Pass the outpoint through so externally selected inputs are not underestimated.
Co-authored-by: Antoine Poinsot <darosior@protonmail.com>
Github-Pull: #35228
Rebased-From: cd8d3bd937
The FreeBSD, NetBSD, and OpenBSD build guides state that ZMQ support is compiled in when the package is installed. Since WITH_ZMQ defaults to OFF, update the wording to mention the required CMake option.
Github-Pull: #35283
Rebased-From: ca93ab808c
9ad085bd83 validation: correct lifetime of precomputed tx data (Antoine Poinsot)
Pull request description:
This backports #35209 to the version 29 branch.
ACKs for top commit:
achow101:
ACK 9ad085bd83
theuni:
ACK 9ad085bd83
Tree-SHA512: e9dd4d43ee348015d3b7b485bef4617c69ad8e0c70589f84a16eb2b1d97bdefd21ce5d72ad6a2d8d25bc90d98f65d53374ca921210cab3b0db773072126459f6
This makes sure `txsdata` always outlives the Script check queue (since local
objects are destructed in reverse order of construction).
This is the root cause for a security vulnerability reported by Cory Fields in
2024 that could be exploited by crafting an invalid block to cause nodes to
read freed memory. The vulnerability was covertly fixed in commit
`492e1f09943fcb6145c21d470299305a19e17d8b`.
See security advisory for CVE-2024-52911 for more details.
Github-Pull: #35209
Rebased-From: 1ed799fb21
This effectively reverts a3cb309e7c from PR #30194.
That PR reduced the multi_index type signatures as recommended upstream, but
this is no longer supported as of boost 1.91 because it is no longer necessary.
1.91 drops support for the pre-c++11 work-arounds that bloated the type
signatures to begin with.
The upstream `BOOST_MULTI_INDEX_ENABLE_MPL_SUPPORT` define is meant to provide
compatibility with removed features, but it does not work for this case. Using
`indexed_by` directly when defining the `multi_index` (as opposed to inheriting
from it) works with all versions, and avoids the use of the back-compat define.
This is a slight regression when building against boost < 1.91 because the
bloated type signatures are reintroduced in that case, but it's not significant
enough to go to the trouble of introducing version detection and ifdefs.
Github-Pull: #35175
Rebased-From: 0bc9d354df
CMake version 4.3 deprecated the imported target `Sqlite::Sqlite3`.
Use the preferred name `Sqlite3::Sqlite3` instead and provide an
alias for older versions of CMake. Also define the same alias when
using vcpkg.
Github-Pull: #34848
Rebased-From: 498b6eb6b5
The function signature for the `send` RPC is:
```
send [{"address":amount,...},{"data":"hex"},...] ( conf_target "estimate_mode" fee_rate options version )
```
The last example in the manpage is missing the `fee_rate` arg, but is trying to specify the `options` arg, by index.
The parser confuses the intended `options` arg as the missing `fee_rate` arg.
See:
```
$ bitcoin-cli -rpcuser=doggman -rpcpassword=donkey -rpcport=18554 -regtest send '{"bcrt1qusm48zmlzwr32csxdw4ar7atw260h22c9ten9l": 0.1}' 1 economical '{"add_to_wallet": false, "inputs": [{"txid":"0b7e1a471dc948b7a6187936b16e6d7d9833629b2f9dd8a392eb89928f63aaad", "vout":0}]}'
error code: -8
error message:
Cannot specify both conf_target and fee_rate. Please provide either a confirmation target in blocks for automatic fee estimation, or an explicit fee rate.
```
vs
```
$ bitcoin-cli -rpcuser=doggman -rpcpassword=donkey -rpcport=18554 -regtest send '{"bcrt1qusm48zmlzwr32csxdw4ar7atw260h22c9ten9l": 0.1}' 1 economical null '{"add_to_wallet": false, "inputs": [{"txid":"0b7e1a471dc948b7a6187936b16e6d7d9833629b2f9dd8a392eb89928f63aaad", "vout":0}]}'
{
"psbt": "cHNidP8BAHECAAAAAa2qY4+SieuSo9idL5tiM5h9bW6xNnkYprdIyR1HGn4LAAAAAAD9////AkR2DwQAAAAAFgAUpLDwJu+wFRHLQAgKAb0psk7UVd2AlpgAAAAAABYAFOQ3U4t/E4cVYgZrq9H7q3K0+6lYAAAAAAABAIUCAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wQC4wMA/////wLIF6gEAAAAABYAFLMY1zihXrefAA0DA5nld4MCPjkrAAAAAAAAAAAmaiSqIant4vYcP3HR3v0/qZnfo2lTdVxpBol5mWK0i+vYNpdOjPkAAAAAAQEfyBeoBAAAAAAWABSzGNc4oV63nwANAwOZ5XeDAj45KwEIawJHMEQCIElTV4pbUrsPR9qHWcioowVv3QVWHizxwevfD0u/I8YyAiBCY3OzF81PSLM00h4ueQkehYuxDFZu7Jk51iejphKnnwEhA0VKdYVSyBpWoxBwTDOupB58Fi3mEBs+u+OOqEYVd2sZACICA98YLWyH7dBCfXVxe7woiLSTgV1mJN8Zc8KgZ77pVSg+GNBMeT5UAACAAQAAgAAAAIABAAAAbAAAAAAA",
"txid": "625b71b314a6ac4f738634e29dc007cd5edc0427c1ae96ab706d06a62910cea2",
"hex": "02000000000101adaa638f9289eb92a3d89d2f9b6233987d6d6eb1367918a6b748c91d471a7e0b0000000000fdffffff0244760f0400000000160014a4b0f026efb01511cb40080a01bd29b24ed455dd8096980000000000160014e437538b7f13871562066babd1fbab72b4fba9580247304402204953578a5b52bb0f47da8759c8a8a3056fdd05561e2cf1c1ebdf0f4bbf23c6320220426373b317cd4f48b334d21e2e79091e858bb10c566eec9939d627a3a612a79f012103454a758552c81a56a310704c33aea41e7c162de6101b3ebbe38ea84615776b1900000000",
"complete": true
}
```
Github-Pull: #34561
Rebased-From: 50cf6838e6
The bpftrace project moved from iovisor/bpftrace to bpftrace/bpftraceand
removed the separate INSTALL.md file. Installation instructionsare now
in the README.md Quick Start section.
Github-Pull: #34510
Rebased-From: 42ee31e80c
c9a7bd90eb doc: Update man pages for v29.3 (sedited)
ea6af54a63 doc: Update release notes for 29.3 (sedited)
824de1e6b3 build: Bump version to 29.3 (sedited)
Pull request description:
Bump version in build and doc, and regenerate the man pages in preparation for the release of v29.3.
ACKs for top commit:
fanquake:
ACK c9a7bd90eb
willcl-ark:
ACK c9a7bd90eb
Tree-SHA512: ff05dcb829fb2d9e7a8053cd275415c53148db0b826845a556fff4d3ef04320ef063188c50c5dd24e62142e15763cbabe68ce1cb8eac9d78ccb605ec10519f9a
65173944ed QA: tool_wallet: Check that db.log is deleted with a lone legacy wallet, but not with a shared db environment (Luke Dashjr)
69a6b9b115 Bugfix: Wallet/Migration: Move backup into wallet directory when migrating from non-directory (Luke Dashjr)
cef01d0be5 Wallet/Migration: Skip moving the backup file back and forth for no reason (Luke Dashjr)
60f529027c Wallet/Migration: If loading the new watchonly or solvables wallet fails, log the correct wallet name in error message (Luke Dashjr)
7475d134f6 Wallet/bdb: Safely and correctly list files only used by the single wallet (Luke Dashjr)
Pull request description:
ACKs for top commit:
achow101:
ACK 65173944ed
furszy:
light ACK 65173944ed
Tree-SHA512: c10fe00dde512ca78cd6939a748b3875d0b40e9714997aedfd939a1dffdc7eaa2fd1779f3972a34b1c1d9a97d8f1ee1e082c970de15ac0e2ef5d9bbf3dc1d89a
We can use vswhere.exe directly to create a vs developer
prompt and so can remove this third party dependency.
Co-authored-by: David Gumberg <davidzgumberg@gmail.com>
Github-Pull: #32513
Rebased-From: 7ae0497eef
