465bca734e contrib: reject divergent verify-commits history (Lőrinc)
b3d1dca338 contrib: fail on verify-commits ancestry errors (Lőrinc)
Pull request description:
**Problem:** `verify-commits.py` checks a Git commit's history for trusted signatures and tree hashes back to configured roots.
The documented workflow runs this check after fetching a commit and before checkout, proceeding only when the script succeeds.
A commit that is an ancestor of a configured root is intentionally accepted without checking earlier history.
The script also takes this success path after Git errors or for divergent commits, even though neither establishes that relationship.
**Fix:** Require Git to prove the ancestor relationship before taking this success path.
**Reproducers:** Each commit can be validated manually.
<details><summary>Manual reproducer: Git error</summary>
Run this on `master` and at this PR's head:
```bash
contrib/verify-commits/verify-commits.py 0000000000000000000000000000000000000000 && echo ❌ || echo ✅
```
`master` exits successfully without verifying the missing commit, while the PR head rejects the Git error.
</details>
<details><summary>Manual reproducer: divergent history</summary>
On `master` and at this PR's head, create an unreferenced sibling of the trusted root and run the verifier:
```bash
root=$(head -n1 contrib/verify-commits/trusted-git-root)
divergent_commit=$(git commit-tree "$root^{tree}" -p "$root^" -m 'divergent commit')
contrib/verify-commits/verify-commits.py "$divergent_commit" && echo ❌ || echo ✅
```
`master` exits successfully without verifying the sibling commit, while the PR head rejects divergent history.
</details>
This issue was also found and disclosed responsibly by the Red Team 🟥.
ACKs for top commit:
151henry151:
tACK 465bca734e
jeanpablojp:
tACK 465bca734e
achow101:
ACK 465bca734e
sedited:
ACK 465bca734e
maflcko:
review ACK 465bca734e🥜
Tree-SHA512: 72b8cd9902d881e59a1d99fda8e5d511806826fa27c05a2c21a7d2eb62b2a5a0b1b6bdc67e8d19d57f9171278f4858fd019eb7890b960df0475ba4713683f0ac
de9b436ba3 depends: Switch from multilib to platform-specific toolchains (Hennadii Stepanov)
Pull request description:
Using the multilib GCC toolchain, as currently documented in [`depends/README.md`](4c1906a500/depends/README.md), has several issues, such as:
1. The [`g++-multilib`](https://packages.ubuntu.com/noble/g++-multilib) package conflicts with platform-specific cross-compiler packages. This means it is not possible to cross compile for `i686` and other platforms using the same set of installed packages.
2. The [`g++-multilib`](https://packages.ubuntu.com/noble/g++-multilib) package is not available for `arm64`:
```sh
$ sudo apt install g++-multilib
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package g++-multilib
```
3. Managing the multilib GCC toolchain requires additional code in both depends and Guix scripts.
This PR addresses all the issues mentioned above by switching from multilib to platform-specific toolchains.
Also see https://github.com/bitcoin/bitcoin/pull/22456.
---
Here are examples of building for different scenarions:
- Linux, `x86_64` or `arm64`, building with depends natively:
```sh
$ gmake -C depends -j $(nproc)
$ cmake -B build --toolchain depends/$(./depends/config.sub $(./depends/config.guess))/toolchain.cmake
$ cmake --build build -j $(nproc)
```
- Linux, `x86_64` or `arm64`, cross compiling for `i686-pc-linux-gnu`:
```sh
$ sudo apt install g++-i686-linux-gnu binutils-i686-linux-gnu
$ export HOST=i686-linux-gnu
$ gmake -C depends -j $(nproc)
$ cmake -B build-${HOST} --toolchain depends/${HOST}/toolchain.cmake
$ cmake --build build-${HOST} -j $(nproc)
```
- Linux, `x86_64`, cross compiling for `arm64`:
```sh
$ sudo apt install g++-aarch64-linux-gnu binutils-aarch64-linux-gnu
$ export HOST=aarch64-linux-gnu
$ gmake -C depends -j $(nproc)
$ cmake -B build-${HOST} --toolchain depends/${HOST}/toolchain.cmake
$ cmake --build build-${HOST} -j $(nproc)
```
- Linux, `arm64`, cross compiling for `x86_64`:
```sh
$ sudo apt install g++-x86-64-linux-gnu binutils-x86-64-linux-gnu
$ export HOST=x86_64-linux-gnu
$ gmake -C depends -j $(nproc)
$ cmake -B build-${HOST} --toolchain depends/${HOST}/toolchain.cmake
$ cmake --build build-${HOST} -j $(nproc)
```
ACKs for top commit:
fanquake:
ACK de9b436ba3
BrandonOdiwuor:
ACK de9b436ba3
Tree-SHA512: 453b4744974cdf56d6edfdbe93bb11e3bae3f9bc9cd99b9c57aee74e65fcdd3ac011a1dcc19f485ea3be427f4e9c6c6b0d704881369f719620f0cb299123e561
`verify-commits.py` must not authorize checkout for a commit whose history diverges from configured trust roots.
Require proof that the commit is an ancestor of a root before skipping checks, and identify the failing root in errors.
Co-authored-by: Rob Hamilton <6456095+Rob1Ham@users.noreply.github.com>
`verify-commits.py` must not authorize checkout when Git cannot inspect the requested commit or its ancestry.
Reject ancestry command errors and validate the exact trusted root through Git before reporting success.
Co-authored-by: Rob Hamilton <6456095+Rob1Ham@users.noreply.github.com>
Otherwise the build will fail with warnings about missing Python (removed
in the next commit). However the non-gui build does not need Python.
Note that #25573 contains the same change, because the use of -static-pie
causes warnings when combined with our/CMakes own PIE handling.
Can do this now that the GUI has been split out.
riscv64-linux-gnu failus due to
https://github.com/boostorg/test/issues/345:
```bash
[102%] Linking CXX executable ../../bin/test_bitcoin
/gnu/store/r03804zpq5i6wsalx0yaqrr5jb7pqrmv-binutils-cross-riscv64-linux-gnu-2.46.0/bin/riscv64-linux-gnu-ld: CMakeFiles/test_bitcoin.dir/main.cpp.o: in function `boost::fpe::disable(unsigned int)':
/bitcoin/depends/riscv64-linux-gnu/boost/include/boost/test/impl/execution_monitor.ipp:1538:(.text+0x9dc8): warning: fedisableexcept is not implemented and will always fail
/gnu/store/r03804zpq5i6wsalx0yaqrr5jb7pqrmv-binutils-cross-riscv64-linux-gnu-2.46.0/bin/riscv64-linux-gnu-ld: CMakeFiles/test_bitcoin.dir/main.cpp.o: in function `boost::fpe::enable(unsigned int)':
/bitcoin/depends/riscv64-linux-gnu/boost/include/boost/test/impl/execution_monitor.ipp:1502:(.text+0x9d76): warning: feenableexcept is not implemented and will always fail
collect2: error: ld returned 1 exit status
```
Darwin could be done after something like
https://github.com/bitcoin/bitcoin/pull/35756.
a92e93429e guix: Drop unused `(guix licenses)` import from `manifest_build.scm` (Hennadii Stepanov)
Pull request description:
This was overlooked in bitcoin/bitcoin#34948.
ACKs for top commit:
fanquake:
ACK a92e93429e
Tree-SHA512: ab1ff63b49da104b21c4731ea06d3a4db009712be7c50f907cd46d0ad2936f7a236915bbe436801f02ea32bd781dfb5f38d349dda0a99a563701bef1089f35a6
We don't need support for transactional memory.
> Disable TM clone registry in libgcc. It is enabled in libgcc by default.
> This option helps to reduce code size for embedded targets which do
> not use transactional memory.
https://gcc.gnu.org/install/configure.html
2cb3bfa8df scripted-diff: Use long form of shell options in Guix scripts (Hennadii Stepanov)
711eb10f08 guix: Add copyright headers to Guix scripts (Hennadii Stepanov)
80f831494e guix: Fix `glibc` version in comment (Hennadii Stepanov)
8916f7967e scripted-diff: Use C.UTF-8 locale in Guix scripts (Hennadii Stepanov)
Pull request description:
The C.UTF-8 locale is set by default in `guix shell`, and there is no reason to avoid it nowadays. This PR also silences superfluous warnings from Qt tools, making build logs cleaner and other issues easier to spot. For example:
```
Detected locale "C" with character encoding "ANSI_X3.4-1968", which is not UTF-8.
Qt depends on a UTF-8 locale, and has switched to "C.UTF-8" instead.
If this causes problems, reconfigure your locale. See the locale(1) manual
for more information.
```
Locales in the `guix-*` launch scripts have been updated as well for consistency with the rest of the codebase.
Additionally, the headers of the Guix scripts have been adjusted for [uniformity](https://github.com/bitcoin/bitcoin/pull/35775#discussion_r3636959268).
ACKs for top commit:
fanquake:
ACK 2cb3bfa8df
Tree-SHA512: 9e21d4ad50f5d583efdd8f79d9f96d45a92f1982ea3a422565bceea38aa50bbd9a9360972b37a1b49907e67523d68f3d1f889cb26179ce735453eb4fae4d3fa4
The C.UTF-8 locale is set by default in `guix shell`, and there is no
reason to avoid it nowadays. This change also silences superfluous
warnings from Qt tools, making build logs cleaner and other issues
easier to spot.
Locales in the `guix-*` launch scripts have been updated as well for
consistency with the rest of the codebase.
-BEGIN VERIFY SCRIPT-
sed -i "s/\<export LC_ALL=C\>/export LC_ALL=C.UTF-8/g" \
$( git grep -l "export LC_ALL=C" ./contrib/guix/* )
-END VERIFY SCRIPT-
Using shell parameter expansion ensures that on non-Linux hosts the
build script won't pass an empty string argument to CMake. This resolves
the "Ignoring empty string" warning.
701bc2dc02 contrib: Fix NameError in signet miner gbt() (Torkel Rogstad)
Pull request description:
The logging.warning call referenced `bci["bestblockhash"]`, a variable from the calling scope `do_generate()` that is not available inside the `Generate.gbt()` method. This would crash with a NameError when getblocktemplate returned a template based on an unexpected previous block.
Use the `bestblockhash` parameter that was already being passed in and used correctly in the comparison on the line above.
The bug was introduced in 7b31332370 when the gbt logic was extracted into its own method — the if-condition was updated but the logging call was not.
ACKs for top commit:
kevkevinpal:
ACK [701bc2d](701bc2dc02)
sedited:
ACK 701bc2dc02
Tree-SHA512: 53764db954dd40b68d2f92279fd609732402fdbac4a1838aeee81d366b2b7fbc8da86b07f6f05cee62cb8ecdc514f6e0af1e59c65380eec38cbeaa58ec3c10ed
9fa4076b20 test: Test merging implicit PSBTv0 with explicit PSBTv0 (w0xlt)
1660c18232 doc: Release notes for psbtv2 (Ava Chow)
470e52a5f8 fuzz: Enforce additional version invariants in PSBT fuzzer (Antoine Poinsot)
5bd0579c09 test: Tests for PSBT AddInput and AddOutput (Ava Chow)
b8b6e7f0c2 tests: Add PSBT unit test for ComputeTimeLock (Ava Chow)
0bc1c2e508 tests: Add test vectors from BIP 370 (Ava Chow)
e0e4dbdeb5 psbt: Change default psbt version to 2 (Ava Chow)
bcc1dca77b Add psbt_version to PSBT RPCs and default to v2 (Ava Chow)
ab38c30195 Implement PSBTv2 field merging (Ava Chow)
93e339e29f Implement PSBTv2 AddInput and AddOutput (Ava Chow)
b39c86ae60 Allow specifying PSBT version in constructor (Ava Chow)
dcc9a3c8df Implement PSBTv2 in decodepsbt (Ava Chow)
5770dbd39f Add PSBT::ComputeLockTime() (Ava Chow)
863cf47b33 Update test_framework/psbt.py for PSBTv2 (Ava Chow)
925161eaf0 Implement PSBTv2 fields de/ser (Ava Chow)
d9cf658ee0 Restrict joinpsbts to PSBTv0 only (Ava Chow)
3da0e16012 Replace PSBT.tx with PSBT::GetUnsignedTx and PSBT::GetUniqueID (Ava Chow)
c568624ff2 psbt: Return std::optional from PrecomputePSBTData (Ava Chow)
092de4f1f6 Replace PSBT::GetInputUTXO with PSBTInput::GetUTXO (Ava Chow)
82c9fe3179 psbt: Use PSBTInput and PSBTOutput fields instead of accessing global tx (Ava Chow)
95897507e9 psbt: AddInput and AddOutput should take only PSBTInput and PSBTOutput (Ava Chow)
1b7d323a72 Add PSBTInput::GetOutPoint (Ava Chow)
543d3e1cdc psbt: add PSBTv2 global tx fields (Ava Chow)
c01c7f068c psbt: Remove default constructor (Ava Chow)
9671aa08c2 psbt: add tx input and output fields in PSBTInput and PSBTOutput (Ava Chow)
990b084f11 Have PSBTInput and PSBTOutput know the PSBT's version (Ava Chow)
7eacc21ff6 psbt: make PSBT structs into classes (Ava Chow)
f926c326bb gui: Store PSBT in std::optional in PSBTOperationsDialog (Ava Chow)
1e2d146b47 psbt: Refactor duplicate key lookup and size checks (Ava Chow)
88384180d3 test: PSBTs should roundtrip through RPCs that do nothing (Ava Chow)
001877500d test: construct psbt with unknown field programmatically (David Gumberg)
0cb884e6df psbt: Fill hash preimages and taproot builder from SignatureData (Ava Chow)
Pull request description:
BIP 370 PSBTv2 introduces several new fields and different invariants for PSBT. This PR implements those new fields and restructures the PSBT implementation to match PSBTv2 but still remain compatible with PSBTv0.
ACKs for top commit:
nervana21:
re-ACK 9fa4076b20
theStack:
re-ACK 9fa4076b20
w0xlt:
ACK 9fa4076b20
Tree-SHA512: ab0a5ada4fa5fca27ba9ec9c291a44b30e69d6db11971957572d86c58c71c4caa4557dc25f403e1170ba4fac751306d074cc582defefc6e2fdd37be51c3d9dd0
758f208cc1 contrib: override system locale in gen-manpages.py (Sjors Provoost)
Pull request description:
`bitcoin-qt --help` emits a translation of "version", which creates a diff when updating or verifying man pages.
The script aborts earlier however, because `bitcoin-qt --version` also emits a localized output, which triggers the `Copyright (C)` assertion on a translated term like "Auteursrecht".
Fix this by passing `--lang=en` to both `bitcoin-qt` invocations.
None of the actual command options are translated, so this commit does not affect the actual manual page.
Noticed while verifying the manual updates in #34800 on macOS with Dutch system locale.
See also https://github.com/bitcoin/bitcoin/blob/master/test/lint/lint-locale-dependence.py notes about localization (though the issue here is translation).
ACKs for top commit:
achow101:
ACK 758f208cc1
sedited:
ACK 758f208cc1
hebasto:
ACK 758f208cc1.
Tree-SHA512: 0947391b85a637ae6d1d0cf4b8de4ab74b42e772bf7d70991cf2b4e035c2cb99f0f46c6cb2cd8aac9cc1be22a9e8329594536688622d3bacab22289c8151e89d
