Files
bitcoin/.github/workflows/ci.yml
merge-script 4da01123df Merge bitcoin/bitcoin#30595: kernel: Introduce C header API
6c7a34f3b0 kernel: Add Purpose section to header documentation (TheCharlatan)
7e9f00bcc1 kernel: Allowing reducing exports (TheCharlatan)
7990463b10 kernel: Add pure kernel bitcoin-chainstate (TheCharlatan)
36ec9a3ea2 Kernel: Add functions for working with outpoints (TheCharlatan)
5eec7fa96a kernel: Add block hash type and block tree utility functions to C header (TheCharlatan)
f5d5d1213c kernel: Add function to read block undo data from disk to C header (TheCharlatan)
09d0f62638 kernel: Add functions to read block from disk to C header (TheCharlatan)
a263a4caf2 kernel: Add function for copying block data to C header (TheCharlatan)
b30e15f432 kernel: Add functions for the block validation state to C header (TheCharlatan)
aa262da7bc kernel: Add validation interface to C header (TheCharlatan)
d27e27758d kernel: Add interrupt function to C header (TheCharlatan)
1976b13be9 kernel: Add import blocks function to C header (TheCharlatan)
a747ca1f51 kernel: Add chainstate load options for in-memory dbs in C header (TheCharlatan)
070e77732c kernel: Add options for reindexing in C header (TheCharlatan)
ad80abc73d kernel: Add block validation to C header (TheCharlatan)
cb1590b05e kernel: Add chainstate loading when instantiating a ChainstateManager (TheCharlatan)
e2c1bd3d71 kernel: Add chainstate manager option for setting worker threads (TheCharlatan)
65571c36a2 kernel: Add chainstate manager object to C header (TheCharlatan)
c62f657ba3 kernel: Add notifications context option to C header (TheCharlatan)
9e1bac4585 kernel: Add chain params context option to C header (TheCharlatan)
337ea860df kernel: Add kernel library context object (TheCharlatan)
28d679bad9 kernel: Add logging to kernel library C header (TheCharlatan)
2cf136dec4 kernel: Introduce initial kernel C header API (TheCharlatan)

Pull request description:

  This is a first attempt at introducing a C header for the libbitcoinkernel library that may be used by external applications for interfacing with Bitcoin Core's validation logic. It currently is limited to operations on blocks. This is a conscious choice, since it already offers a lot of powerful functionality, but sits just on the cusp of still being reviewable scope-wise while giving some pointers on how the rest of the API could look like.

  The current design was informed by the development of some tools using the C header:

  * A re-implementation (part of this pull request) of [bitcoin-chainstate](https://github.com/bitcoin/bitcoin/blob/master/src/bitcoin-chainstate.cpp).
  * A re-implementation of the python [block linearize](https://github.com/bitcoin/bitcoin/tree/master/contrib/linearize) scripts: https://github.com/TheCharlatan/bitcoin/tree/kernelLinearize
  * A silent payment scanner: https://github.com/josibake/silent-payments-scanner
  * An electrs index builder: https://github.com/josibake/electrs/commits/electrs-kernel-integration
  * A rust bitcoin node: https://github.com/TheCharlatan/kernel-node
  * A reindexer: https://github.com/TheCharlatan/bitcoin/tree/kernelApi_Reindexer

  The library has also been used by other developers already:

  * A historical block analysis tool: https://github.com/ismaelsadeeq/mining-analysis
  * A swiftsync hints generator: https://github.com/theStack/swiftsync-hints-gen
  * Fast script validation in floresta: https://github.com/vinteumorg/Floresta/pull/456
  * A swiftsync node implementation: https://github.com/2140-dev/swiftsync/tree/master/node

  Next to the C++ header also made available in this pull request, bindings for other languages are available here:

  * Rust: https://github.com/TheCharlatan/rust-bitcoinkernel
  * Python: https://github.com/stickies-v/py-bitcoinkernel
  * Go: https://github.com/stringintech/go-bitcoinkernel
  * Java: https://github.com/yuvicc/java-bitcoinkernel

  The rust bindings include unit and fuzz tests for the API.

  The header currently exposes logic for enabling the following functionality:
  * Feature-parity with the now deprecated libbitcoin-consensus
  * Optimized sha256 implementations that were not available to previous users of libbitcoin-consensus thanks to a static kernel context
  * Full support for logging as well as control over categories and severity
  * Feature parity with the existing experimental bitcoin-chainstate
  * Traversing the block index as well as using block index entries for reading block and undo data.
  * Running the chainstate in memory
  * Reindexing (both full and chainstate-only)
  * Interrupting long-running functions

  The pull request introduces a new kernel-only test binary that purely relies on the kernel C header and the C++ standard library. This is intentionally done to show its capabilities without relying on other code inside the project. This may be relaxed to include some of the existing utilities, or even be merged into the existing test suite.

  The complete docs for the API as well as some usage examples are hosted on [thecharlatan.ch/kernel-docs](https://thecharlatan.ch/kernel-docs/index.html). The docs are generated from the following repository (which also holds the examples): [github.com/TheCharlatan/kernel-docs](https://github.com/TheCharlatan/kernel-docs).

  #### How can I review this PR?

  Scrutinize the commit messages, run the tests, write your own little applications using the library, let your favorite code sanitizer loose on it, hook it up to your fuzzing infrastructure, profile the difference between the existing bitcoin-chainstate and the bitcoin-chainstate introduced here, be nitty on the documentation, police the C interface, opine on your own API design philosophy.

  To get a feeling for the API, read through the tests, or one of the examples.

  To configure this PR for making the shared library and the bitcoin-chainstate and test_kernel utilities available:
  ```
  cmake -B build -DBUILD_KERNEL_LIB=ON -DBUILD_UTIL_CHAINSTATE=ON
  ```

  Once compiled the library is part of the build artifacts that can be installed with:
  ```
  cmake --install build
  ```

  #### Why a C header (and not a C++ header)

  * Shipping a shared library with a C++ header is hard, because of name mangling and an unstable ABI.
  * Mature and well-supported tooling for integrating C exists for nearly every popular language.
  * C offers a reasonably stable ABI

  Also see https://github.com/bitcoin/bitcoin/pull/30595#issuecomment-2285719575.

  #### What about versioning?

  The header and library are still experimental and I would expect this to remain so for some time, so best not to worry about versioning yet.

  #### Potential future additions

  In future, the C header could be expanded to support (some of these have been roughly implemented):

  * Handling transactions, block headers, coins cache, utxo set, meta data, and the mempool
  * Adapters for an abstract coins store
  * Adapters for an abstract block store
  * Adapters for an abstract block tree store
  * Allocators and buffers for more efficient memory usage
  * An "[io-less](https://sans-io.readthedocs.io/how-to-sans-io.html)" interface
  * Hooks for an external mempool, or external policy rules

  #### Current drawbacks

  * For external applications to read the block index of an existing Bitcoin Core node, Bitcoin Core needs to shut down first, since leveldb does not support reading across multiple processes. Other than migrating away from leveldb, there does not seem to be a solution for this problem. Such a migration is implemented in #32427.
  * The fatal error handling through the notifications is awkward. This is partly improved through #29642.
  * Handling shared pointers in the interfaces is unfortunate. They make ownership and freeing of the resources fuzzy and poison the interfaces with additional types and complexity. However, they seem to be an artifact of the current code that interfaces with the validation engine. The validation engine itself does not seem to make extensive use of these shared pointers.
  * If multiple instances of the same type of objects are used, there is no mechanism for distinguishing the log messages produced by each of them. A potential solution is #30342.
  * The background leveldb compaction thread may not finish in time leading to a non-clean exit. There seems to be nothing we can do about this, outside of patching leveldb.

ACKs for top commit:
  alexanderwiederin:
    re-ACK 6c7a34f3b0
  stringintech:
    re-ACK 6c7a34f
  laanwj:
    Code review ACK 6c7a34f3b0
  ismaelsadeeq:
    reACK 6c7a34f3b0 👾
  fanquake:
    ACK 6c7a34f3b0 - soon we'll be running bitcoin (kernel)

Tree-SHA512: ffe7d4581facb7017d06da8b685b81f4b5e4840576e878bb6845595021730eab808d8f9780ed0eb0d2b57f2647c85dcb36b6325180caaac469eaf339f7258030
2025-11-04 15:38:42 +00:00

610 lines
26 KiB
YAML

# Copyright (c) 2023-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or https://opensource.org/license/mit.
name: CI
on:
# See: https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#pull_request.
pull_request:
# See: https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#push.
push:
branches:
- '**'
tags-ignore:
- '**'
concurrency:
group: ${{ github.event_name != 'pull_request' && github.run_id || github.ref }}
cancel-in-progress: true
env:
CI_FAILFAST_TEST_LEAVE_DANGLING: 1 # GHA does not care about dangling processes and setting this variable avoids killing the CI script itself on error
CIRRUS_CACHE_HOST: http://127.0.0.1:12321/ # When using Cirrus Runners this host can be used by the docker `gha` build cache type.
REPO_USE_CIRRUS_RUNNERS: 'bitcoin/bitcoin' # Use cirrus runners and cache for this repo, instead of falling back to the slow GHA runners
defaults:
run:
# Enforce fail-fast behavior for all platforms.
# See: https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#exit-codes-and-error-action-preference
shell: bash
jobs:
runners:
name: 'determine runners'
runs-on: ubuntu-latest
outputs:
provider: ${{ steps.runners.outputs.provider }}
steps:
- name: Annotate with pull request number
# This annotation is machine-readable and can be used to assign a check
# run to its corresponding pull request. Running in one check run is
# sufficient for each check suite.
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
echo "::notice title=debug_pull_request_number_str::${{ github.event.number }}"
fi
- id: runners
run: |
if [[ "${REPO_USE_CIRRUS_RUNNERS}" == "${{ github.repository }}" ]]; then
echo "provider=cirrus" >> "$GITHUB_OUTPUT"
echo "::notice title=Runner Selection::Using Cirrus Runners"
else
echo "provider=gha" >> "$GITHUB_OUTPUT"
echo "::notice title=Runner Selection::Using GitHub-hosted runners"
fi
test-each-commit:
name: 'test each commit'
runs-on: ubuntu-24.04
if: github.event_name == 'pull_request' && github.event.pull_request.commits != 1
timeout-minutes: 360 # Use maximum time, see https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes. Assuming a worst case time of 1 hour per commit, this leads to a --max-count=6 below.
env:
MAX_COUNT: 6
steps:
- name: Determine fetch depth
run: echo "FETCH_DEPTH=$((${{ github.event.pull_request.commits }} + 2))" >> "$GITHUB_ENV"
- uses: actions/checkout@v5
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: ${{ env.FETCH_DEPTH }}
- name: Determine commit range
run: |
# Checkout HEAD~ and find the test base commit
# Checkout HEAD~ because it would be wasteful to rerun tests on the PR
# head commit that are already run by other jobs.
git checkout HEAD~
# Figure out test base commit by listing ancestors of HEAD, excluding
# ancestors of the most recent merge commit, limiting the list to the
# newest MAX_COUNT ancestors, ordering it from oldest to newest, and
# taking the first one.
#
# If the branch contains up to MAX_COUNT ancestor commits after the
# most recent merge commit, all of those commits will be tested. If it
# contains more, only the most recent MAX_COUNT commits will be
# tested.
#
# In the command below, the ^@ suffix is used to refer to all parents
# of the merge commit as described in:
# https://git-scm.com/docs/git-rev-parse#_other_rev_parent_shorthand_notations
# and the ^ prefix is used to exclude these parents and all their
# ancestors from the rev-list output as described in:
# https://git-scm.com/docs/git-rev-list
MERGE_BASE=$(git rev-list -n1 --merges HEAD)
EXCLUDE_MERGE_BASE_ANCESTORS=
# MERGE_BASE can be empty due to limited fetch-depth
if test -n "$MERGE_BASE"; then
EXCLUDE_MERGE_BASE_ANCESTORS=^${MERGE_BASE}^@
fi
echo "TEST_BASE=$(git rev-list -n$((${{ env.MAX_COUNT }} + 1)) --reverse HEAD $EXCLUDE_MERGE_BASE_ANCESTORS | head -1)" >> "$GITHUB_ENV"
- run: |
git fetch origin "${GITHUB_BASE_REF}"
git config user.email "ci@example.com"
git config user.name "CI"
- run: |
sudo apt-get update
sudo apt-get install clang mold ccache build-essential cmake ninja-build pkgconf python3-zmq libevent-dev libboost-dev libsqlite3-dev systemtap-sdt-dev libzmq3-dev qt6-base-dev qt6-tools-dev qt6-l10n-tools libqrencode-dev capnproto libcapnp-dev -y
sudo pip3 install --break-system-packages pycapnp
- name: Compile and run tests
run: |
# Run tests on commits after the last merge commit and before the PR head commit
git rebase --exec "git merge --no-commit origin/${GITHUB_BASE_REF} && python3 ./.github/ci-test-each-commit-exec.py && git reset --hard" ${{ env.TEST_BASE }}
macos-native-arm64:
name: ${{ matrix.job-name }}
# Use any image to support the xcode-select below, but hardcode version to avoid silent upgrades (and breaks).
# See: https://github.com/actions/runner-images#available-images.
runs-on: macos-15
# When a contributor maintains a fork of the repo, any pull request they make
# to their own fork, or to the main repository, will trigger two CI runs:
# one for the branch push and one for the pull request.
# This can be avoided by setting SKIP_BRANCH_PUSH=true as a custom env variable
# in Github repository settings.
if: ${{ vars.SKIP_BRANCH_PUSH != 'true' || github.event_name == 'pull_request' }}
timeout-minutes: 120
strategy:
fail-fast: false
matrix:
job-type: [standard, fuzz]
include:
- job-type: standard
file-env: './ci/test/00_setup_env_mac_native.sh'
job-name: 'macOS native, no depends, sqlite only, gui'
- job-type: fuzz
file-env: './ci/test/00_setup_env_mac_native_fuzz.sh'
job-name: 'macOS native, fuzz'
env:
DANGER_RUN_CI_ON_HOST: 1
BASE_ROOT_DIR: ${{ github.workspace }}
steps:
- &CHECKOUT
name: Checkout
uses: actions/checkout@v5
with:
# Ensure the latest merged pull request state is used, even on re-runs.
ref: &CHECKOUT_REF_TMPL ${{ github.event_name == 'pull_request' && github.ref || '' }}
- name: Clang version
run: |
# Use the earliest Xcode supported by the version of macOS denoted in
# doc/release-notes-empty-template.md and providing at least the
# minimum clang version denoted in doc/dependencies.md.
# See: https://developer.apple.com/documentation/xcode-release-notes/xcode-16-release-notes
sudo xcode-select --switch /Applications/Xcode_16.0.app
clang --version
- name: Install Homebrew packages
env:
HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK: 1
run: |
# A workaround for "The `brew link` step did not complete successfully" error.
brew install --quiet python@3 || brew link --overwrite python@3
brew install --quiet coreutils ninja pkgconf gnu-getopt ccache boost libevent zeromq qt@6 qrencode capnp
- name: Set Ccache directory
run: echo "CCACHE_DIR=${RUNNER_TEMP}/ccache_dir" >> "$GITHUB_ENV"
- name: Restore Ccache cache
id: ccache-cache
uses: actions/cache/restore@v4
with:
path: ${{ env.CCACHE_DIR }}
key: ${{ github.job }}-${{ matrix.job-type }}-ccache-${{ github.run_id }}
restore-keys: ${{ github.job }}-${{ matrix.job-type }}-ccache-
- name: CI script
run: ./ci/test_run_all.sh
env:
FILE_ENV: ${{ matrix.file-env }}
- name: Save Ccache cache
uses: actions/cache/save@v4
if: github.event_name != 'pull_request' && steps.ccache-cache.outputs.cache-hit != 'true'
with:
path: ${{ env.CCACHE_DIR }}
# https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache
key: ${{ github.job }}-${{ matrix.job-type }}-ccache-${{ github.run_id }}
windows-native-dll:
name: ${{ matrix.job-name }}
runs-on: windows-2022
if: ${{ vars.SKIP_BRANCH_PUSH != 'true' || github.event_name == 'pull_request' }}
env:
PYTHONUTF8: 1
TEST_RUNNER_TIMEOUT_FACTOR: 40
strategy:
fail-fast: false
matrix:
job-type: [standard, fuzz]
include:
- job-type: standard
generate-options: '-DBUILD_GUI=ON -DWITH_ZMQ=ON -DBUILD_BENCH=ON -DBUILD_KERNEL_LIB=ON -DBUILD_UTIL_CHAINSTATE=ON -DWERROR=ON'
job-name: 'Windows native, VS 2022'
- job-type: fuzz
generate-options: '-DVCPKG_MANIFEST_NO_DEFAULT_FEATURES=ON -DVCPKG_MANIFEST_FEATURES="wallet" -DBUILD_GUI=OFF -DBUILD_FOR_FUZZING=ON -DWERROR=ON'
job-name: 'Windows native, fuzz, VS 2022'
steps:
- *CHECKOUT
- &SET_UP_VS
name: Set up VS Developer Prompt
shell: pwsh -Command "$PSVersionTable; $PSNativeCommandUseErrorActionPreference = $true; $ErrorActionPreference = 'Stop'; & '{0}'"
run: |
$vswherePath = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe"
$installationPath = & $vswherePath -latest -property installationPath
& "${env:COMSPEC}" /s /c "`"$installationPath\Common7\Tools\vsdevcmd.bat`" -arch=x64 -no_logo && set" | foreach-object {
$name, $value = $_ -split '=', 2
echo "$name=$value" >> $env:GITHUB_ENV
}
- name: Get tool information
shell: pwsh
run: |
cmake -version | Tee-Object -FilePath "cmake_version"
Write-Output "---"
msbuild -version | Tee-Object -FilePath "msbuild_version"
$env:VCToolsVersion | Tee-Object -FilePath "toolset_version"
py -3 --version
Write-Host "PowerShell version $($PSVersionTable.PSVersion.ToString())"
bash --version
- name: Using vcpkg with MSBuild
run: |
echo "set(VCPKG_BUILD_TYPE release)" >> "${VCPKG_INSTALLATION_ROOT}/triplets/x64-windows.cmake"
# Workaround for libevent, which requires CMake 3.1 but is incompatible with CMake >= 4.0.
sed -i '1s/^/set(ENV{CMAKE_POLICY_VERSION_MINIMUM} 3.5)\n/' "${VCPKG_INSTALLATION_ROOT}/scripts/ports.cmake"
- name: vcpkg tools cache
uses: actions/cache@v4
with:
path: C:/vcpkg/downloads/tools
key: ${{ github.job }}-vcpkg-tools
- name: Restore vcpkg binary cache
uses: actions/cache/restore@v4
id: vcpkg-binary-cache
with:
path: ~/AppData/Local/vcpkg/archives
key: ${{ github.job }}-vcpkg-binary-${{ hashFiles('cmake_version', 'msbuild_version', 'toolset_version', 'vcpkg.json') }}
- name: Generate build system
run: |
cmake -B build -Werror=dev --preset vs2022 -DCMAKE_TOOLCHAIN_FILE="${VCPKG_INSTALLATION_ROOT}/scripts/buildsystems/vcpkg.cmake" ${{ matrix.generate-options }}
- name: Save vcpkg binary cache
uses: actions/cache/save@v4
if: github.event_name != 'pull_request' && steps.vcpkg-binary-cache.outputs.cache-hit != 'true' && matrix.job-type == 'standard'
with:
path: ~/AppData/Local/vcpkg/archives
key: ${{ github.job }}-vcpkg-binary-${{ hashFiles('cmake_version', 'msbuild_version', 'toolset_version', 'vcpkg.json') }}
- name: Build
working-directory: build
run: |
cmake --build . -j $NUMBER_OF_PROCESSORS --config Release
- name: Check executable manifests
if: matrix.job-type == 'standard'
working-directory: build
shell: pwsh -Command "$PSVersionTable; $PSNativeCommandUseErrorActionPreference = $true; $ErrorActionPreference = 'Stop'; & '{0}'"
run: |
mt.exe -nologo -inputresource:bin\Release\bitcoind.exe -out:bitcoind.manifest
Get-Content bitcoind.manifest
Get-ChildItem -Filter "bin\Release\*.exe" | ForEach-Object {
$exeName = $_.Name
# Skip as they currently do not have manifests
if ($exeName -eq "fuzz.exe" -or $exeName -eq "bench_bitcoin.exe" -or $exeName -eq "test_bitcoin-qt.exe" -or $exeName -eq "test_kernel.exe" -or $exeName -eq "bitcoin-chainstate.exe") {
Write-Host "Skipping $exeName (no manifest present)"
return
}
Write-Host "Checking $exeName"
& mt.exe -nologo -inputresource:$_.FullName -validate_manifest
}
- name: Run test suite
if: matrix.job-type == 'standard'
working-directory: build
env:
QT_PLUGIN_PATH: '${{ github.workspace }}\build\vcpkg_installed\x64-windows\Qt6\plugins'
run: |
ctest --output-on-failure --stop-on-failure -j $NUMBER_OF_PROCESSORS -C Release
- name: Run functional tests
if: matrix.job-type == 'standard'
working-directory: build
env:
BITCOIN_BIN: '${{ github.workspace }}\build\bin\Release\bitcoin.exe'
BITCOIND: '${{ github.workspace }}\build\bin\Release\bitcoind.exe'
BITCOINCLI: '${{ github.workspace }}\build\bin\Release\bitcoin-cli.exe'
BITCOINTX: '${{ github.workspace }}\build\bin\Release\bitcoin-tx.exe'
BITCOINUTIL: '${{ github.workspace }}\build\bin\Release\bitcoin-util.exe'
BITCOINWALLET: '${{ github.workspace }}\build\bin\Release\bitcoin-wallet.exe'
BITCOINCHAINSTATE: '${{ github.workspace }}\build\bin\Release\bitcoin-chainstate.exe'
TEST_RUNNER_EXTRA: ${{ github.event_name != 'pull_request' && '--extended' || '' }}
run: py -3 test/functional/test_runner.py --jobs $NUMBER_OF_PROCESSORS --ci --quiet --tmpdirprefix="${RUNNER_TEMP}" --combinedlogslen=99999999 --timeout-factor=${TEST_RUNNER_TIMEOUT_FACTOR} ${TEST_RUNNER_EXTRA}
- name: Clone corpora
if: matrix.job-type == 'fuzz'
run: |
git clone --depth=1 https://github.com/bitcoin-core/qa-assets "${RUNNER_TEMP}/qa-assets"
cd "${RUNNER_TEMP}/qa-assets"
echo "Using qa-assets repo from commit ..."
git log -1
- name: Run fuzz tests
if: matrix.job-type == 'fuzz'
working-directory: build
env:
BITCOINFUZZ: '${{ github.workspace }}\build\bin\Release\fuzz.exe'
run: |
py -3 test/fuzz/test_runner.py --par $NUMBER_OF_PROCESSORS --loglevel DEBUG "${RUNNER_TEMP}/qa-assets/fuzz_corpora"
windows-cross:
name: 'Linux->Windows cross, no tests'
needs: runners
runs-on: ${{ needs.runners.outputs.provider == 'cirrus' && 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-sm' || 'ubuntu-24.04' }}
if: ${{ vars.SKIP_BRANCH_PUSH != 'true' || github.event_name == 'pull_request' }}
env:
FILE_ENV: './ci/test/00_setup_env_win64.sh'
DANGER_CI_ON_HOST_FOLDERS: 1
steps:
- *CHECKOUT
- name: Configure environment
uses: ./.github/actions/configure-environment
- name: Restore caches
id: restore-cache
uses: ./.github/actions/restore-caches
- name: Configure Docker
uses: ./.github/actions/configure-docker
with:
cache-provider: ${{ needs.runners.outputs.provider }}
- name: CI script
run: ./ci/test_run_all.sh
- name: Save caches
uses: ./.github/actions/save-caches
- name: Upload built executables
uses: actions/upload-artifact@v4
with:
name: x86_64-w64-mingw32-executables-${{ github.run_id }}
path: |
${{ env.BASE_BUILD_DIR }}/bin/*.exe
${{ env.BASE_BUILD_DIR }}/src/secp256k1/bin/*.exe
${{ env.BASE_BUILD_DIR }}/src/univalue/*.exe
${{ env.BASE_BUILD_DIR }}/test/config.ini
windows-native-test:
name: 'Windows, test cross-built'
runs-on: windows-2022
needs: windows-cross
env:
PYTHONUTF8: 1
TEST_RUNNER_TIMEOUT_FACTOR: 40
steps:
- *CHECKOUT
- name: Download built executables
uses: actions/download-artifact@v5
with:
name: x86_64-w64-mingw32-executables-${{ github.run_id }}
- name: Run bitcoind.exe
run: ./bin/bitcoind.exe -version
- *SET_UP_VS
- name: Check executable manifests
shell: pwsh -Command "$PSVersionTable; $PSNativeCommandUseErrorActionPreference = $true; $ErrorActionPreference = 'Stop'; & '{0}'"
run: |
mt.exe -nologo -inputresource:bin\bitcoind.exe -out:bitcoind.manifest
Get-Content bitcoind.manifest
Get-ChildItem -Filter "bin\*.exe" | ForEach-Object {
$exeName = $_.Name
# Skip as they currently do not have manifests
if ($exeName -eq "fuzz.exe" -or $exeName -eq "bench_bitcoin.exe" -or $exeName -eq "test_kernel.exe") {
Write-Host "Skipping $exeName (no manifest present)"
return
}
Write-Host "Checking $exeName"
& mt.exe -nologo -inputresource:$_.FullName -validate_manifest
}
- name: Run unit tests
# Can't use ctest here like other jobs as we don't have a CMake build tree.
run: |
./bin/test_bitcoin.exe -l test_suite # Intentionally run sequentially here, to catch test case failures caused by dirty global state from prior test cases.
./src/secp256k1/bin/exhaustive_tests.exe
./src/secp256k1/bin/noverify_tests.exe
./src/secp256k1/bin/tests.exe
./src/univalue/object.exe
./src/univalue/unitester.exe
- name: Run benchmarks
run: ./bin/bench_bitcoin.exe -sanity-check
- name: Adjust paths in test/config.ini
shell: pwsh
run: |
(Get-Content "test/config.ini") -replace '(?<=^SRCDIR=).*', '${{ github.workspace }}' -replace '(?<=^BUILDDIR=).*', '${{ github.workspace }}' -replace '(?<=^RPCAUTH=).*', '${{ github.workspace }}/share/rpcauth/rpcauth.py' | Set-Content "test/config.ini"
Get-Content "test/config.ini"
- name: Set previous release directory
run: |
echo "PREVIOUS_RELEASES_DIR=${{ runner.temp }}/previous_releases" >> "$GITHUB_ENV"
- name: Get previous releases
run: ./test/get_previous_releases.py --target-dir $PREVIOUS_RELEASES_DIR
- name: Run functional tests
env:
# TODO: Fix the excluded test and re-enable it.
# feature_unsupported_utxo_db.py fails on windows because of emojis in the test data directory
EXCLUDE: '--exclude wallet_multiwallet.py,feature_unsupported_utxo_db.py'
TEST_RUNNER_EXTRA: ${{ github.event_name != 'pull_request' && '--extended' || '' }}
run: py -3 test/functional/test_runner.py --jobs $NUMBER_OF_PROCESSORS --ci --quiet --tmpdirprefix="$RUNNER_TEMP" --combinedlogslen=99999999 --timeout-factor=$TEST_RUNNER_TIMEOUT_FACTOR $EXCLUDE $TEST_RUNNER_EXTRA
ci-matrix:
name: ${{ matrix.name }}
needs: runners
runs-on: ${{ needs.runners.outputs.provider == 'cirrus' && matrix.cirrus-runner || matrix.fallback-runner }}
if: ${{ vars.SKIP_BRANCH_PUSH != 'true' || github.event_name == 'pull_request' }}
timeout-minutes: ${{ matrix.timeout-minutes }}
env:
DANGER_CI_ON_HOST_FOLDERS: 1
FILE_ENV: ${{ matrix.file-env }}
strategy:
fail-fast: false
matrix:
include:
- name: '32 bit ARM, unit tests, no functional tests'
cirrus-runner: 'ubuntu-24.04-arm' # Cirrus' Arm runners are Apple (with virtual Linux aarch64), which doesn't support 32-bit mode
fallback-runner: 'ubuntu-24.04-arm'
timeout-minutes: 120
file-env: './ci/test/00_setup_env_arm.sh'
provider: 'gha'
- name: 'ASan + LSan + UBSan + integer, no depends, USDT'
cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-md' # has to match container in ci/test/00_setup_env_native_asan.sh for tracing tools
fallback-runner: 'ubuntu-24.04'
timeout-minutes: 120
file-env: './ci/test/00_setup_env_native_asan.sh'
- name: 'macOS-cross to arm64'
cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-sm'
fallback-runner: 'ubuntu-24.04'
timeout-minutes: 120
file-env: './ci/test/00_setup_env_mac_cross.sh'
- name: 'macOS-cross to x86_64'
cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-sm'
fallback-runner: 'ubuntu-24.04'
timeout-minutes: 120
file-env: './ci/test/00_setup_env_mac_cross_intel.sh'
- name: 'No wallet, libbitcoinkernel'
cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-sm'
fallback-runner: 'ubuntu-24.04'
timeout-minutes: 120
file-env: './ci/test/00_setup_env_native_nowallet_libbitcoinkernel.sh'
- name: 'no IPC, i686, DEBUG'
cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-md'
fallback-runner: 'ubuntu-24.04'
timeout-minutes: 120
file-env: './ci/test/00_setup_env_i686_no_ipc.sh'
- name: 'fuzzer,address,undefined,integer, no depends'
cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-lg'
fallback-runner: 'ubuntu-24.04'
timeout-minutes: 240
file-env: './ci/test/00_setup_env_native_fuzz.sh'
- name: 'Valgrind, fuzz'
cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-md'
fallback-runner: 'ubuntu-24.04'
timeout-minutes: 240
file-env: './ci/test/00_setup_env_native_fuzz_with_valgrind.sh'
- name: 'previous releases, depends DEBUG'
cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-md'
fallback-runner: 'ubuntu-24.04'
timeout-minutes: 120
file-env: './ci/test/00_setup_env_native_previous_releases.sh'
- name: 'Alpine (musl), depends, gui'
cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-md'
fallback-runner: 'ubuntu-24.04'
timeout-minutes: 120
file-env: './ci/test/00_setup_env_native_alpine_musl.sh'
- name: 'tidy'
cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-md'
fallback-runner: 'ubuntu-24.04'
timeout-minutes: 120
file-env: './ci/test/00_setup_env_native_tidy.sh'
- name: 'TSan, depends, no gui'
cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-md'
fallback-runner: 'ubuntu-24.04'
timeout-minutes: 120
file-env: './ci/test/00_setup_env_native_tsan.sh'
- name: 'MSan, fuzz'
cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-md'
fallback-runner: 'ubuntu-24.04'
timeout-minutes: 150
file-env: './ci/test/00_setup_env_native_fuzz_with_msan.sh'
- name: 'MSan, depends'
cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-lg'
fallback-runner: 'ubuntu-24.04'
timeout-minutes: 120
file-env: './ci/test/00_setup_env_native_msan.sh'
steps:
- *CHECKOUT
- name: Configure environment
uses: ./.github/actions/configure-environment
- name: Restore caches
id: restore-cache
uses: ./.github/actions/restore-caches
- name: Configure Docker
uses: ./.github/actions/configure-docker
with:
cache-provider: ${{ matrix.provider || needs.runners.outputs.provider }}
- name: Enable bpfcc script
if: ${{ env.CONTAINER_NAME == 'ci_native_asan' }}
# In the image build step, no external environment variables are available,
# so any settings will need to be written to the settings env file:
run: sed -i "s|\${INSTALL_BCC_TRACING_TOOLS}|true|g" ./ci/test/00_setup_env_native_asan.sh
- name: Set mmap_rnd_bits
if: ${{ env.CONTAINER_NAME == 'ci_native_tsan' || env.CONTAINER_NAME == 'ci_native_msan' || env.CONTAINER_NAME == 'ci_native_fuzz_msan' }}
# Prevents crashes due to high ASLR entropy
run: sudo sysctl -w vm.mmap_rnd_bits=28
- name: CI script
run: ./ci/test_run_all.sh
- name: Save caches
uses: ./.github/actions/save-caches
lint:
name: 'lint'
needs: runners
runs-on: ${{ needs.runners.outputs.provider == 'cirrus' && 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-xs' || 'ubuntu-24.04' }}
if: ${{ vars.SKIP_BRANCH_PUSH != 'true' || github.event_name == 'pull_request' }}
timeout-minutes: 20
env:
CONTAINER_NAME: "bitcoin-linter"
steps:
- name: Checkout
uses: actions/checkout@v5
with:
ref: *CHECKOUT_REF_TMPL
fetch-depth: 0
- name: Configure Docker
uses: ./.github/actions/configure-docker
with:
cache-provider: ${{ needs.runners.outputs.provider }}
- name: CI script
run: |
set -o xtrace
docker buildx build -t "$CONTAINER_NAME" $DOCKER_BUILD_CACHE_ARG --file "./ci/lint_imagefile" .
CIRRUS_PR_FLAG=""
if [ "${{ github.event_name }}" = "pull_request" ]; then
CIRRUS_PR_FLAG="-e CIRRUS_PR=1"
fi
docker run --rm $CIRRUS_PR_FLAG -v "$(pwd)":/bitcoin "$CONTAINER_NAME"