0f01e1577f
a4f929696490 Merge bitcoin-core/libmultiprocess#224: doc: fix typos f4344ae87da0 Merge bitcoin-core/libmultiprocess#222: test, ci: Fix threadsanitizer errors in mptest 1434642b3804 doc: fix typos 73d22ba2e930 test: Fix tsan race in thread busy test b74e1bba014d ci: Use tsan-instrumented cap'n proto in sanitizers job c332774409ad test: Fix failing exception check in new thread busy test ca3c05d56709 test: Use KJ_LOG instead of std::cout for logging 7eb1da120ab6 ci: Use tsan-instrumented libcxx in sanitizers job ec86e4336e98 Merge bitcoin-core/libmultiprocess#220: Add log levels and advertise them to users via logging callback 515ce93ad349 Logging: Pass LogData struct to logging callback 213574ccc43d Logging: reclassify remaining log messages e4de0412b430 Logging: Break out expensive log messages and classify them as Trace 408874a78fdc Logging: Use new logging macros 67b092d835cd Logging: Disable logging if messsage level is less than the requested level d0a1ba7ebf21 Logging: add log levels to mirror Core's 463a8296d188 Logging: Disable moving or copying Logger 83a2e10c0b03 Logging: Add an EventLoop constructor to allow for user-specified log options 58cf47a7fc8c Merge bitcoin-core/libmultiprocess#221: test default PassField impl handles output parameters db03a663f514 Merge bitcoin-core/libmultiprocess#214: Fix crash on simultaneous IPC calls using the same thread afcc40b0f1e8 Merge bitcoin-core/libmultiprocess#213: util+doc: Clearer errors when attempting to run examples + polished docs 6db669628387 test In|Out parameter 29cf2ada75ea test default PassField impl handles output parameters 1238170f68e8 test: simultaneous IPC calls using same thread eb069ab75d83 Fix crash on simultaneous IPC calls using the same thread ec03a9639ab5 doc: Precision and typos 2b4348193551 doc: Where possible, remove links to ryanofsky/bitcoin/ 286fe469c9c9 util: Add helpful error message when failing to execute file git-subtree-dir: src/ipc/libmultiprocess git-subtree-split: a4f92969649018ca70f949a09148bccfeaecd99a
CI Scripts
This directory contains scripts for each build step in each build stage.
Running a Stage Locally
Be aware that the tests will be built and run in-place, so please run at your own risk. If the repository is not a fresh git clone, you might have to clean files from previous builds or test runs first.
The ci needs to perform various sysadmin tasks such as installing packages or writing to the user's home directory. While it should be fine to run the ci system locally on your development box, the ci scripts can generally be assumed to have received less review and testing compared to other parts of the codebase. If you want to keep the work tree clean, you might want to run the ci system in a virtual machine with a Linux operating system of your choice.
To allow for a wide range of tested environments, but also ensure reproducibility to some extent, the test stage
requires bash, docker, and python3 to be installed. To run on different architectures than the host qemu is also required. To install all requirements on Ubuntu, run
sudo apt install bash docker.io python3 qemu-user-static
It is recommended to run the ci system in a clean env. To run the test stage with a specific configuration,
env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
Configurations
The test files (FILE_ENV) are constructed to test a wide range of
configurations, rather than a single pass/fail. This helps to catch build
failures and logic errors that present on platforms other than the ones the
author has tested.
Some builders use the dependency-generator in ./depends, rather than using
the system package manager to install build dependencies. This guarantees that
the tester is using the same versions as the release builds, which also use
./depends.
It is also possible to force a specific configuration without modifying the file. For example,
env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'MAKEJOBS="-j1" FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
The files starting with 0n (n greater than 0) are the scripts that are run
in order.
Cache
In order to avoid rebuilding all dependencies for each build, the binaries are cached and reused when possible. Changes in the dependency-generator will trigger cache-invalidation and rebuilds as necessary.
Configuring a repository for CI
Primary repository
To configure the primary repository, follow these steps:
- Register with Cirrus Runners and purchase runners.
- Install the Cirrus Runners GitHub app against the GitHub organization.
- Enable organisation-level runners to be used in public repositories:
Org settings -> Actions -> Runner Groups -> Default -> Allow public repos
- Permit the following actions to run:
- cirruslabs/cache/restore@*
- cirruslabs/cache/save@*
- docker/setup-buildx-action@*
- actions/github-script@*
Forked repositories
When used in a fork the CI will run on GitHub's free hosted runners by default. In this case, due to GitHub's 10GB-per-repo cache size limitations caches will be frequently evicted and missed, but the workflows will run (slowly).
It is also possible to use your own Cirrus Runners in your own fork with an appropriate patch to the REPO_USE_CIRRUS_RUNNERS variable in ../.github/workflows/ci.yml
NB that Cirrus Runners only work at an organisation level, therefore in order to use your own Cirrus Runners, the fork must be within your own organisation.