6251949443
666016e56bci: use --usecli in one of the CI jobs (Martin Zumsande)7ea248a020test: Disable several (sub)tests with cli (Martin Zumsande)f420b6356btest: skip subtests that check for wrong types with cli (Martin Zumsande)6530d0015btest: add function to convert to json for height_or_hash params (Martin Zumsande)54d28722batest: Don't send empty named args with cli (Martin Zumsande)cca422060etest: convert tuple to json for cli (Martin Zumsande)af34e98086test: make rpc_psbt.py usable with --usecli (Martin Zumsande)8f8ce9e174test: rename .rpc to ._rpc and remove unnecessary uses (Martin Zumsande)5b08885986test: enable functional tests with large rpc args for cli (Martin Zumsande)7d5352ac73test: use -stdin for large rpc commands (Martin Zumsande)6c364e0c10test: Enable various tests for usage with cli (Martin Zumsande) Pull request description: Fixes #32264 I looked into all current failures listed in the issue, as well all tests that are already disabled for the cli with `self.supports_cli = False`. There are several reasons why existing tests fail with `--usecli` on many systems, the most important ones are: - Most common reason is that the test executes a RPC call with a large arg that exceeds `MAX_ARG_STRLEN` of the OS, which is usually 128kb on linux: This is fixed by using `-stdin` for these large calls (idea by 0xB10C) - they test specifically the rpc interface - nothing to do there except disabling. - Some functional test submit wrong types to params on purpose to test the error message (which is different when using the cli) - deactivated these specific subtests locally for the cli when there is just one or two of them, deactivated the entire tests when there are more spots - When python sets `None` for an arg, the cli converts this to 'null' in `arg_to_cli`. This is fine e.g. for boolean args, but doesn't work for strings where it's interpreted as the string 'null'. Bypass this for named args by not including args in case the value is `None` for the cli is used (it's effectively the same as leaving the optional arg out). - the `height_or_hash` param used in some RPC needs to be converted to a JSON (effectively adding full quotes). - Some tests were marked with `self.supports_cli = False` in the past but run fine on master today - enabled those. In total, this PR fixes all tests that fail on master and reduces the number of tests that are deactivated (`self.supports_cli = False`) from 40 to 21. It also adds `--usecli` to one CI job (multiprocess, i686, DEBUG) to detect regressions. ACKs for top commit: maflcko: re-ACK666016e56b🔀 pinheadmz: re-ACK666016e56bTree-SHA512: 7a1efd212649ca100b236a1239294d40ecd36e2720e3b173a230b14545bb40b135111db7fed8a0d1448120f5387da146a03f1912e2028c8d03a0b6a3ca8761b0
CI Scripts
This directory contains scripts for each build step in each build stage.
Running a Stage Locally
Be aware that the tests will be built and run in-place, so please run at your own risk. If the repository is not a fresh git clone, you might have to clean files from previous builds or test runs first.
The ci needs to perform various sysadmin tasks such as installing packages or writing to the user's home directory. While it should be fine to run the ci system locally on your development box, the ci scripts can generally be assumed to have received less review and testing compared to other parts of the codebase. If you want to keep the work tree clean, you might want to run the ci system in a virtual machine with a Linux operating system of your choice.
To allow for a wide range of tested environments, but also ensure reproducibility to some extent, the test stage
requires bash, docker, and python3 to be installed. To run on different architectures than the host qemu is also required. To install all requirements on Ubuntu, run
sudo apt install bash docker.io python3 qemu-user-static
It is recommended to run the ci system in a clean env. To run the test stage with a specific configuration,
env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
Configurations
The test files (FILE_ENV) are constructed to test a wide range of
configurations, rather than a single pass/fail. This helps to catch build
failures and logic errors that present on platforms other than the ones the
author has tested.
Some builders use the dependency-generator in ./depends, rather than using
the system package manager to install build dependencies. This guarantees that
the tester is using the same versions as the release builds, which also use
./depends.
It is also possible to force a specific configuration without modifying the file. For example,
env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'MAKEJOBS="-j1" FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
The files starting with 0n (n greater than 0) are the scripts that are run
in order.
Cache
In order to avoid rebuilding all dependencies for each build, the binaries are cached and reused when possible. Changes in the dependency-generator will trigger cache-invalidation and rebuilds as necessary.