09a9bb3536
2424e52836lint: doc: detail lint tool install methods (will)5fefa5a654Don't pin Python patch version (Sjors Provoost)fd15b55c2elint: use requirements.txt (will)5f4d3383dalint: switch to ruff for formatting and linting (will)a53b81ce4elint: switch to uv for python management in linter (will) Pull request description: Modernise our lint tooling by: \- Replacing pyenv + pip with [uv](https://docs.astral.sh/uv/) for better Python environment and dependency management \- Move uv ruff and ty to install via `COPY --from` multi-stage Docker image imports \- Moving ruff lint rules from hardcoded Rust array (in lint_py.rs) into a top-level ruff.toml \- Extracting all remaining pip dependencies into dedicated ci/lint/requirements.txt Extra rationale: `COPY --from` pulls pre-built binaries from upstream images instead of compiling/downloading at runtime. Containerfile layer optimisations reduce rebuild frequency further. Pinning tool versions in the dockerfile makes it more excplicit and easier to find. The tradeoff we make here is that there is no longer a single install script to install tooling on a local machine. However I think this is OK, as it currently only works for `apt`-based OSes anyway, and I don't think running the linter outside of the container is such a valuable use-case as it is with some of the other CI jobs. ACKs for top commit: maflcko: review ACK2424e52836🗿 sedited: ACK2424e52836Tree-SHA512: 32ef989c1e241cebe5f13da10abd23f6f63306591fd1f81880d688b886082bca17987591dc592c41fbb72278eba57b3cc6e786de7cfa80eb490ab34465d0119b
This folder contains lint scripts.
Running locally
To run linters locally with the same versions as the CI environment use the lint.py helper script which runs checks inside the CI container:
./ci/lint.py
Extra arguments are passed to cargo run -- ... in the container so you can do:
./ci/lint.py --help
./ci/lint.py --lint=py_lint
test runner
To run all the lint checks in the test runner outside the container you first need to install the rust toolchain using your package manager of choice or rustup.
Then you can use:
( cd ./test/lint/test_runner/ && cargo fmt && cargo clippy && RUST_BACKTRACE=1 cargo run )
If you wish to run individual lint checks, run the test_runner with
--lint=TEST_TO_RUN arguments. If running with cargo run, arguments after
-- are passed to the binary you are running e.g.:
( cd ./test/lint/test_runner/ && RUST_BACKTRACE=1 cargo run -- --lint=doc --lint=trailing_whitespace )
To see a list of all individual lint checks available in test_runner, use -h
or --help:
( cd ./test/lint/test_runner/ && RUST_BACKTRACE=1 cargo run -- --help )
Dependencies
| Lint test | Dependency |
|---|---|
lint-python.py |
lief |
lint-python.py |
mypy |
lint-python.py |
pyzmq |
lint-shell.py |
ShellCheck |
py_lint |
ruff |
| markdown link check | mlc |
Dependency versions and installation instructions are available in the CI setup and the lint_imagefile (for tools where an OCI imagefile exists).
Please be aware that on Linux distributions all dependencies are usually available as packages, but could be outdated.
Running the tests
Individual tests can be run by directly calling the test script, e.g.:
test/lint/lint-files.py
check-doc.py
Check for missing documentation of command line options.
commit-script-check.sh
Verification of scripted diffs. Scripted diffs are only assumed to run on the latest LTS release of Ubuntu. Running them on other operating systems might require installing GNU tools, such as GNU sed.
git-subtree-check.sh
Run this script from the root of the repository to verify that a subtree matches the contents of the commit it claims to have been updated to.
Usage: test/lint/git-subtree-check.sh [-r] DIR [COMMIT]
test/lint/git-subtree-check.sh -?
DIRis the prefix within the repository to check.COMMITis the commit to check, if it is not provided, HEAD will be used.-rchecks that subtree commit is present in repository.
To do a full check with -r, make sure that you have fetched the upstream repository branch in which the subtree is
maintained:
- for
src/crc32c: https://github.com/bitcoin-core/crc32c-subtree.git (branch bitcoin-fork) - for
src/crypto/ctaes: https://github.com/bitcoin-core/ctaes.git (branch master) - for
src/ipc/libmultiprocess: https://github.com/bitcoin-core/libmultiprocess (branch master) - for
src/leveldb: https://github.com/bitcoin-core/leveldb-subtree.git (branch bitcoin-fork) - for
src/minisketch: https://github.com/bitcoin-core/minisketch.git (branch master) - for
src/secp256k1: https://github.com/bitcoin-core/secp256k1.git (branch master)
Keep this list in sync with fn get_subtrees() in the lint runner.
To do so, add the upstream repository as remote:
git remote add --fetch secp256k1 https://github.com/bitcoin-core/secp256k1.git
lint_ignore_dirs.py
Add list of common directories to ignore when running tests