mirror of https://github.com/bitcoin/bitcoin.git synced 2026-02-19 22:04:38 +01:00

Files

Ava Chow f640b323bd Merge bitcoin/bitcoin#30723 : lint: Speed up and fix flake8 checks

fafdb7df34 lint: Speed up flake8 checks (MarcoFalke)
faf17df7fb lint: Document missing py_lint dependency (MarcoFalke)
faebeb828f lint: Remove python whitespace and shadowing lint rules (MarcoFalke)
7777047835 lint: Remove python lint rules that are SyntaxError (MarcoFalke)
faaf3e53f0 test: [refactor] Fix F841 flake8 (MarcoFalke)
444421db69 test: [refactor] Fix E714 pycodestyle (MarcoFalke)

Pull request description:

  The checks have many issues:

  * Some checks that could in theory hide bugs are not applied -> Fix them and apply them going forward
  * Some checks are redundant Python 2 checks, or of low value -> Remove them
  * The checks are slow -> Speed them up from ~10 seconds to about ~20 milliseconds

ACKs for top commit:
  davidgumberg:
    review and tested reACK fafdb7df34
  kevkevinpal:
    ACK [fafdb7d](fafdb7df34)
  achow101:
    ACK fafdb7df34

Tree-SHA512: a0488b722cfaf7071bd6848cd3be002e0b6c38af80d8b5cbb08613c0b174ef63277289f960db8ac31adb09fe563a4973203b8fb10b83cbcfdc6f0ef39bd04410

2024-09-04 15:35:52 -04:00

test_runner

lint: Speed up flake8 checks

2024-08-29 07:03:42 +02:00

check-doc.py

init: completely remove -zapwallettxes (remaining hidden option)

2023-11-03 20:00:44 +01:00

commit-script-check.sh

lint: scripted-diff verification also requires GNU grep

2024-03-21 13:12:19 +01:00

git-subtree-check.sh

scripted-diff: Bump copyright headers

2021-12-30 19:36:57 +02:00

lint_ignore_dirs.py

test: Refactor subtree exclusion in lint tests

2024-03-26 13:49:47 +03:00

lint-assertions.py

rpc: Use CHECK_NONFATAL over Assert

2024-07-12 09:27:41 +02:00

lint-circular-dependencies.py

refactor: Remove unused circular include dependency from validation.cpp

2023-11-01 17:45:48 +01:00

lint-files.py

test: use built-in collection types for type hints (Python 3.9 / PEP 585)

2023-10-25 01:10:21 +02:00

lint-format-strings.py

refactor: Remove unused LogPrint

2024-08-29 15:58:27 +02:00

lint-git-commit-check.py

lint: Fix COMMIT_RANGE issues

2024-03-21 20:15:08 +01:00

lint-include-guards.py

test: Refactor subtree exclusion in lint tests

2024-03-26 13:49:47 +03:00

lint-includes.py

external_signer: replace boost::process with cpp-subprocess

2024-03-27 14:16:37 +00:00

lint-locale-dependence.py

fuzz: Remove legacy int parse fuzz tests

2023-03-27 16:37:31 +02:00

lint-python-dead-code.py

lint: Convert Python dead code linter to Python

2022-04-06 00:55:22 +02:00

lint-python-utf8-encoding.py

lint: Ignore check_fileopens failure on **kwargs

2023-07-20 09:15:43 +02:00

lint-python.py

lint: Speed up flake8 checks

2024-08-29 07:03:42 +02:00

lint-qt-translation.py

lint: Add lint-qt-translation.py

2023-10-23 15:07:02 +01:00

lint-shell-locale.py

refactor: cleanups post unsubtree'ing univalue

2022-06-15 12:56:44 +01:00

lint-shell.py

lint: Include test_utxo_snapshots in lint_shell

2023-10-13 14:06:06 +02:00

lint-spelling.py

Fix typos in subprocess.hpp

2024-04-11 14:03:37 +02:00

lint-submodule.py

scripted-diff: Use new python 3.7 keywords

2023-01-18 13:00:34 +01:00

lint-tests.py

scripted-diff: Use new python 3.7 keywords

2023-01-18 13:00:34 +01:00

README.md

lint: Speed up flake8 checks

2024-08-29 07:03:42 +02:00

run-lint-format-strings.py

scripted-diff: LogPrint -> LogDebug

2024-08-29 13:49:57 +02:00

spelling.ignore-words.txt

Update spelling.ignore-words

2024-08-28 18:42:19 +02:00

README.md

This folder contains lint scripts.

Running locally

To run linters locally with the same versions as the CI environment, use the included Dockerfile:

DOCKER_BUILDKIT=1 docker build -t bitcoin-linter --file "./ci/lint_imagefile" ./ && docker run --rm -v $(pwd):/bitcoin -it bitcoin-linter

Building the container can be done every time, because it is fast when the result is cached and it prevents issues when the image changes.

test runner

To run all the lint checks in the test runner outside the docker you first need to install the rust toolchain using your package manager of choice or rustup.

Then you can use:

( cd ./test/lint/test_runner/ && cargo fmt && cargo clippy && RUST_BACKTRACE=1 cargo run )

If you wish to run individual lint checks, run the test_runner with --lint=TEST_TO_RUN arguments. If running with cargo run, arguments after -- are passed to the binary you are running e.g.:

( cd ./test/lint/test_runner/ && RUST_BACKTRACE=1 cargo run -- --lint=doc --lint=trailing_whitespace )

To see a list of all individual lint checks available in test_runner, use -h or --help:

( cd ./test/lint/test_runner/ && RUST_BACKTRACE=1 cargo run -- --help )

Dependencies

Lint test	Dependency
`lint-python.py`	lief
`lint-python.py`	mypy
`lint-python.py`	pyzmq
`lint-python-dead-code.py`	vulture
`lint-shell.py`	ShellCheck
`lint-spelling.py`	codespell
`py_lint`	ruff
markdown link check	mlc

In use versions and install instructions are available in the CI setup.

Please be aware that on Linux distributions all dependencies are usually available as packages, but could be outdated.

Running the tests

Individual tests can be run by directly calling the test script, e.g.:

test/lint/lint-files.py

check-doc.py

Check for missing documentation of command line options.

commit-script-check.sh

Verification of scripted diffs. Scripted diffs are only assumed to run on the latest LTS release of Ubuntu. Running them on other operating systems might require installing GNU tools, such as GNU sed.

git-subtree-check.sh

Run this script from the root of the repository to verify that a subtree matches the contents of the commit it claims to have been updated to.

Usage: test/lint/git-subtree-check.sh [-r] DIR [COMMIT]
       test/lint/git-subtree-check.sh -?

DIR is the prefix within the repository to check.
COMMIT is the commit to check, if it is not provided, HEAD will be used.
-r checks that subtree commit is present in repository.

To do a full check with -r, make sure that you have fetched the upstream repository branch in which the subtree is maintained:

for src/secp256k1: https://github.com/bitcoin-core/secp256k1.git (branch master)
for src/leveldb: https://github.com/bitcoin-core/leveldb-subtree.git (branch bitcoin-fork)
for src/crypto/ctaes: https://github.com/bitcoin-core/ctaes.git (branch master)
for src/crc32c: https://github.com/bitcoin-core/crc32c-subtree.git (branch bitcoin-fork)
for src/minisketch: https://github.com/sipa/minisketch.git (branch master)

To do so, add the upstream repository as remote:

git remote add --fetch secp256k1 https://github.com/bitcoin-core/secp256k1.git

lint_ignore_dirs.py

Add list of common directories to ignore when running tests