From 3c69b071ac4964870abb347f9d9c15df2e60aa62 Mon Sep 17 00:00:00 2001 From: Prabhat Verma Date: Tue, 25 Feb 2025 22:15:26 +0530 Subject: [PATCH] add clang/llvm based coverage report generation Signed-off-by: Prabhat Verma --- doc/developer-notes.md | 56 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/doc/developer-notes.md b/doc/developer-notes.md index c8851a8dd09..0f2399e0980 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -484,6 +484,8 @@ $ ./build/test/functional/test_runner.py --valgrind ### Compiling for test coverage +#### Using LCOV + LCOV can be used to generate a test coverage report based upon `ctest` execution. LCOV must be installed on your system (e.g. the `lcov` package on Debian/Ubuntu). @@ -513,6 +515,60 @@ To enable test parallelism: cmake -DJOBS=$(nproc) -P build/Coverage.cmake ``` +#### Using LLVM/Clang toolchain + +The following generates a coverage report for unit tests and functional tests. + +Configure the build with the following flags: + +```shell +# MacOS may require -DCMAKE_CXX_COMPILER="$(brew --prefix llvm)/bin/clang++" and -DCMAKE_C_COMPILER="$(brew --prefix llvm)/bin/clang" +cmake -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_C_COMPILER="clang" \ + -DCMAKE_CXX_COMPILER="clang++" \ + -DAPPEND_CFLAGS="-fprofile-instr-generate -fcoverage-mapping" \ + -DAPPEND_CXXFLAGS="-fprofile-instr-generate -fcoverage-mapping" \ + -DAPPEND_LDFLAGS="-fprofile-instr-generate -fcoverage-mapping" +cmake --build build # Use "-j N" here for N parallel jobs. +``` + +Generating the raw profile data based on `ctest` and functional tests execution: + +```shell +# Create directory for raw profile data +mkdir -p $(pwd)/build/raw_profile_data + +# Run tests to generate profiles +LLVM_PROFILE_FILE="$(pwd)/build/raw_profile_data/%m_%p.profraw" ctest --test-dir build # Use "-j N" here for N parallel jobs. +LLVM_PROFILE_FILE="$(pwd)/build/raw_profile_data/%m_%p.profraw" build/test/functional/test_runner.py # Use "-j N" here for N parallel jobs + +# merge all the raw profile data into a single file +find build -name "*.profraw" | xargs llvm-profdata merge -o build/coverage.profdata +``` + +> **Note:** The "counter mismatch" warning can be safely ignored, though it can be resolved by updating to Clang 19. +> The warning occurs due to version mismatches but doesn't affect the coverage report generation. + +Generating the coverage report: + +```shell +llvm-cov show \ + --object=build/bin/test_bitcoin \ + --object=build/bin/bitcoind \ + --instr-profile=build/coverage.profdata \ + --ignore-filename-regex="src/crc32c/|src/leveldb/|src/minisketch/|src/secp256k1/|src/test/" \ + --format=html \ + --show-instantiation-summary \ + --show-line-counts-or-regions \ + --show-expansions \ + --output-dir=build/coverage_report \ + --project-title="Bitcoin Core Coverage Report" +``` + +> **Note:** The "functions have mismatched data" warning can be safely ignored, the coverage report will still be generated correctly despite this warning. +> This warning occurs due to profdata mismatch created during the merge process for shared libraries. + +The generated coverage report can be accessed at `build/coverage_report/index.html`. + ### Performance profiling with perf Profiling is a good way to get a precise idea of where time is being spent in