84de8c93e7ci: Add `deploy` target for native macOS CI job (Hennadii Stepanov)fad57e9e0fbuild: Fix `macdeployqtplus` after switching to Qt 6 (Hennadii Stepanov)938208d91abuild: Resolve `@rpath` in `macdeployqtplus` (Hennadii Stepanov) Pull request description: Homebrew's Qt 6 package — namely `qt` or `qt@6` — introduces a few differences that must be properly handled by the `macdeployqtplus` script: 1. Use of `@rpath` references: ``` % objdump --macho --dylibs-used $(brew --prefix qt@5)/Frameworks/QtGui.framework/QtGui /usr/local/opt/qt@5/Frameworks/QtGui.framework/QtGui: /usr/local/opt/qt@5/lib/QtGui.framework/Versions/5/QtGui (compatibility version 5.15.0, current version 5.15.16) /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 2575.30.19) /System/Library/Frameworks/Metal.framework/Versions/A/Metal (compatibility version 1.0.0, current version 367.6.0) /usr/local/Cellar/qt@5/5.15.16_1/lib/QtCore.framework/Versions/5/QtCore (compatibility version 5.15.0, current version 5.15.16) /System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration (compatibility version 1.0.0, current version 1.0.0) /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0) /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics (compatibility version 64.0.0, current version 1889.2.7) /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 3208.0.0) /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0) /System/Library/Frameworks/AGL.framework/Versions/A/AGL (compatibility version 1.0.0, current version 1.0.0) /usr/local/opt/libpng/lib/libpng16.16.dylib (compatibility version 64.0.0, current version 64.0.0) /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.12) /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 65.0.0) /usr/local/opt/md4c/lib/libmd4c.0.dylib (compatibility version 0.0.0, current version 0.5.2) /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1800.105.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1351.0.0) /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 3208.0.0) /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText (compatibility version 1.0.0, current version 844.2.0) /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0) % objdump --macho --dylibs-used $(brew --prefix qt@6)/Frameworks/QtGui.framework/QtGui /usr/local/opt/qt/Frameworks/QtGui.framework/QtGui: /usr/local/opt/qt/lib/QtGui.framework/Versions/A/QtGui (compatibility version 6.0.0, current version 6.9.0) /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 2575.30.19) /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 170.0.0) /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0) /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO (compatibility version 1.0.0, current version 1.0.0) /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 3208.0.0) /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics (compatibility version 64.0.0, current version 1889.2.7) /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText (compatibility version 1.0.0, current version 844.2.0) /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 3208.0.0) /System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore (compatibility version 1.2.0, current version 1.11.0) /System/Library/Frameworks/Metal.framework/Versions/A/Metal (compatibility version 1.0.0, current version 367.6.0) /usr/local/opt/glib/lib/libglib-2.0.0.dylib (compatibility version 8401.0.0, current version 8401.0.0) @rpath/QtDBus.framework/Versions/A/QtDBus (compatibility version 6.0.0, current version 6.9.0) /System/Library/Frameworks/AGL.framework/Versions/A/AGL (compatibility version 1.0.0, current version 1.0.0) /usr/local/opt/libpng/lib/libpng16.16.dylib (compatibility version 64.0.0, current version 64.0.0) /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.12) /usr/local/opt/harfbuzz/lib/libharfbuzz.0.dylib (compatibility version 61100.0.0, current version 61100.0.0) /usr/local/opt/md4c/lib/libmd4c.0.dylib (compatibility version 0.0.0, current version 0.5.2) /usr/local/opt/freetype/lib/libfreetype.6.dylib (compatibility version 27.0.0, current version 27.2.0) /usr/local/opt/glib/lib/libgthread-2.0.0.dylib (compatibility version 8401.0.0, current version 8401.0.0) @rpath/QtCore.framework/Versions/A/QtCore (compatibility version 6.0.0, current version 6.9.0) /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0) /System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration (compatibility version 1.0.0, current version 1.0.0) /System/Library/Frameworks/UniformTypeIdentifiers.framework/Versions/A/UniformTypeIdentifiers (compatibility version 1.0.0, current version 709.0.0) /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1800.105.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1351.0.0) /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 1226.0.0) /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0) ``` 2. Different directory layout: ``` % ls -l $(brew --prefix qt@5)/ total 544 drwxr-xr-x 79 hebasto admin 2528 13 Nov 06:22 Frameworks -rw-r--r-- 1 hebasto admin 7533 16 Apr 09:09 INSTALL_RECEIPT.json -rw-r--r-- 1 hebasto admin 22961 13 Nov 06:22 LICENSE.FDL -rw-r--r-- 1 hebasto admin 36363 13 Nov 06:22 LICENSE.GPL3-EXCEPT -rw-r--r-- 1 hebasto admin 15351 13 Nov 06:22 LICENSE.GPLv2 -rw-r--r-- 1 hebasto admin 35641 13 Nov 06:22 LICENSE.GPLv3 -rw-r--r-- 1 hebasto admin 26828 13 Nov 06:22 LICENSE.LGPLv21 -rw-r--r-- 1 hebasto admin 8174 13 Nov 06:22 LICENSE.LGPLv3 -rw-r--r-- 1 hebasto admin 106262 13 Nov 06:22 LICENSE.QT-LICENSE-AGREEMENT -rw-r--r-- 1 hebasto admin 3842 13 Nov 06:22 README drwxr-xr-x 57 hebasto admin 1824 16 Apr 09:09 bin drwxr-xr-x 4 hebasto admin 128 13 Nov 06:22 doc drwxr-xr-x 95 hebasto admin 3040 13 Nov 06:22 include drwxr-xr-x 119 hebasto admin 3808 16 Apr 09:09 lib drwxr-xr-x 8 hebasto admin 256 13 Nov 06:22 libexec drwxr-xr-x 79 hebasto admin 2528 16 Apr 09:09 mkspecs drwxr-xr-x 15 hebasto admin 480 13 Nov 06:22 phrasebooks drwxr-xr-x 31 hebasto admin 992 13 Nov 06:22 plugins drwxr-xr-x 28 hebasto admin 896 13 Nov 06:22 qml -rw-r--r-- 1 hebasto admin 6952 16 Apr 09:09 sbom.spdx.json drwxr-xr-x 3 hebasto admin 96 13 Nov 06:22 share drwxr-xr-x 347 hebasto admin 11104 13 Nov 06:22 translations % ls -l $(brew --prefix qt@6)/share/qt/ total 0 drwxr-xr-x 4 hebasto admin 128 30 Mar 09:49 doc drwxr-xr-x 35 hebasto admin 1120 16 Apr 09:16 libexec drwxr-xr-x 167 hebasto admin 5344 30 Mar 09:49 metatypes drwxr-xr-x 70 hebasto admin 2240 16 Apr 09:16 mkspecs drwxr-xr-x 178 hebasto admin 5696 30 Mar 09:49 modules drwxr-xr-x 15 hebasto admin 480 30 Mar 09:49 phrasebooks drwxr-xr-x 31 hebasto admin 992 30 Mar 09:49 plugins drwxr-xr-x 34 hebasto admin 1088 30 Mar 09:49 qml drwxr-xr-x 45 hebasto admin 1440 30 Mar 09:49 sbom drwxr-xr-x 285 hebasto admin 9120 30 Mar 09:49 translations ``` This PR addresses both issues and additionally adds a `deploy` target to the native macOS CI job to prevent any similar recessions in the future. Fixes https://github.com/bitcoin/bitcoin/issues/32267. ACKs for top commit: fanquake: ACK84de8c93e7Tree-SHA512: 27a0eff3cd9317647529ff4571bd79c5dd8f007775b19415c8c27ca4912a60d85074c840cf0443be314d9a404f78bb015029d46dab18e292462249a5d90c6c47
CI Scripts
This directory contains scripts for each build step in each build stage.
Running a Stage Locally
Be aware that the tests will be built and run in-place, so please run at your own risk. If the repository is not a fresh git clone, you might have to clean files from previous builds or test runs first.
The ci needs to perform various sysadmin tasks such as installing packages or writing to the user's home directory. While it should be fine to run the ci system locally on your development box, the ci scripts can generally be assumed to have received less review and testing compared to other parts of the codebase. If you want to keep the work tree clean, you might want to run the ci system in a virtual machine with a Linux operating system of your choice.
To allow for a wide range of tested environments, but also ensure reproducibility to some extent, the test stage
requires bash, docker, and python3 to be installed. To run on different architectures than the host qemu is also required. To install all requirements on Ubuntu, run
sudo apt install bash docker.io python3 qemu-user-static
It is recommended to run the ci system in a clean env. To run the test stage with a specific configuration,
env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
Configurations
The test files (FILE_ENV) are constructed to test a wide range of
configurations, rather than a single pass/fail. This helps to catch build
failures and logic errors that present on platforms other than the ones the
author has tested.
Some builders use the dependency-generator in ./depends, rather than using
the system package manager to install build dependencies. This guarantees that
the tester is using the same versions as the release builds, which also use
./depends.
It is also possible to force a specific configuration without modifying the file. For example,
env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'MAKEJOBS="-j1" FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
The files starting with 0n (n greater than 0) are the scripts that are run
in order.
Cache
In order to avoid rebuilding all dependencies for each build, the binaries are cached and reused when possible. Changes in the dependency-generator will trigger cache-invalidation and rebuilds as necessary.