mirror of
https://github.com/bitcoin/bitcoin.git
synced 2025-04-02 08:59:08 +02:00
Merge bitcoin/bitcoin#31657: ci: Supply --platform argument to docker commands.
6e29de21010fc5213176a6ba29f754ca72612ea0 ci: Supply `platform` argument to docker commands. (David Gumberg)
Pull request description:
I ran into this issue when following the instructions in `ci/README.md` for running CI locally.
Newer versions of docker require a `--platform` argument when building from a platform-specific image that differs from the host platform, I'm not sure when this change took place, but trying to build any of the cross-platform CI images on Docker 27.5.0 fails in the following manner:
```console
$ # From ci/README.md
$ env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64/v4) and no specific platform was requested
Creating docker.io/arm64v8/debian:bookworm container to run in
+ docker build --file $BITCOIN_SRC/ci/test_imagefile --build-arg CI_IMAGE_NAME_TAG=docker.io/arm64v8/debian:bookworm --build-arg FILE_ENV=./ci/test/00_setup_env_arm.sh --label=bitcoin-ci-test --tag=ci_arm_linux $BITCOIN_SRC
[+] Building 0.6s (2/2) FINISHED docker:default
=> [internal] load build definition from test_imagefile 0.0s
=> => transferring dockerfile: 600B 0.0s
=> WARN: InvalidDefaultArgInFrom: Default value for ARG ${CI_IMAGE_NAME_TAG} results in empty or invalid base image name (line 8) 0.0s
=> ERROR [internal] load metadata for docker.io/arm64v8/debian:bookworm 0.5s
------
> [internal] load metadata for docker.io/arm64v8/debian:bookworm:
------
1 warning found (use docker --debug to expand):
- InvalidDefaultArgInFrom: Default value for ARG ${CI_IMAGE_NAME_TAG} results in empty or invalid base image name (line 8)
test_imagefile:8
--------------------
6 |
7 | ARG CI_IMAGE_NAME_TAG
8 | >>> FROM ${CI_IMAGE_NAME_TAG}
9 |
10 | ARG FILE_ENV
--------------------
ERROR: failed to solve: docker.io/arm64v8/debian:bookworm: failed to resolve source metadata for docker.io/arm64v8/debian:bookworm: no match for platform in manifest: not found
```
This branch fixes this by setting the `--platform` argument of `docker build` and `docker run` with an environment variable `CI_IMAGE_PLATFORM` for each platform specific job, and `linux/{$cpuarch}` for any native jobs.
Thi
## Steps to reproduce
1. Install relevant dependencies, on Ubuntu:
```bash
sudo apt install bash docker.io python3 qemu-user-static
```
2. Run one of the platform-specific CI images, e.g.:
```bash
env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
```
ACKs for top commit:
maflcko:
lgtm ACK 6e29de21010fc5213176a6ba29f754ca72612ea0
hebasto:
ACK 6e29de21010fc5213176a6ba29f754ca72612ea0
Tree-SHA512: 81b9fa8ec1f3d21619d37d864047c8d7917ef2c8536851f80facf7f1973dfe14628d7755f12d2a9c6edebb6cb16877c582d4d41cdab52b73b23c44f08c6e6b30
This commit is contained in:
merge-script
commit
6dc6012670
@ -68,3 +68,6 @@ export CI_BASE_PACKAGES=${CI_BASE_PACKAGES:-build-essential pkgconf curl ca-cert
|
||||
export GOAL=${GOAL:-install}
|
||||
export DIR_QA_ASSETS=${DIR_QA_ASSETS:-${BASE_SCRATCH_DIR}/qa-assets}
|
||||
export CI_RETRY_EXE=${CI_RETRY_EXE:-"retry --"}
|
||||
|
||||
# The --platform argument used with `docker build` and `docker run`.
|
||||
export CI_IMAGE_PLATFORM=${CI_IMAGE_PLATFORM:-"linux"} # Force linux, but use native arch by default
|
||||
|
||||
@ -10,7 +10,8 @@ export HOST=arm-linux-gnueabihf
|
||||
export DPKG_ADD_ARCH="armhf"
|
||||
export PACKAGES="python3-zmq g++-arm-linux-gnueabihf busybox libc6:armhf libstdc++6:armhf libfontconfig1:armhf libxcb1:armhf"
|
||||
export CONTAINER_NAME=ci_arm_linux
|
||||
export CI_IMAGE_NAME_TAG="docker.io/arm64v8/debian:bookworm" # Check that https://packages.debian.org/bookworm/g++-arm-linux-gnueabihf (version 12.2, similar to guix) can cross-compile
|
||||
export CI_IMAGE_NAME_TAG="docker.io/debian:bookworm" # Check that https://packages.debian.org/bookworm/g++-arm-linux-gnueabihf (version 12.2, similar to guix) can cross-compile
|
||||
export CI_IMAGE_PLATFORM="linux/arm64"
|
||||
export USE_BUSY_BOX=true
|
||||
export RUN_UNIT_TESTS=true
|
||||
export RUN_FUNCTIONAL_TESTS=false
|
||||
|
||||
@ -8,7 +8,8 @@ export LC_ALL=C.UTF-8
|
||||
|
||||
export HOST=i686-pc-linux-gnu
|
||||
export CONTAINER_NAME=ci_i686_multiprocess
|
||||
export CI_IMAGE_NAME_TAG="docker.io/amd64/ubuntu:24.04"
|
||||
export CI_IMAGE_NAME_TAG="docker.io/ubuntu:24.04"
|
||||
export CI_IMAGE_PLATFORM="linux/amd64"
|
||||
export PACKAGES="llvm clang g++-multilib"
|
||||
export DEP_OPTS="DEBUG=1 MULTIPROCESS=1"
|
||||
export GOAL="install"
|
||||
|
||||
@ -9,7 +9,8 @@ export LC_ALL=C.UTF-8
|
||||
export HOST=s390x-linux-gnu
|
||||
export PACKAGES="python3-zmq"
|
||||
export CONTAINER_NAME=ci_s390x
|
||||
export CI_IMAGE_NAME_TAG="docker.io/s390x/ubuntu:24.04"
|
||||
export CI_IMAGE_NAME_TAG="docker.io/ubuntu:24.04"
|
||||
export CI_IMAGE_PLATFORM="linux/s390x"
|
||||
export TEST_RUNNER_EXTRA="--exclude rpc_bind,feature_bind_extra" # Excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
|
||||
export RUN_FUNCTIONAL_TESTS=true
|
||||
export GOAL="install"
|
||||
|
||||
@ -7,7 +7,8 @@
|
||||
export LC_ALL=C.UTF-8
|
||||
|
||||
export CONTAINER_NAME=ci_win64
|
||||
export CI_IMAGE_NAME_TAG="docker.io/amd64/debian:bookworm" # Check that https://packages.debian.org/bookworm/g++-mingw-w64-x86-64-posix (version 12.2, similar to guix) can cross-compile
|
||||
export CI_IMAGE_NAME_TAG="docker.io/debian:bookworm" # Check that https://packages.debian.org/bookworm/g++-mingw-w64-x86-64-posix (version 12.2, similar to guix) can cross-compile
|
||||
export CI_IMAGE_PLATFORM="linux/amd64"
|
||||
export HOST=x86_64-w64-mingw32
|
||||
export DPKG_ADD_ARCH="i386"
|
||||
export PACKAGES="nsis g++-mingw-w64-x86-64-posix wine-binfmt wine64 wine32 file"
|
||||
|
||||
@ -14,7 +14,7 @@ if [ -z "$DANGER_RUN_CI_ON_HOST" ]; then
|
||||
# Though, exclude those with newlines to avoid parsing problems.
|
||||
python3 -c 'import os; [print(f"{key}={value}") for key, value in os.environ.items() if "\n" not in value and "HOME" != key and "PATH" != key and "USER" != key]' | tee "/tmp/env-$USER-$CONTAINER_NAME"
|
||||
# System-dependent env vars must be kept as is. So read them from the container.
|
||||
docker run --rm "${CI_IMAGE_NAME_TAG}" bash -c "env | grep --extended-regexp '^(HOME|PATH|USER)='" | tee --append "/tmp/env-$USER-$CONTAINER_NAME"
|
||||
docker run --platform="${CI_IMAGE_PLATFORM}" --rm "${CI_IMAGE_NAME_TAG}" bash -c "env | grep --extended-regexp '^(HOME|PATH|USER)='" | tee --append "/tmp/env-$USER-$CONTAINER_NAME"
|
||||
|
||||
# Env vars during the build can not be changed. For example, a modified
|
||||
# $MAKEJOBS is ignored in the build process. Use --cpuset-cpus as an
|
||||
@ -31,6 +31,7 @@ if [ -z "$DANGER_RUN_CI_ON_HOST" ]; then
|
||||
--build-arg "CI_IMAGE_NAME_TAG=${CI_IMAGE_NAME_TAG}" \
|
||||
--build-arg "FILE_ENV=${FILE_ENV}" \
|
||||
$MAYBE_CPUSET \
|
||||
--platform="${CI_IMAGE_PLATFORM}" \
|
||||
--label="${CI_IMAGE_LABEL}" \
|
||||
--tag="${CONTAINER_NAME}" \
|
||||
"${BASE_READ_ONLY_DIR}"
|
||||
@ -100,6 +101,7 @@ if [ -z "$DANGER_RUN_CI_ON_HOST" ]; then
|
||||
--env-file /tmp/env-$USER-$CONTAINER_NAME \
|
||||
--name "$CONTAINER_NAME" \
|
||||
--network ci-ip6net \
|
||||
--platform="${CI_IMAGE_PLATFORM}" \
|
||||
"$CONTAINER_NAME")
|
||||
export CI_CONTAINER_ID
|
||||
export CI_EXEC_CMD_PREFIX="docker exec ${CI_CONTAINER_ID}"
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user