Instead of doing the awkward /bin path prepending at config.site
creation time, set darwin_{CC,CXX} in a way that fully determines the
program's path (clang/clang++) similar to how AC_PATH_{TOOL,PROG} would
do.
Also see the added comment block in depends/Makefile for more context on
determining $PATH for our config.site.
clang warns when a command line option is unused, and some of our tests
use Werror, so unfortunately we cannot use this flag to pin our linker
for now. Leaving this commit in for future reference, as it would be
great if there's more granularity to Werror and we can be explicit about
what linker we want to use.
Prior to this commit, when int_vars was called for packages, it would
immediately expand the "single-dollar variables", which may be defined
in terms of variables which are not yet determined (e.g. variables
defined in package/*.mk, which are included after int_vars is called).
This is required for the next commit as after that commit, for darwin
cross-builds:
0. int_vars is defined in terms of $(1)_cc
1. $(1)_cc is defined in terms of darwin_CC
2. ... which is defined in terms of clang_resource_dir
3. ... which is defined in terms of native_cctools_clang_version
4. which is undetermined at the time when int_vars is being expanded and evaluated
1112035d32ffe73a4522226c8cb2f6a5878d3ada doc: fix various typos (Ikko Ashimine)
e8640849c775efcf202dbd34736fed8d61379c49 doc: Use https URLs where possible (Sawyer Billings)
Pull request description:
Consolidates / fixes the changes from #20762, #20836, #20810. There is no output when `test/lint/lint-all.sh` is run.
Closes#20807.
ACKs for top commit:
MarcoFalke:
ACK 1112035d32ffe73a4522226c8cb2f6a5878d3ada
Tree-SHA512: 22ca824688758281a74e5ebc6a84a358142351434e34c88c6b36045d2d241ab95fd0958565fd2060f98317e62e683323b5320cc7ec13592bf340e6922294ed78
87fe104537eab5ccd8728321fe1c9ba39f7dda78 depends: Use more legible qmake commands in qt package (Hennadii Stepanov)
bf35a8da6ec2791dedf36f459add69ac67b11ff9 depends: Do not set build_subdir for qt package (Hennadii Stepanov)
Pull request description:
Rather than using `cd` to jump all over the place, perform all `(q)make` commands from the top level directory.
Looking at bash like `cd ../../../..` gives me a headache.
Credits to **fanquake**.
This PR is an alternative to #20504 that works without any additional [non-trivial hack](https://github.com/bitcoin/bitcoin/pull/20504#issuecomment-734730336).
ACKs for top commit:
promag:
Tested ACK 87fe104537eab5ccd8728321fe1c9ba39f7dda78.
fanquake:
ACK 87fe104537eab5ccd8728321fe1c9ba39f7dda78
Tree-SHA512: 1d2a13b5358fc7406c5363ddd62fd363dbc0ec5ace68946e4d3e6e8620419afaa64ef2837488aaed226174e01e8897495085540f7126b80f8b2372d21b5b29f9
a4118c6e200e02e7560f8bc213697aa2909d95b1 Add patch to make codesign_allocate compatible with Apple's (Pieter Wuille)
Pull request description:
This is an alternative to #20638.
The problem is that Apple's codesign(_allocate) apparently rounds the "vmsize" attribute on the __LINKEDIT section to a multiple of 0x2000 on x86_64 rather than 0x1000 (as their published source code does). This divergence means that the binary signed by codesign is slightly different from the one recreated by our reattach-sig-to-gitian-output process, and the signature being invalid.
This fixes it by patching our codesign_allocate source code to also use 0x2000. In tests, this appears to result in matching binaries.
ACKs for top commit:
jonasschnelli:
Tested ACK a4118c6e200e02e7560f8bc213697aa2909d95b1 - removed the osx cache, built commit a4118c6e200e02e7560f8bc213697aa2909d95b1 for osx in gitian (dependency where built, patch was applied), signed on my signing mac (detach-sig-create), ran gitian osx signer with the produces signature and the a4118c6e200e02e7560f8bc213697aa2909d95b1 build (detach-sig-apply), signature then was successful verified on my Mac (codesign -v /Volumes/Bitcoin-Core/Bitcoin-Qt.app)
MarcoFalke:
Concept ACK a4118c6e200e02e7560f8bc213697aa2909d95b1
Tree-SHA512: 07b8cdf8216249ddfe4bd38b39f2b48b2e190d4002b84d8981e62197bbbc9f25ac5c137bcc32057b23fbf38cbb2889ef95101ce008edfbf608cd170b88b3acbc
7587d11ec959f15f469bd396d4ad2697729b4ccd build: remove cdrkit package from depends (fanquake)
0df98191268fe641dd5d12e3a9f517c0c5cfacdd build: Replace genisoimage with xorriso (fanquake)
22437fc72e78ba3845a3953853d40093de32c395 build: Run libdmg-hfsplus's DMG tool in make deploy (Carl Dong)
Pull request description:
This is a redo of fanquake's https://github.com/bitcoin/bitcoin/pull/18151, which, aside from switching us from the deprecated `genisoimage` to the maintained `xorriso`, is also necessary for Guix to achieve determinism without using faketime.
> xorriso and its mkisofs/genisoimage emulation alter-ego xorrisofs are
> more maintained, and has the right toggles for us to achieve output
> determinism without using blunt tools like faketime.
>
> In this commit, we use xorrisofs from the build environment rather than
> building it ourselves using depends. This is not necessary and can be
> changed in the future.
>
> From wiki.debian.org/genisoimage?action=recall&rev=11 :
>
> > The classical command line interface for production of ISO 9660
> > filesystem images is the option set established by program mkisofs.
> > For reasons of licensing and other problems with its author, Debian
> > ships a fork of mkisofs, called genisoimage, which was split off in
> > 2006 and then developed independently.
> >
> > Meanwhile, genisoimage gets no new features and not even bug fixes. It
> > is first choice only if its options -udf or -hfs are needed.
> >
> > Replacement in most uses cases, especially for bootable ISO 9660
> > filesystems, archiving, and backup, is xorrisofs which starts the -as
> > mkisofs emulation mode of program xorriso.
ACKs for top commit:
laanwj:
ACK 7587d11ec959f15f469bd396d4ad2697729b4ccd
Tree-SHA512: 62f3aad08fa8bf21192e951d7dd33b24975586d76834cfa3498f4b8cdb586cefec8cab2c073d1951a0884b5e182fd71ef2cf3accad98f84455016776ad3c5422
This change gets rid of multiple `../` that makes reasoning about the
script and its maintaining much easier.
Co-authored-by: fanquake <fanquake@gmail.com>
7d0271b5c30f86e4af175a5ab7df5e593cd85195 depends: Set CMAKE_INSTALL_RPATH for native packages (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
---
After #19685 started setting `LDFLAGS`, the `INSTALL_RPATH_USE_LINK_PATH` cmake option used in the libmultiprocess build no longer works, so it is neccessary to set `CMAKE_INSTALL_RPATH` as a fallback.
It's unclear currently whether the bad interaction between `INSTALL_RPATH_USE_LINK_PATH` and `LDFLAGS` is a bug, but the issue is reported:
- https://github.com/bitcoin/bitcoin/issues/19981#issuecomment-696680877
- https://discourse.cmake.org/t/install-rpath-use-link-path-not-working-when-cmake-exe-linker-flags-ldflags-is-set/1892Fixes#19981
ACKs for top commit:
fanquake:
ACK 7d0271b5c30f86e4af175a5ab7df5e593cd85195 - I haven't looked in depth, but I've re-read through #19981 and checked the failure by testing #19160 (with this reverted):
dongcarl:
ACK 7d0271b Looked into this a bit, it makes sense that for the things we build in depends, we want the library search to start in depends. It seems reasonable to expect this to happen automatically when `CMAKE_INSTALL_PREFIX` and `INSTALL_RPATH_USE_LINK_PATH` are set, but oh well...
Tree-SHA512: 97cc5801c3204c14cd33004423631456ca0701e2127ee5146810a76e2f4aac9de1f4b5437402a4329cda54e022dc99270fee7e38c2995765f36b3848215fa78e
b685f60a08007e0ae8a5564ee68cd94f9015d899 build: mac_alias 2.1.1 (fanquake)
5d2cbdf772030b203ab1b32c65481ce3fc524b22 macdeploy: use Python 3.6 (fanquake)
a42aa94c54699833723076f3bfaeeac668523a69 macdeploy: remove runHDIUtil in favor of directly calling subprocess.run (fanquake)
adaa26202b965346566b5adff2fc5cee65955dfe macdeploy: remove existing Bitcoin-Core.dmg if present (fanquake)
ccb0325b1bd1cee5a76382a16901dc80ea8f50d8 macdeploy: move qt_conf to where it's used (fanquake)
6390a04862c043cd2bdf3610f3bcf9cb5526659f macdeploy: consolidate .DS_Store generation (fanquake)
32347cd56aaae95f3f4c78be9270565285280d72 macdeploy: assume plistlib is available (fanquake)
0ab4018c1217f82dffd65e973d9cccf13af2ef50 macdeploy: have a single level of logging output (fanquake)
827d382aa79d503470cc7abb0000cc365db06f12 macdeploy: remove add-resources argument (fanquake)
464b34d4c328d5109b8dd197da9e7f00d1b843c2 macdeploy: remove codesigning argument (fanquake)
4d70d3d7fe29db38a1f9c84a3a6167ca57b38479 build: automatically determine macOS translations (fanquake)
Pull request description:
This consolidates our macOS build code so that `.DS_Store` generation is the same when running `make deploy` for macOS when building on Linux and macOS, rather than maintaining two version of code that essentially do the same thing (just slightly differently).
It also removes unused code and any AppleScript usage, automates finding translation files and generally simplifies `macdeployqtplus`. It also gets rid of the annoying "popping up" behaviour during DMG generation, names the created image `Bitcoin-Core.dmg` rather than `Bitcoin-Qt.dmg`.
ACKs for top commit:
dergoegge:
ACK b685f60a08007e0ae8a5564ee68cd94f9015d899 - Less and cleaner code looks good. I tested this with `make deploy` and everything still works + the popup during DMG generation is gone.
Tree-SHA512: dcd38344e2dfcfa7ffbccf6226a71425c4d16b421a4881d5ee37b8e7ef393b3e8077262444c39b11912269d8cf688aba897e6518cba8361eb24a03fdd03b8caf
c23f6f84efa2fe7e7168a5d41341f3a7c5598f70 Add depends qt fix for ARM macs (Jonas Schnelli)
Pull request description:
With this, depends builds fine on macOS 11 on an Apple Silicon Mac (ARM64).
ACKs for top commit:
laanwj:
Code review ACK c23f6f84efa2fe7e7168a5d41341f3a7c5598f70
Tree-SHA512: a8354cec99969cff9e7dab150c335050ddb4b3c93a9f12a4db5e8046f02b11ce692ac17c2b96cbbe7f380c1aa110b15b8d6d48d51bc9c560282c702e99fd8a8d
c82d15b6d189983ebf82f35ade33d830870f25c6 depends: Do not force Precompiled Headers (PCH) for building Qt on Linux (Hennadii Stepanov)
Pull request description:
On CentOS 8 (Cirrus CI job) the forced `-pch` option breaks Qt build.
Removing `-pch` option does not affect build time for other systems:
- master (e2ff5e7b35d71195278d2a2ed9485f141de33d7a):
```
$ time make -j 9 -C depends/ qt
...
Caching qt...
make: Leaving directory '/home/hebasto/guix/GitHub/bitcoin/depends'
real 4m22,359s
user 18m3,719s
sys 1m24,769s
```
- this PR:
```
$ time make -j 9 -C depends/ qt
...
Caching qt...
make: Leaving directory '/home/hebasto/guix/GitHub/bitcoin/depends'
real 4m14,862s
user 18m3,355s
sys 1m24,506s
```
Qt docs: https://doc.qt.io/qt-5/qmake-precompiledheaders.htmlFixes#20423
ACKs for top commit:
MarcoFalke:
review ACK c82d15b6d189983ebf82f35ade33d830870f25c6
Tree-SHA512: 0f2a3712e90de881d00f8e56c363edde33dd4f5c117df5744ab4e51d0a8146331de7236bc8329d68ddd91535cd853e68ee80ef4cceb6a909786abfd8881b01e8
Rather than using OSX_QT_TRANSLATIONS which must be manually updated,
and we forget to update anyway, i.e: #19059, automatically find and copy
available translations from the translations directory.
2f5dfe4a7ff12b6b57427374142cdf7e266b73bc depends: build qt in c++17 mode (fanquake)
104e859c9755aee5708ea1934454d88b10c266ff builds: don't pass -silent to qt when building in debug mode (fanquake)
e2c500636cb767347ae2b913345788ad3c3e8279 depends: build zeromq with -std=c++17 (fanquake)
2374f2fbef4359476fe3184e2402a2cc741cefad depends: build Boost with -std=c++17 (fanquake)
2dde55702da30ea568cac8a1d1cbddd652d6958e depends: build bdb with -std=c++17 (fanquake)
Pull request description:
In packages where we are passing `-std=c++11` switch to `-std=c++17`, or, `-std=c++1z` in the case of Qt.
This PR also contains a [commit](104e859c97) that improves debug output when building Qt for debugging (`DEBUG=1`).
Now we'll get output like this:
```bash
g++ -c -pipe -ffunction-sections -O2 -fPIC -std=c++11 -fno-exceptions <lots more> ../../corelib/kernel/qcoreapplication.cpp
```
rather than just:
```bash
compiling ../../corelib/kernel/qcoreapplication.cpp
```
Note that when you look at the DEBUG output for these changes when building Qt, you'll see objects being compiled with a mix of C++11 and C++17. The breakdown is roughly:
1. `qmake` built with `-std=c++11`:
```bash
Creating qmake...
make[1]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qtbase/qmake'
g++ -c -o project.o -std=c++11 -ffunction-sections -O2 -g <trim> <trim>/qt/5.9.8-4110fa99945/qtbase/qmake/project.cpp
# when qmake, Qt also builds some of it's corelib, such as corelib/global/qmalloc.cpp
g++ -c -o qmalloc.o -std=c++11 -ffunction-sections -O2 -g <trim> <trim>/qt/5.9.8-4110fa99945/qtbase/src/corelib/global/qmalloc.cpp
```
2. `qmake` is run, and passed our build options, including `-c++std`:
```bash
make[1]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qtbase'
<trim>qt/5.9.8-4110fa99945/qtbase/bin/qmake -o Makefile qtbase.pro -- -bindir <trim>/native/bin -c++std c++1z -confirm-license <trim>
```
3. After some cleaning and configuring, we actually start to build Qt, as well as it's tools and internal libs:
```bash
Building qt...
make[1]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qtbase/src'
# build libpng, zlib etc
gcc -c -m64 -pipe -pipe -O1 <trim> -o .obj/png.o png.c
# build libQt5Bootstrap, using C++11, which again compiles qmalloc.cpp
make[2]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qtbase/src/tools/bootstrap'
g++ -c -pipe -ffunction-sections -O2 -fPIC -std=c++11 <trim> -o .obj/qmalloc.o ../../corelib/global/qmalloc.cpp
# build a bunch of tools like moc, rcc, uic, qfloat16-tables, qdbuscpp2xml, using C++11
g++ -c -pipe -O2 -std=c++11 -fno-exceptions -Wall -W <trim> -o .obj/rcc.o rcc.cpp
# from here, Qt is compiled with -std=c++1z, including qmalloc.cpp, for the third and final time:
g++ -c -include .pch/Qt5Core <trim> -g -Og -fPIC -std=c++1z -fvisibility=hidden <trim> -o .obj/qmalloc.o global/qmalloc.cpp
```
4. Finally, build tools like `lrelease`, `lupdate`, etc, but back to using -std=c++11
```bash
make[1]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qttools/src/linguist/lrelease'
g++ -c -pipe -O2 -std=c++11 -fno-exceptions -Wall -W <trim> -o .obj/translator.o ../shared/translator.cpp
```
If you dump the debug info from the built Qt libs, they should also tell you that they were compiled with `C++17`:
```bash
objdump -g bitcoin/depends/x86_64-pc-linux-gnu/lib/libQt5Core.a
GNU C++17 9.3.0 -m64 -mtune=generic -march=x86-64 -g -O1 -Og -std=c++17 -fPIC -fvisibility=hidden -fvisibility-inlines-hidden -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection
```
ACKs for top commit:
laanwj:
Code review ACK 2f5dfe4a7f
practicalswift:
cr ACK 2f5dfe4a7ff12b6b57427374142cdf7e266b73bc: patch looks correct
fjahr:
Code review ACK 2f5dfe4a7ff12b6b57427374142cdf7e266b73bc
hebasto:
ACK 2f5dfe4a7ff12b6b57427374142cdf7e266b73bc, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: fc5e9d7c7518c68349c8228fb1aead829850373efc960c9b8c079096a83d1dad19c62a9730fce5802322bf07e320960fd47851420d429eda0a87c307f4e8b03a
8f7d1b39efbe65ab2747c593cc3560d4a449a333 Fix QPainter non-determinism on macOS (Andrew Chow)
Pull request description:
Aplies a patch to Qt that fixes the non-determinism by modifying Qt. The source of the non-determinism is how LLVM 8 optimizes qt_intersect_spans when compiling. The particular optimization that seems to be causing the problems is that a temp variable is being added for spans->y. For some reason, when it does this, it chooses different instructions to use when making that variable. We bypass this problem by patching qt_intersect_spans to always make and use this local variable.
Potential alternative to #20436 and #20440
ACKs for top commit:
hebasto:
re-ACK 8f7d1b39efbe65ab2747c593cc3560d4a449a333 ~for merging into the 0.21 branch, but [not into the master](https://github.com/bitcoin/bitcoin/pull/20454) branch.~
fanquake:
ACK 8f7d1b39efbe65ab2747c593cc3560d4a449a333
Tree-SHA512: b0d00a77643554021736524fb64611462ef2ec849a220543c12d99edb0f52f2e8128d2cc61fa82176b7e13b294574774a92d6b649badf8b7630c6d6a7e70ce10
This means we'll get build output like this when building with DEBUG=1:
g++ -c -pipe -ffunction-sections -O2 -fPIC -std=c++11 -fno-exceptions <lots more> ../../corelib/kernel/qcoreapplication.cpp
rather than just:
compiling ../../corelib/kernel/qcoreapplication.cpp
Aplies a patch to Qt that fixes the non-determinism by modifying Qt. The
source of the non-determinism is how LLVM 8 optimizes qt_intersect_spans
when compiling. The particular optimization that seems to be causing the
problems is that a temp variable is being added for spans->y. For some
reason, when it does this, it chooses different instructions to use when
making that variable. We bypass this problem by patching
qt_intersect_spans to always make and use this local variable.
a52ecc936a267cae97c2f313743bf75d5af07700 build: set minimum supported macOS to 10.14 (fanquake)
Pull request description:
This is a requirement for C++17 support. See my comments [here](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-643722538):
> You cannot use std::get with std::variant on macOS < 10.14, because Apples libc++ doesn't support the std::bad_variant_access exception. [Relevant comment](https://github.com/bitcoin/bitcoin/pull/19183#discussion_r439794318) in #19183.
> While we could work around this in our own code, using std::get_if, this would still be a problem for 3rd-party dependencies.
> I've been testing Qt 5.15LTS (we'll have to enable C++17 in qt, and may upgrade to a newer version at the same time), and you can't enable -std c++17, while targeting a macOS deployment version < 10.14, configuring will fail. They are making use of std::get with std::variant throughout their cocoa code.
We would have to had to have bumped to at least 10.13 in any case, as Qt 5.15 (#19716) [requires 10.13+](https://doc.qt.io/qt-5/supported-platforms.html).
ACKs for top commit:
hebasto:
ACK a52ecc936a267cae97c2f313743bf75d5af07700, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: f669b2fc777aeea1e9afdbbc7bd9afe3997418211db6ba53c934cae0e62a9b999603da539518c229f34961d275c9e2f315c7b022cf5fb97bd201a69c85d470cc
