893 Commits

Author SHA1 Message Date
Hennadii Stepanov
223b1ba7d9
doc: Use CONFIG_SITE instead of --prefix 2021-04-09 12:25:18 +03:00
Carl Dong
fa872c9af3 depends: Fix id_string invocations
We now use a script named gen_id to generate the base build_id/host_id.
This solves 2 problems:

1. GNU Make special-casing exit code 127 (command not found) meant that
   warnings about missing tools would propagate to the user's terminal
   and broke our opportunistic build_id construction.
2. This change ensures that we don't have arbitrary characters in our
   make variables that would be misinterpreted by Make.

See comments in depends/Makefile and depends/gen_id for more
information.
2021-04-08 20:17:20 -04:00
fanquake
a5491882a0
build: fix configuring when building depends with NO_BDB=1
Currently, if you build depends using `NO_BDB=1` (only sqlite wallets),
./configure will fail as it still tries to find bdb. i.e:
```bash
checking for Berkeley DB C++ headers... default
configure: error: Found Berkeley DB other than 4.8, required for portable BDB wallets (--with-incompatible-bdb to ignore or --without-bdb to disable BDB wallet support)
```

This PR fixes the build such that you can build depends, opting out of
bdb without opting out of wallets entirely, and still configure
successfully.
2021-04-07 20:50:10 +08:00
Carl Dong
84912d4b24 build: Remove spaces from variable-printing rules
This simplifies parsing when using these rules from scripts.
2021-04-05 19:13:54 -04:00
W. J. van der Laan
0102f80b51
Merge : guix: Misc feedback-based fixes + hier restructuring
7476b46f1893a4858616d2a8456a7c43238851ed guix: Build dmg as a static binary (Carl Dong)
06d6cf6784421290e6235fe8684d5e08ed6f1b62 depends: libdmg-hfsplus: Skip CMake RPATH patching (Carl Dong)
65176ab5730dff34466caaecdd292625ef8294fc guix: Remove codesign_allocate+pagestuff from unsigned tarball (Carl Dong)
ca85679eb43b8375a95d82101977829d08fb1e1b guix: Use clang-toolchain instead of clang (Carl Dong)
1aec0eda8fd31a57b0621eea616398017c2ead98 guix: Fallback to local build for substitute-enabled Guix users (Carl Dong)
1742f8e12d163852df09575e03edcd3db73198ee guix: Add early health check for guix-daemon (Carl Dong)
c1ae726a13ecfa5e7e9fdc3030a8110b8bb263f8 guix: More thoroughly control native toolchain (Carl Dong)
39741128d3775d198dbee34dc827353bfd18acd8 guix: Supply --link-profile (Carl Dong)
d55a1056ee565afed64e42d6f6efb6b0adc5599b guix: Add troubleshooting documentation entries (Carl Dong)
7f401c953f8bb3574cec48561e13ef3b47dedc6e guix: Adapt guix-build to prelude, restructure hier (Carl Dong)
4eccf063b252bfe256cf72d363a24cf0183e926e guix: Remove guix-build.sh filename extension (Carl Dong)
7753357a7bae98ec775c707b9dec4cea1e945802 guix: Add source-able bash prelude and utils (Carl Dong)
e5b49a01f5d0f631e7f08f86ca8a2c2b8213319f guix: Create windeploy inside distsrc-* (Carl Dong)
3e9982ab3877eb8fe0a8c0cb3d847ac0913c7336 contrib: Silence git-describe when looking for tag (Carl Dong)
d5a71e97853ea9e1b879e8c76bfb01d4bef33172 guix: Use --cores instead of --max-jobs (Carl Dong)

Pull request description:

  This PR addresses a few hiccups encountered by the brave souls who've been experimenting with the Guix scripts:
  - Resolves confusion between `--cores=` and `--max-jobs=`
    - `guix`'s `--cores=` actually corresponds to make's `--jobs=`, so let's just control `--cores=` with our overridable env var
  - `git-describe` will scream `fatal: no tag exactly matches '<hash>'` when looking for a tag, but we don't care, so silence that
  - `windeploy/unsigned` should be inside `distsrc-*` and created idempotently (sorry I know this one annoyed people)
  - Add troubleshooting documentation to `README.md`
  - Add early health check for `guix-daemon` in case user forgot to start a `guix-daemon`
  - Depending on configuration, a `--fallback` flag may be needed to tell Guix to not fail if substitutes fail but fallback to building locally
  - `codesign_allocate` and `pagestuff` are now unnecessary for codesigning as we're now using `signapple`

  A few robustness changes are also included:
  - We supply the `--link-profile` flag, as some Guix packages may expect the profile to be available under `$HOME/.guix-profile`
  - We now clear and manually set all toolchain-related env vars (e.g. `C*_INCLUDE_PATH`) ourselves, after patching a Qt::moc bug
  - We use the native `clang-toolchain` package for darwin builds instead of `clang`, lining up with all our other toolchain packages.

  Finally, we restructure the guix building hierarchy such that it looks something like:
  ```
  guix-build-<short-hash-or-version-tag>
  ├── distsrc-<short-hash-or-version-tag>-${HOST}
  │   ├── contrib
  │   ├── depends
  │   ├── src
  │   └── ...
  ├── distsrc-<short-hash-or-version-tag>-...
  └── output
      ├── dist-archive
      │   └── bitcoin-<short-hash-or-version-tag>.tar.gz
      ├── *-linux-*
      │   ├── bitcoin-<short-hash-or-version-tag>-*-linux-*-debug.tar.gz
      │   └── bitcoin-<short-hash-or-version-tag>-*-linux-*.tar.gz
      ├── x86_64-apple-darwin18
      │   ├── bitcoin-<short-hash-or-version-tag>-osx64.tar.gz
      │   ├── bitcoin-<short-hash-or-version-tag>-osx-unsigned.dmg
      │   └── bitcoin-<short-hash-or-version-tag>-osx-unsigned.tar.gz
      └── x86_64-w64-mingw32
          ├── bitcoin-<short-hash-or-version-tag>-win64-debug.zip
          ├── bitcoin-<short-hash-or-version-tag>-win64-setup-unsigned.exe
          ├── bitcoin-<short-hash-or-version-tag>-win64.zip
          └── bitcoin-<short-hash-or-version-tag>-win-unsigned.tar.gz
  ```
  Separating guix builds by their version identifier (basically namespacing them) allows us to change the layout in the future without worry about potential naming conflicts.

ACKs for top commit:
  sipa:
    ACK 7476b46f1893a4858616d2a8456a7c43238851ed
  laanwj:
    ACK 7476b46f1893a4858616d2a8456a7c43238851ed

Tree-SHA512: 0e899aa941aafdf552b2a7e8a08131ee9283180bbef7334439e2461a02aa7235ab7b9ca9c149b80fc5d0a9f4bbd35bc80fcee26197c0836ba8eaf2d86ffa0386
2021-04-06 01:00:27 +02:00
Carl Dong
7476b46f18 guix: Build dmg as a static binary
This relatively easy change eliminates all runtime dependencies (except
for the kernel) for dmg, which is the only native build tool that gets
put in our output tarballs.

This allows much more flexibility when constructing the codesigning
environment, and is much more robust.
2021-04-05 11:00:39 -04:00
Carl Dong
06d6cf6784 depends: libdmg-hfsplus: Skip CMake RPATH patching
CMake's RPATH patching apparently causes non-reproducibility in the
executables which are produced, manifesting in a difference in padding
in the .dynstr section (we found this while investigating
non-reproducibility in the "dmg" tool). This RPATH patching can be
safely skipped for executables which don't depend on internal shared
libraries.

Documentation sources:
1. https://gitlab.kitware.com/cmake/community/-/wikis/doc/cmake/RPATH-handling
2. https://reproducible-builds.org/docs/deterministic-build-systems/#cmake-notes

Prior debugging art:
1. https://stackoverflow.com/questions/63438206/cmake-g-reproducible-build-issue-with-changing-build-path
2. https://github.com/NXPmicro/mfgtools/pull/229/files
2021-04-05 11:00:39 -04:00
Carl Dong
c1ae726a13 guix: More thoroughly control native toolchain 2021-04-05 11:00:39 -04:00
Carl Dong
39741128d3 guix: Supply --link-profile 2021-04-05 11:00:39 -04:00
Hennadii Stepanov
b95f7f8ac0
build, qt, refactor: Drop sed commands for win32-g++/qmake.conf 2021-04-04 14:40:06 +03:00
fanquake
735610940c
build: set --build when configuring packages in depends
After reading
https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.70/autoconf.html#Specifying-Target-Triplets,
my understanding is that this change should mostly be a no-op, as
--build defaults to the output of config.guess, however, this may be
slightly more correct
>  For historical reasons, whenever you specify --host, be sure to
> specify --build too; this will be fixed in the future.

and will quell some warnings in depends (). If anything, this
also explicitly enables cross-compilation mode when `--host` differs
from `--build`.

As for "fixed in the future", this is the case for Autoconf 2.70+.
2021-03-31 17:05:52 +08:00
fanquake
765e0be534
build: split native_cctools 2021-03-30 14:54:08 +08:00
Hennadii Stepanov
ec76bad655
build, qt: Fix static builds on macOS Big Sur
See details and the patch: https://bugreports.qt.io/browse/QTBUG-87014
2021-03-25 03:17:40 +02:00
Wladimir J. van der Laan
23b15601df
Merge : Qt: Add Android packaging support
246774e26459cb3652e308880abdd140e8e9d204 depends: fix Qt precompiled headers bug (Igor Cota)
8e7ad4146d55f472e3d1dacaabb6b7dee704a896 depends: disable Qt Vulkan support on Android (Igor Cota)
ba46adaa1abd51798394b5bad3799021adc237d2 CI: add Android APK build to cirrus (Igor Cota)
7563720e30a3052b7ee390f1b3d2874856fd073a CI: add Android APK build script (Igor Cota)
ebfb10cb75adb704418d08197681c1e742e63bd5 Qt: add Android packaging support (Igor Cota)

Pull request description:

  ![bitcoin-qt](https://user-images.githubusercontent.com/762502/67396157-62f3d000-f5a7-11e9-8a6f-9425823fcd6c.gif)
  This PR is the third and final piece of the basic Android support puzzle - it depends on https://github.com/bitcoin/bitcoin/pull/16110 and is related to https://github.com/bitcoin/bitcoin/pull/16883. It introduces an `android` directory under `qt` and a simple way to build an Android package of `bitcoin-qt`:

  1. Build depends for Android as described in the [README](https://github.com/bitcoin/bitcoin/blob/master/depends/README.md)
  2. Configure with one of the resulting prefixes
  3. Run `make && make apk` in `src/qt`

  The resulting APK files will be in `android/build/outputs/apk`. You can install them manually or with [adb](https://developer.android.com/studio/command-line/adb). One can also open the `android` directory in Android Studio for that integrated development and debugging experience. `BitcoinQtActivity` is your starting point.

  Under the hood makefile `apk` target:

  1. Renames the `bitcoin-qt` binary to `libbitcoin-qt.so` and copies it over to a folder under `android/libs` depending on which prefix and corresponding [ABI](https://developer.android.com/ndk/guides/abis.html#sa) `bitcoin-qt` was built for
  2. Takes `libc++_shared.so` from the Android NDK and puts in the same place. It [must be included](https://developer.android.com/ndk/guides/cpp-support) in the APK
  3. Extracts Qt for Android Java support files from the `qtbase` archive in `depends/sources` to `android/src`

  There is also just a tiny bit of `ifdef`'d code to make the Qt Widgets menus usable. It's not pretty but it works and is a stepping stone towards https://github.com/bitcoin/bitcoin/pull/16883.

ACKs for top commit:
  MarcoFalke:
    cr ACK 246774e264
  laanwj:
    Code review ACK 246774e26459cb3652e308880abdd140e8e9d204

Tree-SHA512: ba30a746576a167545223c35a51ae60bb0838818779fc152c210f5af1413961b2a6ab6af520ff92cbc8dcd5dcb663e81ca960f021218430c1f76397ed4cead6c
2021-03-24 19:02:01 +01:00
Wladimir J. van der Laan
fa2a5b8f3a
Merge : build: miniupnpc 2.2.2
180dc3c8863fc42e93657eda839001a2a35ef634 build: miniupnpc 2.2.2 (fanquake)

Pull request description:

  Creating the dll subdir is no-longer required.
  We can drop our wingen patch.

  One issue that came up in [](https://github.com/bitcoin/bitcoin/pull/19867#discussion_r483516359):
  > unrelated to this change but: why are we inserting the architecture in here, seems like something not necessary to reveal

  > My assumption is that it was being inserted to make depends more deterministic. However I think we can improve this, as there's no reason to reveal more of the version information either. Could leave the version as is /2.0 and either drop the architecture, or insert something else?

  I've dropped our `sed` and added a patch that just removes the OS string and miniupnpc version from the User-Agent. i.e:
  ```bash
  # master
  strings depends/x86_64-apple-darwin19.6.0/lib/libminiupnpc.a | rg -i User-Agent
  User-Agent: x86_64-apple-darwin19.6.0, UPnP/1.1, MiniUPnPc/2.0.20180203
  User-Agent: x86_64-apple-darwin19.6.0, UPnP/1.1, MiniUPnPc/2.0.20180203

  # this PR
  strings depends/x86_64-apple-darwin19.6.0/lib/libminiupnpc.a | rg -i User-Agent
  User-Agent: UPnP/1.1
  User-Agent: UPnP/1.1
  ```

  Note that built unmodified (22c1386351), the User-Agent would be:
  ```bash
  strings libminiupnpc.dylib | rg User-Agent
  User-Agent: Darwin/19.6.0, UPnP/1.1, MiniUPnPc/2.2.0
  User-Agent: Darwin/19.6.0, UPnP/1.1, MiniUPnPc/2.2.0
  ```

ACKs for top commit:
  laanwj:
    Code review ACK 180dc3c8863fc42e93657eda839001a2a35ef634
  hebasto:
    ACK 180dc3c8863fc42e93657eda839001a2a35ef634.

Tree-SHA512: b0b6e623dbc5499e28faedf992d84278d6a11887a45a3806957b9e08886c5e56044cdfa2e7d7ec81cb1dd55f89be99834367905315d6bc611ba530e91d889ad1
2021-03-23 21:06:41 +01:00
fanquake
180dc3c886
build: miniupnpc 2.2.2
Creating the dll subdir is no-longer required.
We can also drop our wingen patch.
2021-03-23 08:39:16 +08:00
Hennadii Stepanov
810b1310d6
build: Do not build unused CoreWLAN stuff in depends for macOS 2021-03-21 23:45:26 +02:00
Igor Cota
246774e264 depends: fix Qt precompiled headers bug
Fixed in 5.14, see QTBUG-85214
2021-03-21 22:33:35 +01:00
Igor Cota
8e7ad4146d depends: disable Qt Vulkan support on Android 2021-03-21 22:33:35 +01:00
Hennadii Stepanov
cc25f892d2
build: Cleanup libxkbcommon_postprocess_cmds
There is no "share" directory in the staging one.
2021-03-12 14:51:10 +02:00
fanquake
550ed1bed2
build: update qt qpaint non determinism patch for 5.12.10 2021-03-10 12:57:27 +08:00
fanquake
6093ae4d30
build: update qt no-xlib patch for 5.12.10 2021-03-10 12:57:27 +08:00
fanquake
84928c4e73
build: update qt android jni static patch for 5.12.10 2021-03-10 12:57:27 +08:00
fanquake
cc6f47d51a
build: update qt lrelease patch for 5.12.10 2021-03-10 12:57:27 +08:00
Hennadii Stepanov
286d07ff17
build, qt: Fix lib paths in *.pc files
See:
- QTBUG-72903, commit 9864d2c6f3b628ca9f07a56b197e77bd43931cca
- QTBUG-78873, commit e55a61a77f0c87c05661a0335dfdb12673c6a27f

Could be dropped for Qt 5.14+.
2021-03-10 12:57:27 +08:00
fanquake
fa5e97e8c2
build: disable qt SDK version checking
This tries to invoke xcrun, which is not available when cross-compiling.
Given we are in control of the SDK versions being used, removing this
check has minimal-no effect.
2021-03-10 12:57:26 +08:00
fanquake
e674e94302
build: revert to using Qts internal zlib 2021-03-10 12:57:26 +08:00
fanquake
06cd0da21f
build: qt 5.12.10
remove fix_configure_mac.patch
Fixed upstream: https://bugreports.qt.io/browse/QTBUG-67286

remove fix_riscv64_arch.patch
Was fixed upstream in 6a39e49a6cdeb28a04a3657bb6a22f848d5dfa9d

remove fix_rcc_determinism.patch
Fixed upstream in https://bugreports.qt.io/browse/QTBUG-62511

remove freetype_back_compat.patch
By the time we ship a release with Qt 5.12, we'll certainly no-longer be
supporting Ubuntu 14.04 and Ubuntu 16.04 ships with FreeType 2.6.1,
which is new enough that using the symbol is no-longer an issue.
The renaming of FT_Get_X11_Font_Format() happened in FreeType 2.6

remove xkb-default.patch
This was removed upstream in d5abf545971da717014d316127045fc19edbcd65

Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2021-03-10 12:57:22 +08:00
Hennadii Stepanov
3272e34f9c
build: Add xkbcommon 0.8.4
Co-authored-by: fanquake <fanquake@gmail.com>
2021-03-10 08:23:47 +08:00
fanquake
d769b3372d
build: only pass -optimized-tools to qt in debug mode
Qt's configure tells us that "-optimized-tools is not useful in -release
mode.", so don't use it there.
2021-03-10 08:04:04 +08:00
Hennadii Stepanov
173ef8980d
build: Small libxcb.mk improvements 2021-03-07 23:17:28 +02:00
Hennadii Stepanov
5129b36573
build: Clean remnants of QTBUG-34748 fix
A fix for QTBUG-34748 was introduced in  (v0.11.0, Qt 5.2.1).
QTBUG-34748 was fixed in version 5.3.0.
The separated patch file, provided by , was dropped in  while
bumping Qt to 5.9.4 (5.9.6). But libxcb.mk remained unchanged.

This change reverts  for libxcb.mk.
2021-03-07 23:03:38 +02:00
fanquake
48725e64fb
Merge : build: use newer source for libnatpmp
7af25024e9563241c72797d4eeabdf660e548f53 build: compile libnatpmp with -DNATPMP_STATICLIB on Windows (fanquake)
ee35745754075d862e1855c77b20a09b260cce75 build: use newer source for libnatpmp (fanquake)

Pull request description:

  The source we are currently using is from 2015. The upstream repo has
  received a small number of bug fixes and improvements since then.
  Including one that fixes an issue for Windows users:  https://github.com/miniupnp/libnatpmp/pull/13.

  The source we are currently using is the most recent "official" release,
  however I don't think it's worth waiting for a new one. The maintainer
  was prompted to do so in Oct 2020, then again in Jan of this year, and
  no release has eventuated. Given libnatpmp is a new inclusion into our
  repository, I think we should be using this newer source.

  This also cleans up a few warnings we currently see in Windows depends builds:
  ```bash
  Extracting libnatpmp...
  /home/ubuntu/bitcoin/depends/sources/libnatpmp-20150609.tar.gz: OK
  Preprocessing libnatpmp...
  Configuring libnatpmp...
  Building libnatpmp...
  make[1]: Entering directory '/home/ubuntu/bitcoin/depends/work/build/x86_64-w64-mingw32/libnatpmp/20150609-13efa1beb87'
  x86_64-w64-mingw32-gcc -Os -fPIC -Wall -DENABLE_STRNATPMPERR   -c -o natpmp.o natpmp.c
  x86_64-w64-mingw32-gcc -Os -fPIC -Wall -DENABLE_STRNATPMPERR   -c -o getgateway.o getgateway.c
  natpmp.c:42: warning: "EWOULDBLOCK" redefined
     42 | #define EWOULDBLOCK WSAEWOULDBLOCK
        |
  In file included from natpmp.c:38:
  /usr/share/mingw-w64/include/errno.h:166: note: this is the location of the previous definition
    166 | #define EWOULDBLOCK 140
        |
  natpmp.c:43: warning: "ECONNREFUSED" redefined
     43 | #define ECONNREFUSED WSAECONNREFUSED
        |
  In file included from natpmp.c:38:
  /usr/share/mingw-w64/include/errno.h:110: note: this is the location of the previous definition
    110 | #define ECONNREFUSED 107
        |
  natpmp.c:271:5: warning: ‘readnatpmpresponseorretry’ redeclared without dllimport attribute: previous dllimport ignored [-Wattributes]
    271 | int readnatpmpresponseorretry(natpmp_t * p, natpmpresp_t * response)
        |     ^~~~~~~~~~~~~~~~~~~~~~~~~
  ar crs libnatpmp.a natpmp.o getgateway.o
  make[1]: Leaving directory '/home/ubuntu/bitcoin/depends/work/build/x86_64-w64-mingw32/libnatpmp/20150609-13efa1beb87'
  Staging libnatpmp...
  Postprocessing libnatpmp...
  Caching libnatpmp...
  ```

ACKs for top commit:
  hebasto:
    ACK 7af25024e9563241c72797d4eeabdf660e548f53

Tree-SHA512: 6939014ea986149a5bfdd42b516d563a65ae643516e234579d3f28e7c2f877b0270cc4305ae7c7cb131d6d946a6e0aedc84b4cc880a412612a878a333398b9d7
2021-03-06 09:20:03 +08:00
fanquake
7af25024e9
build: compile libnatpmp with -DNATPMP_STATICLIB on Windows
This fixes linking issues and mirrors what we do with miniupnpc.
2021-03-04 12:34:46 +08:00
fanquake
ee35745754
build: use newer source for libnatpmp
The source we are currently using is from 2015. The upstream repo has
received a small number of bug fixes and improvements since then.
Including one that fixes an issue for Windows users:
https://github.com/miniupnp/libnatpmp/pull/13.

The source we are currently using is the most recent "official" release,
however I don't think it's worth waiting for a new one. The maintainer
was prompted to do so in Oct 2020, then again in Jan of this year, and
no release has eventuated. Given libnatpmp is a new inclusion into our
repository, I think we should be using this newer source.

This also cleans up a few warnings we currently see in depends builds:
```bash
Extracting libnatpmp...
/home/ubuntu/bitcoin/depends/sources/libnatpmp-20150609.tar.gz: OK
Preprocessing libnatpmp...
Configuring libnatpmp...
Building libnatpmp...
make[1]: Entering directory '/home/ubuntu/bitcoin/depends/work/build/x86_64-w64-mingw32/libnatpmp/20150609-13efa1beb87'
x86_64-w64-mingw32-gcc -Os -fPIC -Wall -DENABLE_STRNATPMPERR   -c -o natpmp.o natpmp.c
x86_64-w64-mingw32-gcc -Os -fPIC -Wall -DENABLE_STRNATPMPERR   -c -o getgateway.o getgateway.c
natpmp.c:42: warning: "EWOULDBLOCK" redefined
   42 | #define EWOULDBLOCK WSAEWOULDBLOCK
      |
In file included from natpmp.c:38:
/usr/share/mingw-w64/include/errno.h:166: note: this is the location of the previous definition
  166 | #define EWOULDBLOCK 140
      |
natpmp.c:43: warning: "ECONNREFUSED" redefined
   43 | #define ECONNREFUSED WSAECONNREFUSED
      |
In file included from natpmp.c:38:
/usr/share/mingw-w64/include/errno.h:110: note: this is the location of the previous definition
  110 | #define ECONNREFUSED 107
      |
natpmp.c:271:5: warning: ‘readnatpmpresponseorretry’ redeclared without dllimport attribute: previous dllimport ignored [-Wattributes]
  271 | int readnatpmpresponseorretry(natpmp_t * p, natpmpresp_t * response)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~
ar crs libnatpmp.a natpmp.o getgateway.o
make[1]: Leaving directory '/home/ubuntu/bitcoin/depends/work/build/x86_64-w64-mingw32/libnatpmp/20150609-13efa1beb87'
Staging libnatpmp...
Postprocessing libnatpmp...
Caching libnatpmp...
```
2021-03-04 10:34:18 +08:00
Hennadii Stepanov
f7f3829a68
build, doc: Drop libbz2-dev from macOS cross-compiling dependencies 2021-03-03 22:43:08 +02:00
Hennadii Stepanov
d8239362e2
build, doc: Drop libcap-dev from macOS cross-compiling dependencies 2021-03-03 22:38:45 +02:00
fanquake
97a35f3ae5
Merge : build: fix libnatpmp macos cross compile
bd49ac416881ede3e5132789defd3fec4c6f8685 build: fix libnatpmp macos cross compile (fanquake)

Pull request description:

  Currently, our cross-compile of libnatpmp for macOS doesn't work at all.
  The wrong archiver is used, which produces an archive the linker doesn't like.
  This becomes clear when configuring:
  ```bash
  configure:25722: checking for initnatpmp in -lnatpmp
  configure:25747: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/ubuntu/bitcoin/depends/x86_64-apple-darwin18/native/bin/clang++ --target=x86_64-apple-darwin18 <trim>  -Wl,-headerpad_max_install_names -Wl,-dead_strip -Wl,-dead_strip_dylibs conftest.cpp -lnatpmp   >&5
  ld: archive has no table of contents for architecture x86_64
  clang: error: linker command failed with exit code 1 (use -v to see invocation)
  ```

  Fix this by using the right `ar` (we do the same for upnp).

  While we're at it, fix the build so that we are using our c/ppflags.
  In practice this basically means building with `-O2` rather than `-Os`.

  Note that this fixes an issue that is also fixed by . However, given there are reservations about updating to use a newer libnatpmp source, we should just fix this for now.

ACKs for top commit:
  hebasto:
    ACK bd49ac416881ede3e5132789defd3fec4c6f8685, tested:

Tree-SHA512: 2efc2c788ef3ebebfbf564ef07b6cf63a72d8a0bccc22b0ba36537216aa575436b7e87088477e85f6d9191ad34f0b13f1c22cf88c90e1cb81641bfee5dc3058a
2021-03-03 19:58:25 +08:00
fanquake
fca3e98f64
Merge : build: Fix Qt processing of configure script for depends with DEBUG=1
76f52e3da359c3738d36bc20ac13a8ccd17e4e9f build: Fix Qt processing of configure script for depends with DEBUG=1 (Hennadii Stepanov)

Pull request description:

  This PR:
  - makes the `configure` script correctly pickup Qt if depends is built with `DEBUG=1`:
      - for Windows -- fix 
      - for macOS -- fix 
  - is an alternative to  (without downsides)

ACKs for top commit:
  fanquake:
    ACK 76f52e3da359c3738d36bc20ac13a8ccd17e4e9f. Tested native darwin, and darwin/win cross compile with `DEBUG=1`.

Tree-SHA512: 8fde99302b4b06faf109315bddba9e3063b156c50f8f9863c2bd51718538c719429a63fdced071730c18022f2e559d3b25c1dcec3efa81fe79f657253680956a
2021-03-03 15:48:11 +08:00
fanquake
bd49ac4168
build: fix libnatpmp macos cross compile
Currently, our cross-compile of libnatpmp for macOS doesn't work at all.
The wrong archiver is used, which produces an archive the linker doesn't like.
This becomes clear when configuring:
```bash
configure:25722: checking for initnatpmp in -lnatpmp
configure:25747: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/ubuntu/bitcoin/depends/x86_64-apple-darwin18/native/bin/clang++ --target=x86_64-apple-darwin18 <trim>  -Wl,-headerpad_max_install_names -Wl,-dead_strip -Wl,-dead_strip_dylibs conftest.cpp -lnatpmp   >&5
ld: archive has no table of contents for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```

Fix this by using the right `ar` (we do the same for upnp).

While we're at it, we fixe the build so that we are using our c/ppflags.
This  means building with `-O2` rather than `-Os`.

Note that this fixes an issue that is also fixed by .
However, given there are reservations about updating to use a newer libnatpmp source, we should just fix this for now.
2021-03-01 14:00:36 +08:00
Wladimir J. van der Laan
51397c0ff7
Merge : depends: Improve id string robustness
5200929bfe26c549d7da92c0adf8adf61e143416 depends: Include GUIX_ENVIRONMENT in id string (Carl Dong)
4c7d41858821e4fecf7cb0cec3fcad002365e6c9 depends: Improve id string robustness (Carl Dong)
b3bdff42b5a7b4b956da700b187a7254daac54ae build: Proper quoting for var printing targets (Carl Dong)

Pull request description:

  ```
  Environment variables and search paths can drastically effect the
  operation of build tools.

  Include these in our id string to mitigate against false cache hits.
  ```

  Note to builders: This will invalidate all depends output caches in `BASE_CACHE`

ACKs for top commit:
  laanwj:
    re-ACK 5200929bfe26c549d7da92c0adf8adf61e143416

Tree-SHA512: e70c98da89cde90dc54bc3be89b925787cf94bbf246e27cc9345816b312073d78a02215448f731f21d8cf033c455234a2377ff1d66c00e1f3db69c9c9687d027
2021-02-15 11:43:00 +01:00
Wladimir J. van der Laan
9996b1806a
Merge : refactor: use std::shared_mutex & remove Boost Thread
060a2a64d40d75fecb60b7d2b9946a67e46aa6fc ci: remove boost thread installation (fanquake)
06e1d7d81d5a56d136c6fc88f09a2b0654a164f9 build: don't build or use Boost Thread (fanquake)
7097add83c8596f81be9edd66971ffd2486357eb refactor: replace Boost shared_mutex with std shared_mutex in sigcache (fanquake)
8e55981ef834490c438436719f95cbaf888c4914 refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests (fanquake)

Pull request description:

  This replaces `boost::shared_mutex` and `boost::unique_lock` with [`std::shared_mutex`](https://en.cppreference.com/w/cpp/thread/shared_mutex) & [`std::unique_lock`](https://en.cppreference.com/w/cpp/thread/unique_lock).

  Even though [some concerns were raised](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) in  with regard to `std::shared_mutex` being unsafe to use across some glibc versions, I still think this change is an improvement. As I mentioned in , I also think trying to restrict standard library feature usage based on bugs in glibc is not only hard to do, but it's not currently clear exactly how we do that in practice (does it also extend to patching out use in our dependencies, should we be implementing more runtime checks for features we are using, when do we consider an affected glibc "old enough" not to worry about? etc). If you take a look through the [glibc bug tracker](https://sourceware.org/bugzilla/describecomponents.cgi?product=glibc) you'll no doubt find plenty of (active) bug reports for standard library code we already using. Obviously not to say we shouldn't try and avoid buggy code where possible.

  Two other points:

  [Cory mentioned in ](https://github.com/bitcoin/bitcoin/pull/21022#issuecomment-769274179):
  > It also seems reasonable to me to worry that boost hits the same underlying glibc bug, and we've just not happened to trigger the right conditions yet.

  Moving away from Boost to the standard library also removes the potential for differences related to Boosts configuration. Boost has multiple versions of `shared_mutex`, and what you end up using, and what it's backed by depends on:
  * The version of Boost.
  * The platform you're building for.
  * Which version of `BOOST_THREAD_VERSION` is defined: (2,3,4 or 5) default=2. (see [here](https://www.boost.org/doc/libs/1_70_0/doc/html/thread/build.html#thread.build.configuration) for some of the differences).
  * Is `BOOST_THREAD_V2_SHARED_MUTEX` defined? (not by default). If so, you might get the ["less performant, but more robust"](https://github.com/boostorg/thread/issues/230#issuecomment-475937761) version of `shared_mutex`.

  A lot of these factors are eliminated by our use of depends, but users will have varying configurations. It's also not inconceivable to think that a distro, or some package manager might start defining something like `BOOST_THREAD_VERSION=3`. Boost tried to change the default from 2 to 3 at one point.

  With this change, we no longer use Boost Thread, so this PR also removes it from depends, the build system, CI etc.

  Previous similar PRs were  & . The authors are included in the commits here.
  Also related to  - pthread sanity checking.

ACKs for top commit:
  laanwj:
    Code review ACK 060a2a64d40d75fecb60b7d2b9946a67e46aa6fc
  vasild:
    ACK 060a2a64d40d75fecb60b7d2b9946a67e46aa6fc

Tree-SHA512: 572d14d8c9de20bc434511f20d3f431836393ff915b2fe9de5a47a02dca76805ad5c3fc4cceecb4cd43f3ba939a0508178c4e60e62abdbaaa6b3e8db20b75b03
2021-02-12 11:39:36 +01:00
Carl Dong
5200929bfe depends: Include GUIX_ENVIRONMENT in id string 2021-02-03 12:10:10 -05:00
Carl Dong
4c7d418588 depends: Improve id string robustness
Environment variables and search paths can drastically effect the
operation of build tools.

Include these in our id string to mitigate against false cache hits.
2021-02-03 12:10:10 -05:00
Carl Dong
b3bdff42b5 build: Proper quoting for var printing targets
Previously, if the value contained syntax that was meaningful to make,
the printing would fail. Quoting properly avoids this.
2021-02-03 12:10:02 -05:00
fanquake
f22a3ec140
build: make macOS HOST in download-osx generic
This was missed in , and the update before that, so just
make this un-versioned so that we don't have to worry about it.
This is fine, because it's just for downloading sources.
2021-02-02 20:28:17 +08:00
fanquake
06e1d7d81d
build: don't build or use Boost Thread 2021-02-02 12:38:22 +08:00
fanquake
5b41d84b34
doc: add xorriso to macOS depends packages
This was missed in .
2021-01-22 16:22:11 +08:00
Hennadii Stepanov
76f52e3da3
build: Fix Qt processing of configure script for depends with DEBUG=1
Now, if depends is built with DEBUG=1, the configure script correctly
finds Qt for macOS and Windows.
2021-01-08 10:48:22 +02:00
Carl Dong
196b727649 depends: Add comment about cache invalidation 2021-01-07 14:24:06 -05:00