Compare commits

..

181 Commits

Author SHA1 Message Date
fanquake
af591f2068 Merge bitcoin/bitcoin#22858: [0.21] Final changes for 0.21.2
6c102a696e release: bump RC to 0 (-final) (fanquake)
f78570ef87 doc: mention bech32m/BIP350 in doc/descriptors.md (Pieter Wuille)

Pull request description:

  Backports:
  * #22837 - doc: mention bech32m/BIP350 in doc/descriptors.md

ACKs for top commit:
  laanwj:
    ACK 6c102a696e
  glozow:
    ACK 6c102a696e looks correct to me
  jnewbery:
    ACK 6c102a696e

Tree-SHA512: 8cdfee955e1abdbc6900df3d01982cb77ec57ef008b75487ac9a63d571cc21e4ae0f107849aa575d239a8662619553778955ffbebf57201de5afcd34298d57b4
2021-09-23 18:07:49 +08:00
fanquake
6c102a696e release: bump RC to 0 (-final) 2021-09-23 15:07:14 +08:00
Pieter Wuille
f78570ef87 doc: mention bech32m/BIP350 in doc/descriptors.md
Github-Pull: #22837
Rebased-From: d2a09c8355
2021-09-23 14:50:29 +08:00
fanquake
97d3500601 Merge bitcoin/bitcoin#22823: [0.21] Update version and release notes for 0.21.2rc2
b8f5fb7c6b build: bump version to 0.21.2rc2 (fanquake)
82c5208ddc doc: update release notes for rc2 (fanquake)

Pull request description:

  Bump version to `0.21.2rc2` and add the new backports to the release notes.

ACKs for top commit:
  MarcoFalke:
    ACK b8f5fb7c6b
  hebasto:
    ACK b8f5fb7c6b

Tree-SHA512: 7bcdf6c42ac75fb24df8d6b60bddcac5f14363a3f7dd89a239f798bb14b5c911c2d7535a0372c2998719d33a561d0d28b0b6764aaf1f2ec330d4035ce965997b
2021-08-28 16:25:43 +08:00
fanquake
b8f5fb7c6b build: bump version to 0.21.2rc2 2021-08-28 16:01:20 +08:00
fanquake
82c5208ddc doc: update release notes for rc2 2021-08-28 15:55:03 +08:00
fanquake
6ebb9d0e46 Merge bitcoin/bitcoin#22808: [0.21] 0.21.2rc2 backports
d9b18c1290 Fix build with Boost 1.77.0 (Rafael Sadowski)
2d7f2606c1 ci: Run fuzzer task for the master branch only (Hennadii Stepanov)

Pull request description:

  Backported:

  1) #22730
  1) #22713

ACKs for top commit:
  fanquake:
    ACK d9b18c1290 - Checked the backports and tested building this branch with system Boost 1.76.0 and depends Boost 1.77.0.

Tree-SHA512: dc3e0a2b3c1e3e80f6570e329a08ebc5103c233c30562f660432891c90bacb4d88d5373e32b9ac34fd143be46b8e63900ce9f52786b04bc799a4d17ba9fd8499
2021-08-28 15:33:34 +08:00
fanquake
55a8a5b2df Merge bitcoin/bitcoin#22810: [0.21] qt: Pre-0.21.2rc2 translations update
2a7568999c qt: Pre-0.21.2rc2 translations update (Hennadii Stepanov)

Pull request description:

  New translations with insignificant amount of translated strings were ignored: `ach`, `cmn`, `gd`, `ig`, `sc`.

ACKs for top commit:
  fanquake:
    ACK 2a7568999c - didn't review any specific translations.

Tree-SHA512: fab299a7a3691b38f85812e9302ae3a6fda2b7bd95a824aaf5537f1bc066417f912c042fa4366432ed091244f051cc7d4c57161d8da1e4bd6261a85570f8f12e
2021-08-28 14:44:02 +08:00
Rafael Sadowski
d9b18c1290 Fix build with Boost 1.77.0
BOOST_FILESYSTEM_C_STR changed to accept the path as an argument

Github-Pull: bitcoin/bitcoin#22713
Rebased-From: acb7aad27e
2021-08-27 09:54:05 +03:00
Hennadii Stepanov
2d7f2606c1 ci: Run fuzzer task for the master branch only
Github-Pull: bitcoin/bitcoin#22730
Rebased-From: 5a9e255e5a
2021-08-27 09:54:04 +03:00
Hennadii Stepanov
2a7568999c qt: Pre-0.21.2rc2 translations update 2021-08-27 00:29:27 +03:00
W. J. van der Laan
3118a5eedf Merge bitcoin/bitcoin#22721: doc: add inital PR and author list to 0.21.2 release notes
2873e01c6a doc: add inital PR and author list to 0.21.2 release notes (fanquake)

Pull request description:

  Any further updates / version number adjustments can be done prior to final.

ACKs for top commit:
  laanwj:
    ACK 2873e01c6a

Tree-SHA512: 57135f64ccfc41ec63aa265eb25978a2afd68b6b8719e79358c52ec7d2dc0cee04b81f720b827263c381529f457667b0b1a8f7d835118dc56bf2a6aa9d321859
2021-08-17 15:42:43 +02:00
fanquake
2873e01c6a doc: add inital PR and author list to 0.21.2 release notes
Any further updates / version number adjustments can be done prior to
final.
2021-08-17 13:57:17 +08:00
W. J. van der Laan
89d148c8c6 qt: Translations update for 0.21.2rc1
Tree-SHA512: f0a74deb84711645ba112b364fbe958db51992f28d0ec87262dd0a21f1052a270f5fc8c13dfd6dc6f5a8a5580e7f1991b3de2d44e3a44252cd3c315dfc2e00ba
2021-08-11 13:23:05 +02:00
W. J. van der Laan
e94e433762 doc: Update manual pages for 0.21.2
Tree-SHA512: 26870ec4b8423879ccdbe0d74670985bf73f4814d32ff5d2c312acdd766e93e07e6a69bfb58b0288be8ba402d283ce493684420b254f8535936cb061ee11c0d2
2021-08-11 13:20:35 +02:00
W. J. van der Laan
bf672ce25a build: Bump version to 0.21.2rc1
Tree-SHA512: 73d4df24ad516ca54ab23bcbd686223447a208904bb18a7a34400daab6d4b07322a7f244504547c772ede08b9e8606291df9d101236d8fd6debcb43563d069ac
2021-08-11 12:08:43 +02:00
MarcoFalke
4b523c8f0a Merge bitcoin/bitcoin#22569: [0.21] Rate limit the processing of rumoured addresses
2a57108051 Avoid Appveyor compilation failure (Pieter Wuille)
a653aacbd6 Add logging and addr rate limiting statistics (Pieter Wuille)
aaa4833fc9 Functional tests for addr rate limiting (Pieter Wuille)
8df3e5bd84 Randomize the order of addr processing (Pieter Wuille)
83dfe6c65e Rate limit the processing of incoming addr messages (Pieter Wuille)

Pull request description:

  Backport of #22387.

  The rate at which IP addresses are rumoured (through ADDR and ADDRV2 messages) on the network seems to vary from 0 for some non-participating nodes, to 0.005-0.025 addr/s for recent Bitcoin Core nodes. However, the current codebase will happily accept and process an effectively unbounded rate from attackers. There are measures to limit the influence attackers can have on the addrman database (bucket restrictions based on source IPs), but still - there is no need to permit them to feed us addresses at a rate that's orders of magnitude larger than what is common on the network today, especially as it will cause us to spam our peers too.

  This PR implements a [token bucket](https://en.wikipedia.org/wiki/Token_bucket) based rate limiter, allowing an average of 0.1 addr/s per connection, with bursts up to 1000 addresses at once. Whitelisted peers as well as responses to GETADDR requests are exempt from the limit. New connections start with 1 token, so as to not interfere with the common practice of peers' self-announcement.

  Due to the lack of the `Peer` struct in 0.21, the relevant fields have been added to `CNodeState` instead, necessitating additional locks, and slightly different structure to avoid too much `cs_main` grabbing. The last test-improving commit has also been dropped, as the code has changed too much. Most of the behavior is still tested however, just not the part that compares with RPC statistics.

ACKs for top commit:
  achow101:
    ACK 2a57108051
  GeneFerneau:
    Approach + code review ACK [2a57108](2a57108051)
  jnewbery:
    reACK 2a57108051

Tree-SHA512: ecf4891ac6173d732aa40b4d05fc0dce94127a613cb9051bf6188a2f95824f8234b17d386dd0b352ddf3d352202cc2ff07915ae35657d8e64907e3f80703d1d9
2021-08-06 19:46:04 +02:00
Pieter Wuille
2a57108051 Avoid Appveyor compilation failure 2021-08-05 09:48:25 -07:00
Pieter Wuille
a653aacbd6 Add logging and addr rate limiting statistics
Includes logging improvements by Vasil Dimov and John Newbery.

Github-Pull: #22387
Rebased-From: f424d601e1
2021-08-05 09:40:04 -07:00
Pieter Wuille
aaa4833fc9 Functional tests for addr rate limiting
Github-Pull: #22387
Rebased-From: b4ece8a1cd
2021-08-05 09:40:04 -07:00
Pieter Wuille
8df3e5bd84 Randomize the order of addr processing
Github-Pull: #22387
Rebased-From: 5648138f59
2021-08-05 09:40:04 -07:00
Pieter Wuille
83dfe6c65e Rate limit the processing of incoming addr messages
While limitations on the influence of attackers on addrman already
exist (affected buckets are restricted to a subset based on incoming
IP / network group), there is no reason to permit them to let them
feed us addresses at more than a multiple of the normal network
rate.

This commit introduces a "token bucket" rate limiter for the
processing of addresses in incoming ADDR and ADDRV2 messages.
Every connection gets an associated token bucket. Processing an
address in an ADDR or ADDRV2 message from non-whitelisted peers
consumes a token from the bucket. If the bucket is empty, the
address is ignored (it is not forwarded or processed). The token
counter increases at a rate of 0.1 tokens per second, and will
accrue up to a maximum of 1000 tokens (the maximum we accept in a
single ADDR or ADDRV2). When a GETADDR is sent to a peer, it
immediately gets 1000 additional tokens, as we actively desire many
addresses from such peers (this may temporarily cause the token
count to exceed 1000).

The rate limit of 0.1 addr/s was chosen based on observation of
honest nodes on the network. Activity in general from most nodes
is either 0, or up to a maximum around 0.025 addr/s for recent
Bitcoin Core nodes. A few (self-identified, through subver) crawler
nodes occasionally exceed 0.1 addr/s.

Github-Pull: #22387
Rebased-From: 0d64b8f709
2021-08-05 09:40:00 -07:00
MarcoFalke
068ac69b56 Merge bitcoin/bitcoin#22578: [0.21] Additional backports
55e941f5df test: Fix intermittent feature_taproot issue (MarcoFalke)
681f728a35 ci: Build with --enable-werror by default, and document exceptions (Hennadii Stepanov)
89426c43fb ci: Fix macOS brew install command (Hennadii Stepanov)

Pull request description:

  This backports a few changes to fix CI failures we are seeing with the 0.21 branch.

  Backports #21663, this might be the easiest way to fix the macOS CI failures we're seeing. i.e in #22569. The underlying issue is that the older CI images are using a version of brew that without running `brew update` first, is trying to download packages like Boost, from bintray (which no-longer works).

  This also includes #20182, as by fixing macOS failure, via running `brew upgrade`, we end up using a newer version of miniupnpc, which emits a GNU extension related warning, and causes the build to fail, because we use `-Werror`.

  Backporting #20535 should fix #22581.

ACKs for top commit:
  hebasto:
    ACK 55e941f5df, I verified changes by backporting locally.

Tree-SHA512: 3ab2c5c73c707d0f5b862264f3a0179cdeee30ae55aae872f3c3e0bb81d71a5027c39ba830210c99a21f98cc86c4167c4f215e24d1a8891ec79ce512debf82df
2021-07-30 09:15:14 +02:00
fanquake
52778f4245 Merge bitcoin/bitcoin#22580: [0.21] Backport: Reset scantxoutset progress before inferring descriptors
080b47d9ce rpc: reset scantxoutset progress on finish (Pavol Rusnak)

Pull request description:

  Backport of #19362.

ACKs for top commit:
  achow101:
    Code Review ACK 080b47d9ce

Tree-SHA512: 1f58965393663bfc0796fc06ef846bc87521d04d79c2a62227acf4d466de649db7d244c8992a32aa1086e6bf0ac92fb8a5aeadf4673441fefa505e40c02d5daf
2021-07-30 10:22:08 +08:00
MarcoFalke
55e941f5df test: Fix intermittent feature_taproot issue
Github-Pull: #20535
Rebased-From: fa275e1539
2021-07-29 20:25:28 +08:00
Hennadii Stepanov
681f728a35 ci: Build with --enable-werror by default, and document exceptions
Github-Pull: #20182
Rebased-From: 2f6fe4e4e9
2021-07-29 20:25:03 +08:00
Hennadii Stepanov
89426c43fb ci: Fix macOS brew install command
Details: https://github.com/Homebrew/discussions/discussions/691

Github-Pull: #21663
Rebased-From: b7381552cd
2021-07-29 16:52:15 +08:00
Pavol Rusnak
080b47d9ce rpc: reset scantxoutset progress on finish
Github-Pull: #19362
Rebased-From: 8c4129b454
2021-07-29 16:02:26 +08:00
fanquake
997e528a34 Merge bitcoin/bitcoin#22427: [0.21] gui: Backports for 0.21.2
e3f1da4bf3 qt: Draw "eye" sign at the beginning of watch-only addresses (Hennadii Stepanov)
6ca54ce2ae qt: Do not extend recent transaction width to address/label string (Hennadii Stepanov)
f220368220 qt: Do not use QClipboard::Selection on Windows and macOS. (Hennadii Stepanov)

Pull request description:

  Backports https://github.com/bitcoin-core/gui/pull/277, https://github.com/bitcoin-core/gui/pull/365.

ACKs for top commit:
  fanquake:
    ACK e3f1da4bf3
  jarolrod:
    ACK e3f1da4bf3

Tree-SHA512: 43cc2ac48f4e5014bfdbe86cc904bb36d2be9fcd257f0fc0800c384bd727bb98466723e450a8909b06708784ad91184be599c49cf60de2e4377202774cb878f6
2021-07-29 11:20:57 +08:00
Hennadii Stepanov
e3f1da4bf3 qt: Draw "eye" sign at the beginning of watch-only addresses
Github-Pull: bitcoin-core/gui#365
Rebased-From: cd46c11577
2021-07-11 12:03:33 +03:00
Hennadii Stepanov
6ca54ce2ae qt: Do not extend recent transaction width to address/label string
Github-Pull: bitcoin-core/gui#365
Rebased-From: 9ea1da6fc9
2021-07-11 12:03:12 +03:00
Hennadii Stepanov
f220368220 qt: Do not use QClipboard::Selection on Windows and macOS.
Windows and macOS do not support the global mouse selection.

Github-Pull: bitcoin-core/gui#277
Rebased-From: 7f3a5980c1
2021-07-11 12:03:11 +03:00
fanquake
bd2f4164c6 Merge bitcoin/bitcoin#22366: [0.21] fuzz: add missing ECCVerifyHandle to base_encode_decode
da816247f0 util: Properly handle -noincludeconf on command line (MarcoFalke)
513613d8a8 Cleanup -includeconf error message (MarcoFalke)
70eac6fcd0 Fix crash when parsing command line with -noincludeconf=0 (MarcoFalke)
c5357fa415 fuzz: add missing ECCVerifyHandle to base_encode_decode (Andrew Poelstra)

Pull request description:

  Backports #22279, #22002 and #22137 to fix fuzzing issues in the 0.21 branch: https://github.com/bitcoin/bitcoin/runs/2864012729.

ACKs for top commit:
  achow101:
    ACK da816247f0

Tree-SHA512: ab8751387e42e03ff43594ae34be8ed0dba903d7da1aaecb9f19c08366570d8995abe89ba0c9bafe37662940f3e83bef1e9e50f330e86114cd6a773becd1fd21
2021-07-08 11:52:07 +08:00
MarcoFalke
4c29b63cfb Merge bitcoin/bitcoin#22255: [0.21] wallet: Do not iterate a directory if having an error while accessing it
7b0b201d10 wallet: Do not iterate a directory if having an error while accessing it (Hennadii Stepanov)

Pull request description:

  This change prevents infinite looping for, for example, system folders
  on Windows.

  Github-Pull: #21907
  Rebased-From: 29c9e2c2d2

  Note: Trivial backport, but in a differently-named function in another file

ACKs for top commit:
  hebasto:
    ACK 7b0b201d10, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: d41ab267250e8bfd9289cacf1fd804cc1a3bb20fc479dc9da5a69ebf26530b552b11b2ee6b11e17a1c146ca792ee65bd64eeb2269fa5e73a70361da8a2a09925
2021-07-01 08:35:53 +02:00
MarcoFalke
da816247f0 util: Properly handle -noincludeconf on command line
This bug was introduced in commit
fad0867d6a.

Unit test
Co-Authored-By: Russell Yanofsky <russ@yanofsky.org>

Github-Pull: #22137
Rebased-From: fa910b4765
2021-06-30 10:39:33 +08:00
MarcoFalke
513613d8a8 Cleanup -includeconf error message
Remove the erroneous trailing newline '\n'. Also, print only the first
value to remove needless redundancy in the error message.

Github-Pull: #22002
Rebased-From: fad0867d6a
2021-06-30 10:23:54 +08:00
MarcoFalke
70eac6fcd0 Fix crash when parsing command line with -noincludeconf=0
Github-Pull: #22002
Rebased-From: fa9f711c37
2021-06-30 10:23:51 +08:00
Andrew Poelstra
c5357fa415 fuzz: add missing ECCVerifyHandle to base_encode_decode
GitHub Pull: #22279
Rebased-From: 906d791311
2021-06-29 11:03:27 +08:00
MarcoFalke
926f76cb20 Merge bitcoin/bitcoin#22191: [0.21] gitian: Use custom MacOS code signing tool
0fe60a84ae Use latest signapple commit (Andrew Chow)
5313d6aed2 gitian: Remove codesign_allocate and pagestuff from MacOS build (Andrew Chow)
27d691b6b5 gitian: use signapple to create the MacOS code signature (Andrew Chow)
2f33e339a8 gitian: use signapple to apply the MacOS code signature (Andrew Chow)
65ce833042 gitian: install signapple in gitian-osx-signer.yml (Andrew Chow)

Pull request description:

  Backport of #20880 and #22190

ACKs for top commit:
  MarcoFalke:
    cherry-pick-only ACK 0fe60a84ae 🍀

Tree-SHA512: e864048fab02a1857161602dd53abba552ca3f859c133a47a5e62c28d3e4de9cd099bce86123a1b5892042b09f51cc1ddd2ed1b0c71bfba162710eaee3f5bf91
2021-06-19 09:50:11 +02:00
Hennadii Stepanov
7b0b201d10 wallet: Do not iterate a directory if having an error while accessing it
This change prevents infinite looping for, for example, system folders
on Windows.

Github-Pull: #21907
Rebased-From: 29c9e2c2d2
2021-06-15 19:02:45 +00:00
Andrew Chow
0fe60a84ae Use latest signapple commit
Update gitian and guix to use the same latest signapple commit

Github-Pull: #22190
Rebased-From: 683d197970
2021-06-08 17:04:36 -04:00
Andrew Chow
5313d6aed2 gitian: Remove codesign_allocate and pagestuff from MacOS build
Github-Pull: #20880
Rebased-From: 2c403279e2
2021-06-08 16:58:40 -04:00
Andrew Chow
27d691b6b5 gitian: use signapple to create the MacOS code signature
Github-Pull: #20880
Rebased-From: f55eed2514
2021-06-08 16:57:32 -04:00
Andrew Chow
2f33e339a8 gitian: use signapple to apply the MacOS code signature
Github-Pull: #20880
Rebased-From: 95b06d2185
2021-06-08 16:57:28 -04:00
Andrew Chow
65ce833042 gitian: install signapple in gitian-osx-signer.yml
Github-Pull: #20880
Rebased-From: 42bb1ea363
2021-06-08 16:56:08 -04:00
fanquake
419f9b3b3b Merge bitcoin/bitcoin#22022: Final backports for 0.21.2rc1
09620b89f5 Update Windows code signing certificate (Andrew Chow)
46320ba72f Remove user input from URI error message (unknown)
f2a88986a1 p2p, bugfix: use NetPermissions::HasFlag() in CConnman::Bind() (Jon Atack)

Pull request description:

ACKs for top commit:
  achow101:
    ACK 09620b89f5 Diffs match.
  hebasto:
    ACK 09620b89f5, tested https://github.com/bitcoin-core/gui/pull/280 behavior.
  fanquake:
    ACK 09620b89f5

Tree-SHA512: 1c4aaec42ea047261b5d30851bca605540eccf572708403335b38016127d3230b5380b3f5ef03921ed62192239b0d3da9787d51f557ed7911bf6bb2a7c172753
2021-05-31 15:31:31 +08:00
Andrew Chow
09620b89f5 Update Windows code signing certificate
Github-Pull: bitcoin/bitcoin#22017
Rebased-From: 167fb1fc72
2021-05-22 10:45:21 +02:00
unknown
46320ba72f Remove user input from URI error message
+ Detailed error messages for invalid address
+ Used `IsValidDestination` instead of `IsValidDestinationString`
+ Referred to https://github.com/bitcoin/bitcoin/pull/20832 for solution

Github-Pull: bitcoin-core/gui#280
Rebased-From: 3bad0b3fad
2021-05-22 10:45:20 +02:00
Jon Atack
f2a88986a1 p2p, bugfix: use NetPermissions::HasFlag() in CConnman::Bind()
PF_NOBAN is a multi-flag that includes PF_DOWNLOAD, so the conditional
in CConnman::Bind() using a bitwise AND will return the same result
for both the "noban" status and the "download" status.

Example:

`PF_DOWNLOAD` is `0b1000000`
`PF_NOBAN`    is `0b1010000`

This makes a check like `flags & PF_NOBAN` return `true` even if `flags`
is equal to `PF_DOWNLOAD`.

If `-whitebind=download@1.1.1.1:8765` is specified, then `1.1.1.1:8765`
should be added to the list of local addresses. We only want to avoid
adding to local addresses (that are advertised) a whitebind that has a
`noban@` flag.

As a result of a mis-check in `CConnman::Bind()` we would not have added
`1.1.1.1:8765` to the local addresses in the example above.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>

Github-Pull: bitcoin/bitcoin#21644
Rebased-From: dde69f20a0
2021-05-22 09:58:26 +02:00
W. J. van der Laan
58c0742632 Merge bitcoin/bitcoin#21932: [0.21] depends: update Qt 5.9 source url
deff4e763e depends: update Qt 5.9 source url (Kittywhiskers Van Gogh)

Pull request description:

  ## Expected Behavior

  Running `make` in the depends directory should result in the successful building of host platform dependencies

  ## Actual Behavior

  `make` terminates as `qt.mk` points to (now) outdated URL path for Qt 5.9 sources, returning error 404

  ## Remedy

  Update `qt.mk` to point to updated source location as Qt has relocated them

ACKs for top commit:
  RandyMcMillan:
    tACK deff4e7
  hebasto:
    ACK deff4e763e, I have verified the new link, and checked that old one is unavailable:
  jarolrod:
    tACK deff4e763e

Tree-SHA512: d5b11a678c871a87e6884a35641178ee349728ead51da06018953be36aab5a30aef9c80b549da49802e87903eb0b7836cc343301325bd8c505cb4246f6752505
2021-05-13 13:43:35 +02:00
Kittywhiskers Van Gogh
deff4e763e depends: update Qt 5.9 source url 2021-05-12 22:35:32 +05:30
W. J. van der Laan
8584a4460d doc: Archive and clean out release notes post-0.21.1
Tree-SHA512: 0ae93cb9dae1449318913a7a6f32c233aefddf77335dc1ad416e84beab39ab45cd863073ddc69a243bbb704951d99646f734717f71d9599a54592699f15d84ab
2021-05-05 16:18:38 +02:00
MarcoFalke
bbd89d23b3 Merge bitcoin/bitcoin#21860: [0.21] Backport update to Boost download URL
856de5bd5e build,boost: update download url. (fdov)

Pull request description:

  Backports #21662 to the 0.21 branch. Boost has migrated it's download URLs due to bintrays imminent closure. The Boost site has also been updated to point to the new URLs. i.e: https://www.boost.org/users/history/version_1_70_0.html.

  Github-Pull: #21662
  Rebased-From: 36c10b9f4b

ACKs for top commit:
  jarolrod:
    ACK 856de5bd5e

Tree-SHA512: cae7262f963dd6e00af699c3e1bce2ee3ff9f76c5dcc24c8a5bf37025b53de28e4cb80de65a380208ee1841a89736d5c35605e40b728b6c7a9f026d8165d74c8
2021-05-05 08:54:13 +02:00
fdov
856de5bd5e build,boost: update download url.
- bintray is closing.
    - updated to jfrog.io.

Github-Pull: #21662
Rebased-From: 36c10b9f4b
2021-05-05 09:41:09 +08:00
W. J. van der Laan
194b9b8792 build: Bump RC to 0 (-final)
Tree-SHA512: b63d8c2514fa34d4503da8a37fb42948e03610e06dae6aaef7ba3d2568efd3bb138e7510da569a49d83c2618fa697ba949d55880fc2884a75a87028259d7c544
2021-04-29 21:31:20 +02:00
W. J. van der Laan
d97d0d31a6 doc: Merge release notes fragment, merge taproot description from wiki
Co-authored-by: David A. Harding <dave@dtrt.org>
Co-authored-by: Jon Atack <jon@atack.com>
Co-authored-by: Pieter Wuille <pieter@wuille.net>

Tree-SHA512: dd9ac416ff22276833111198445d76cf8417012a6faad0c3560276f1dcf24586ff41c65ac3430fbf1e840aaa563d3dd101671cff306b0fd92aa2ee03bb7f926b
2021-04-22 22:13:28 +02:00
W. J. van der Laan
5577e0a486 doc: Add PR and author list to release notes for 0.21.1
Tree-SHA512: c74b12dee645bd8d3dcff8d572a82369ea0895339d0d12e5360182421fa8cef4d3eba309ff7668e97dc24cd6afc800ab9ceb4eca5458085acf12c368f6c2f859
2021-04-19 06:01:38 +02:00
W. J. van der Laan
329eafa7f4 doc: Regenerate manual pages for 0.21.1rc1
Tree-SHA512: cc9cd502dc40d89f34b1b043e96df180f0e16ba7c6e039866b349d19aff582d1c4b4ab8e8960b4f427d72ad5f97c7c1e8fec3f2e008a0107dea33a2c8f13febc
2021-04-16 13:41:41 +02:00
W. J. van der Laan
e85dfdbf31 build: Bump version to 0.21.1rc1 2021-04-16 13:23:58 +02:00
W. J. van der Laan
ab20518191 gui: Pre-rc1 translations update
Tree-SHA512: 6978293bda56b0cb1370f3ddf201477a2f12e0b8c9462d3f4703b837d9be4be65142a5e97c4a473fb9fa90edd83dba8a98ea3b0ecc335062868d1e6c550241b2
2021-04-16 13:21:07 +02:00
fanquake
f8bbee425e Merge #21614: [0.21] test: Backports
b8af67eeef fuzz: cleanups for versionbits fuzzer (Anthony Towns)
79cdb4a198 test: make sure non-IP peers get discouraged and disconnected (Vasil Dimov)
b765f41164 test: also check disconnect in denialofservice_tests/peer_discouragement (Vasil Dimov)
dfeb6c10bb test: use pointers in denialofservice_tests/peer_discouragement (Vasil Dimov)

Pull request description:

  Backport tests

ACKs for top commit:
  vasild:
    ACK b8af67eeef
  jnewbery:
    ACK b8af67eeef
  ajtowns:
    ACK b8af67eeef ; visually compared individual commits to originals, checked original commits are in master

Tree-SHA512: 22f665560f9d452993b12508d93d93ff54e3e91dcf39f731e27aedfb891570168066c185413d455bee4fa082c011b65ea1b0eee51e3633392b07a0db008d51c8
2021-04-16 17:57:18 +08:00
Anthony Towns
b8af67eeef fuzz: cleanups for versionbits fuzzer
Github-Pull: #21489
Rebased-From: aa7f418fe3
2021-04-16 11:21:02 +02:00
Vasil Dimov
79cdb4a198 test: make sure non-IP peers get discouraged and disconnected
Github-Pull: #21571
Rebased-From: 81747b2171
2021-04-16 11:20:04 +02:00
Vasil Dimov
b765f41164 test: also check disconnect in denialofservice_tests/peer_discouragement
Use `CConnmanTest` instead of `CConnman` and add the nodes to it
so that their `fDisconnect` flag is set during disconnection.

Github-Pull: #21571
Rebased-From: 637bb6da36
2021-04-16 11:20:04 +02:00
Vasil Dimov
dfeb6c10bb test: use pointers in denialofservice_tests/peer_discouragement
This is a non-functional change that replaces the `CNode` on-stack
variables with `CNode` pointers.

The reason for this is that it would allow us to add those `CNode`s
to `CConnman::vNodes[]` which in turn would allow us to check that they
are disconnected properly - a `CNode` object must be in
`CConnman::vNodes[]` in order for its `fDisconnect` flag to be set.

If we store pointers to the on-stack variables in `CConnman` then it
would crash at the end, trying to `delete` them.

Github-Pull: #21571
Rebased-From: 4d6e246fa4
2021-04-16 11:20:04 +02:00
MarcoFalke
e3b76b6c13 Merge #21701: [0.21] Speedy trial activation for Taproot
cbd64c3a28 Add mainnet and testnet taproot activation params (Andrew Chow)
ec7824396b chainparams: drop versionbits threshold to 90% for mainnnet and signet (Anthony Towns)
600357306e versionbits: simplify state transitions (Anthony Towns)
3acf0379e0 versionbits: Add explicit NEVER_ACTIVE deployments (Anthony Towns)
b529222ad1 fuzz: test versionbits delayed activation (Anthony Towns)
71917e01eb tests: test versionbits delayed activation (Anthony Towns)
4cab84cfdf versionbits: Add support for delayed activation (Anthony Towns)
f9517e6014 tests: clean up versionbits test (Anthony Towns)
1c0164544c tests: test ComputeBlockVersion for all deployments (Anthony Towns)
2e9e7f4329 tests: pull ComputeBlockVersion test into its own function (Anthony Towns)

Pull request description:

  Backport of #21377 and #21686

ACKs for top commit:
  instagibbs:
    cherry-pick ACK cbd64c3a28
  jnewbery:
    ACK cbd64c3a28
  Sjors:
    tACK cbd64c3
  MarcoFalke:
    cherry-pick-only ACK cbd64c3a28 🌾

Tree-SHA512: e9efb0ca9986d685161bcba5ed43efdc5f1dca88322cf65faccf17009b567c2d930c2aba4d1541539fc65347574ed4faa3d4558b907c779d1c128b3d2c681f31
2021-04-16 11:12:35 +02:00
fanquake
0fe5b6130c Merge #21520: [0.21] wallet: Avoid requesting fee rates multiple times during coin selection
d61fb07da7 Rename CoinSelectionParams::effective_fee to m_effective_feerate (Andrew Chow)
5fc381e443 wallet: Move discard feerate fetching to CreateTransaction (Andrew Chow)
bcd716670b wallet: Move long term feerate setting to CreateTransaction (Andrew Chow)
34c89f92f3 wallet: Replace nFeeRateNeeded with effective_fee (Andrew Chow)
48fc675163 wallet: Use existing feerate instead of getting a new one (Andrew Chow)

Pull request description:

  Backport of #21083

ACKs for top commit:
  MarcoFalke:
    cherry-pick-only re-ACK d61fb07da7 🔙
  instagibbs:
    utACK d61fb07da7

Tree-SHA512: 23b212301bb467153dd9723903918ae01dd520525c81d541c411e7a4381e46594fe032e2a7c06ddcff7dc56dcb546991d50187c33fcff08ec45bd835cc01bd19
2021-04-16 10:10:50 +08:00
Andrew Chow
cbd64c3a28 Add mainnet and testnet taproot activation params
Github-Pull: #21686
Rebased-From: f979b3237f
2021-04-15 20:59:21 -04:00
Anthony Towns
ec7824396b chainparams: drop versionbits threshold to 90% for mainnnet and signet
Github-Pull: #21377
Rebased-From: ffe33dfbd4
2021-04-15 20:59:19 -04:00
Anthony Towns
600357306e versionbits: simplify state transitions
This removes the DEFINED->FAILED transition and changes the
STARTED->FAILED transition to only occur if signalling didn't pass the
threshold. This ensures that it is always possible for activation to
occur, no matter what settings are chosen, or the speed at which blocks
are found.

Github-Pull: #21377
Rebased-From: f054f6bcd2
2021-04-15 20:59:17 -04:00
Anthony Towns
3acf0379e0 versionbits: Add explicit NEVER_ACTIVE deployments
Previously we used deployments that would timeout prior to Bitcoin's
invention, which allowed the deployment to still be activated in unit
tests. This switches those deployments to be truly never active.

Github-Pull: #21377
Rebased-From: 55ac5f568a
2021-04-15 20:59:13 -04:00
Anthony Towns
b529222ad1 fuzz: test versionbits delayed activation
Github-Pull: #21377
Rebased-From: dd07e6da48
2021-04-15 20:58:49 -04:00
Anthony Towns
71917e01eb tests: test versionbits delayed activation
Github-Pull: #21377
Rebased-From: dd85d5411c
2021-04-15 20:58:42 -04:00
Anthony Towns
4cab84cfdf versionbits: Add support for delayed activation
Github-Pull: #21377
Rebased-From: 73d4a70639
2021-04-15 12:03:27 -04:00
Anthony Towns
f9517e6014 tests: clean up versionbits test
Simplify the versionbits unit test slightly to make the next set of
changes a little easier to follow.

Github-Pull: #21377
Rebased-From: 9e6b65f6fa
2021-04-15 12:03:25 -04:00
Anthony Towns
1c0164544c tests: test ComputeBlockVersion for all deployments
This generalises the ComputeBlockVersion test so that it can apply to
any activation parameters we might set, and checks all the parameters
set for each deployment on each chain, to simultaneously ensure that the
deployments we have configured work sensibly, and that the test code
does not suffer bitrot in the event that all interesting deployments
are buried.

Github-Pull: #21377
Rebased-From: 5932744450
2021-04-15 12:03:24 -04:00
Anthony Towns
2e9e7f4329 tests: pull ComputeBlockVersion test into its own function
The intent here is to allow checking ComputeBlockVersion behaviour with
each deployment, rather than only testdummy on mainnet. This commit does
the trivial refactoring component of that change.

Github-Pull: #21377
Rebased-From: 63879f0a47
2021-04-15 12:03:22 -04:00
fanquake
e358b43f7d Merge #21640: [0.21] Introduce DeferredSignatureChecker and have SignatureExtractorClass subclass it
f79189ca54 Test that signrawtx works when a signed CSV and CLTV inputs are present (Andrew Chow)
7de019bc61 Introduce DeferringSignatureChecker and inherit with SignatureExtractor (Andrew Chow)

Pull request description:

  Backport of #21166

ACKs for top commit:
  MarcoFalke:
    checked this is a clean cherry-pick did not review ACK f79189ca54 🐖
  instagibbs:
    ACK f79189ca54

Tree-SHA512: 51e945c9b353713423d3886c557066c66a6517d2300523832e5a5471ab91a8943385096d9bf5b46910477cb4c47470431690cf3da09b9f6956fe030f13ddff51
2021-04-12 09:28:18 +08:00
Andrew Chow
f79189ca54 Test that signrawtx works when a signed CSV and CLTV inputs are present
Github-Pull: #21166
Rebased-From: a97a9298ce
2021-04-08 18:19:02 -04:00
Andrew Chow
7de019bc61 Introduce DeferringSignatureChecker and inherit with SignatureExtractor
Introduces a DeferringSignatureChecker which simply takes a
BaseSignatureChecker and passes through everything.
SignatureExtractorChecker now subclasses DeferringSignatureChecker. This
allows for all BaseSignatureChecker functions to be implemented for
SignatureExtractorChecker, while allowing for future signature checkers
which opreate similarly to SignatureExtractorChecker.

Github-Pull: #21166
Rebased-From: 6965456c10
2021-04-08 18:18:55 -04:00
MarcoFalke
0b9a046c1a Merge #21616: [0.21] build: link against -lsocket if required for *ifaddrs
1a9a2cb7dc net: add ifaddrs.h include (fanquake)
f6896dfde7 build: check if -lsocket is required with *ifaddrs (fanquake)
e99d6d0c7c rand: only try and use freeifaddrs if available (fanquake)

Pull request description:

  Backports #21486 to the 0.21 branch. Related to #21485.

ACKs for top commit:
  MarcoFalke:
    cherry-pick-only ACK 1a9a2cb7dc did not test or review 🚶

Tree-SHA512: 400dcf0f0bffa9b2c820403936b894969113a2bd82a32ae29d3fb73c4ef9ffafb55f5325fcc4929f6e685f8932be6f566f5dd2163ecf8a64b154b0c401109311
2021-04-07 07:32:44 +02:00
fanquake
1a9a2cb7dc net: add ifaddrs.h include
Github-Pull: #21486
Rebased-From: 4783115fd4
2021-04-06 17:36:11 +08:00
fanquake
f6896dfde7 build: check if -lsocket is required with *ifaddrs
Github-Pull: #21486
Rebased-From: 879215e665
2021-04-06 17:35:41 +08:00
fanquake
e99d6d0c7c rand: only try and use freeifaddrs if available
Github-Pull: #21486
Rebased-From: 87deac66aa
2021-04-06 17:34:57 +08:00
Andrew Chow
d61fb07da7 Rename CoinSelectionParams::effective_fee to m_effective_feerate
It's a feerate, not a fee. Also follow the style guide for member names.

Github-Pull: #21083
Rebased-From: f9cd2bfbcc
2021-04-01 12:44:00 -04:00
MarcoFalke
65fa43bda1 Merge #21469: BIP 350: Implement Bech32m and use it for v1+ segwit addresses (0.21 backport)
f2195d7c4a Backport invalid address tests (Pieter Wuille)
1e9671116f naming nits (Fabian Jahr)
7dfe406e20 Add signet support to gen_key_io_test_vectors.py (Pieter Wuille)
593e206627 Use Bech32m encoding for v1+ segwit addresses (Pieter Wuille)
8944aaa6d6 Add Bech32m test vectors (Pieter Wuille)
1485533092 Implement Bech32m encoding/decoding (Pieter Wuille)

Pull request description:

  Backport of #20861. Also includes #21471.

ACKs for top commit:
  jnewbery:
    utACK f2195d7c4a
  MarcoFalke:
    cherry-pick re-ACK f2195d7c4a , only change is version number in doc/bips and new test commit 🍝
  fanquake:
    ACK f2195d7c4a - performed the backport, changes look sane. Have not tested extensively.

Tree-SHA512: 7dc043e44d7cda07d73331a7b49666b9db98c99f2635dab0cfeb45422dbfbe75a7b44d0aff85ef6369d412d8a5041ed0826c86ffdfc13c5fbff74adfe4d91c1a
2021-04-01 12:34:08 +02:00
Pieter Wuille
f2195d7c4a Backport invalid address tests
Reduced version of the test from master/#20861 by John Newbery.

Github-Pull: #20861
Rebased-From: fe5e495c31
2021-03-27 12:09:48 -07:00
Fabian Jahr
1e9671116f naming nits
Github-Pull: #20861
Rebased-From: 03346022d6
2021-03-27 12:09:48 -07:00
Pieter Wuille
7dfe406e20 Add signet support to gen_key_io_test_vectors.py
Github-Pull: #20861
Rebased-From: 2e7c80fb5b
2021-03-27 12:09:48 -07:00
Pieter Wuille
593e206627 Use Bech32m encoding for v1+ segwit addresses
This also includes updates to the Python test framework implementation,
test vectors, and release notes.

Github-Pull: #20861
Rebased-From: fe5e495c31
2021-03-27 12:09:44 -07:00
Pieter Wuille
8944aaa6d6 Add Bech32m test vectors
Github-Pull: #20861
Rebased-From: 25b1c6e13d
2021-03-27 12:09:13 -07:00
Pieter Wuille
1485533092 Implement Bech32m encoding/decoding
Github-Pull: #20861
Rebased-From: da2bb6976d
2021-03-27 12:09:13 -07:00
Andrew Chow
5fc381e443 wallet: Move discard feerate fetching to CreateTransaction
Instead of fetching the discard feerate for each SelectCoinsMinConf
iteration, fetch and cache it once during CreateTransaction so that it
is shared for each SelectCoinsMinConf through
coin_selection_params.m_discard_feerate.

Does not change behavior.

Github-Pull: #21083
Rebased-From: bdd0c2934b
2021-03-24 00:34:54 -04:00
Andrew Chow
bcd716670b wallet: Move long term feerate setting to CreateTransaction
Instead of setting the long term feerate for each SelectCoinsMinConf
iteration, set it once during CreateTransaction and let it be shared
with each SelectCoinsMinConf through
coin_selection_params.m_long_term_feerate.

Does not change behavior.

Github-Pull: #21083
Rebased-From: 448d04b931
2021-03-24 00:23:02 -04:00
Andrew Chow
34c89f92f3 wallet: Replace nFeeRateNeeded with effective_fee
Make sure that all fee calculations use the same feerate.
coin_selection_params.effective_fee is the variable we use for all fee
calculations, so get rid of remaining nFeeRateNeeded usages and just
directly set coin_selection_params.effective_fee.

Does not change behavior.

Github-Pull: #21083
Rebased-From: e2f429e6bb
2021-03-24 00:19:34 -04:00
Andrew Chow
48fc675163 wallet: Use existing feerate instead of getting a new one
During each loop of CreateTransaction, instead of constantly getting a
new feerate, use the feerate that we have already fetched for all
fee calculations. Thix fixes a race condition where the feerate required
changes during each iteration of the loop.

This commit changes behavior as the "Fee estimation failed" error will
now take priority over "Signing transaction failed".

Github-Pull: #21083
Rebased-From: 1a6a0b0dfb
2021-03-24 00:19:27 -04:00
fanquake
a30fd40735 Merge #20901: [0.21.1]: rc1 Backports
5a2d98c640 doc: Remove outdated comment (Hennadii Stepanov)
8426e3a8a1 fuzz: Bump FuzzedDataProvider.h (MarcoFalke)
14e3f2a1c9 fuzz: Bump FuzzedDataProvider.h (MarcoFalke)
a48c9d3161 fuzz: Update FuzzedDataProvider.h from upstream (LLVM) (practicalswift)
6746cd078b doc: add signet to doc/bitcoin-conf.md (Jon Atack)
58975d5c0a doc: add signet to share/examples/bitcoin.conf (Jon Atack)
b35711efde Update vcpkg checkout commit. (Aaron Clauson)
3a12672419 GUI: Write PSBTs to file with binary mode (Andrew Chow)
36ecf5eb87 tests: Test that a fully signed tx given to signrawtx is unchanged (Andrew Chow)
4ef1e4bd40 test: disallow sendtoaddress/sendmany when private keys disabled (Jon Atack)
d6b5eb5fcc Disallow sendtoaddress and sendmany when private keys disabled (Andrew Chow)
08dada8456 util: Disallow negative mocktime (MarcoFalke)
95218ee95c net: Avoid UBSan warning in ProcessMessage(...) (practicalswift)
4607019798 fix the unreachable code at feature_taproot (Bruno Garcia)
6dc58e9945 qt: Use "fusion" style on macOS Big Sur with old Qt (Hennadii Stepanov)
e2ebc8567a raise helpMessageDialog (randymcmillan)
a98f211940 Fix MSVC build after gui#176 (Hennadii Stepanov)
bdc64c9030 qt: Stop the effect of hidden widgets on the size of QStackedWidget (Hennadii Stepanov)
7bc4498234 qt: Fix TxViewDelegate layout (Hennadii Stepanov)
b7086e69ff qt: Add TransactionOverviewWidget class (Hennadii Stepanov)
0dba346a56 qt: Use layout manager for Create Wallet dialog (Hennadii Stepanov)
7bf3ed495b Bugfix: GUI: Restore SendConfirmationDialog button default to "Yes" (Luke Dashjr)
bdce029191 test: add test for banning of non-IP addresses (Vasil Dimov)
c33fbab25c net: allow CSubNet of non-IP networks (Vasil Dimov)

Pull request description:

  Current backports for *0.21.1*.

  One conflict was in the test case.

ACKs for top commit:
  ajtowns:
    ACK 5a2d98c640 -- checked 'rebased-from' patches are in master, and rebased patches are clean rebases (except for the first one which changes `""s` to `std::string("")` to avoid c++17 dependency). commits seem fine, but haven't reviewed in detail.
  fanquake:
    ACK 5a2d98c640 - branched off `0.21` and redid the backports. Minor conflict in c33fbab25c. The diff between my branch and #20901 was just in release notes, `_CLIENT_VERSION_RC` (#20901 branched before 95ea54ba08) and #21490 which has already been merged into `0.21`.

Tree-SHA512: 75d16d3cf9066a45759758b8185dc3b9dad6a6102c2ac9921f758a310e48d5d3122f0dafa515df42475235fc66a42cc04dd156ee1e61c86a1238bd11707642ea
2021-03-24 11:54:06 +08:00
MarcoFalke
1bad33f952 Merge #21490: [0.21] Backport versionbits tests
e775b0a6dd tests: Add fuzzing harness for versionbits (Anthony Towns)
0c471a5f30 tests: check never active versionbits (Anthony Towns)
3ba9283a47 tests: more helpful errors for failing versionbits tests (Anthony Towns)

Pull request description:

  Backport of unit test (#21334) and fuzz test (#21380) changes for versionbits.

Top commit has no ACKs.

Tree-SHA512: b68b570e48e0076bb2ade3b91c59612029235d2c9e39048d548aa141fa0906343fa492e9a981065fbdbbebecbbb3dcbaf39ec69228c7581178fcca567e8201b8
2021-03-21 09:55:21 +01:00
Hennadii Stepanov
5a2d98c640 doc: Remove outdated comment
The removed commit is wrong since v0.21.0.

Github-Pull: #21342
Rebased-From: f1f63ac3f8
2021-03-21 09:39:27 +01:00
MarcoFalke
8426e3a8a1 fuzz: Bump FuzzedDataProvider.h
Latest version from 0cccccf0d2/compiler-rt/include/fuzzer/FuzzedDataProvider.h

Github-Pull: #21397
Rebased-From: fa7dc7ae95
2021-03-21 09:35:15 +01:00
MarcoFalke
14e3f2a1c9 fuzz: Bump FuzzedDataProvider.h
Latest version from https://raw.githubusercontent.com/llvm/llvm-project/70de7e0d9a95b7fcd7c105b06bd90fdf4e01f563/compiler-rt/include/fuzzer/FuzzedDataProvider.h

Github-Pull: #20812
Rebased-From: fafce49336
2021-03-21 09:34:29 +01:00
practicalswift
a48c9d3161 fuzz: Update FuzzedDataProvider.h from upstream (LLVM)
Upstream revision: 6d0488f75b/compiler-rt/include/fuzzer/FuzzedDataProvider.h

Changes:
* [compiler-rt] FuzzedDataProvider: add ConsumeData and method.
* [compiler-rt] Fix a typo in a comment in FuzzedDataProvider.h.
* [compiler-rt] Add ConsumeRandomLengthString() version without arguments.
* [compiler-rt] Refactor FuzzedDataProvider for better readability.
* [compiler-rt] FuzzedDataProvider: make linter happy.
* [compiler-rt] Mark FDP non-template methods inline to avoid ODR violations.

Github-Pull: #20740
Rebased-From: e3d2ba7c70
2021-03-21 09:33:42 +01:00
Jon Atack
6746cd078b doc: add signet to doc/bitcoin-conf.md
Github-Pull: #21384
Rebased-From: 4a285107c1
2021-03-21 09:32:33 +01:00
Jon Atack
58975d5c0a doc: add signet to share/examples/bitcoin.conf
Github-Pull: #21384
Rebased-From: 21b6a23373
2021-03-21 09:32:31 +01:00
Aaron Clauson
b35711efde Update vcpkg checkout commit.
Previously vcpkg was relying on https://repo.msys2.org/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz which is no longer available. The vcpkg source has been updated to use http://repo.msys2.org/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-2-any.pkg.tar.zst. This PR updates the commit ID used to checkout vcpkg for the updated URL.

Github-Pull: #21446
Rebased-From: b9e3f35306
2021-03-21 08:40:51 +01:00
Anthony Towns
e775b0a6dd tests: Add fuzzing harness for versionbits
Github-Pull: #21380
Rebased-From: 1639c3b76c
2021-03-21 14:24:41 +10:00
Andrew Chow
3a12672419 GUI: Write PSBTs to file with binary mode
Github-Pull: #bitcoin-core/gui#188
Rebased-From: cc3971c9ff
2021-03-11 21:15:09 +01:00
Andrew Chow
36ecf5eb87 tests: Test that a fully signed tx given to signrawtx is unchanged
Tests that a fully signed transaction given to
signrawtransactionwithwallet is both unchanged and marked as complete.
This tests for a regression in 0.20 where the transaction would not be
marked as complete.

Github-Pull: #20562
Rebased-From: 773c42b265
2021-03-06 08:18:16 +01:00
Jon Atack
4ef1e4bd40 test: disallow sendtoaddress/sendmany when private keys disabled
Github-Pull: #21201
Rebased-From: 6bfbc97d71
2021-02-19 12:47:43 +01:00
Andrew Chow
d6b5eb5fcc Disallow sendtoaddress and sendmany when private keys disabled
Github-Pull: #21201
Rebased-From: 0997019e76
2021-02-19 12:47:40 +01:00
MarcoFalke
08dada8456 util: Disallow negative mocktime
Signed-off-by: practicalswift <practicalswift@users.noreply.github.com>

Github-Pull: #21043
Rebased-From: 3ddbf22ed1
2021-02-11 12:42:40 +01:00
practicalswift
95218ee95c net: Avoid UBSan warning in ProcessMessage(...)
Github-Pull: #21043
Rebased-From: f5f2f97168
2021-02-11 12:42:31 +01:00
Bruno Garcia
4607019798 fix the unreachable code at feature_taproot
Github-Pull: #21081
Rebased-From: 5e0cd25e29
2021-02-08 08:51:02 +01:00
Hennadii Stepanov
6dc58e9945 qt: Use "fusion" style on macOS Big Sur with old Qt
The "macintosh" style is broken on macOS Big Sur at least for Qt 5.9.8.

Github-Pull: #bitcoin-core/gui#177
Rebased-From: 4e1154dfd1
2021-01-28 10:15:59 +01:00
randymcmillan
e2ebc8567a raise helpMessageDialog
Github-Pull: bitcoin-core/gui#167
Rebased-From: 77114462f2
2021-01-26 09:51:56 +01:00
Hennadii Stepanov
a98f211940 Fix MSVC build after gui#176
Github-Pull: #20983
Rebased-From: c5354e4641
2021-01-22 07:41:18 +01:00
Hennadii Stepanov
bdc64c9030 qt: Stop the effect of hidden widgets on the size of QStackedWidget
Layouts of the hidden widgets, those are children of QStackedWidget,
could prevent to adjust the size of the parent widget in the
WalletFrame widget.

Github-Pull: bitcoin-core/gui#176
Rebased-From: af58f5b12c
2021-01-21 18:58:49 +01:00
Hennadii Stepanov
7bc4498234 qt: Fix TxViewDelegate layout
This change (1) prevents overlapping date and amount strings,
and (2) guaranties that "eye" sign at the end of the watch-only
address/label is always visible.

Github-Pull: bitcoin-core/gui#176
Rebased-From: f0d04795e2
2021-01-21 18:58:45 +01:00
Hennadii Stepanov
b7086e69ff qt: Add TransactionOverviewWidget class
Github-Pull: bitcoin-core/gui#176
Rebased-From: d439921406
2021-01-21 18:58:42 +01:00
Hennadii Stepanov
0dba346a56 qt: Use layout manager for Create Wallet dialog
Github-Pull: bitcoin-core/gui#171
Rebased-From: d4feb6812a
2021-01-21 18:57:10 +01:00
Wladimir J. van der Laan
7d8a10a6f4 Merge #20933: [0.21] doc: Archive release notes, Add template for minor release
b6d3502993 doc: Archive release notes, Add template for minor release (MarcoFalke)

Pull request description:

  Same as #20931, but also clearing the template

ACKs for top commit:
  laanwj:
    ACK b6d3502993

Tree-SHA512: 929d2f774904b58d62dd60d4dcf8447f7e40d959596f3f97462076627d89549ad39dc39c56456f4c8861353cfdf6633abca0be3e84a783f174be1ee1b8d213a4
2021-01-14 11:30:54 +01:00
MarcoFalke
b6d3502993 doc: Archive release notes, Add template for minor release 2021-01-14 10:51:15 +01:00
Wladimir J. van der Laan
95ea54ba08 build: Bump RC to 0 (-final)
Tree-SHA512: 1a626b9bb7fb04ece4acb24ffcfed178cc5128d16af6860ed37097b6ed36be8b33797bd70a6da0364cd6a9246ebc02fbb83f9523abf5f8dc3c7a524a58416a8f
2021-01-13 22:52:34 +01:00
Wladimir J. van der Laan
91f0632464 Merge #20929: doc: Move 0.21.0 release notes from wiki
66e6742a27 doc: Move 0.21.0 release notes from wiki (Wladimir J. van der Laan)

Pull request description:

  These were the contents of https://github.com/bitcoin-core/bitcoin-devwiki/wiki/0.21.0-Release-Notes-Draft, minus the paragraph about gauging how fast taproot can be deployed.

Top commit has no ACKs.

Tree-SHA512: d7bb61a76d87cb9d9e43602bf81b51ea463e3a098748e42440950d08e6f6e84bf6c02b07b297ac959f979aa72682775dd2ae2be6d30b22977ed60ed454b9c66d
2021-01-13 22:51:42 +01:00
Wladimir J. van der Laan
66e6742a27 doc: Move 0.21.0 release notes from wiki 2021-01-13 19:43:29 +01:00
Luke Dashjr
7bf3ed495b Bugfix: GUI: Restore SendConfirmationDialog button default to "Yes"
The SendConfirmationDialog is used for bumping the fee, where "Send" doesn't really make sense

Github-Pull: #bitcoin-core/gui#148
Rebased-From: 8775691383
2021-01-13 17:56:34 +01:00
Vasil Dimov
bdce029191 test: add test for banning of non-IP addresses
Co-authored-by: Jon Atack <jon@atack.com>

Github-Pull: #20852
Rebased-From: 39b43298d9
2021-01-11 11:57:21 +01:00
Vasil Dimov
c33fbab25c net: allow CSubNet of non-IP networks
Allow creation of valid `CSubNet` objects of non-IP networks and only
match the single address they were created from (like /32 for IPv4 or
/128 for IPv6).

This fixes a deficiency in `CConnman::DisconnectNode(const CNetAddr& addr)`
and in `BanMan` which assume that creating a subnet from any address
using the `CSubNet(CNetAddr)` constructor would later match that address
only. Before this change a non-IP subnet would be invalid and would not
match any address.

Github-Pull: #20852
Rebased-From: 94d335da7f
2021-01-11 11:55:17 +01:00
Wladimir J. van der Laan
15877d160c qt: Pre-rc5 translations update
Tree-SHA512: f6986d00a9565ee7fbf8dd2766dca9c90af931e2dc2fdc0faf5734a2e9be1a0e0774031a20207aa2afa645f1d6229aac9751c84ada324a8dcb463497fb9f237f
2021-01-05 16:46:14 +01:00
Wladimir J. van der Laan
4e7b4ce7eb build: Bump RC to rc5
Tree-SHA512: ed01a7a66e5d064c77c948b87afce832b33852005ec1dc556bf4fcc69fa3743aaa6dea4c03a5d2d520682d8de9c7596f1eec3db326caf754726d1ca4aab2cec3
2021-01-05 15:25:43 +01:00
Wladimir J. van der Laan
6e28714da3 Merge #20850: [0.21] final rc5 backports
3308718a6b Revert "Add patch to make codesign_allocate compatible with Apple's" (Pieter Wuille)

Pull request description:

ACKs for top commit:
  jonasschnelli:
    ACK 3308718a6b
  fanquake:
    ACK 3308718a6b

Tree-SHA512: 2a1002499e6f0da6e9ac7ae393e478bd34e55390ff1a41c5b16b4e91e339fef4b82642237af6725507337cadf9ddc1f2f23b2512d8dd47e5a345770c6c0d1b02
2021-01-05 15:23:48 +01:00
Pieter Wuille
3308718a6b Revert "Add patch to make codesign_allocate compatible with Apple's"
This reverts commit a4118c6e20.

Github-Pull: #20855
Rebased-From: a0eb4c551e
2021-01-05 09:47:47 +01:00
Wladimir J. van der Laan
e6ad8a6220 doc: Generate manual pages for 0.21.0rc4
Tree-SHA512: c2ff50dbb54c42db9d56f1fcb0d8a2ce9c89089254f1a6b6f2aa98959f47699cd72c7771b53490232938b45d9bb030ce7fe08707c62190e68ada0ddbd8b6e13a
2021-01-03 16:16:56 +01:00
Wladimir J. van der Laan
212525337e build: Bump RC to rc4
Tree-SHA512: f22375573f1737d64b62351cdec58f0f3e17d9f21b044430f366cbf272514369b73f24cc34e499e2637f0a0fa57ce5b925dd2df8ba0265b1b9c37beb7c8b8e22
2021-01-03 13:43:03 +01:00
Wladimir J. van der Laan
93ce429f0e qt: Pre-rc4 translations update
Tree-SHA512: 14259d2f469ea588f364f64fdd89b90cbcde8f66ddc011d96d656a06920584c0577e64fa15a41a1dc1c8c8a98a1f955748c61def6807b1bb71b0113bb28268f2
2021-01-03 13:12:39 +01:00
Wladimir J. van der Laan
ac125e960f Merge #20669: [0.21] final rc4 backports
b1c0f97483 [doc] Add permissions to the getpeerinfo help. (Amiti Uttarwar)
1fda7db64f rpc: Add missing description of vout in getrawtransaction help text (Ben Carman)
ef7a155cf0 qt: Align layout of checkboxes (Hennadii Stepanov)
35a10e4ebc Add patch to make codesign_allocate compatible with Apple's (Pieter Wuille)
e70ccb0bc4 doc: update -externalip documentation in tor.md (Jon Atack)
2c8482d0a2 doc: add tor.md section on how to get tor info via bitcoind (Jon Atack)
0c1fa78af1 doc: update tor.md address examples from onion v2 to v3 (Jon Atack)
84e8d5467f doc: warn that incoming conns are unlikely when not using default ports (Adam Jonas)
e4440eb67b doc: Add warnings for http interfaces limitations (Fabian Jahr)
85dabd1249 Removed redundant git pull from appveyor config. (Aaron Clauson)
249d61a382 Adjusted msvc compiler and linker settings to remove optimisations that are causing sporadic ABI issues on Visual Studio updates. (Aaron Clauson)
e7b53d4721 This change to the appveyor CI config for msvc builds reverses a change introduced in #19960. It re-applies a setting to inform vcpkg to only build release vesions of the dependencies rather than the default of debug and release. (Aaron Clauson)
8273ea3b8d Move signet onion seed from v2 to v3 (Sjors Provoost)

Pull request description:

  The remaining backports to get rc4 out. Currently only waiting on the macOS build fix.

ACKs for top commit:
  benthecarman:
    ACK b1c0f97483
  Sjors:
    ACK b1c0f97

Tree-SHA512: 53eaecd531ba461678917cb630d67f1e6bb737d64022abe971eaced6eca366c9ed593e44276bd9c7ad7b3aebe3850d2d29282eb310e10b547986d10fe77a8482
2021-01-02 15:01:09 +01:00
Amiti Uttarwar
b1c0f97483 [doc] Add permissions to the getpeerinfo help.
This field was already being returned, but the RPCHelpMan did not indicate
this. So, this PR updates the help text to match.

Github-Pull: #20756
Rebased-From: 667d203687
2020-12-26 21:28:15 +01:00
Ben Carman
1fda7db64f rpc: Add missing description of vout in getrawtransaction help text
Github-Pull: #20731
Rebased-From: b23349b880
2020-12-21 20:01:02 +01:00
Hennadii Stepanov
ef7a155cf0 qt: Align layout of checkboxes
Github-Pull: bitcoin-core/gui#155
Rebased-From: e71b656f31
2020-12-18 08:56:36 +01:00
Pieter Wuille
35a10e4ebc Add patch to make codesign_allocate compatible with Apple's
Github-Pull: #20644
Rebased-From: a4118c6e20
2020-12-17 21:38:57 +01:00
Jon Atack
e70ccb0bc4 doc: update -externalip documentation in tor.md
Github-Pull: #19961
Rebased-From: a34eceb4cc
2020-12-17 12:29:23 +01:00
Jon Atack
2c8482d0a2 doc: add tor.md section on how to get tor info via bitcoind
Github-Pull: #19961
Rebased-From: dc8a591222
2020-12-17 12:29:22 +01:00
Jon Atack
0c1fa78af1 doc: update tor.md address examples from onion v2 to v3
Github-Pull: #19961
Rebased-From: e1765d8b04
2020-12-17 12:29:15 +01:00
Adam Jonas
84e8d5467f doc: warn that incoming conns are unlikely when not using default ports
Github-Pull: #20668
Rebased-From: 010eed3ce0
2020-12-17 12:28:37 +01:00
Fabian Jahr
e4440eb67b doc: Add warnings for http interfaces limitations
Github-Pull: #19050
Rebased-From: 5c3eaf9983
2020-12-17 10:02:47 +01:00
Aaron Clauson
85dabd1249 Removed redundant git pull from appveyor config.
Github-Pull: #20506
Rebased-From: 2c69381f3d
2020-12-16 13:27:31 +01:00
Aaron Clauson
249d61a382 Adjusted msvc compiler and linker settings to remove optimisations that are causing sporadic ABI issues on Visual Studio updates.
Tidied up debug and release configuration blocks in common project file to avoid duplication.

Updated appveyor config to use latest Visual Studio 2019 image.

Changed appveyor config file hash to use a new version of Qt pre-compiled binaries built for Visual Studio 2019 v16.8.1.

Bumped vcpkg version to tag '2020.11-1' for binary caching feature.

See #20392 for related discussion.

Github-Pull: #20506
Rebased-From: 8b99e609e7
2020-12-16 13:27:29 +01:00
Aaron Clauson
e7b53d4721 This change to the appveyor CI config for msvc builds reverses a change introduced in #19960. It re-applies a setting to inform vcpkg to only build release vesions of the dependencies rather than the default of debug and release.
It had been expected that the vcpkg manifest mechanism introduced in #19960 would do this automatically but it turns out not to be the case.

Github-Pull: #20489
Rebased-From: fa18e7cbc5
2020-12-16 13:27:11 +01:00
Sjors Provoost
8273ea3b8d Move signet onion seed from v2 to v3
Github-Pull: #20660
Rebased-From: 3e6657a14d
2020-12-16 07:36:44 +01:00
Wladimir J. van der Laan
9ae536d368 build: Bump RC to rc3
Tree-SHA512: a6d81259e17c6b55b33c638a9054eaee2f82322ce0482a73f85562cd56efeb73e2b19b3f7b44fd97e7caa6e757573a8c13ae8967b1e6492f2483d2e057d96c87
2020-12-10 20:00:43 +01:00
Wladimir J. van der Laan
e6d0fa3ef6 gui: pre-rc3 translations update
Tree-SHA512: 2aba5a743ecc1f9e336b8db036640182eb3e85449d9ab4c883c97e6ac75466760939b29c18560205d25489d58091212f7f1dcf651388deb675f824703c943cbf
2020-12-10 19:28:24 +01:00
Wladimir J. van der Laan
3fee499bc3 Merge #20612: [0.21] final rc3 backports
48134a09ad doc: Update wallet database installation guide for macOS (Hennadii Stepanov)
f51e1cb291 build: Use Homebrew's sqlite package if it is available (Hennadii Stepanov)
48f8929aad build, refactor: Check that Homebrew's qt5 package is actually installed (Hennadii Stepanov)
96124a2041 build: Check that Homebrew's berkeley-db4 package is actually installed (Hennadii Stepanov)
61e316e661 Don't set BDB flags when configuring without (Jonas Schnelli)
ce13b99020 Add regression test for incorrect decoding (Pieter Wuille)
1caa32e3f2 Improve heuristic hex transaction decoding (Pieter Wuille)
0d3c140c4d test: add coverage for passing fee rate as a string (Jon Atack)
06c84232b3 wallet, bugfix: allow send to take string fee rate values (Jon Atack)
bead935470 Send and require SENDADDRV2 before VERACK (Pieter Wuille)
9e806887a8 Don't send 'sendaddrv2' to pre-70016 software (Pieter Wuille)

Pull request description:

ACKs for top commit:
  laanwj:
    ACK 48134a09ad

Tree-SHA512: 92f1199b96ab7775f88e882ec7fedf43118a4b8452d1c8d0b1cf072d8de153bbb601c7381bc1c5c80c93803c6f9942d54646e9c74e3a6703ce13854fb383fd5e
2020-12-10 19:18:38 +01:00
Hennadii Stepanov
48134a09ad doc: Update wallet database installation guide for macOS
Github-Pull: #20527
Rebased-From: c932e0d67e
2020-12-10 13:54:59 +01:00
Hennadii Stepanov
f51e1cb291 build: Use Homebrew's sqlite package if it is available
Github-Pull: #20527
Rebased-From: ee7b84e63c
2020-12-10 13:54:52 +01:00
Hennadii Stepanov
48f8929aad build, refactor: Check that Homebrew's qt5 package is actually installed
This change unifies Homebrew packages workflow, and does not change
behavior.

Github-Pull: #20527
Rebased-From: c96d1f65a5
2020-12-10 13:54:50 +01:00
Hennadii Stepanov
96124a2041 build: Check that Homebrew's berkeley-db4 package is actually installed
Github-Pull: #20563
Rebased-From: d3ef947524
2020-12-10 13:54:46 +01:00
Jonas Schnelli
61e316e661 Don't set BDB flags when configuring without
Github-Pull: #20478
Rebased-From: 982e548a9a
2020-12-10 13:54:42 +01:00
Pieter Wuille
ce13b99020 Add regression test for incorrect decoding
Github-Pull: #20595
Rebased-From: 0f949cde3d
2020-12-10 11:37:57 +01:00
Pieter Wuille
1caa32e3f2 Improve heuristic hex transaction decoding
Whenever both encodings are permitted, try both, and if only one succeeds,
return that one. Otherwise prefer the one for which the heuristic sanity
check passes. If that is the case for neither or for both, return the
extended-permitting deserialization.

Github-Pull: #20595
Rebased-From: 39c42c4420
2020-12-10 11:37:53 +01:00
Jon Atack
0d3c140c4d test: add coverage for passing fee rate as a string
Github-Pull: #20573
Rebased-From: 6fa72ceb80
2020-12-10 11:37:41 +01:00
Jon Atack
06c84232b3 wallet, bugfix: allow send to take string fee rate values
Github-Pull: #20573
Rebased-From: ce207d6b93
2020-12-10 11:37:36 +01:00
Pieter Wuille
bead935470 Send and require SENDADDRV2 before VERACK
See the corresponding BIP change: https://github.com/bitcoin/bips/pull/1043

Github-Pull: #20564
Rebased-From: 1583498fb6
2020-12-10 11:37:22 +01:00
Pieter Wuille
9e806887a8 Don't send 'sendaddrv2' to pre-70016 software
Github-Pull: #20564
Rebased-From: c5a8919660
2020-12-10 11:37:18 +01:00
MarcoFalke
aa4b8ebfec Merge #20510: [backport] wallet: allow zero-fee fundrawtransaction/walletcreatefundedpsbt and other fixes
6313362553 Use the correct incremental fee constant in bumpfee help (Jon Atack)
6e4969f76f Update feeRate (BTC/kvB) to fee_rate (sat/vB) in wallet_bumpfee (Jon Atack)
54e1edcc2b Allow zero-fee fundrawtxn and walletcreatefundedpsbt calls (Jon Atack)

Pull request description:

  Github-Pull: #20426
  Rebased-From: 1b3d700928

  Github-Pull: #20426
  Rebased-From: 3f1e10b2b1

  Github-Pull: #20426
  Rebased-From: 9f08780dd7

Top commit has no ACKs.

Tree-SHA512: 89556f69ca4a36d86d6ab5df740b6d5de809f13d45b03450fb526362b1eabd3d2d5285f97c552dc0bcb07e17ac2e6c557ff46335962e6bcf366d36ad412f257c
2020-12-04 18:14:15 +01:00
fanquake
68bd88597a Merge #20505: [backport] build: Avoid secp256k1.h include from system
01b647b1a2 build: Avoid secp256k1.h include from system (Niklas Gögge)

Pull request description:

  Backports #20469 to the 0.21 branch.

ACKs for top commit:
  hebasto:
    ACK 01b647b1a2, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: c098055b5e413be6f438d1d43e80c1943329ebb708531d8d82e72de402bddeb6f8b812303f9ae5a45abf62b3ff87fa909fbbf7fb56dca7959ecb9061febae4a1
2020-12-02 10:11:37 +08:00
Jon Atack
6313362553 Use the correct incremental fee constant in bumpfee help
and remove redundant units ("Must be at least 1.000 sat/vB sat/vB" -> "1.00 sat vB")

Github-Pull: #20426
Rebased-From: 9f08780dd7
2020-11-26 19:32:30 +01:00
Jon Atack
6e4969f76f Update feeRate (BTC/kvB) to fee_rate (sat/vB) in wallet_bumpfee
as the feeRate argument should soon be deprecated.

Also loosen one test (and a similar one) that caused a one-off CI failure with:
expected message
'Insufficient total fee 0.00000141, must be at least 0.00001704 (oldFee 0.00000999 + incrementalFee 0.00000705)'
actual message
'Insufficient total fee 0.00000141, must be at least 0.00001712 (oldFee 0.00001007 + incrementalFee 0.00000705)'

Github-Pull: #20426
Rebased-From: 3f1e10b2b1
2020-11-26 19:32:06 +01:00
Jon Atack
54e1edcc2b Allow zero-fee fundrawtxn and walletcreatefundedpsbt calls
A check to raise an error on zero-fee txns was mistakenly extended in commit
a0d4957 from the bumpfee and send{toaddress, many} RPCs to also include
fundrawtransaction and walletcreatefundedpsbt.

This commit overrides zero fee rate checking for these two RPCs, not only for
the feeRate (BTC/kvB) arg to return to previous behavior, but also for the new
fee_rate (sat/vB) arg.

Github-Pull: #20426
Rebased-From: 1b3d700928
2020-11-26 19:31:26 +01:00
Niklas Gögge
01b647b1a2 build: Avoid secp256k1.h include from system
GitHub-Pull: #20469
Rebase-From: e95aaefe25
2020-11-26 09:38:44 +01:00
MarcoFalke
9facca3ce0 Merge #20490: [backport] wallet: upgradewallet fixes, improvements, test coverage
ca8cd893bb wallet: fix and improve upgradewallet error responses (Jon Atack)
99d56e3571 wallet: fix and improve upgradewallet result responses (Jon Atack)
2498b04ce8 Don't upgrade to HD split if it is already supported (Andrew Chow)
c46c18b788 wallet: refactor GetClosestWalletFeature() (Jon Atack)

Pull request description:

  Github-Pull: #20403
  Rebased-From: c46c18b788

  Github-Pull: #20403
  Rebased-From: 2498b04ce8

  Github-Pull: #20403
  Rebased-From: 99d56e3571

  Github-Pull: #20403
  Rebased-From: ca8cd893bb

Top commit has no ACKs.

Tree-SHA512: b18a1d015c963298740c585385eaa056988464112c88a519fe619be22dc78a8f6a102365cf799c50b781a77a09bec82b58ce411ab007b48f8b5de876e9c75060
2020-11-25 18:03:53 +01:00
MarcoFalke
d47d16025e Merge #20485: [backport] wallet: Do not treat default constructed types as None-type
fa69c2c784 wallet: Do not treat default constructed types as None-type (MarcoFalke)
fac4e136fa refactor: Change pointer to reference because it can not be null (MarcoFalke)

Pull request description:

  Github-Pull: #20410
  Rebased-From: fac4e136fa

  Github-Pull: #20410
  Rebased-From: fa69c2c784

Top commit has no ACKs.

Tree-SHA512: 05c3fe29677710b57dcc482fd529b0ab79475519f60f9cfde19f956c4e2212d09b042af458ec4f1272c581360ce841b735dca4df144e0798b3ccf16547de9cd0
2020-11-25 11:27:42 +01:00
MarcoFalke
4e964094db Merge #20486: [backport] test: Fix intermittent issue in mempool_compatibility
fa05d19bd6 test: Fix intermittent issue in mempool_compatibility (MarcoFalke)

Pull request description:

  Github-Pull: #20456
  Rebased-From: fa05d19bd6

Top commit has no ACKs.

Tree-SHA512: c07ed3ffab315da2bf70427882e2da8216964fef97dd538d7c46c5d5b3563b0732626ffbe4b0e19537ad49a8542eedb3c08850cccacdef466f24582defe99d22
2020-11-25 11:25:33 +01:00
Wladimir J. van der Laan
7264bcd20f qt: Pre-rc2 translations update
Tree-SHA512: 465a7b586f3eff476bd9648ff70e37e11f51e67f048dc1265ffcc1132c4e063515495ec6cbdf6ebb4abd5f5b08615595ce12273ce9abebc8ffef9c7b8375ac20
2020-11-25 08:57:45 +01:00
Wladimir J. van der Laan
ba19f9b490 build: Bump RC to rc2
Tree-SHA512: 122ccac91708dda65a30aab6313c770008ccc0571402d92bc24fbefed578002626899c1754676118f5a456a63dbea84d1fe2fef46af57b77837f438c6e637b26
2020-11-25 08:50:47 +01:00
fanquake
17294c1820 Merge #20479: Fix QPainter non-determinism on macOS (0.21 backport)
ab23a83400 Fix QPainter non-determinism on macOS (Andrew Chow)

Pull request description:

  Aplies a patch to Qt that fixes the non-determinism by modifying Qt. The
  source of the non-determinism is how LLVM 8 optimizes qt_intersect_spans
  when compiling. The particular optimization that seems to be causing the
  problems is that a temp variable is being added for spans->y. For some
  reason, when it does this, it chooses different instructions to use when
  making that variable. We bypass this problem by patching
  qt_intersect_spans to always make and use this local variable.

  Github-Pull: #20447
  Rebased-From: 8f7d1b39ef
  Tree-SHA512: 558da5c2bb0373e2a89f2c219170f802036e0e87cc8e808336b23d074152cb893007a440f46ec957156b0921355cd18502710f2d224f27bc26e934c50ebebc41

ACKs for top commit:
  jonasschnelli:
    codereview ACK ab23a83400
  achow101:
    ACK ab23a83400

Tree-SHA512: 10991fe2b5452b1393678c315281cfdca011e9bb2cd8094a002746e690890ace148ac2dbf39c5fbe5e7f4cd39eeebfa0a715c065cff150cf70e9733cb0ff32d6
2020-11-25 10:07:28 +08:00
MarcoFalke
3fae2f95f1 Merge #20475: [backport] RPC: unloadwallet: Clarify docs/error when both the RPC endpoint and wallet_name parameter specify a wallet
b1f59d55d9 RPC/Wallet: unloadwallet: Clarify docs/error when both the RPC endpoint and wallet_name parameter specify a wallet (Luke Dashjr)

Pull request description:

  Github-Pull: #20462
  Rebased-From: b1f59d55d9

Top commit has no ACKs.

Tree-SHA512: 2ee0a8a280f56baf196a3a48a59620f297075d23898e6aa3b3e677cdde74826688614d27a477a1448306234c2109fa39083946f691ced10d8bbc53006730784e
2020-11-24 17:28:19 +01:00
Andrew Chow
ab23a83400 Fix QPainter non-determinism on macOS
Aplies a patch to Qt that fixes the non-determinism by modifying Qt. The
source of the non-determinism is how LLVM 8 optimizes qt_intersect_spans
when compiling. The particular optimization that seems to be causing the
problems is that a temp variable is being added for spans->y. For some
reason, when it does this, it chooses different instructions to use when
making that variable. We bypass this problem by patching
qt_intersect_spans to always make and use this local variable.

Github-Pull: #20447
Rebased-From: 8f7d1b39ef
Tree-SHA512: 558da5c2bb0373e2a89f2c219170f802036e0e87cc8e808336b23d074152cb893007a440f46ec957156b0921355cd18502710f2d224f27bc26e934c50ebebc41
2020-11-24 16:32:42 +01:00
MarcoFalke
bf9548bc59 Merge #20431: [backport 0.21] tests: shrink feature_taproot transfer of funds tx
7ffac12545 tests: shrink feature_taproot transfer of funds tx (Anthony Towns)

Pull request description:

  Github-Pull: #20428
  Rebased-From: 7ffac12545

Top commit has no ACKs.

Tree-SHA512: 4e6b37a44dca3e29d5168b7eb9238a7ce0bbb9b0924a21671537a7c534790fb6b05b1a30a404db434fade030b4f369adfc73694ef85d91884bb7349adddc5f6a
2020-11-20 11:50:10 +01:00
Wladimir J. van der Laan
80496f9e81 build: Set msvc builds's CLIENT_VERSION_IS_RELEASE
Of course, this one was in another place too.

Tree-SHA512: 87784829b1f700dcf5fd22daad0c920cfb25485ae17eff0b3e236513dc543c8643e568f39d418d43ea0eeb330fcac93ab2276cda8253ec6538d01e20d102d10c
2020-11-19 09:38:57 +01:00
Wladimir J. van der Laan
6cde7bb9b2 doc: Generate manual pages for 0.21.0rc1
Tree-SHA512: d3e4f927d39f38317242b376f7cbd04ff3ab2b5795ffb5912a1e193b3bc27fa0e7fe55446b62c22df6a213509c2c489a66cde866992706a6ddd0ebbddc713303
2020-11-18 10:43:41 +01:00
Wladimir J. van der Laan
463b316822 build: Bump version for 0.21.0rc1
Tree-SHA512: 5bfe150de6e808a6af499e1f01fbfde91fca3fd726e3d457172e94161c4a18fb508824a4b2a58a35ac582d505e4e9da1287a349b51be863dc5bd4fa858400863
2020-11-18 10:33:34 +01:00
1390 changed files with 154607 additions and 152032 deletions

View File

@@ -7,9 +7,9 @@ clone_depth: 5
environment:
PATH: 'C:\Python37-x64;C:\Python37-x64\Scripts;%PATH%'
PYTHONUTF8: 1
QT_DOWNLOAD_URL: 'https://github.com/sipsorcery/qt_win_binary/releases/download/qt51211x64_static_vs2019_16101/Qt5.12.11_x64_static_vs2019_16101.zip'
QT_DOWNLOAD_HASH: 'cf1b58107fadbf0d9a957d14dab16cde6b6eb6936a1908472da1f967dda34a3a'
QT_LOCAL_PATH: 'C:\Qt5.12.11_x64_static_vs2019_16101'
QT_DOWNLOAD_URL: 'https://github.com/sipsorcery/qt_win_binary/releases/download/qt598x64_vs2019_v1681/qt598_x64_vs2019_1681.zip'
QT_DOWNLOAD_HASH: '00cf7327818c07d74e0b1a4464ffe987c2728b00d49d4bf333065892af0515c3'
QT_LOCAL_PATH: 'C:\Qt5.9.8_x64_static_vs2019'
VCPKG_TAG: '75522bb1f2e7d863078bcd06322348f053a9e33f'
install:
# Disable zmq test for now since python zmq library on Windows would cause Access violation sometimes.

View File

@@ -1,25 +1,33 @@
### Global defaults
timeout_in: 120m # https://cirrus-ci.org/faq/#instance-timed-out
container:
# https://cirrus-ci.org/faq/#are-there-any-limits
# Each project has 16 CPU in total, assign 2 to each container, so that 8 tasks run in parallel
cpu: 2
memory: 8G # Set to 8GB to avoid OOM. https://cirrus-ci.org/guide/linux/#linux-containers
kvm: true # Use kvm to avoid spurious CI failures in the default virtualization cluster, see https://github.com/bitcoin/bitcoin/issues/20093
env:
PACKAGE_MANAGER_INSTALL: "apt-get update && apt-get install -y"
MAKEJOBS: "-j4"
DANGER_RUN_CI_ON_HOST: "1" # Containers will be discarded after the run, so there is no risk that the ci scripts modify the system
TEST_RUNNER_PORT_MIN: "14000" # Must be larger than 12321, which is used for the http cache. See https://cirrus-ci.org/guide/writing-tasks/#http-cache
CCACHE_SIZE: "200M"
CCACHE_DIR: "/tmp/ccache_dir"
CCACHE_NOHASHDIR: "1" # Debug info might contain a stale path if the build dir changes, but this is fine
cirrus_ephemeral_worker_template_env: &CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
DANGER_RUN_CI_ON_HOST: "1" # Containers will be discarded after the run, so there is no risk that the ci scripts modify the system
persistent_worker_template_env: &PERSISTENT_WORKER_TEMPLATE_ENV
RESTART_CI_DOCKER_BEFORE_RUN: "1"
persistent_worker_template: &PERSISTENT_WORKER_TEMPLATE
persistent_worker: {} # https://cirrus-ci.org/guide/persistent-workers/
### Global task template
# https://cirrus-ci.org/guide/tips-and-tricks/#sharing-configuration-between-tasks
base_template: &BASE_TEMPLATE
global_task_template: &GLOBAL_TASK_TEMPLATE
skip: $CIRRUS_REPO_FULL_NAME == "bitcoin-core/gui" && $CIRRUS_PR == "" # No need to run on the read-only mirror, unless it is a PR. https://cirrus-ci.org/guide/writing-tasks/#conditional-task-execution
ccache_cache:
folder: "/tmp/ccache_dir"
depends_built_cache:
folder: "/tmp/cirrus-ci-build/depends/built"
depends_sdk_cache:
folder: "/tmp/cirrus-ci-build/depends/sdk-sources"
depends_releases_cache:
folder: "/tmp/cirrus-ci-build/releases"
merge_base_script:
- if [ "$CIRRUS_PR" = "" ]; then exit 0; fi
- bash -c "$PACKAGE_MANAGER_INSTALL git"
@@ -27,32 +35,9 @@ base_template: &BASE_TEMPLATE
- git config --global user.email "ci@ci.ci"
- git config --global user.name "ci"
- git merge FETCH_HEAD # Merge base to detect silent merge conflicts
stateful: false # https://cirrus-ci.org/guide/writing-tasks/#stateful-tasks
global_task_template: &GLOBAL_TASK_TEMPLATE
<< : *BASE_TEMPLATE
timeout_in: 120m # https://cirrus-ci.org/faq/#instance-timed-out
container:
# https://cirrus-ci.org/faq/#are-there-any-limits
# Each project has 16 CPU in total, assign 2 to each container, so that 8 tasks run in parallel
cpu: 2
memory: 8G # Set to 8GB to avoid OOM. https://cirrus-ci.org/guide/linux/#linux-containers
ccache_cache:
folder: "/tmp/ccache_dir"
depends_built_cache:
folder: "depends/built"
ci_script:
- ./ci/test_run_all.sh
depends_sdk_cache_template: &DEPENDS_SDK_CACHE_TEMPLATE
depends_sdk_cache:
folder: "depends/sdk-sources"
compute_credits_template: &CREDITS_TEMPLATE
# https://cirrus-ci.org/pricing/#compute-credits
# Only use credits for pull requests to the main repo
use_compute_credits: $CIRRUS_REPO_FULL_NAME == 'bitcoin/bitcoin' && $CIRRUS_PR != ""
#task:
# name: "Windows"
# windows_container:
@@ -70,153 +55,92 @@ compute_credits_template: &CREDITS_TEMPLATE
# - choco install python --version=3.7.7 -y
task:
name: 'lint [bionic]'
<< : *BASE_TEMPLATE
container:
image: ubuntu:bionic # For python 3.6, oldest supported version according to doc/dependencies.md
cpu: 1
memory: 1G
# For faster CI feedback, immediately schedule the linters
<< : *CREDITS_TEMPLATE
lint_script:
- ./ci/lint_run_all.sh
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
task:
name: 'ARM [unit tests, no functional tests] [buster]'
name: 'ARM [GOAL: install] [buster] [unit tests, no functional tests]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: debian:buster
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_arm.sh"
task:
name: 'Win64 [unit tests, no gui tests, no boost::process, no functional tests] [focal]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_win64.sh"
task:
name: '32-bit + dash [gui] [CentOS 8]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: quay.io/centos/centos:stream8
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
PACKAGE_MANAGER_INSTALL: "yum install -y"
FILE_ENV: "./ci/test/00_setup_env_i686_centos.sh"
task:
name: '[previous releases, uses qt5 dev package and some depends packages, DEBUG] [unsigned char] [bionic]'
previous_releases_cache:
folder: "releases"
<< : *GLOBAL_TASK_TEMPLATE
<< : *PERSISTENT_WORKER_TEMPLATE
env:
<< : *PERSISTENT_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_qt5.sh"
task:
name: '[depends, sanitizers: thread (TSan), no gui] [jammy]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:jammy
cpu: 6 # Increase CPU and Memory to avoid timeout
memory: 24G
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
MAKEJOBS: "-j8"
FILE_ENV: "./ci/test/00_setup_env_native_tsan.sh"
task:
name: '[depends, sanitizers: memory (MSan)] [focal]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_msan.sh"
task:
name: '[no depends, sanitizers: address/leak (ASan + LSan) + undefined (UBSan) + integer] [jammy]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:jammy
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_asan.sh"
task:
name: '[no depends, sanitizers: fuzzer,address,undefined,integer] [focal]'
only_if: $CIRRUS_BRANCH == $CIRRUS_DEFAULT_BRANCH || $CIRRUS_BASE_BRANCH == $CIRRUS_DEFAULT_BRANCH
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
cpu: 4 # Increase CPU and memory to avoid timeout
memory: 16G
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
MAKEJOBS: "-j8"
FILE_ENV: "./ci/test/00_setup_env_native_fuzz.sh"
task:
name: '[multiprocess, DEBUG] [focal]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
cpu: 4
memory: 16G # The default memory is sometimes just a bit too small, so double everything
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
MAKEJOBS: "-j8"
FILE_ENV: "./ci/test/00_setup_env_native_multiprocess.sh"
task:
name: '[no wallet] [bionic]'
name: 'Win64 [GOAL: deploy] [unit tests, no gui, no boost::process, no functional tests]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:bionic
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_nowallet.sh"
FILE_ENV: "./ci/test/00_setup_env_win64.sh"
task:
name: 'macOS 10.14 [gui, no tests] [focal]'
<< : *DEPENDS_SDK_CACHE_TEMPLATE
name: 'x86_64 Linux [GOAL: install] [bionic] [C++17, previous releases, uses qt5 dev package and some depends packages] [unsigned char]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:bionic
env:
FILE_ENV: "./ci/test/00_setup_env_native_qt5.sh"
task:
name: 'x86_64 Linux [GOAL: install] [focal] [depends, sanitizers: thread (TSan), no gui]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
cpu: 4 # Double CPU and Memory to avoid timeout
memory: 16G
env:
MAKEJOBS: "-j8"
FILE_ENV: "./ci/test/00_setup_env_native_tsan.sh"
task:
name: 'x86_64 Linux [GOAL: install] [focal] [depends, sanitizers: memory (MSan)]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_msan.sh"
task:
name: 'x86_64 Linux [GOAL: install] [focal] [no depends, only system libs, sanitizers: address/leak (ASan + LSan) + undefined (UBSan) + integer]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
env:
FILE_ENV: "./ci/test/00_setup_env_native_asan.sh"
task:
name: 'x86_64 Linux [GOAL: install] [focal] [no depends, only system libs, sanitizers: fuzzer,address,undefined]'
only_if: $CIRRUS_BRANCH == $CIRRUS_DEFAULT_BRANCH || $CIRRUS_BASE_BRANCH == $CIRRUS_DEFAULT_BRANCH
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
env:
FILE_ENV: "./ci/test/00_setup_env_native_fuzz.sh"
task:
name: 'x86_64 Linux [GOAL: install] [focal] [multiprocess]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
env:
FILE_ENV: "./ci/test/00_setup_env_native_multiprocess.sh"
task:
name: 'macOS 10.12 [GOAL: deploy] [no functional tests]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:bionic
env:
FILE_ENV: "./ci/test/00_setup_env_mac.sh"
task:
name: 'macOS 11 native [gui] [no depends]'
name: 'macOS 10.14 native [GOAL: install] [GUI] [no depends]'
brew_install_script:
- brew install boost libevent berkeley-db4 qt@5 miniupnpc libnatpmp ccache zeromq qrencode sqlite libtool automake pkg-config gnu-getopt
- brew update
- brew install boost libevent berkeley-db4 qt@5 miniupnpc ccache zeromq qrencode sqlite libtool automake pkg-config gnu-getopt
<< : *GLOBAL_TASK_TEMPLATE
osx_instance:
# Use latest image, but hardcode version to avoid silent upgrades (and breaks)
image: big-sur-xcode-12.5 # https://cirrus-ci.org/guide/macOS
image: catalina-xcode-12.1 # https://cirrus-ci.org/guide/macOS
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
DANGER_RUN_CI_ON_HOST: "true"
CI_USE_APT_INSTALL: "no"
PACKAGE_MANAGER_INSTALL: "echo" # Nothing to do
FILE_ENV: "./ci/test/00_setup_env_mac_host.sh"
task:
name: 'ARM64 Android APK [focal]'
<< : *DEPENDS_SDK_CACHE_TEMPLATE
depends_sources_cache:
folder: "depends/sources"
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_android.sh"

View File

@@ -1,26 +0,0 @@
# This is the top-most EditorConfig file.
root = true
# For all files.
[*]
charset = utf-8
end_of_line = lf
indent_style = space
insert_final_newline = true
trim_trailing_whitespace = true
# Source code files
[*.{h,cpp,py,sh}]
indent_size = 4
# .cirrus.yml, .appveyor.yml, .fuzzbuzz.yml, etc.
[*.yml]
indent_size = 2
# Makefiles
[{*.am,Makefile.*.include}]
indent_style = tab
# Autoconf scripts
[configure.ac]
indent_size = 2

16
.fuzzbuzz.yml Normal file
View File

@@ -0,0 +1,16 @@
base: ubuntu:16.04
language: c++
engine: libFuzzer
environment:
- CXXFLAGS=-fcoverage-mapping -fno-omit-frame-pointer -fprofile-instr-generate -gline-tables-only -O1
setup:
- sudo apt-get update
- sudo apt-get install -y autoconf bsdmainutils clang git libboost-all-dev libboost-program-options-dev libc++1 libc++abi1 libc++abi-dev libc++-dev libclang1 libclang-dev libdb5.3++ libevent-dev libllvm-ocaml-dev libomp5 libomp-dev libprotobuf-dev libqt5core5a libqt5dbus5 libqt5gui5 libssl-dev libtool llvm llvm-dev llvm-runtime pkg-config protobuf-compiler qttools5-dev qttools5-dev-tools software-properties-common
- ./autogen.sh
- CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
- make
- git clone https://github.com/bitcoin-core/qa-assets
auto_targets:
find_targets_command: find src/test/fuzz/ -executable -type f ! -name "*.cpp" ! -name "*.h"
base_corpus_dir: qa-assets/fuzz_seed_corpus/
memory_limit: none

9
.gitignore vendored
View File

@@ -8,9 +8,9 @@ src/bitcoin-cli
src/bitcoin-gui
src/bitcoin-node
src/bitcoin-tx
src/bitcoin-util
src/bitcoin-wallet
src/test/fuzz/fuzz
src/test/fuzz/*
!src/test/fuzz/*.*
src/test/test_bitcoin
src/qt/test/test_bitcoin-qt
@@ -61,7 +61,7 @@ src/qt/bitcoin-qt.includes
.dirstamp
.libs
.*.swp
*~
*.*~*
*.bak
*.rej
*.orig
@@ -76,7 +76,6 @@ src/qt/bitcoin-qt.includes
*.log
*.trs
*.dmg
*.iso
*.json.h
*.raw.h
@@ -149,5 +148,3 @@ db4/
osx_volname
dist/
*.background.tiff
/guix-build-*

View File

@@ -1 +1 @@
3.6.12
3.5.6

76
.travis.yml Normal file
View File

@@ -0,0 +1,76 @@
# Travis caches can be manually removed if necessary. This is one of the very
# few manual operations that is possible with Travis, and it can be done by a
# Bitcoin Core GitHub member via the Travis web interface [0].
#
# Travis CI uploads the cache after the script phase of the build [1].
# However, the build is terminated without saving the cache if it takes over
# 50 minutes [2]. Thus, if we spent too much time in early build stages, fail
# with an error and save the cache.
#
# [0] https://travis-ci.org/bitcoin/bitcoin/caches
# [1] https://docs.travis-ci.com/user/caching/#build-phases
# [2] https://docs.travis-ci.com/user/customizing-the-build#build-timeouts
version: ~> 1.0
dist: bionic
os: linux
language: minimal
arch: amd64
cache:
directories:
- $TRAVIS_BUILD_DIR/depends/built
- $TRAVIS_BUILD_DIR/depends/sdk-sources
- $TRAVIS_BUILD_DIR/ci/scratch/.ccache
- $TRAVIS_BUILD_DIR/releases/$HOST
stages:
- lint
- test
env:
global:
- CI_RETRY_EXE="travis_retry"
- CACHE_ERR_MSG="Error! Initial build successful, but not enough time remains to run later build stages and tests. See https://docs.travis-ci.com/user/customizing-the-build#build-timeouts . Please manually re-run this job by using the travis restart button. The next run should not time out because the build cache has been saved."
before_install:
- set -o errexit; source ./ci/test/00_setup_env.sh
- set -o errexit; source ./ci/test/03_before_install.sh
install:
- set -o errexit; source ./ci/test/04_install.sh
before_script:
# Temporary workaround for https://github.com/bitcoin/bitcoin/issues/16368
- for i in {1..4}; do echo "$(sleep 500)" ; done &
- set -o errexit; source ./ci/test/05_before_script.sh &> "/dev/null"
script:
- export CONTINUE=1
- if [ $SECONDS -gt 1200 ]; then export CONTINUE=0; fi # Likely the depends build took very long
- if [ $TRAVIS_REPO_SLUG = "bitcoin/bitcoin" ]; then export CONTINUE=1; fi # continue on repos with extended build time (90 minutes)
- if [ $CONTINUE = "1" ]; then set -o errexit; source ./ci/test/06_script_a.sh; else set +o errexit; echo "$CACHE_ERR_MSG"; false; fi
- if [[ $SECONDS -gt 50*60-$EXPECTED_TESTS_DURATION_IN_SECONDS ]]; then export CONTINUE=0; fi
- if [ $TRAVIS_REPO_SLUG = "bitcoin/bitcoin" ]; then export CONTINUE=1; fi # continue on repos with extended build time (90 minutes)
- if [ $CONTINUE = "1" ]; then set -o errexit; source ./ci/test/06_script_b.sh; else set +o errexit; echo "$CACHE_ERR_MSG"; false; fi
after_script:
- echo $TRAVIS_COMMIT_RANGE
jobs:
include:
- stage: lint
name: 'lint'
env:
cache: pip
language: python
python: '3.5' # Oldest supported version according to doc/dependencies.md
install:
- set -o errexit; source ./ci/lint/04_install.sh
before_script:
- set -o errexit; source ./ci/lint/05_before_script.sh
script:
- set -o errexit; source ./ci/lint/06_script.sh
- stage: test
name: '32-bit + dash [GOAL: install] [CentOS 7] [gui]'
env: >-
FILE_ENV="./ci/test/00_setup_env_i686_centos.sh"
- stage: test
name: 'x86_64 Linux [GOAL: install] [xenial] [no wallet]'
env: >-
FILE_ENV="./ci/test/00_setup_env_native_nowallet.sh"

View File

@@ -1,7 +1,7 @@
[main]
host = https://www.transifex.com
[o:bitcoin:p:bitcoin:r:qt-translation-022x]
file_filter = src/qt/locale/bitcoin_<lang>.xlf
source_file = src/qt/locale/bitcoin_en.xlf
[bitcoin.qt-translation-021x]
file_filter = src/qt/locale/bitcoin_<lang>.ts
source_file = src/qt/locale/bitcoin_en.ts
source_lang = en

View File

@@ -1,15 +1,20 @@
# ==============================================================================
# Bitcoin Core REVIEWERS
# Bitcoin Core CODEOWNERS
# ==============================================================================
# Configuration of automated review requests for the bitcoin/bitcoin repo
# via DrahtBot.
# Configuration of code ownership and review approvals for the bitcoin/bitcoin
# repo.
# Order is not important; if a modified file or directory matches a fnmatch,
# the reviewer will be mentioned in a PR comment requesting a review.
# Order is important; the last matching pattern takes the most precedence.
# More info on how this file works can be found at:
# https://help.github.com/articles/about-codeowners/
# Regular contributors are free to add their names to specific directories or
# files provided that they are willing to provide a review.
# This file is called CODEOWNERS because it is a magic file for GitHub to
# automatically suggest reviewers. In this project's case, the names below
# should be thought of as code reviewers rather than owners. Regular
# contributors are free to add their names to specific directories or files
# provided that they are willing to provide a review when automatically
# assigned.
# Absence from this list should not be interpreted as a discouragement to
# review a pull request. Peer review is always welcome and is a critical
@@ -18,13 +23,12 @@
# Maintainers
# @fanquake
# @hebasto
# @jonasschnelli
# @laanwj
# @sipa
# @fanquake
# @jonasschnelli
# @marcofalke
# @meshcollider
# @sipa
# Docs
/doc/*[a-zA-Z-].md @harding
@@ -57,17 +61,14 @@
/src/util/settings.* @ryanofsky
# Fuzzing
/src/test/fuzz/ @practicalswift
/doc/fuzzing.md @practicalswift
# Tests
/src/test/net_peer_eviction_tests.cpp @jonatack
# Test framework
/test/functional/mempool_updatefromblock.py @hebasto
/test/functional/feature_asmap.py @jonatack
/test/functional/interface_bitcoin_cli.py @jonatack
# Backwards compatibility tests
*_compatibility.py @sjors
/test/functional/wallet_upgradewallet.py @sjors @achow101
/test/get_previous_releases.py @sjors
/test/functional/tool_wallet.py @jonatack
# Translations
/src/util/translation.h @hebasto
@@ -102,11 +103,6 @@
# Descriptors
*descriptor* @achow101 @sipa
# External signer
*external_signer* @sjors
/doc/external-signer.md @sjors
*signer.py @sjors
# Interfaces
/src/interfaces/ @ryanofsky
@@ -114,7 +110,9 @@
/src/txdb.* @jamesob
/src/dbwrapper.* @jamesob
# Linter
# Scripts/Linter
*.sh @practicalswift
/test/lint/ @practicalswift
/test/lint/lint-shell.sh @hebasto
# Bech32

View File

@@ -57,8 +57,8 @@ Communication Channels
----------------------
Most communication about Bitcoin Core development happens on IRC, in the
`#bitcoin-core-dev` channel on Libera Chat. The easiest way to participate on IRC is
with the web client, [web.libera.chat](https://web.libera.chat/#bitcoin-core-dev). Chat
`#bitcoin-core-dev` channel on Freenode. The easiest way to participate on IRC is
with the web client, [webchat.freenode.net](https://webchat.freenode.net/). Chat
history logs can be found
on [http://www.erisian.com.au/bitcoin-core-dev/](http://www.erisian.com.au/bitcoin-core-dev/)
and [http://gnusha.org/bitcoin-core-dev/](http://gnusha.org/bitcoin-core-dev/).
@@ -197,7 +197,7 @@ Note: Code review is a burdensome but important part of the development process,
If your pull request contains fixup commits (commits that change the same line of code repeatedly) or too fine-grained
commits, you may be asked to [squash](https://git-scm.com/docs/git-rebase#_interactive_mode) your commits
before it will be reviewed. The basic squashing workflow is shown below.
before it will be merged. The basic squashing workflow is shown below.
git checkout your_branch_name
git rebase -i HEAD~n

View File

@@ -1,7 +1,7 @@
The MIT License (MIT)
Copyright (c) 2009-2021 The Bitcoin Core developers
Copyright (c) 2009-2021 Bitcoin Developers
Copyright (c) 2009-2020 The Bitcoin Core developers
Copyright (c) 2009-2020 Bitcoin Developers
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

View File

@@ -3,8 +3,8 @@
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
# Pattern rule to print variables, e.g. make print-top_srcdir
print-%: FORCE
@echo '$*'='$($*)'
print-%:
@echo $* = $($*)
ACLOCAL_AMFLAGS = -I build-aux/m4
SUBDIRS = src
@@ -12,7 +12,6 @@ if ENABLE_MAN
SUBDIRS += doc/man
endif
.PHONY: deploy FORCE
.INTERMEDIATE: $(OSX_TEMP_ISO) $(COVERAGE_INFO)
export PYTHONPATH
@@ -25,10 +24,7 @@ BITCOIND_BIN=$(top_builddir)/src/$(BITCOIN_DAEMON_NAME)$(EXEEXT)
BITCOIN_QT_BIN=$(top_builddir)/src/qt/$(BITCOIN_GUI_NAME)$(EXEEXT)
BITCOIN_CLI_BIN=$(top_builddir)/src/$(BITCOIN_CLI_NAME)$(EXEEXT)
BITCOIN_TX_BIN=$(top_builddir)/src/$(BITCOIN_TX_NAME)$(EXEEXT)
BITCOIN_UTIL_BIN=$(top_builddir)/src/$(BITCOIN_UTIL_NAME)$(EXEEXT)
BITCOIN_WALLET_BIN=$(top_builddir)/src/$(BITCOIN_WALLET_TOOL_NAME)$(EXEEXT)
BITCOIN_NODE_BIN=$(top_builddir)/src/$(BITCOIN_MP_NODE_NAME)$(EXEEXT)
BITCOIN_GUI_BIN=$(top_builddir)/src/$(BITCOIN_MP_GUI_NAME)$(EXEEXT)
BITCOIN_WIN_INSTALLER=$(PACKAGE)-$(PACKAGE_VERSION)-win64-setup$(EXEEXT)
empty :=
@@ -37,18 +33,17 @@ space := $(empty) $(empty)
OSX_APP=Bitcoin-Qt.app
OSX_VOLNAME = $(subst $(space),-,$(PACKAGE_NAME))
OSX_DMG = $(OSX_VOLNAME).dmg
OSX_TEMP_ISO = $(OSX_DMG:.dmg=).temp.iso
OSX_BACKGROUND_SVG=background.svg
OSX_BACKGROUND_IMAGE=background.tiff
OSX_BACKGROUND_IMAGE_DPIS=36 72
OSX_DSSTORE_GEN=$(top_srcdir)/contrib/macdeploy/custom_dsstore.py
OSX_DEPLOY_SCRIPT=$(top_srcdir)/contrib/macdeploy/macdeployqtplus
OSX_FANCY_PLIST=$(top_srcdir)/contrib/macdeploy/fancy.plist
OSX_INSTALLER_ICONS=$(top_srcdir)/src/qt/res/icons/bitcoin.icns
OSX_PLIST=$(top_builddir)/share/qt/Info.plist #not installed
OSX_QT_TRANSLATIONS = ar,bg,ca,cs,da,de,es,fa,fi,fr,gd,gl,he,hu,it,ja,ko,lt,lv,pl,pt,ru,sk,sl,sv,uk,zh_CN,zh_TW
DIST_CONTRIB = \
$(top_srcdir)/test/sanitizer_suppressions/lsan \
$(top_srcdir)/test/sanitizer_suppressions/tsan \
$(top_srcdir)/test/sanitizer_suppressions/ubsan \
$(top_srcdir)/contrib/linearize/linearize-data.py \
$(top_srcdir)/contrib/linearize/linearize-hashes.py
@@ -57,17 +52,16 @@ DIST_SHARE = \
$(top_srcdir)/share/rpcauth
BIN_CHECKS=$(top_srcdir)/contrib/devtools/symbol-check.py \
$(top_srcdir)/contrib/devtools/security-check.py \
$(top_srcdir)/contrib/devtools/utils.py \
$(top_srcdir)/contrib/devtools/pixie.py
$(top_srcdir)/contrib/devtools/security-check.py
WINDOWS_PACKAGING = $(top_srcdir)/share/pixmaps/bitcoin.ico \
$(top_srcdir)/share/pixmaps/nsis-header.bmp \
$(top_srcdir)/share/pixmaps/nsis-wizard.bmp \
$(top_srcdir)/doc/README_windows.txt
OSX_PACKAGING = $(OSX_DEPLOY_SCRIPT) $(OSX_INSTALLER_ICONS) \
OSX_PACKAGING = $(OSX_DEPLOY_SCRIPT) $(OSX_FANCY_PLIST) $(OSX_INSTALLER_ICONS) \
$(top_srcdir)/contrib/macdeploy/$(OSX_BACKGROUND_SVG) \
$(OSX_DSSTORE_GEN) \
$(top_srcdir)/contrib/macdeploy/detached-sig-apply.sh \
$(top_srcdir)/contrib/macdeploy/detached-sig-create.sh
@@ -79,7 +73,6 @@ COVERAGE_INFO = $(COV_TOOL_WRAPPER) baseline.info \
dist-hook:
-$(GIT) archive --format=tar HEAD -- src/clientversion.cpp | $(AMTAR) -C $(top_distdir) -xf -
if TARGET_WINDOWS
$(BITCOIN_WIN_INSTALLER): all-recursive
$(MKDIR_P) $(top_builddir)/release
STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(BITCOIND_BIN) $(top_builddir)/release
@@ -87,15 +80,10 @@ $(BITCOIN_WIN_INSTALLER): all-recursive
STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(BITCOIN_CLI_BIN) $(top_builddir)/release
STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(BITCOIN_TX_BIN) $(top_builddir)/release
STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(BITCOIN_WALLET_BIN) $(top_builddir)/release
STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(BITCOIN_UTIL_BIN) $(top_builddir)/release
@test -f $(MAKENSIS) && echo 'OutFile "$@"' | cat $(top_builddir)/share/setup.nsi - | $(MAKENSIS) -V2 - || \
echo error: could not build $@
@echo built $@
deploy: $(BITCOIN_WIN_INSTALLER)
endif
if TARGET_DARWIN
$(OSX_APP)/Contents/PkgInfo:
$(MKDIR_P) $(@D)
@echo "APPL????" > $@
@@ -129,7 +117,7 @@ osx_volname:
if BUILD_DARWIN
$(OSX_DMG): $(OSX_APP_BUILT) $(OSX_PACKAGING) $(OSX_BACKGROUND_IMAGE)
$(PYTHON) $(OSX_DEPLOY_SCRIPT) $(OSX_APP) $(OSX_VOLNAME) -translations-dir=$(QT_TRANSLATION_DIR) -dmg
$(PYTHON) $(OSX_DEPLOY_SCRIPT) $(OSX_APP) -add-qt-tr $(OSX_QT_TRANSLATIONS) -translations-dir=$(QT_TRANSLATION_DIR) -dmg -fancy $(OSX_FANCY_PLIST) -verbose 2 -volname $(OSX_VOLNAME)
$(OSX_BACKGROUND_IMAGE).png: contrib/macdeploy/$(OSX_BACKGROUND_SVG)
sed 's/PACKAGE_NAME/$(PACKAGE_NAME)/' < "$<" | $(RSVG_CONVERT) -f png -d 36 -p 36 -o $@
@@ -139,7 +127,7 @@ $(OSX_BACKGROUND_IMAGE): $(OSX_BACKGROUND_IMAGE).png $(OSX_BACKGROUND_IMAGE)@2x.
tiffutil -cathidpicheck $^ -out $@
deploydir: $(OSX_DMG)
else !BUILD_DARWIN
else
APP_DIST_DIR=$(top_builddir)/dist
APP_DIST_EXTRAS=$(APP_DIST_DIR)/.background/$(OSX_BACKGROUND_IMAGE) $(APP_DIST_DIR)/.DS_Store $(APP_DIST_DIR)/Applications
@@ -149,11 +137,8 @@ $(APP_DIST_DIR)/Applications:
$(APP_DIST_EXTRAS): $(APP_DIST_DIR)/$(OSX_APP)/Contents/MacOS/Bitcoin-Qt
$(OSX_TEMP_ISO): $(APP_DIST_EXTRAS)
$(XORRISOFS) -D -l -V "$(OSX_VOLNAME)" -no-pad -r -dir-mode 0755 -o $@ $(APP_DIST_DIR) -- $(if $(SOURCE_DATE_EPOCH),-volume_date all_file_dates =$(SOURCE_DATE_EPOCH))
$(OSX_DMG): $(OSX_TEMP_ISO)
$(DMG) dmg "$<" "$@"
$(OSX_DMG): $(APP_DIST_EXTRAS)
$(GENISOIMAGE) -no-cache-inodes -D -l -probe -V "$(OSX_VOLNAME)" -no-pad -r -dir-mode 0755 -apple -o $@ dist
dpi%.$(OSX_BACKGROUND_IMAGE): contrib/macdeploy/$(OSX_BACKGROUND_SVG)
sed 's/PACKAGE_NAME/$(PACKAGE_NAME)/' < "$<" | $(RSVG_CONVERT) -f png -d $* -p $* | $(IMAGEMAGICK_CONVERT) - $@
@@ -162,15 +147,22 @@ $(APP_DIST_DIR)/.background/$(OSX_BACKGROUND_IMAGE): $(OSX_BACKGROUND_IMAGE_DPIF
$(MKDIR_P) $(@D)
$(TIFFCP) -c none $(OSX_BACKGROUND_IMAGE_DPIFILES) $@
$(APP_DIST_DIR)/.DS_Store: $(OSX_DSSTORE_GEN)
$(PYTHON) $< "$@" "$(OSX_VOLNAME)"
$(APP_DIST_DIR)/$(OSX_APP)/Contents/MacOS/Bitcoin-Qt: $(OSX_APP_BUILT) $(OSX_PACKAGING)
INSTALLNAMETOOL=$(INSTALLNAMETOOL) OTOOL=$(OTOOL) STRIP=$(STRIP) $(PYTHON) $(OSX_DEPLOY_SCRIPT) $(OSX_APP) $(OSX_VOLNAME) -translations-dir=$(QT_TRANSLATION_DIR)
INSTALLNAMETOOL=$(INSTALLNAMETOOL) OTOOL=$(OTOOL) STRIP=$(STRIP) $(PYTHON) $(OSX_DEPLOY_SCRIPT) $(OSX_APP) -translations-dir=$(QT_TRANSLATION_DIR) -add-qt-tr $(OSX_QT_TRANSLATIONS) -verbose 2
deploydir: $(APP_DIST_EXTRAS)
endif !BUILD_DARWIN
endif
if TARGET_DARWIN
appbundle: $(OSX_APP_BUILT)
deploy: $(OSX_DMG)
endif
if TARGET_WINDOWS
deploy: $(BITCOIN_WIN_INSTALLER)
endif
$(BITCOIN_QT_BIN): FORCE
$(MAKE) -C src qt/$(@F)
@@ -184,18 +176,9 @@ $(BITCOIN_CLI_BIN): FORCE
$(BITCOIN_TX_BIN): FORCE
$(MAKE) -C src $(@F)
$(BITCOIN_UTIL_BIN): FORCE
$(MAKE) -C src $(@F)
$(BITCOIN_WALLET_BIN): FORCE
$(MAKE) -C src $(@F)
$(BITCOIN_NODE_BIN): FORCE
$(MAKE) -C src $(@F)
$(BITCOIN_GUI_BIN): FORCE
$(MAKE) -C src $(@F)
if USE_LCOV
LCOV_FILTER_PATTERN = \
-p "/usr/local/" \
@@ -343,6 +326,8 @@ EXTRA_DIST += \
CLEANFILES = $(OSX_DMG) $(BITCOIN_WIN_INSTALLER)
.INTERMEDIATE: $(COVERAGE_INFO)
DISTCHECK_CONFIGURE_FLAGS = --enable-man
doc/doxygen/.stamp: doc/Doxyfile FORCE
@@ -367,14 +352,11 @@ clean-local: clean-docs
test-security-check:
if TARGET_DARWIN
$(AM_V_at) CC='$(CC)' $(PYTHON) $(top_srcdir)/contrib/devtools/test-security-check.py TestSecurityChecks.test_MACHO
$(AM_V_at) CC='$(CC)' $(PYTHON) $(top_srcdir)/contrib/devtools/test-symbol-check.py TestSymbolChecks.test_MACHO
$(AM_V_at) $(PYTHON) $(top_srcdir)/contrib/devtools/test-security-check.py TestSecurityChecks.test_MACHO
endif
if TARGET_WINDOWS
$(AM_V_at) CC='$(CC)' $(PYTHON) $(top_srcdir)/contrib/devtools/test-security-check.py TestSecurityChecks.test_PE
$(AM_V_at) CC='$(CC)' $(PYTHON) $(top_srcdir)/contrib/devtools/test-symbol-check.py TestSymbolChecks.test_PE
$(AM_V_at) $(PYTHON) $(top_srcdir)/contrib/devtools/test-security-check.py TestSecurityChecks.test_PE
endif
if TARGET_LINUX
$(AM_V_at) CC='$(CC)' $(PYTHON) $(top_srcdir)/contrib/devtools/test-security-check.py TestSecurityChecks.test_ELF
$(AM_V_at) CC='$(CC)' CPPFILT='$(CPPFILT)' $(PYTHON) $(top_srcdir)/contrib/devtools/test-symbol-check.py TestSymbolChecks.test_ELF
$(AM_V_at) $(PYTHON) $(top_srcdir)/contrib/devtools/test-security-check.py TestSecurityChecks.test_ELF
endif

View File

@@ -3,11 +3,6 @@ Bitcoin Core integration/staging tree
https://bitcoincore.org
For an immediately usable, binary version of the Bitcoin Core software, see
https://bitcoincore.org/en/download/.
Further information about Bitcoin Core is available in the [doc folder](/doc).
What is Bitcoin?
----------------
@@ -17,7 +12,9 @@ with no central authority: managing transactions and issuing money are carried
out collectively by the network. Bitcoin Core is the name of open source
software which enables the use of this currency.
For more information read the original Bitcoin whitepaper.
For more information, as well as an immediately usable, binary version of
the Bitcoin Core software, see https://bitcoincore.org/en/download/, or read the
[original whitepaper](https://bitcoincore.org/bitcoin.pdf).
License
-------
@@ -56,11 +53,10 @@ submit new unit tests for old code. Unit tests can be compiled and run
and extending unit tests can be found in [/src/test/README.md](/src/test/README.md).
There are also [regression and integration tests](/test), written
in Python.
in Python, that are run automatically on the build server.
These tests can be run (if the [test dependencies](/test) are installed) with: `test/functional/test_runner.py`
The CI (Continuous Integration) systems make sure that every pull request is built for Windows, Linux, and macOS,
and that unit/sanity tests are run automatically.
The Travis CI system makes sure that every pull request is built for Windows, Linux, and macOS, and that unit/sanity tests are run automatically.
### Manual Quality Assurance (QA) Testing
@@ -80,3 +76,5 @@ Translations are periodically pulled from Transifex and merged into the git repo
**Important**: We do not accept translation changes as GitHub pull requests because the next
pull from Transifex would automatically overwrite them again.
Translators should also subscribe to the [mailing list](https://groups.google.com/forum/#!forum/bitcoin-translators).

View File

@@ -11,7 +11,7 @@
# Test for the Boost C++ libraries of a particular version (or newer)
#
# If no path to the installed boost library is given the macro searchs
# under /usr, /usr/local, /opt, /opt/local and /opt/homebrew and evaluates the
# under /usr, /usr/local, /opt and /opt/local and evaluates the
# $BOOST_ROOT environment variable. Further documentation is available at
# <http://randspringer.de/boost/index.html>.
#
@@ -151,7 +151,7 @@ AC_DEFUN([_AX_BOOST_BASE_RUNDETECT],[
else
search_libsubdirs="$multiarch_libsubdir $libsubdirs"
fi
for _AX_BOOST_BASE_boost_path_tmp in /usr /usr/local /opt /opt/local /opt/homebrew/; do
for _AX_BOOST_BASE_boost_path_tmp in /usr /usr/local /opt /opt/local ; do
if test -d "$_AX_BOOST_BASE_boost_path_tmp/include/boost" && test -r "$_AX_BOOST_BASE_boost_path_tmp/include/boost" ; then
for libsubdir in $search_libsubdirs ; do
if ls "$_AX_BOOST_BASE_boost_path_tmp/$libsubdir/libboost_"* >/dev/null 2>&1 ; then break; fi
@@ -227,7 +227,7 @@ AC_DEFUN([_AX_BOOST_BASE_RUNDETECT],[
fi
else
if test "x$cross_compiling" != "xyes" ; then
for _AX_BOOST_BASE_boost_path in /usr /usr/local /opt /opt/local /opt/homebrew ; do
for _AX_BOOST_BASE_boost_path in /usr /usr/local /opt /opt/local ; do
if test -d "$_AX_BOOST_BASE_boost_path" && test -r "$_AX_BOOST_BASE_boost_path" ; then
for i in `ls -d $_AX_BOOST_BASE_boost_path/include/boost-* 2>/dev/null`; do
_version_tmp=`echo $i | sed "s#$_AX_BOOST_BASE_boost_path##" | sed 's/\/include\/boost-//' | sed 's/_/./'`

View File

@@ -0,0 +1,121 @@
# ===========================================================================
# https://www.gnu.org/software/autoconf-archive/ax_boost_process.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_BOOST_PROCESS
#
# DESCRIPTION
#
# Test for Process library from the Boost C++ libraries. The macro
# requires a preceding call to AX_BOOST_BASE. Further documentation is
# available at <http://randspringer.de/boost/index.html>.
#
# This macro calls:
#
# AC_SUBST(BOOST_PROCESS_LIB)
#
# And sets:
#
# HAVE_BOOST_PROCESS
#
# LICENSE
#
# Copyright (c) 2008 Thomas Porschberg <thomas@randspringer.de>
# Copyright (c) 2008 Michael Tindal
# Copyright (c) 2008 Daniel Casimiro <dan.casimiro@gmail.com>
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice
# and this notice are preserved. This file is offered as-is, without any
# warranty.
#serial 2
AC_DEFUN([AX_BOOST_PROCESS],
[
AC_ARG_WITH([boost-process],
AS_HELP_STRING([--with-boost-process@<:@=special-lib@:>@],
[use the Process library from boost - it is possible to specify a certain library for the linker
e.g. --with-boost-process=boost_process-gcc-mt ]),
[
if test "$withval" = "no"; then
want_boost_process="no"
elif test "$withval" = "yes"; then
want_boost_process="yes"
ax_boost_user_process_lib=""
else
want_boost_process="yes"
ax_boost_user_process_lib="$withval"
fi
],
[want_boost_process="yes"]
)
if test "x$want_boost_process" = "xyes"; then
AC_REQUIRE([AC_PROG_CC])
AC_REQUIRE([AC_CANONICAL_BUILD])
CPPFLAGS_SAVED="$CPPFLAGS"
CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS"
export CPPFLAGS
LDFLAGS_SAVED="$LDFLAGS"
LDFLAGS="$LDFLAGS $BOOST_LDFLAGS"
export LDFLAGS
AC_CACHE_CHECK(whether the Boost::Process library is available,
ax_cv_boost_process,
[AC_LANG_PUSH([C++])
CXXFLAGS_SAVE=$CXXFLAGS
CXXFLAGS=
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include <boost/process.hpp>]],
[[boost::process::child* child = new boost::process::child; delete child;]])],
ax_cv_boost_process=yes, ax_cv_boost_process=no)
CXXFLAGS=$CXXFLAGS_SAVE
AC_LANG_POP([C++])
])
if test "x$ax_cv_boost_process" = "xyes"; then
AC_SUBST(BOOST_CPPFLAGS)
AC_DEFINE(HAVE_BOOST_PROCESS,,[define if the Boost::Process library is available])
BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\/@:>@*//'`
LDFLAGS_SAVE=$LDFLAGS
if test "x$ax_boost_user_process_lib" = "x"; then
for libextension in `ls -r $BOOSTLIBDIR/libboost_process* 2>/dev/null | sed 's,.*/lib,,' | sed 's,\..*,,'` ; do
ax_lib=${libextension}
AC_CHECK_LIB($ax_lib, exit,
[BOOST_PROCESS_LIB="-l$ax_lib"; AC_SUBST(BOOST_PROCESS_LIB) link_process="yes"; break],
[link_process="no"])
done
if test "x$link_process" != "xyes"; then
for libextension in `ls -r $BOOSTLIBDIR/boost_process* 2>/dev/null | sed 's,.*/,,' | sed -e 's,\..*,,'` ; do
ax_lib=${libextension}
AC_CHECK_LIB($ax_lib, exit,
[BOOST_PROCESS_LIB="-l$ax_lib"; AC_SUBST(BOOST_PROCESS_LIB) link_process="yes"; break],
[link_process="no"])
done
fi
else
for ax_lib in $ax_boost_user_process_lib boost_process-$ax_boost_user_process_lib; do
AC_CHECK_LIB($ax_lib, exit,
[BOOST_PROCESS_LIB="-l$ax_lib"; AC_SUBST(BOOST_PROCESS_LIB) link_process="yes"; break],
[link_process="no"])
done
fi
if test "x$ax_lib" = "x"; then
AC_MSG_ERROR(Could not find a version of the Boost::Process library!)
fi
if test "x$link_process" = "xno"; then
AC_MSG_ERROR(Could not link against $ax_lib !)
fi
fi
CPPFLAGS="$CPPFLAGS_SAVED"
LDFLAGS="$LDFLAGS_SAVED"
fi
])

View File

@@ -0,0 +1,187 @@
# ===========================================================================
# https://www.gnu.org/software/autoconf-archive/ax_boost_thread.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_BOOST_THREAD
#
# DESCRIPTION
#
# Test for Thread library from the Boost C++ libraries. The macro requires
# a preceding call to AX_BOOST_BASE. Further documentation is available at
# <http://randspringer.de/boost/index.html>.
#
# This macro calls:
#
# AC_SUBST(BOOST_THREAD_LIB)
#
# And sets:
#
# HAVE_BOOST_THREAD
#
# LICENSE
#
# Copyright (c) 2009 Thomas Porschberg <thomas@randspringer.de>
# Copyright (c) 2009 Michael Tindal
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice
# and this notice are preserved. This file is offered as-is, without any
# warranty.
#serial 33
AC_DEFUN([AX_BOOST_THREAD],
[
AC_ARG_WITH([boost-thread],
AS_HELP_STRING([--with-boost-thread@<:@=special-lib@:>@],
[use the Thread library from boost -
it is possible to specify a certain library for the linker
e.g. --with-boost-thread=boost_thread-gcc-mt ]),
[
if test "$withval" = "yes"; then
want_boost="yes"
ax_boost_user_thread_lib=""
else
want_boost="yes"
ax_boost_user_thread_lib="$withval"
fi
],
[want_boost="yes"]
)
if test "x$want_boost" = "xyes"; then
AC_REQUIRE([AC_PROG_CC])
AC_REQUIRE([AC_CANONICAL_BUILD])
CPPFLAGS_SAVED="$CPPFLAGS"
CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS"
export CPPFLAGS
LDFLAGS_SAVED="$LDFLAGS"
LDFLAGS="$LDFLAGS $BOOST_LDFLAGS"
export LDFLAGS
AC_CACHE_CHECK(whether the Boost::Thread library is available,
ax_cv_boost_thread,
[AC_LANG_PUSH([C++])
CXXFLAGS_SAVE=$CXXFLAGS
case "x$host_os" in
xsolaris )
CXXFLAGS="-pthreads $CXXFLAGS"
break;
;;
xmingw32 )
CXXFLAGS="-mthreads $CXXFLAGS"
break;
;;
*android* )
break;
;;
* )
CXXFLAGS="-pthread $CXXFLAGS"
break;
;;
esac
AC_COMPILE_IFELSE([
AC_LANG_PROGRAM(
[[@%:@include <boost/thread/thread.hpp>]],
[[boost::thread_group thrds;
return 0;]])],
ax_cv_boost_thread=yes, ax_cv_boost_thread=no)
CXXFLAGS=$CXXFLAGS_SAVE
AC_LANG_POP([C++])
])
if test "x$ax_cv_boost_thread" = "xyes"; then
case "x$host_os" in
xsolaris )
BOOST_CPPFLAGS="-pthreads $BOOST_CPPFLAGS"
break;
;;
xmingw32 )
BOOST_CPPFLAGS="-mthreads $BOOST_CPPFLAGS"
break;
;;
*android* )
break;
;;
* )
BOOST_CPPFLAGS="-pthread $BOOST_CPPFLAGS"
break;
;;
esac
AC_SUBST(BOOST_CPPFLAGS)
AC_DEFINE(HAVE_BOOST_THREAD,,
[define if the Boost::Thread library is available])
BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\/@:>@*//'`
LDFLAGS_SAVE=$LDFLAGS
case "x$host_os" in
*bsd* )
LDFLAGS="-pthread $LDFLAGS"
break;
;;
esac
if test "x$ax_boost_user_thread_lib" = "x"; then
for libextension in `ls -r $BOOSTLIBDIR/libboost_thread* 2>/dev/null | sed 's,.*/lib,,' | sed 's,\..*,,'`; do
ax_lib=${libextension}
AC_CHECK_LIB($ax_lib, exit,
[link_thread="yes"; break],
[link_thread="no"])
done
if test "x$link_thread" != "xyes"; then
for libextension in `ls -r $BOOSTLIBDIR/boost_thread* 2>/dev/null | sed 's,.*/,,' | sed 's,\..*,,'`; do
ax_lib=${libextension}
AC_CHECK_LIB($ax_lib, exit,
[link_thread="yes"; break],
[link_thread="no"])
done
fi
else
for ax_lib in $ax_boost_user_thread_lib boost_thread-$ax_boost_user_thread_lib; do
AC_CHECK_LIB($ax_lib, exit,
[link_thread="yes"; break],
[link_thread="no"])
done
fi
if test "x$ax_lib" = "x"; then
AC_MSG_ERROR(Could not find a version of the Boost::Thread library!)
fi
if test "x$link_thread" = "xno"; then
AC_MSG_ERROR(Could not link against $ax_lib !)
else
BOOST_THREAD_LIB="-l$ax_lib"
case "x$host_os" in
*bsd* )
BOOST_LDFLAGS="-pthread $BOOST_LDFLAGS"
break;
;;
xsolaris )
BOOST_THREAD_LIB="$BOOST_THREAD_LIB -lpthread"
break;
;;
xmingw32 )
break;
;;
*android* )
break;
;;
* )
BOOST_THREAD_LIB="$BOOST_THREAD_LIB -lpthread"
break;
;;
esac
AC_SUBST(BOOST_THREAD_LIB)
fi
fi
CPPFLAGS="$CPPFLAGS_SAVED"
LDFLAGS="$LDFLAGS_SAVED"
fi
])

View File

@@ -0,0 +1,223 @@
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_gcc_func_attribute.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_GCC_FUNC_ATTRIBUTE(ATTRIBUTE)
#
# DESCRIPTION
#
# This macro checks if the compiler supports one of GCC's function
# attributes; many other compilers also provide function attributes with
# the same syntax. Compiler warnings are used to detect supported
# attributes as unsupported ones are ignored by default so quieting
# warnings when using this macro will yield false positives.
#
# The ATTRIBUTE parameter holds the name of the attribute to be checked.
#
# If ATTRIBUTE is supported define HAVE_FUNC_ATTRIBUTE_<ATTRIBUTE>.
#
# The macro caches its result in the ax_cv_have_func_attribute_<attribute>
# variable.
#
# The macro currently supports the following function attributes:
#
# alias
# aligned
# alloc_size
# always_inline
# artificial
# cold
# const
# constructor
# constructor_priority for constructor attribute with priority
# deprecated
# destructor
# dllexport
# dllimport
# error
# externally_visible
# flatten
# format
# format_arg
# gnu_inline
# hot
# ifunc
# leaf
# malloc
# noclone
# noinline
# nonnull
# noreturn
# nothrow
# optimize
# pure
# unused
# used
# visibility
# warning
# warn_unused_result
# weak
# weakref
#
# Unsuppored function attributes will be tested with a prototype returning
# an int and not accepting any arguments and the result of the check might
# be wrong or meaningless so use with care.
#
# LICENSE
#
# Copyright (c) 2013 Gabriele Svelto <gabriele.svelto@gmail.com>
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice
# and this notice are preserved. This file is offered as-is, without any
# warranty.
#serial 3
AC_DEFUN([AX_GCC_FUNC_ATTRIBUTE], [
AS_VAR_PUSHDEF([ac_var], [ax_cv_have_func_attribute_$1])
AC_CACHE_CHECK([for __attribute__(($1))], [ac_var], [
AC_LINK_IFELSE([AC_LANG_PROGRAM([
m4_case([$1],
[alias], [
int foo( void ) { return 0; }
int bar( void ) __attribute__(($1("foo")));
],
[aligned], [
int foo( void ) __attribute__(($1(32)));
],
[alloc_size], [
void *foo(int a) __attribute__(($1(1)));
],
[always_inline], [
inline __attribute__(($1)) int foo( void ) { return 0; }
],
[artificial], [
inline __attribute__(($1)) int foo( void ) { return 0; }
],
[cold], [
int foo( void ) __attribute__(($1));
],
[const], [
int foo( void ) __attribute__(($1));
],
[constructor_priority], [
int foo( void ) __attribute__((__constructor__(65535/2)));
],
[constructor], [
int foo( void ) __attribute__(($1));
],
[deprecated], [
int foo( void ) __attribute__(($1("")));
],
[destructor], [
int foo( void ) __attribute__(($1));
],
[dllexport], [
__attribute__(($1)) int foo( void ) { return 0; }
],
[dllimport], [
int foo( void ) __attribute__(($1));
],
[error], [
int foo( void ) __attribute__(($1("")));
],
[externally_visible], [
int foo( void ) __attribute__(($1));
],
[flatten], [
int foo( void ) __attribute__(($1));
],
[format], [
int foo(const char *p, ...) __attribute__(($1(printf, 1, 2)));
],
[format_arg], [
char *foo(const char *p) __attribute__(($1(1)));
],
[gnu_inline], [
inline __attribute__(($1)) int foo( void ) { return 0; }
],
[hot], [
int foo( void ) __attribute__(($1));
],
[ifunc], [
int my_foo( void ) { return 0; }
static int (*resolve_foo(void))(void) { return my_foo; }
int foo( void ) __attribute__(($1("resolve_foo")));
],
[leaf], [
__attribute__(($1)) int foo( void ) { return 0; }
],
[malloc], [
void *foo( void ) __attribute__(($1));
],
[noclone], [
int foo( void ) __attribute__(($1));
],
[noinline], [
__attribute__(($1)) int foo( void ) { return 0; }
],
[nonnull], [
int foo(char *p) __attribute__(($1(1)));
],
[noreturn], [
void foo( void ) __attribute__(($1));
],
[nothrow], [
int foo( void ) __attribute__(($1));
],
[optimize], [
__attribute__(($1(3))) int foo( void ) { return 0; }
],
[pure], [
int foo( void ) __attribute__(($1));
],
[unused], [
int foo( void ) __attribute__(($1));
],
[used], [
int foo( void ) __attribute__(($1));
],
[visibility], [
int foo_def( void ) __attribute__(($1("default")));
int foo_hid( void ) __attribute__(($1("hidden")));
int foo_int( void ) __attribute__(($1("internal")));
int foo_pro( void ) __attribute__(($1("protected")));
],
[warning], [
int foo( void ) __attribute__(($1("")));
],
[warn_unused_result], [
int foo( void ) __attribute__(($1));
],
[weak], [
int foo( void ) __attribute__(($1));
],
[weakref], [
static int foo( void ) { return 0; }
static int bar( void ) __attribute__(($1("foo")));
],
[
m4_warn([syntax], [Unsupported attribute $1, the test may fail])
int foo( void ) __attribute__(($1));
]
)], [])
],
dnl GCC doesn't exit with an error if an unknown attribute is
dnl provided but only outputs a warning, so accept the attribute
dnl only if no warning were issued.
[AS_IF([test -s conftest.err],
[AS_VAR_SET([ac_var], [no])],
[AS_VAR_SET([ac_var], [yes])])],
[AS_VAR_SET([ac_var], [no])])
])
AS_IF([test yes = AS_VAR_GET([ac_var])],
[AC_DEFINE_UNQUOTED(AS_TR_CPP(HAVE_FUNC_ATTRIBUTE_$1), 1,
[Define to 1 if the system has the `$1' function attribute])], [])
AS_VAR_POPDEF([ac_var])
])

View File

@@ -6,9 +6,7 @@ AC_DEFUN([BITCOIN_FIND_BDB48],[
AC_ARG_VAR(BDB_CFLAGS, [C compiler flags for BerkeleyDB, bypasses autodetection])
AC_ARG_VAR(BDB_LIBS, [Linker flags for BerkeleyDB, bypasses autodetection])
if test "x$use_bdb" = "xno"; then
use_bdb=no
elif test "x$BDB_CFLAGS" = "x"; then
if test "x$BDB_CFLAGS" = "x"; then
AC_MSG_CHECKING([for Berkeley DB C++ headers])
BDB_CPPFLAGS=
bdbpath=X
@@ -46,30 +44,25 @@ AC_DEFUN([BITCOIN_FIND_BDB48],[
],[])
done
if test "x$bdbpath" = "xX"; then
use_bdb=no
AC_MSG_RESULT([no])
AC_MSG_ERROR([libdb_cxx headers missing, ]AC_PACKAGE_NAME[ requires this library for BDB wallet support (--without-bdb to disable BDB wallet support)])
AC_MSG_ERROR([libdb_cxx headers missing, ]AC_PACKAGE_NAME[ requires this library for wallet functionality (--disable-wallet to disable wallet functionality)])
elif test "x$bdb48path" = "xX"; then
BITCOIN_SUBDIR_TO_INCLUDE(BDB_CPPFLAGS,[${bdbpath}],db_cxx)
AC_ARG_WITH([incompatible-bdb],[AS_HELP_STRING([--with-incompatible-bdb], [allow using a bdb version other than 4.8])],[
AC_MSG_WARN([Found Berkeley DB other than 4.8; BDB wallets opened by this build will not be portable!])
AC_MSG_WARN([Found Berkeley DB other than 4.8; wallets opened by this build will not be portable!])
],[
AC_MSG_ERROR([Found Berkeley DB other than 4.8, required for portable BDB wallets (--with-incompatible-bdb to ignore or --without-bdb to disable BDB wallet support)])
AC_MSG_ERROR([Found Berkeley DB other than 4.8, required for portable wallets (--with-incompatible-bdb to ignore or --disable-wallet to disable wallet functionality)])
])
use_bdb=yes
else
BITCOIN_SUBDIR_TO_INCLUDE(BDB_CPPFLAGS,[${bdb48path}],db_cxx)
bdbpath="${bdb48path}"
use_bdb=yes
fi
else
BDB_CPPFLAGS=${BDB_CFLAGS}
fi
AC_SUBST(BDB_CPPFLAGS)
if test "x$use_bdb" = "xno"; then
use_bdb=no
elif test "x$BDB_LIBS" = "x"; then
if test "x$BDB_LIBS" = "x"; then
# TODO: Ideally this could find the library version and make sure it matches the headers being used
for searchlib in db_cxx-4.8 db_cxx db4_cxx; do
AC_CHECK_LIB([$searchlib],[main],[
@@ -78,12 +71,8 @@ AC_DEFUN([BITCOIN_FIND_BDB48],[
])
done
if test "x$BDB_LIBS" = "x"; then
AC_MSG_ERROR([libdb_cxx missing, ]AC_PACKAGE_NAME[ requires this library for BDB wallet support (--without-bdb to disable BDB wallet support)])
AC_MSG_ERROR([libdb_cxx missing, ]AC_PACKAGE_NAME[ requires this library for wallet functionality (--disable-wallet to disable wallet functionality)])
fi
fi
if test "x$use_bdb" != "xno"; then
AC_SUBST(BDB_LIBS)
AC_DEFINE([USE_BDB], [1], [Define if BDB support should be compiled in])
use_bdb=yes
fi
AC_SUBST(BDB_LIBS)
])

View File

@@ -64,13 +64,6 @@ AC_DEFUN([BITCOIN_QT_INIT],[
],
[bitcoin_qt_want_version=auto])
AS_IF([test "x$with_gui" = xqt5_debug],
[AS_CASE([$host],
[*darwin*], [qt_lib_suffix=_debug],
[*mingw*], [qt_lib_suffix=d],
[qt_lib_suffix= ]); bitcoin_qt_want_version=qt5],
[qt_lib_suffix= ])
AC_ARG_WITH([qt-incdir],[AS_HELP_STRING([--with-qt-incdir=INC_DIR],[specify qt include path (overridden by pkgconfig)])], [qt_include_path=$withval], [])
AC_ARG_WITH([qt-libdir],[AS_HELP_STRING([--with-qt-libdir=LIB_DIR],[specify qt lib path (overridden by pkgconfig)])], [qt_lib_path=$withval], [])
AC_ARG_WITH([qt-plugindir],[AS_HELP_STRING([--with-qt-plugindir=PLUGIN_DIR],[specify qt plugin path (overridden by pkgconfig)])], [qt_plugin_path=$withval], [])
@@ -108,10 +101,13 @@ AC_DEFUN([BITCOIN_QT_CONFIGURE],[
BITCOIN_QT_CHECK([_BITCOIN_QT_FIND_LIBS])
dnl This is ugly and complicated. Yuck. Works as follows:
dnl We check a header to find out whether Qt is built statically.
dnl When Qt is built statically, some plugins must be linked into
dnl the final binary as well. _BITCOIN_QT_CHECK_STATIC_PLUGIN does
dnl a quick link-check and appends the results to QT_LIBS.
dnl For Qt5, we can check a header to find out whether Qt is build
dnl statically. When Qt is built statically, some plugins must be linked into
dnl the final binary as well.
dnl With Qt5, languages moved into core and the WindowsIntegration plugin was
dnl added.
dnl _BITCOIN_QT_CHECK_STATIC_PLUGINS does a quick link-check and appends the
dnl results to QT_LIBS.
BITCOIN_QT_CHECK([
TEMP_CPPFLAGS=$CPPFLAGS
TEMP_CXXFLAGS=$CXXFLAGS
@@ -119,50 +115,24 @@ AC_DEFUN([BITCOIN_QT_CONFIGURE],[
CXXFLAGS="$PIC_FLAGS $CXXFLAGS"
_BITCOIN_QT_IS_STATIC
if test "x$bitcoin_cv_static_qt" = xyes; then
_BITCOIN_QT_CHECK_STATIC_LIBS
if test "x$qt_plugin_path" != x; then
if test -d "$qt_plugin_path/platforms"; then
QT_LIBS="$QT_LIBS -L$qt_plugin_path/platforms"
fi
if test -d "$qt_plugin_path/styles"; then
QT_LIBS="$QT_LIBS -L$qt_plugin_path/styles"
fi
if test -d "$qt_plugin_path/accessible"; then
QT_LIBS="$QT_LIBS -L$qt_plugin_path/accessible"
fi
if test -d "$qt_plugin_path/platforms/android"; then
QT_LIBS="$QT_LIBS -L$qt_plugin_path/platforms/android -lqtfreetype -lEGL"
fi
fi
_BITCOIN_QT_FIND_STATIC_PLUGINS
AC_DEFINE(QT_STATICPLUGIN, 1, [Define this symbol if qt plugins are static])
if test "x$TARGET_OS" != xandroid; then
_BITCOIN_QT_CHECK_STATIC_PLUGIN([QMinimalIntegrationPlugin], [-lqminimal])
_BITCOIN_QT_CHECK_STATIC_PLUGINS([Q_IMPORT_PLUGIN(QMinimalIntegrationPlugin)],[-lqminimal])
AC_DEFINE(QT_QPA_PLATFORM_MINIMAL, 1, [Define this symbol if the minimal qt platform exists])
fi
if test "x$TARGET_OS" = xwindows; then
dnl Linking against wtsapi32 is required. See #17749 and
dnl https://bugreports.qt.io/browse/QTBUG-27097.
AX_CHECK_LINK_FLAG([-lwtsapi32], [QT_LIBS="$QT_LIBS -lwtsapi32"], [AC_MSG_ERROR([could not link against -lwtsapi32])])
_BITCOIN_QT_CHECK_STATIC_PLUGIN([QWindowsIntegrationPlugin], [-lqwindows])
_BITCOIN_QT_CHECK_STATIC_PLUGIN([QWindowsVistaStylePlugin], [-lqwindowsvistastyle])
_BITCOIN_QT_CHECK_STATIC_PLUGINS([Q_IMPORT_PLUGIN(QWindowsIntegrationPlugin)],[-lqwindows])
AC_DEFINE(QT_QPA_PLATFORM_WINDOWS, 1, [Define this symbol if the qt platform is windows])
elif test "x$TARGET_OS" = xlinux; then
dnl workaround for https://bugreports.qt.io/browse/QTBUG-74874
AX_CHECK_LINK_FLAG([-lxcb-shm], [QT_LIBS="$QT_LIBS -lxcb-shm"], [AC_MSG_ERROR([could not link against -lxcb-shm])])
_BITCOIN_QT_CHECK_STATIC_PLUGIN([QXcbIntegrationPlugin], [-lqxcb])
_BITCOIN_QT_CHECK_STATIC_PLUGINS([Q_IMPORT_PLUGIN(QXcbIntegrationPlugin)],[-lqxcb -lxcb-static])
AC_DEFINE(QT_QPA_PLATFORM_XCB, 1, [Define this symbol if the qt platform is xcb])
elif test "x$TARGET_OS" = xdarwin; then
AX_CHECK_LINK_FLAG([[-framework Carbon]],[QT_LIBS="$QT_LIBS -framework Carbon"],[AC_MSG_ERROR(could not link against Carbon framework)])
AX_CHECK_LINK_FLAG([[-framework IOSurface]],[QT_LIBS="$QT_LIBS -framework IOSurface"],[AC_MSG_ERROR(could not link against IOSurface framework)])
AX_CHECK_LINK_FLAG([[-framework Metal]],[QT_LIBS="$QT_LIBS -framework Metal"],[AC_MSG_ERROR(could not link against Metal framework)])
AX_CHECK_LINK_FLAG([[-framework QuartzCore]],[QT_LIBS="$QT_LIBS -framework QuartzCore"],[AC_MSG_ERROR(could not link against QuartzCore framework)])
_BITCOIN_QT_CHECK_STATIC_PLUGIN([QCocoaIntegrationPlugin], [-lqcocoa])
_BITCOIN_QT_CHECK_STATIC_PLUGIN([QMacStylePlugin], [-lqmacstyle])
AX_CHECK_LINK_FLAG([[-framework IOKit]],[QT_LIBS="$QT_LIBS -framework IOKit"],[AC_MSG_ERROR(could not iokit framework)])
_BITCOIN_QT_CHECK_STATIC_PLUGINS([Q_IMPORT_PLUGIN(QCocoaIntegrationPlugin)],[-lqcocoa])
AC_DEFINE(QT_QPA_PLATFORM_COCOA, 1, [Define this symbol if the qt platform is cocoa])
elif test "x$TARGET_OS" = xandroid; then
QT_LIBS="-Wl,--export-dynamic,--undefined=JNI_OnLoad -lqtforandroid -ljnigraphics -landroid -lqtfreetype $QT_LIBS"
QT_LIBS="-Wl,--export-dynamic,--undefined=JNI_OnLoad -lqtforandroid -ljnigraphics -landroid -lqtfreetype -lQt5EglSupport $QT_LIBS"
AC_DEFINE(QT_QPA_PLATFORM_ANDROID, 1, [Define this symbol if the qt platform is android])
fi
fi
@@ -171,7 +141,7 @@ AC_DEFUN([BITCOIN_QT_CONFIGURE],[
])
if test "x$qt_bin_path" = x; then
qt_bin_path="`$PKG_CONFIG --variable=host_bins ${qt_lib_prefix}Core 2>/dev/null`"
qt_bin_path="`$PKG_CONFIG --variable=host_bins Qt5Core 2>/dev/null`"
fi
if test "x$use_hardening" != xno; then
@@ -226,14 +196,13 @@ AC_DEFUN([BITCOIN_QT_CONFIGURE],[
BITCOIN_QT_PATH_PROGS([RCC], [rcc-qt5 rcc5 rcc], $qt_bin_path)
BITCOIN_QT_PATH_PROGS([LRELEASE], [lrelease-qt5 lrelease5 lrelease], $qt_bin_path)
BITCOIN_QT_PATH_PROGS([LUPDATE], [lupdate-qt5 lupdate5 lupdate],$qt_bin_path, yes)
BITCOIN_QT_PATH_PROGS([LCONVERT], [lconvert-qt5 lconvert5 lconvert], $qt_bin_path, yes)
MOC_DEFS='-DHAVE_CONFIG_H -I$(srcdir)'
case $host in
*darwin*)
BITCOIN_QT_CHECK([
MOC_DEFS="${MOC_DEFS} -DQ_OS_MAC"
base_frameworks="-framework Foundation -framework AppKit"
base_frameworks="-framework Foundation -framework ApplicationServices -framework AppKit"
AX_CHECK_LINK_FLAG([[$base_frameworks]],[QT_LIBS="$QT_LIBS $base_frameworks"],[AC_MSG_ERROR(could not find base frameworks)])
])
;;
@@ -260,10 +229,7 @@ AC_DEFUN([BITCOIN_QT_CONFIGURE],[
AC_MSG_ERROR([libQtDBus not found. Install libQtDBus or remove --with-qtdbus.])
fi
if test "x$LUPDATE" = x; then
AC_MSG_WARN([lupdate tool is required to update Qt translations.])
fi
if test "x$LCONVERT" = x; then
AC_MSG_WARN([lconvert tool is required to update Qt translations.])
AC_MSG_WARN([lupdate is required to update qt translations])
fi
],[
bitcoin_enable_qt=no
@@ -286,13 +252,12 @@ AC_DEFUN([BITCOIN_QT_CONFIGURE],[
AC_SUBST(MOC_DEFS)
])
dnl All macros below are internal and should _not_ be used from configure.ac.
dnl All macros below are internal and should _not_ be used from the main
dnl configure.ac.
dnl ----
dnl Internal. Check if the linked version of Qt was built statically.
dnl
dnl _BITCOIN_QT_IS_STATIC
dnl ---------------------
dnl
dnl Internal. Check if the linked version of Qt was built as static libs.
dnl Requires: Qt5.
dnl Requires: INCLUDES and LIBS must be populated as necessary.
dnl Output: bitcoin_cv_static_qt=yes|no
AC_DEFUN([_BITCOIN_QT_IS_STATIC],[
@@ -313,84 +278,78 @@ AC_DEFUN([_BITCOIN_QT_IS_STATIC],[
])
])
dnl Internal. Check if the link-requirements for a static plugin are met.
dnl
dnl _BITCOIN_QT_CHECK_STATIC_PLUGIN(PLUGIN, LIBRARIES)
dnl --------------------------------------------------
dnl
dnl Internal. Check if the link-requirements for static plugins are met.
dnl Requires: INCLUDES and LIBS must be populated as necessary.
dnl Inputs: $1: A static plugin name.
dnl Inputs: $1: A series of Q_IMPORT_PLUGIN().
dnl Inputs: $2: The libraries that resolve $1.
dnl Output: QT_LIBS is prepended or configure exits.
AC_DEFUN([_BITCOIN_QT_CHECK_STATIC_PLUGIN], [
AC_MSG_CHECKING([for $1 ($2)])
AC_DEFUN([_BITCOIN_QT_CHECK_STATIC_PLUGINS],[
AC_MSG_CHECKING(for static Qt plugins: $2)
CHECK_STATIC_PLUGINS_TEMP_LIBS="$LIBS"
LIBS="$2${qt_lib_suffix} $QT_LIBS $LIBS"
LIBS="$2 $QT_LIBS $LIBS"
AC_LINK_IFELSE([AC_LANG_PROGRAM([[
#include <QtPlugin>
Q_IMPORT_PLUGIN($1)
]])],
[AC_MSG_RESULT([yes]); QT_LIBS="$2${qt_lib_suffix} $QT_LIBS"],
[AC_MSG_RESULT([no]); BITCOIN_QT_FAIL([$1 not found.])])
#define QT_STATICPLUGIN
#include <QtPlugin>
$1]],
[[return 0;]])],
[AC_MSG_RESULT(yes); QT_LIBS="$2 $QT_LIBS"],
[AC_MSG_RESULT(no); BITCOIN_QT_FAIL(Could not resolve: $2)])
LIBS="$CHECK_STATIC_PLUGINS_TEMP_LIBS"
])
dnl Internal. Check Qt static libs with PKG_CHECK_MODULES.
dnl
dnl _BITCOIN_QT_CHECK_STATIC_LIBS
dnl -----------------------------
dnl
dnl Outputs: QT_LIBS is prepended.
AC_DEFUN([_BITCOIN_QT_CHECK_STATIC_LIBS], [
PKG_CHECK_MODULES([QT_ACCESSIBILITY], [${qt_lib_prefix}AccessibilitySupport${qt_lib_suffix}], [QT_LIBS="$QT_ACCESSIBILITY_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_DEVICEDISCOVERY], [${qt_lib_prefix}DeviceDiscoverySupport${qt_lib_suffix}], [QT_LIBS="$QT_DEVICEDISCOVERY_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_EDID], [${qt_lib_prefix}EdidSupport${qt_lib_suffix}], [QT_LIBS="$QT_EDID_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_EVENTDISPATCHER], [${qt_lib_prefix}EventDispatcherSupport${qt_lib_suffix}], [QT_LIBS="$QT_EVENTDISPATCHER_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_FB], [${qt_lib_prefix}FbSupport${qt_lib_suffix}], [QT_LIBS="$QT_FB_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_FONTDATABASE], [${qt_lib_prefix}FontDatabaseSupport${qt_lib_suffix}], [QT_LIBS="$QT_FONTDATABASE_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_THEME], [${qt_lib_prefix}ThemeSupport${qt_lib_suffix}], [QT_LIBS="$QT_THEME_LIBS $QT_LIBS"])
if test "x$TARGET_OS" = xlinux; then
PKG_CHECK_MODULES([QT_INPUT], [${qt_lib_prefix}InputSupport], [QT_LIBS="$QT_INPUT_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_SERVICE], [${qt_lib_prefix}ServiceSupport], [QT_LIBS="$QT_SERVICE_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_XCBQPA], [${qt_lib_prefix}XcbQpa], [QT_LIBS="$QT_XCBQPA_LIBS $QT_LIBS"])
elif test "x$TARGET_OS" = xdarwin; then
PKG_CHECK_MODULES([QT_CLIPBOARD], [${qt_lib_prefix}ClipboardSupport${qt_lib_suffix}], [QT_LIBS="$QT_CLIPBOARD_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_GRAPHICS], [${qt_lib_prefix}GraphicsSupport${qt_lib_suffix}], [QT_LIBS="$QT_GRAPHICS_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_SERVICE], [${qt_lib_prefix}ServiceSupport${qt_lib_suffix}], [QT_LIBS="$QT_SERVICE_LIBS $QT_LIBS"])
elif test "x$TARGET_OS" = xwindows; then
PKG_CHECK_MODULES([QT_WINDOWSUIAUTOMATION], [${qt_lib_prefix}WindowsUIAutomationSupport${qt_lib_suffix}], [QT_LIBS="$QT_WINDOWSUIAUTOMATION_LIBS $QT_LIBS"])
elif test "x$TARGET_OS" = xandroid; then
PKG_CHECK_MODULES([QT_EGL], [${qt_lib_prefix}EglSupport], [QT_LIBS="$QT_EGL_LIBS $QT_LIBS"])
fi
dnl Internal. Find paths necessary for linking qt static plugins
dnl Inputs: qt_plugin_path. optional.
dnl Outputs: QT_LIBS is appended
AC_DEFUN([_BITCOIN_QT_FIND_STATIC_PLUGINS],[
if test "x$qt_plugin_path" != x; then
QT_LIBS="$QT_LIBS -L$qt_plugin_path/platforms"
if test -d "$qt_plugin_path/accessible"; then
QT_LIBS="$QT_LIBS -L$qt_plugin_path/accessible"
fi
if test -d "$qt_plugin_path/platforms/android"; then
QT_LIBS="$QT_LIBS -L$qt_plugin_path/platforms/android -lqtfreetype -lEGL"
fi
PKG_CHECK_MODULES([QTFONTDATABASE], [Qt5FontDatabaseSupport], [QT_LIBS="-lQt5FontDatabaseSupport $QT_LIBS"])
PKG_CHECK_MODULES([QTEVENTDISPATCHER], [Qt5EventDispatcherSupport], [QT_LIBS="-lQt5EventDispatcherSupport $QT_LIBS"])
PKG_CHECK_MODULES([QTTHEME], [Qt5ThemeSupport], [QT_LIBS="-lQt5ThemeSupport $QT_LIBS"])
PKG_CHECK_MODULES([QTDEVICEDISCOVERY], [Qt5DeviceDiscoverySupport], [QT_LIBS="-lQt5DeviceDiscoverySupport $QT_LIBS"])
PKG_CHECK_MODULES([QTACCESSIBILITY], [Qt5AccessibilitySupport], [QT_LIBS="-lQt5AccessibilitySupport $QT_LIBS"])
PKG_CHECK_MODULES([QTFB], [Qt5FbSupport], [QT_LIBS="-lQt5FbSupport $QT_LIBS"])
if test "x$TARGET_OS" = xlinux; then
PKG_CHECK_MODULES([QTXCBQPA], [Qt5XcbQpa], [QT_LIBS="$QTXCBQPA_LIBS $QT_LIBS"])
elif test "x$TARGET_OS" = xdarwin; then
PKG_CHECK_MODULES([QTCLIPBOARD], [Qt5ClipboardSupport], [QT_LIBS="-lQt5ClipboardSupport $QT_LIBS"])
PKG_CHECK_MODULES([QTGRAPHICS], [Qt5GraphicsSupport], [QT_LIBS="-lQt5GraphicsSupport $QT_LIBS"])
PKG_CHECK_MODULES([QTCGL], [Qt5CglSupport], [QT_LIBS="-lQt5CglSupport $QT_LIBS"])
fi
fi
])
dnl Internal. Find Qt libraries using pkg-config.
dnl
dnl _BITCOIN_QT_FIND_LIBS
dnl ---------------------
dnl
dnl Outputs: All necessary QT_* variables are set.
dnl Outputs: have_qt_test and have_qt_dbus are set (if applicable) to yes|no.
AC_DEFUN([_BITCOIN_QT_FIND_LIBS],[
BITCOIN_QT_CHECK([
PKG_CHECK_MODULES([QT_CORE], [${qt_lib_prefix}Core${qt_lib_suffix} $qt_version], [QT_INCLUDES="$QT_CORE_CFLAGS $QT_INCLUDES" QT_LIBS="$QT_CORE_LIBS $QT_LIBS"],
[BITCOIN_QT_FAIL([${qt_lib_prefix}Core${qt_lib_suffix} $qt_version not found])])
PKG_CHECK_MODULES([QT_CORE], [${qt_lib_prefix}Core $qt_version], [],
[BITCOIN_QT_FAIL([${qt_lib_prefix}Core $qt_version not found])])
])
BITCOIN_QT_CHECK([
PKG_CHECK_MODULES([QT_GUI], [${qt_lib_prefix}Gui${qt_lib_suffix} $qt_version], [QT_INCLUDES="$QT_GUI_CFLAGS $QT_INCLUDES" QT_LIBS="$QT_GUI_LIBS $QT_LIBS"],
[BITCOIN_QT_FAIL([${qt_lib_prefix}Gui${qt_lib_suffix} $qt_version not found])])
PKG_CHECK_MODULES([QT_GUI], [${qt_lib_prefix}Gui $qt_version], [],
[BITCOIN_QT_FAIL([${qt_lib_prefix}Gui $qt_version not found])])
])
BITCOIN_QT_CHECK([
PKG_CHECK_MODULES([QT_WIDGETS], [${qt_lib_prefix}Widgets${qt_lib_suffix} $qt_version], [QT_INCLUDES="$QT_WIDGETS_CFLAGS $QT_INCLUDES" QT_LIBS="$QT_WIDGETS_LIBS $QT_LIBS"],
[BITCOIN_QT_FAIL([${qt_lib_prefix}Widgets${qt_lib_suffix} $qt_version not found])])
PKG_CHECK_MODULES([QT_WIDGETS], [${qt_lib_prefix}Widgets $qt_version], [],
[BITCOIN_QT_FAIL([${qt_lib_prefix}Widgets $qt_version not found])])
])
BITCOIN_QT_CHECK([
PKG_CHECK_MODULES([QT_NETWORK], [${qt_lib_prefix}Network${qt_lib_suffix} $qt_version], [QT_INCLUDES="$QT_NETWORK_CFLAGS $QT_INCLUDES" QT_LIBS="$QT_NETWORK_LIBS $QT_LIBS"],
[BITCOIN_QT_FAIL([${qt_lib_prefix}Network${qt_lib_suffix} $qt_version not found])])
PKG_CHECK_MODULES([QT_NETWORK], [${qt_lib_prefix}Network $qt_version], [],
[BITCOIN_QT_FAIL([${qt_lib_prefix}Network $qt_version not found])])
])
QT_INCLUDES="$QT_CORE_CFLAGS $QT_GUI_CFLAGS $QT_WIDGETS_CFLAGS $QT_NETWORK_CFLAGS"
QT_LIBS="$QT_CORE_LIBS $QT_GUI_LIBS $QT_WIDGETS_LIBS $QT_NETWORK_LIBS"
BITCOIN_QT_CHECK([
PKG_CHECK_MODULES([QT_TEST], [${qt_lib_prefix}Test${qt_lib_suffix} $qt_version], [QT_TEST_INCLUDES="$QT_TEST_CFLAGS"; have_qt_test=yes], [have_qt_test=no])
PKG_CHECK_MODULES([QT_TEST], [${qt_lib_prefix}Test $qt_version], [QT_TEST_INCLUDES="$QT_TEST_CFLAGS"; have_qt_test=yes], [have_qt_test=no])
if test "x$use_dbus" != xno; then
PKG_CHECK_MODULES([QT_DBUS], [${qt_lib_prefix}DBus $qt_version], [QT_DBUS_INCLUDES="$QT_DBUS_CFLAGS"; have_qt_dbus=yes], [have_qt_dbus=no])
fi

View File

@@ -12,17 +12,8 @@ dnl warranty.
m4_define([_CHECK_ATOMIC_testbody], [[
#include <atomic>
#include <cstdint>
#include <chrono>
using namespace std::chrono_literals;
int main() {
std::atomic<bool> lock{true};
std::atomic_exchange(&lock, false);
std::atomic<std::chrono::seconds> t{0s};
t.store(2s);
std::atomic<int64_t> a{};
int64_t v = 5;

View File

@@ -3,7 +3,7 @@ Building Bitcoin Core with Visual Studio
Introduction
---------------------
Solution and project files to build the Bitcoin Core applications `msbuild` or Visual Studio can be found in the `build_msvc` directory. The build has been tested with Visual Studio 2019 (building with earlier versions of Visual Studio should not be expected to work).
Solution and project files to build the Bitcoin Core applications `msbuild` or Visual Studio can be found in the `build_msvc` directory. The build has been tested with Visual Studio 2017 and 2019.
Building with Visual Studio is an alternative to the Linux based [cross-compiler build](https://github.com/bitcoin/bitcoin/blob/master/doc/build-windows.md).
@@ -27,15 +27,11 @@ Options for installing the dependencies in a Visual Studio compatible manner are
- Download the source code, build each dependency, add the required include paths, link libraries and binary tools to the Visual Studio project files.
- Use [nuget](https://www.nuget.org/) packages with the understanding that any binary files have been compiled by an untrusted third party.
The [external dependencies](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md) required for building are listed in the `build_msvc/vcpkg.json` file. To ensure `msbuild` project files automatically install the `vcpkg` dependencies use:
```
vcpkg integrate install
```
The [external dependencies](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md) required for building are listed in the `build_msvc/vcpkg.json` file. The `msbuild` project files are configured to automatically install the `vcpkg` dependencies.
Qt
---------------------
In order to build Bitcoin Core a static build of Qt is required. The runtime library version (e.g. v142) and platform type (x86 or x64) must also match.
In order to build the Bitcoin Core a static build of Qt is required. The runtime library version (e.g. v141, v142) and platform type (x86 or x64) must also match.
Some prebuilt x64 versions of Qt can be downloaded from [here](https://github.com/sipsorcery/qt_win_binary/releases). Please be aware these downloads are NOT officially sanctioned by Bitcoin Core and are provided for developer convenience only. They should NOT be used for builds that will be used in a production environment or with real funds.
@@ -57,13 +53,19 @@ PS >py -3 msvc-autogen.py
- An optional step is to adjust the settings in the `build_msvc` directory and the `common.init.vcxproj` file. This project file contains settings that are common to all projects such as the runtime library version and target Windows SDK version. The Qt directories can also be set.
- To build from the command line with the Visual Studio 2017 toolchain use:
```
msbuild /m bitcoin.sln /p:Platform=x64 /p:Configuration=Release /p:PlatformToolset=v141 /t:build
```
- To build from the command line with the Visual Studio 2019 toolchain use:
```
msbuild /m bitcoin.sln /p:Platform=x64 /p:Configuration=Release /t:build
```
- Alternatively, open the `build_msvc/bitcoin.sln` file in Visual Studio 2019.
- Alternatively open the `build_msvc/bitcoin.sln` file in Visual Studio.
AppVeyor
---------------------
@@ -75,25 +77,3 @@ For safety reasons the Bitcoin Core .appveyor.yml file has the artifact options
#- 7z a bitcoin-%APPVEYOR_BUILD_VERSION%.zip %APPVEYOR_BUILD_FOLDER%\build_msvc\%platform%\%configuration%\*.exe
#- path: bitcoin-%APPVEYOR_BUILD_VERSION%.zip
```
Security
---------------------
[Base address randomization](https://docs.microsoft.com/en-us/cpp/build/reference/dynamicbase-use-address-space-layout-randomization?view=msvc-160) is used to make Bitcoin Core more secure. When building Bitcoin using the `build_msvc` process base address randomization can be disabled by editing `common.init.vcproj` to change `RandomizedBaseAddress` from `true` to `false` and then rebuilding the project.
To check if `bitcoind` has `RandomizedBaseAddress` enabled or disabled run
```
.\dumpbin.exe /headers src/bitcoind.exe
```
If is it enabled then in the output `Dynamic base` will be listed in the `DLL characteristics` under `OPTIONAL HEADER VALUES` as shown below
```
8160 DLL characteristics
High Entropy Virtual Addresses
Dynamic base
NX compatible
Terminal Server Aware
```
This may not disable all stack randomization as versions of windows employ additional stack randomization protections. These protections must be turned off in the OS configuration.

View File

@@ -55,7 +55,6 @@
<AdditionalIncludeDirectories>$(QtIncludes);%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
<Link>
<SubSystem>Windows</SubSystem>
<AdditionalDependencies>$(QtReleaseLibraries);%(AdditionalDependencies)</AdditionalDependencies>
<AdditionalOptions>/ignore:4206 /LTCG:OFF</AdditionalOptions>
</Link>

View File

@@ -1,37 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="..\common.init.vcxproj" />
<PropertyGroup Label="Globals">
<ProjectGuid>{D3022AF6-AD33-4CE3-B358-87CB6A1B29CF}</ProjectGuid>
</PropertyGroup>
<PropertyGroup Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<OutDir>$(SolutionDir)$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<ItemGroup>
<ClCompile Include="..\..\src\bitcoin-util.cpp" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\libbitcoinconsensus\libbitcoinconsensus.vcxproj">
<Project>{2b384fa8-9ee1-4544-93cb-0d733c25e8ce}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_common\libbitcoin_common.vcxproj">
<Project>{7c87e378-df58-482e-aa2f-1bc129bc19ce}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_crypto\libbitcoin_crypto.vcxproj">
<Project>{6190199c-6cf4-4dad-bfbd-93fa72a760c1}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_util\libbitcoin_util.vcxproj">
<Project>{b53a5535-ee9d-4c6f-9a26-f79ee3bc3754}</Project>
</ProjectReference>
<ProjectReference Include="..\libunivalue\libunivalue.vcxproj">
<Project>{5724ba7d-a09a-4ba8-800b-c4c1561b3d69}</Project>
</ProjectReference>
<ProjectReference Include="..\libsecp256k1\libsecp256k1.vcxproj">
<Project>{bb493552-3b8c-4a8c-bf69-a6e7a51d2ea6}</Project>
</ProjectReference>
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<Import Project="..\common.vcxproj" />
</Project>

View File

@@ -32,8 +32,6 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "bench_bitcoin", "bench_bitc
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "bitcoin-tx", "bitcoin-tx\bitcoin-tx.vcxproj", "{D3022AF6-AD33-4CE3-B358-87CB6A1B29CF}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "bitcoin-util", "bitcoin-util\bitcoin-util.vcxproj", "{D3022AF6-AD33-4CE3-B358-87CB6A1B29CF}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "bitcoin-wallet", "bitcoin-wallet\bitcoin-wallet.vcxproj", "{84DE8790-EDE3-4483-81AC-C32F15E861F4}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libbitcoin_wallet_tool", "libbitcoin_wallet_tool\libbitcoin_wallet_tool.vcxproj", "{F91AC55E-6F5E-4C58-9AC5-B40DB7DEEF93}"

View File

@@ -15,10 +15,13 @@
#define CLIENT_VERSION_IS_RELEASE true
/* Major version */
#define CLIENT_VERSION_MAJOR 22
#define CLIENT_VERSION_MAJOR 0
/* Minor version */
#define CLIENT_VERSION_MINOR 0
#define CLIENT_VERSION_MINOR 21
/* Build revision */
#define CLIENT_VERSION_REVISION 2
/* Copyright holder(s) before %s replacement */
#define COPYRIGHT_HOLDERS "The %s developers"
@@ -30,17 +33,11 @@
#define COPYRIGHT_HOLDERS_SUBSTITUTION "Bitcoin Core"
/* Copyright year */
#define COPYRIGHT_YEAR 2021
#define COPYRIGHT_YEAR 2019
/* Define to 1 to enable wallet functions */
#define ENABLE_WALLET 1
/* Define to 1 to enable BDB wallet */
#define USE_BDB 1
/* Define to 1 to enable SQLite wallet */
#define USE_SQLITE 1
/* Define to 1 to enable ZMQ functions */
#define ENABLE_ZMQ 1
@@ -50,12 +47,15 @@
/* define if the Boost::Filesystem library is available */
#define HAVE_BOOST_FILESYSTEM /**/
/* define if external signer support is enabled (requires Boost::Process) */
#define ENABLE_EXTERNAL_SIGNER /**/
/* define if the Boost::Process library is available */
#define HAVE_BOOST_PROCESS /**/
/* define if the Boost::System library is available */
#define HAVE_BOOST_SYSTEM /**/
/* define if the Boost::Thread library is available */
#define HAVE_BOOST_THREAD /**/
/* define if the Boost::Unit_Test_Framework library is available */
#define HAVE_BOOST_UNIT_TEST_FRAMEWORK /**/
@@ -92,9 +92,9 @@
don't. */
#define HAVE_DECL_BSWAP_64 0
/* Define to 1 if you have the declaration of `fork', and to 0 if you don't.
/* Define to 1 if you have the declaration of `daemon', and to 0 if you don't.
*/
#define HAVE_DECL_FORK 0
#define HAVE_DECL_DAEMON 0
/* Define to 1 if you have the declaration of `htobe16', and to 0 if you
don't. */
@@ -132,10 +132,6 @@
don't. */
#define HAVE_DECL_LE64TOH 0
/* Define to 1 if you have the declaration of `setsid', and to 0 if you don't.
*/
#define HAVE_DECL_SETSID 0
/* Define to 1 if you have the declaration of `strerror_r', and to 0 if you
don't. */
#define HAVE_DECL_STRERROR_R 0
@@ -181,6 +177,9 @@
/* Define to 1 if you have the <miniupnpc/miniupnpc.h> header file. */
#define HAVE_MINIUPNPC_MINIUPNPC_H 1
/* Define to 1 if you have the <miniupnpc/miniwget.h> header file. */
#define HAVE_MINIUPNPC_MINIWGET_H 1
/* Define to 1 if you have the <miniupnpc/upnpcommands.h> header file. */
#define HAVE_MINIUPNPC_UPNPCOMMANDS_H 1
@@ -254,7 +253,7 @@
#define PACKAGE_NAME "Bitcoin Core"
/* Define to the full name and version of this package. */
#define PACKAGE_STRING "Bitcoin Core 22.0.0"
#define PACKAGE_STRING "Bitcoin Core 0.21.2"
/* Define to the one symbol short name of this package. */
#define PACKAGE_TARNAME "bitcoin"
@@ -263,7 +262,7 @@
#define PACKAGE_URL "https://bitcoincore.org/"
/* Define to the version of this package. */
#define PACKAGE_VERSION "22.0.0"
#define PACKAGE_VERSION "0.21.2"
/* Define to necessary symbol if this constant uses a non-standard name on
your system. */

View File

@@ -10,9 +10,6 @@
</PropertyGroup>
<ItemGroup>
<ClCompile Include="..\..\src\bitcoind.cpp" />
<ClCompile Include="..\..\src\init\bitcoind.cpp">
<ObjectFileName>$(IntDir)init_bitcoind.obj</ObjectFileName>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\libbitcoinconsensus\libbitcoinconsensus.vcxproj">
@@ -65,10 +62,6 @@
Replace="@EXEEXT@" By=".exe"></ReplaceInFile>
<ReplaceInFile FilePath="$(ConfigIniOut)"
Replace="@ENABLE_WALLET_TRUE@" By=""></ReplaceInFile>
<ReplaceInFile FilePath="$(ConfigIniOut)"
Replace="@USE_BDB_TRUE@" By=""></ReplaceInFile>
<ReplaceInFile FilePath="$(ConfigIniOut)"
Replace="@USE_SQLITE_TRUE@" By=""></ReplaceInFile>
<ReplaceInFile FilePath="$(ConfigIniOut)"
Replace="@BUILD_BITCOIN_CLI_TRUE@" By=""></ReplaceInFile>
<ReplaceInFile FilePath="$(ConfigIniOut)"

View File

@@ -96,8 +96,8 @@
<ClCompile>
<WarningLevel>Level3</WarningLevel>
<PrecompiledHeader>NotUsing</PrecompiledHeader>
<AdditionalOptions>/utf-8 /Zc:__cplusplus /std:c++17 %(AdditionalOptions)</AdditionalOptions>
<DisableSpecificWarnings>4018;4244;4267;4334;4715;4805;4834</DisableSpecificWarnings>
<AdditionalOptions>/utf-8 /std:c++17 %(AdditionalOptions)</AdditionalOptions>
<DisableSpecificWarnings>4018;4221;4244;4267;4334;4715;4805;4834</DisableSpecificWarnings>
<TreatWarningAsError>true</TreatWarningAsError>
<PreprocessorDefinitions>_SILENCE_CXX17_CODECVT_HEADER_DEPRECATION_WARNING;_SILENCE_CXX17_OLD_ALLOCATOR_MEMBERS_DEPRECATION_WARNING;ZMQ_STATIC;NOMINMAX;WIN32;HAVE_CONFIG_H;_CRT_SECURE_NO_WARNINGS;_SCL_SECURE_NO_WARNINGS;_CONSOLE;_WIN32_WINNT=0x0601;_WIN32_IE=0x0501;WIN32_LEAN_AND_MEAN;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>..\..\src;..\..\src\univalue\include;..\..\src\secp256k1\include;..\..\src\leveldb\include;..\..\src\leveldb\helpers\memenv;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
@@ -105,7 +105,6 @@
<Link>
<SubSystem>Console</SubSystem>
<AdditionalDependencies>Iphlpapi.lib;ws2_32.lib;Shlwapi.lib;kernel32.lib;user32.lib;gdi32.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies)</AdditionalDependencies>
<RandomizedBaseAddress>true</RandomizedBaseAddress>
</Link>
<Lib>
<AdditionalOptions>/ignore:4221</AdditionalOptions>

View File

@@ -2,15 +2,15 @@
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup Label="QtGlobals">
<QtBaseDir>C:\Qt5.12.11_x64_static_vs2019_16101</QtBaseDir>
<QtBaseDir>C:\Qt5.9.8_x64_static_vs2019</QtBaseDir>
<QtPluginsLibraryDir>$(QtBaseDir)\plugins</QtPluginsLibraryDir>
<QtLibraryDir>$(QtBaseDir)\lib</QtLibraryDir>
<QtIncludeDir>$(QtBaseDir)\include</QtIncludeDir>
<QtIncludes>$(QtIncludeDir);$(QtIncludeDir)\QtNetwork;$(QtIncludeDir)\QtCore;$(QtIncludeDir)\QtWidgets;$(QtIncludeDir)\QtGui;</QtIncludes>
<GeneratedFilesOutDir>.\QtGeneratedFiles\qt</GeneratedFilesOutDir>
<QtToolsDir>$(QtBaseDir)\bin</QtToolsDir>
<QtReleaseLibraries>$(QtPluginsLibraryDir)\platforms\qminimal.lib;$(QtPluginsLibraryDir)\platforms\qwindows.lib;$(QtPluginsLibraryDir)\styles\qwindowsvistastyle.lib;$(QtLibraryDir)\Qt5WindowsUIAutomationSupport.lib;$(QtLibraryDir)\qtfreetype.lib;$(QtLibraryDir)\qtharfbuzz.lib;$(QtLibraryDir)\qtlibpng.lib;$(QtLibraryDir)\qtpcre2.lib;$(QtLibraryDir)\Qt5AccessibilitySupport.lib;$(QtLibraryDir)\Qt5Core.lib;$(QtLibraryDir)\Qt5Concurrent.lib;$(QtLibraryDir)\Qt5EventDispatcherSupport.lib;$(QtLibraryDir)\Qt5FontDatabaseSupport.lib;$(QtLibraryDir)\Qt5Gui.lib;$(QtLibraryDir)\Qt5Network.lib;$(QtLibraryDir)\Qt5PlatformCompositorSupport.lib;$(QtLibraryDir)\Qt5ThemeSupport.lib;$(QtLibraryDir)\Qt5Widgets.lib;$(QtLibraryDir)\Qt5WinExtras.lib;$(QtLibraryDir)\qtmain.lib;Wtsapi32.lib;userenv.lib;netapi32.lib;imm32.lib;Dwmapi.lib;version.lib;winmm.lib;UxTheme.lib</QtReleaseLibraries>
<QtDebugLibraries>$(QtPluginsLibraryDir)\platforms\qwindowsd.lib;$(QtPluginsLibraryDir)\platforms\qminimald.lib;$(QtPluginsLibraryDir)\styles\qwindowsvistastyled.lib;$(QtLibraryDir)\*d.lib;Wtsapi32.lib;crypt32.lib;userenv.lib;netapi32.lib;imm32.lib;Dwmapi.lib;version.lib;winmm.lib;UxTheme.lib</QtDebugLibraries>
<QtReleaseLibraries>$(QtPluginsLibraryDir)\platforms\qminimal.lib;$(QtPluginsLibraryDir)\platforms\qwindows.lib;$(QtLibraryDir)\qtfreetype.lib;$(QtLibraryDir)\qtharfbuzz.lib;$(QtLibraryDir)\qtlibpng.lib;$(QtLibraryDir)\qtpcre2.lib;$(QtLibraryDir)\Qt5AccessibilitySupport.lib;$(QtLibraryDir)\Qt5Core.lib;$(QtLibraryDir)\Qt5Concurrent.lib;$(QtLibraryDir)\Qt5EventDispatcherSupport.lib;$(QtLibraryDir)\Qt5FontDatabaseSupport.lib;$(QtLibraryDir)\Qt5Gui.lib;$(QtLibraryDir)\Qt5Network.lib;$(QtLibraryDir)\Qt5PlatformCompositorSupport.lib;$(QtLibraryDir)\Qt5ThemeSupport.lib;$(QtLibraryDir)\Qt5Widgets.lib;$(QtLibraryDir)\Qt5WinExtras.lib;$(QtLibraryDir)\qtmain.lib;userenv.lib;netapi32.lib;imm32.lib;Dwmapi.lib;version.lib;winmm.lib;UxTheme.lib</QtReleaseLibraries>
<QtDebugLibraries>$(QtPluginsLibraryDir)\platforms\qwindowsd.lib;$(QtPluginsLibraryDir)\platforms\qminimald.lib;$(QtLibraryDir)\*d.lib;crypt32.lib;userenv.lib;netapi32.lib;imm32.lib;Dwmapi.lib;version.lib;winmm.lib;UxTheme.lib</QtDebugLibraries>
</PropertyGroup>
</Project>

View File

@@ -34,7 +34,6 @@
<ClCompile Include="..\..\src\qt\overviewpage.cpp" />
<ClCompile Include="..\..\src\qt\paymentserver.cpp" />
<ClCompile Include="..\..\src\qt\peertablemodel.cpp" />
<ClCompile Include="..\..\src\qt\peertablesortproxy.cpp" />
<ClCompile Include="..\..\src\qt\platformstyle.cpp" />
<ClCompile Include="..\..\src\qt\psbtoperationsdialog.cpp" />
<ClCompile Include="..\..\src\qt\qrimagewidget.cpp" />
@@ -88,7 +87,6 @@
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_overviewpage.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_paymentserver.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_peertablemodel.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_peertablesortproxy.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_platformstyle.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_psbtoperationsdialog.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_qrimagewidget.cpp" />

View File

@@ -8,9 +8,6 @@
<ConfigurationType>StaticLibrary</ConfigurationType>
</PropertyGroup>
<ItemGroup>
<ClCompile Include="..\..\src\wallet\bdb.cpp" />
<ClCompile Include="..\..\src\wallet\salvage.cpp" />
<ClCompile Include="..\..\src\wallet\sqlite.cpp" />
@SOURCE_FILES@
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />

View File

@@ -51,7 +51,7 @@
<ItemDefinitionGroup>
<ClCompile>
<PreprocessorDefinitions>HAVE_CRC32C=0;HAVE_SNAPPY=0;__STDC_LIMIT_MACROS;LEVELDB_IS_BIG_ENDIAN=0;_UNICODE;UNICODE;_CRT_NONSTDC_NO_DEPRECATE;LEVELDB_PLATFORM_WINDOWS;LEVELDB_ATOMIC_PRESENT;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<DisableSpecificWarnings>4244;4267</DisableSpecificWarnings>
<DisableSpecificWarnings>4244;4267;4312;4722;</DisableSpecificWarnings>
<AdditionalIncludeDirectories>..\..\src\leveldb;..\..\src\leveldb\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
</ItemDefinitionGroup>

View File

@@ -10,14 +10,13 @@
<ItemGroup>
<ClCompile Include="..\..\src\secp256k1\src\secp256k1.c" />
</ItemGroup>
<ItemDefinitionGroup>
<ItemDefinitionGroup>
<ClCompile>
<PreprocessorDefinitions>ENABLE_MODULE_ECDH;ENABLE_MODULE_RECOVERY;ENABLE_MODULE_EXTRAKEYS;ENABLE_MODULE_SCHNORRSIG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>..\..\src\secp256k1;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<DisableSpecificWarnings>4146;4244;4267;4334</DisableSpecificWarnings>
</ClCompile>
<AdditionalIncludeDirectories>..\..\src\secp256k1;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
</ItemDefinitionGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<Import Project="..\common.vcxproj" />
</Project>
</Project>

0
build_msvc/msvc-autogen.py Executable file → Normal file
View File

View File

@@ -11,6 +11,7 @@
<ClCompile Include="..\..\src\test\util\setup_common.cpp" />
<ClCompile Include="..\..\src\qt\test\addressbooktests.cpp" />
<ClCompile Include="..\..\src\qt\test\apptests.cpp" />
<ClCompile Include="..\..\src\qt\test\compattests.cpp" />
<ClCompile Include="..\..\src\qt\test\rpcnestedtests.cpp" />
<ClCompile Include="..\..\src\qt\test\test_main.cpp" />
<ClCompile Include="..\..\src\qt\test\uritests.cpp" />
@@ -19,6 +20,7 @@
<ClCompile Include="..\..\src\wallet\test\wallet_test_fixture.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_addressbooktests.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_apptests.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_compattests.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_rpcnestedtests.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_uritests.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_wallettests.cpp" />
@@ -87,6 +89,7 @@
<ItemGroup>
<MocTestFiles Include="..\..\src\qt\test\addressbooktests.h" />
<MocTestFiles Include="..\..\src\qt\test\apptests.h" />
<MocTestFiles Include="..\..\src\qt\test\compattests.h" />
<MocTestFiles Include="..\..\src\qt\test\rpcnestedtests.h" />
<MocTestFiles Include="..\..\src\qt\test\uritests.h" />
<MocTestFiles Include="..\..\src\qt\test\wallettests.h" />

View File

@@ -1,4 +1,4 @@
// Copyright (c) 2018-2020 The Bitcoin Core developers
// Copyright (c) 2018-2019 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
@@ -45,7 +45,7 @@ int main()
stream << vanillaSpendTx;
bitcoinconsensus_error err;
auto op0Result = bitcoinconsensus_verify_script_with_amount(pubKeyScript.data(), pubKeyScript.size(), amount, stream.data(), stream.size(), 0, bitcoinconsensus_SCRIPT_FLAGS_VERIFY_ALL, &err);
auto op0Result = bitcoinconsensus_verify_script_with_amount(pubKeyScript.data(), pubKeyScript.size(), amount, (const unsigned char*)&stream[0], stream.size(), 0, bitcoinconsensus_SCRIPT_FLAGS_VERIFY_ALL, &err);
std::cout << "Op0 result: " << op0Result << ", error code " << err << std::endl;
getchar();

View File

@@ -8,6 +8,7 @@
"boost-process",
"boost-signals2",
"boost-test",
"boost-thread",
"sqlite3",
"double-conversion",
{

View File

@@ -1,22 +1,20 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2020 The Bitcoin Core developers
# Copyright (c) 2018-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C
${CI_RETRY_EXE} apt-get update
${CI_RETRY_EXE} apt-get install -y clang-format-9 python3-pip curl git gawk jq
update-alternatives --install /usr/bin/clang-format clang-format $(which clang-format-9 ) 100
update-alternatives --install /usr/bin/clang-format-diff clang-format-diff $(which clang-format-diff-9) 100
travis_retry sudo apt update && sudo apt install -y clang-format-9
sudo update-alternatives --install /usr/bin/clang-format clang-format $(which clang-format-9 ) 100
sudo update-alternatives --install /usr/bin/clang-format-diff clang-format-diff $(which clang-format-diff-9) 100
${CI_RETRY_EXE} pip3 install codespell==2.0.0
${CI_RETRY_EXE} pip3 install flake8==3.8.3
${CI_RETRY_EXE} pip3 install yq
${CI_RETRY_EXE} pip3 install mypy==0.781
${CI_RETRY_EXE} pip3 install vulture==2.3
travis_retry pip3 install codespell==1.17.1
travis_retry pip3 install flake8==3.8.3
travis_retry pip3 install yq
travis_retry pip3 install mypy==0.781
SHELLCHECK_VERSION=v0.7.2
SHELLCHECK_VERSION=v0.7.1
curl -sL "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION}/shellcheck-${SHELLCHECK_VERSION}.linux.x86_64.tar.xz" | tar --xz -xf - --directory /tmp/
export PATH="/tmp/shellcheck-${SHELLCHECK_VERSION}:${PATH}"

9
ci/lint/05_before_script.sh Executable file
View File

@@ -0,0 +1,9 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C
git fetch --unshallow

View File

@@ -1,33 +1,30 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2020 The Bitcoin Core developers
# Copyright (c) 2018-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C
GIT_HEAD=$(git rev-parse HEAD)
if [ -n "$CIRRUS_PR" ]; then
COMMIT_RANGE="$CIRRUS_BASE_SHA..$GIT_HEAD"
if [ "$TRAVIS_EVENT_TYPE" = "pull_request" ]; then
# TRAVIS_BRANCH will be present in a Travis environment. For builds triggered
# by a pull request this is the name of the branch targeted by the pull request.
# https://docs.travis-ci.com/user/environment-variables/
COMMIT_RANGE="$TRAVIS_BRANCH..HEAD"
test/lint/commit-script-check.sh $COMMIT_RANGE
fi
export COMMIT_RANGE
# This only checks that the trees are pure subtrees, it is not doing a full
# check with -r to not have to fetch all the remotes.
test/lint/git-subtree-check.sh src/crypto/ctaes
test/lint/git-subtree-check.sh src/secp256k1
test/lint/git-subtree-check.sh src/univalue
test/lint/git-subtree-check.sh src/leveldb
test/lint/git-subtree-check.sh src/crc32c
test/lint/check-doc.py
test/lint/check-rpc-mappings.py .
test/lint/lint-all.sh
if [ "$CIRRUS_REPO_FULL_NAME" = "bitcoin/bitcoin" ] && [ -n "$CIRRUS_CRON" ]; then
if [ "$TRAVIS_REPO_SLUG" = "bitcoin/bitcoin" ] && [ "$TRAVIS_EVENT_TYPE" = "cron" ]; then
git log --merges --before="2 days ago" -1 --format='%H' > ./contrib/verify-commits/trusted-sha512-root-commit
${CI_RETRY_EXE} gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys $(<contrib/verify-commits/trusted-keys) &&
travis_retry gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys $(<contrib/verify-commits/trusted-keys) &&
./contrib/verify-commits/verify-commits.py --clean-merge=2;
fi
echo
git log --no-merges --oneline $COMMIT_RANGE

View File

@@ -1,11 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
set -o errexit; source ./ci/test/00_setup_env.sh
set -o errexit; source ./ci/lint/04_install.sh
set -o errexit; source ./ci/lint/06_script.sh

View File

@@ -11,12 +11,6 @@ export LC_ALL=C.UTF-8
# This is where the depends build is done.
BASE_ROOT_DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )"/../../ >/dev/null 2>&1 && pwd )
export BASE_ROOT_DIR
# The depends dir.
# This folder exists on the ci host and ci guest. Changes are propagated back and forth.
export DEPENDS_DIR=${DEPENDS_DIR:-$BASE_ROOT_DIR/depends}
# A folder for the ci system to put temporary files (ccache, datadirs for tests, ...)
# This folder only exists on the ci host.
export BASE_SCRATCH_DIR=${BASE_SCRATCH_DIR:-$BASE_ROOT_DIR/ci/scratch}
echo "Setting specific values in env"
if [ -n "${FILE_ENV}" ]; then
@@ -28,6 +22,9 @@ fi
echo "Fallback to default values in env (if not yet set)"
# The number of parallel jobs to pass down to make and test_runner.py
export MAKEJOBS=${MAKEJOBS:--j4}
# A folder for the ci system to put temporary files (ccache, datadirs for tests, ...)
# This folder only exists on the ci host.
export BASE_SCRATCH_DIR=${BASE_SCRATCH_DIR:-$BASE_ROOT_DIR/ci/scratch}
# What host to compile for. See also ./depends/README.md
# Tests that need cross-compilation export the appropriate HOST.
# Tests that run natively guess the host
@@ -47,7 +44,7 @@ export RUN_FUZZ_TESTS=${RUN_FUZZ_TESTS:-false}
export EXPECTED_TESTS_DURATION_IN_SECONDS=${EXPECTED_TESTS_DURATION_IN_SECONDS:-1000}
export CONTAINER_NAME=${CONTAINER_NAME:-ci_unnamed}
export DOCKER_NAME_TAG=${DOCKER_NAME_TAG:-ubuntu:20.04}
export DOCKER_NAME_TAG=${DOCKER_NAME_TAG:-ubuntu:18.04}
# Randomize test order.
# See https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/utf_reference/rt_param_reference/random.html
export BOOST_TEST_RANDOM=${BOOST_TEST_RANDOM:-1}
@@ -59,13 +56,16 @@ export CCACHE_COMPRESS=${CCACHE_COMPRESS:-1}
# The cache dir.
# This folder exists on the ci host and ci guest. Changes are propagated back and forth.
export CCACHE_DIR=${CCACHE_DIR:-$BASE_SCRATCH_DIR/.ccache}
# The depends dir.
# This folder exists on the ci host and ci guest. Changes are propagated back and forth.
export DEPENDS_DIR=${DEPENDS_DIR:-$BASE_ROOT_DIR/depends}
# Folder where the build result is put (bin and lib).
export BASE_OUTDIR=${BASE_OUTDIR:-$BASE_SCRATCH_DIR/out/$HOST}
# Folder where the build is done (dist and out-of-tree build).
export BASE_BUILD_DIR=${BASE_BUILD_DIR:-$BASE_SCRATCH_DIR/build}
export PREVIOUS_RELEASES_DIR=${PREVIOUS_RELEASES_DIR:-$BASE_ROOT_DIR/releases/$HOST}
export SDK_URL=${SDK_URL:-https://bitcoincore.org/depends-sources/sdks}
export DOCKER_PACKAGES=${DOCKER_PACKAGES:-build-essential libtool autotools-dev automake pkg-config bsdmainutils curl ca-certificates ccache python3 rsync git procps bison}
export DOCKER_PACKAGES=${DOCKER_PACKAGES:-build-essential libtool autotools-dev automake pkg-config bsdmainutils curl ca-certificates ccache python3 rsync git procps}
export GOAL=${GOAL:-install}
export DIR_QA_ASSETS=${DIR_QA_ASSETS:-${BASE_SCRATCH_DIR}/qa-assets}
export PATH=${BASE_ROOT_DIR}/ci/retry:$PATH

View File

@@ -1,25 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=aarch64-linux-android
export PACKAGES="clang llvm unzip openjdk-8-jdk gradle"
export CONTAINER_NAME=ci_android
export DOCKER_NAME_TAG="ubuntu:focal"
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false
export ANDROID_API_LEVEL=28
export ANDROID_BUILD_TOOLS_VERSION=28.0.3
export ANDROID_NDK_VERSION=21.1.6352462
export ANDROID_TOOLS_URL=https://dl.google.com/android/repository/commandlinetools-linux-6609375_latest.zip
export ANDROID_HOME="${DEPENDS_DIR}/SDKs/android"
export ANDROID_NDK_HOME="${ANDROID_HOME}/ndk/${ANDROID_NDK_VERSION}"
export DEP_OPTS="ANDROID_SDK=${ANDROID_HOME} ANDROID_NDK=${ANDROID_NDK_HOME} ANDROID_API_LEVEL=${ANDROID_API_LEVEL} ANDROID_TOOLCHAIN_BIN=${ANDROID_NDK_HOME}/toolchains/llvm/prebuilt/linux-x86_64/bin/"
export BITCOIN_CONFIG="--disable-ccache"

2
ci/test/00_setup_env_arm.sh Executable file → Normal file
View File

@@ -25,4 +25,4 @@ export RUN_FUNCTIONAL_TESTS=false
export GOAL="install"
# -Wno-psabi is to disable ABI warnings: "note: parameter passing for argument of type ... changed in GCC 7.1"
# This could be removed once the ABI change warning does not show up by default
export BITCOIN_CONFIG="--enable-glibc-back-compat --enable-reduce-exports CXXFLAGS=-Wno-psabi"
export BITCOIN_CONFIG="--enable-glibc-back-compat --enable-reduce-exports CXXFLAGS=-Wno-psabi --with-boost-process"

9
ci/test/00_setup_env_i686_centos.sh Executable file → Normal file
View File

@@ -7,10 +7,9 @@
export LC_ALL=C.UTF-8
export HOST=i686-pc-linux-gnu
export CONTAINER_NAME=ci_i686_centos
export DOCKER_NAME_TAG=quay.io/centos/centos:stream8
export DOCKER_PACKAGES="gcc-c++ glibc-devel.x86_64 libstdc++-devel.x86_64 glibc-devel.i686 libstdc++-devel.i686 ccache libtool make git python3 python3-zmq which patch lbzip2 xz procps-ng dash rsync coreutils bison"
export CONTAINER_NAME=ci_i686_centos_7
export DOCKER_NAME_TAG=centos:7
export DOCKER_PACKAGES="gcc-c++ glibc-devel.x86_64 libstdc++-devel.x86_64 glibc-devel.i686 libstdc++-devel.i686 ccache libtool make git python3 python36-zmq which patch lbzip2 dash"
export GOAL="install"
export BITCOIN_CONFIG="--enable-zmq --with-gui=qt5 --enable-reduce-exports"
export BITCOIN_CONFIG="--enable-zmq --with-gui=qt5 --enable-reduce-exports --with-boost-process"
export CONFIG_SHELL="/bin/dash"
export TEST_RUNNER_ENV="LC_ALL=en_US.UTF-8"

12
ci/test/00_setup_env_mac.sh Executable file → Normal file
View File

@@ -7,12 +7,12 @@
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_macos_cross
export DOCKER_NAME_TAG=ubuntu:20.04 # Check that Focal can cross-compile to macos (Focal is used in the gitian build as well)
export HOST=x86_64-apple-darwin18
export PACKAGES="cmake imagemagick librsvg2-bin libz-dev libtiff-tools libtinfo5 python3-setuptools xorriso"
export XCODE_VERSION=12.1
export XCODE_BUILD_ID=12A7403
export DOCKER_NAME_TAG=ubuntu:18.04 # Check that bionic can cross-compile to macos (bionic is used in the gitian build as well)
export HOST=x86_64-apple-darwin16
export PACKAGES="cmake imagemagick libcap-dev librsvg2-bin libz-dev libbz2-dev libtiff-tools python3-dev python3-setuptools"
export XCODE_VERSION=11.3.1
export XCODE_BUILD_ID=11C505
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false
export GOAL="deploy"
export BITCOIN_CONFIG="--with-gui --enable-reduce-exports"
export BITCOIN_CONFIG="--with-gui --enable-reduce-exports --with-boost-process"

7
ci/test/00_setup_env_mac_host.sh Executable file → Normal file
View File

@@ -6,12 +6,13 @@
export LC_ALL=C.UTF-8
export HOST=x86_64-apple-darwin18
export PIP_PACKAGES="zmq lief"
export HOST=x86_64-apple-darwin16
export PIP_PACKAGES="zmq"
export GOAL="install"
export BITCOIN_CONFIG="--with-gui --enable-reduce-exports"
export BITCOIN_CONFIG="--with-gui --enable-reduce-exports --with-boost-process"
export CI_OS_NAME="macos"
export NO_DEPENDS=1
export OSX_SDK=""
export CCACHE_SIZE=300M
export RUN_SECURITY_TESTS="true"

6
ci/test/00_setup_env_native_asan.sh Executable file → Normal file
View File

@@ -7,8 +7,8 @@
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_asan
export PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev bsdmainutils libboost-dev libboost-system-dev libboost-filesystem-dev libboost-test-dev libdb5.3++-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev libqrencode-dev libsqlite3-dev"
export DOCKER_NAME_TAG=ubuntu:22.04
export PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev libqrencode-dev libsqlite3-dev"
export DOCKER_NAME_TAG=ubuntu:20.04
export NO_DEPENDS=1
export GOAL="install"
export BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --with-gui=qt5 CPPFLAGS='-DARENA_DEBUG -DDEBUG_LOCKORDER' --with-sanitizers=address,integer,undefined CC=clang CXX=clang++"
export BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --with-gui=qt5 CPPFLAGS='-DARENA_DEBUG -DDEBUG_LOCKORDER' --with-sanitizers=address,integer,undefined CC=clang CXX=clang++ --with-boost-process"

4
ci/test/00_setup_env_native_fuzz.sh Executable file → Normal file
View File

@@ -8,11 +8,11 @@ export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:20.04"
export CONTAINER_NAME=ci_native_fuzz
export PACKAGES="clang llvm python3 libevent-dev bsdmainutils libboost-dev libboost-system-dev libboost-filesystem-dev libboost-test-dev"
export PACKAGES="clang llvm python3 libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev"
export NO_DEPENDS=1
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false
export RUN_FUZZ_TESTS=true
export GOAL="install"
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer,address,undefined,integer CC=clang CXX=clang++"
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer,address,undefined CC=clang CXX=clang++ --with-boost-process"
export CCACHE_SIZE=200M

View File

@@ -1,24 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:20.04"
LIBCXX_DIR="${BASE_SCRATCH_DIR}/msan/build/"
export MSAN_FLAGS="-fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -g -O1 -fno-optimize-sibling-calls"
LIBCXX_FLAGS="-nostdinc++ -stdlib=libc++ -L${LIBCXX_DIR}lib -lc++abi -I${LIBCXX_DIR}include -I${LIBCXX_DIR}include/c++/v1 -lpthread -Wl,-rpath,${LIBCXX_DIR}lib -Wno-unused-command-line-argument"
export MSAN_AND_LIBCXX_FLAGS="${MSAN_FLAGS} ${LIBCXX_FLAGS}"
export CONTAINER_NAME="ci_native_msan"
export PACKAGES="clang-9 llvm-9 cmake"
export DEP_OPTS="NO_BDB=1 NO_QT=1 CC='clang' CXX='clang++' CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}' boost_cxxflags='-std=c++17 -fvisibility=hidden -fPIC ${MSAN_AND_LIBCXX_FLAGS}' libevent_cflags='${MSAN_FLAGS}' zeromq_cxxflags='-std=c++17 ${MSAN_AND_LIBCXX_FLAGS}'"
export GOAL="install"
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer,memory --with-asm=no --prefix=${DEPENDS_DIR}/x86_64-pc-linux-gnu/ CC=clang CXX=clang++ CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}'"
export USE_MEMORY_SANITIZER="true"
export RUN_UNIT_TESTS="false"
export RUN_FUNCTIONAL_TESTS="false"
export RUN_FUZZ_TESTS=true
export CCACHE_SIZE=250M

2
ci/test/00_setup_env_native_fuzz_with_valgrind.sh Executable file → Normal file
View File

@@ -8,7 +8,7 @@ export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:20.04"
export CONTAINER_NAME=ci_native_fuzz_valgrind
export PACKAGES="clang llvm python3 libevent-dev bsdmainutils libboost-dev libboost-system-dev libboost-filesystem-dev libboost-test-dev valgrind"
export PACKAGES="clang llvm python3 libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev valgrind"
export NO_DEPENDS=1
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false

6
ci/test/00_setup_env_native_msan.sh Executable file → Normal file
View File

@@ -7,7 +7,7 @@
export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:20.04"
LIBCXX_DIR="${BASE_SCRATCH_DIR}/msan/build/"
LIBCXX_DIR="${BASE_ROOT_DIR}/ci/scratch/msan/build/"
export MSAN_FLAGS="-fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -g -O1 -fno-optimize-sibling-calls"
LIBCXX_FLAGS="-nostdinc++ -stdlib=libc++ -L${LIBCXX_DIR}lib -lc++abi -I${LIBCXX_DIR}include -I${LIBCXX_DIR}include/c++/v1 -lpthread -Wl,-rpath,${LIBCXX_DIR}lib -Wno-unused-command-line-argument"
export MSAN_AND_LIBCXX_FLAGS="${MSAN_FLAGS} ${LIBCXX_FLAGS}"
@@ -15,9 +15,9 @@ export BDB_PREFIX="${BASE_ROOT_DIR}/db4"
export CONTAINER_NAME="ci_native_msan"
export PACKAGES="clang-9 llvm-9 cmake"
export DEP_OPTS="NO_BDB=1 NO_QT=1 CC='clang' CXX='clang++' CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}' boost_cxxflags='-std=c++17 -fvisibility=hidden -fPIC ${MSAN_AND_LIBCXX_FLAGS}' libevent_cflags='${MSAN_FLAGS}' sqlite_cflags='${MSAN_FLAGS}' zeromq_cxxflags='-std=c++17 ${MSAN_AND_LIBCXX_FLAGS}'"
export DEP_OPTS="NO_BDB=1 NO_QT=1 CC='clang' CXX='clang++' CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}' boost_cxxflags='-std=c++11 -fvisibility=hidden -fPIC ${MSAN_AND_LIBCXX_FLAGS}' zeromq_cxxflags='-std=c++11 ${MSAN_AND_LIBCXX_FLAGS}'"
export GOAL="install"
export BITCOIN_CONFIG="--enable-wallet --with-sanitizers=memory --with-asm=no --prefix=${DEPENDS_DIR}/x86_64-pc-linux-gnu/ CC=clang CXX=clang++ CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}' BDB_LIBS='-L${BDB_PREFIX}/lib -ldb_cxx-4.8' BDB_CFLAGS='-I${BDB_PREFIX}/include'"
export BITCOIN_CONFIG="--enable-wallet --with-sanitizers=memory --with-asm=no --prefix=${BASE_ROOT_DIR}/depends/x86_64-pc-linux-gnu/ CC=clang CXX=clang++ CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}' BDB_LIBS='-L${BDB_PREFIX}/lib -ldb_cxx-4.8' BDB_CFLAGS='-I${BDB_PREFIX}/include'"
export USE_MEMORY_SANITIZER="true"
export RUN_FUNCTIONAL_TESTS="false"
export CCACHE_SIZE=250M

7
ci/test/00_setup_env_native_multiprocess.sh Executable file → Normal file
View File

@@ -8,9 +8,8 @@ export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_multiprocess
export DOCKER_NAME_TAG=ubuntu:20.04
export PACKAGES="cmake python3 python3-pip llvm clang"
export DEP_OPTS="DEBUG=1 MULTIPROCESS=1"
export PACKAGES="cmake python3"
export DEP_OPTS="MULTIPROCESS=1"
export GOAL="install"
export BITCOIN_CONFIG="--enable-debug CC=clang CXX=clang++" # Use clang to avoid OOM
export BITCOIN_CONFIG="--with-boost-process"
export TEST_RUNNER_ENV="BITCOIND=bitcoin-node"
export PIP_PACKAGES="lief"

6
ci/test/00_setup_env_native_nowallet.sh Executable file → Normal file
View File

@@ -7,8 +7,8 @@
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_nowallet
export DOCKER_NAME_TAG=ubuntu:18.04 # Use bionic to have one config run the tests in python3.6, see doc/dependencies.md
export PACKAGES="python3-zmq clang-5.0 llvm-5.0" # Use clang-5 to test C++17 compatibility, see doc/dependencies.md
export DOCKER_NAME_TAG=ubuntu:16.04 # Use xenial to have one config run the tests in python3.5, see doc/dependencies.md
export PACKAGES="python3-zmq clang-3.8 llvm-3.8" # Use clang-3.8 to test C++11 compatibility, see doc/dependencies.md
export DEP_OPTS="NO_WALLET=1"
export GOAL="install"
export BITCOIN_CONFIG="--enable-glibc-back-compat --enable-reduce-exports CC=clang-5.0 CXX=clang++-5.0"
export BITCOIN_CONFIG="--enable-glibc-back-compat --enable-reduce-exports CC=clang-3.8 CXX=clang++-3.8 --with-boost-process"

10
ci/test/00_setup_env_native_qt5.sh Executable file → Normal file
View File

@@ -7,13 +7,13 @@
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_qt5
export DOCKER_NAME_TAG=ubuntu:18.04 # Check that bionic gcc-7 can compile our c++17 and run our functional tests in python3, see doc/dependencies.md
export DOCKER_NAME_TAG=ubuntu:18.04 # Check that bionic can compile our c++17 and run our functional tests in python3
export PACKAGES="python3-zmq qtbase5-dev qttools5-dev-tools libdbus-1-dev libharfbuzz-dev"
export DEP_OPTS="NO_QT=1 NO_UPNP=1 NO_NATPMP=1 DEBUG=1 ALLOW_HOST_PACKAGES=1"
export DEP_OPTS="NO_QT=1 NO_UPNP=1 DEBUG=1 ALLOW_HOST_PACKAGES=1"
export TEST_RUNNER_EXTRA="--previous-releases --coverage --extended --exclude feature_dbcrash" # Run extended tests so that coverage does not fail, but exclude the very slow dbcrash
export RUN_SECURITY_TESTS="true"
export RUN_UNIT_TESTS_SEQUENTIAL="true"
export RUN_UNIT_TESTS="false"
export GOAL="install"
export PREVIOUS_RELEASES_TO_DOWNLOAD="v0.15.2 v0.16.3 v0.17.2 v0.18.1 v0.19.1 v0.20.1"
export BITCOIN_CONFIG="--enable-zmq --with-libs=no --with-gui=qt5 --enable-glibc-back-compat --enable-reduce-exports
--enable-debug --disable-fuzz-binary CFLAGS=\"-g0 -O2 -funsigned-char\" CXXFLAGS=\"-g0 -O2 -funsigned-char\""
export PREVIOUS_RELEASES_TO_DOWNLOAD="v0.15.2 v0.16.3 v0.17.2 v0.18.1 v0.19.1"
export BITCOIN_CONFIG="--enable-zmq --with-libs=no --with-gui=qt5 --enable-glibc-back-compat --enable-reduce-exports --enable-c++17 --enable-debug CFLAGS=\"-g0 -O2 -funsigned-char\" CXXFLAGS=\"-g0 -O2 -funsigned-char\" --with-boost-process"

5
ci/test/00_setup_env_native_tsan.sh Executable file → Normal file
View File

@@ -7,8 +7,9 @@
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_tsan
export DOCKER_NAME_TAG=ubuntu:22.04
export DOCKER_NAME_TAG=ubuntu:20.04
export PACKAGES="clang llvm libc++abi-dev libc++-dev python3-zmq"
export DEP_OPTS="CC=clang CXX='clang++ -stdlib=libc++'"
export TEST_RUNNER_EXTRA="--exclude feature_block" # Low memory on Travis machines, exclude feature_block.
export GOAL="install"
export BITCOIN_CONFIG="--enable-zmq --with-gui=no CPPFLAGS='-DARENA_DEBUG -DDEBUG_LOCKORDER' CXXFLAGS='-g' --with-sanitizers=thread CC=clang CXX='clang++ -stdlib=libc++'"
export BITCOIN_CONFIG="--enable-zmq --with-gui=no CPPFLAGS='-DARENA_DEBUG -DDEBUG_LOCKORDER' CXXFLAGS='-g' --with-sanitizers=thread CC=clang CXX='clang++ -stdlib=libc++' --with-boost-process"

2
ci/test/00_setup_env_native_valgrind.sh Executable file → Normal file
View File

@@ -7,7 +7,7 @@
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_valgrind
export PACKAGES="valgrind clang llvm python3-zmq libevent-dev bsdmainutils libboost-dev libboost-system-dev libboost-filesystem-dev libboost-test-dev libdb5.3++-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev libsqlite3-dev"
export PACKAGES="valgrind clang llvm python3-zmq libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev libsqlite3-dev"
export USE_VALGRIND=1
export NO_DEPENDS=1
export TEST_RUNNER_EXTRA="--exclude rpc_bind" # Excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547

2
ci/test/00_setup_env_s390x.sh Executable file → Normal file
View File

@@ -23,4 +23,4 @@ export RUN_UNIT_TESTS=true
export TEST_RUNNER_ENV="LC_ALL=C"
export RUN_FUNCTIONAL_TESTS=true
export GOAL="install"
export BITCOIN_CONFIG="--enable-reduce-exports --with-incompatible-bdb"
export BITCOIN_CONFIG="--enable-reduce-exports --with-incompatible-bdb --with-boost-process"

8
ci/test/00_setup_env_win64.sh Executable file → Normal file
View File

@@ -7,13 +7,13 @@
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_win64
export DOCKER_NAME_TAG=ubuntu:20.04 # Check that Focal can cross-compile to win64 (Focal is used in the gitian build as well)
export DOCKER_NAME_TAG=ubuntu:18.04 # Check that bionic can cross-compile to win64 (bionic is used in the gitian build as well)
export HOST=x86_64-w64-mingw32
export DPKG_ADD_ARCH="i386"
export PACKAGES="python3 nsis g++-mingw-w64-x86-64 wine-binfmt wine64 wine32 file"
export PACKAGES="python3 nsis g++-mingw-w64-x86-64 wine-binfmt wine64 file"
export RUN_FUNCTIONAL_TESTS=false
export RUN_SECURITY_TESTS="true"
export GOAL="deploy"
export BITCOIN_CONFIG="--enable-reduce-exports --disable-gui-tests --disable-external-signer"
export BITCOIN_CONFIG="--enable-reduce-exports --disable-gui-tests --without-boost-process"
# Compiler for MinGW-w64 causes false -Wreturn-type warning.
# See https://sourceforge.net/p/mingw-w64/bugs/306/

22
ci/test/03_before_install.sh Executable file
View File

@@ -0,0 +1,22 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
BEGIN_FOLD () {
echo ""
CURRENT_FOLD_NAME=$1
echo "travis_fold:start:${CURRENT_FOLD_NAME}"
}
END_FOLD () {
RET=$?
echo "travis_fold:end:${CURRENT_FOLD_NAME}"
if [ $RET != 0 ]; then
echo "${CURRENT_FOLD_NAME} failed with status code ${RET}"
fi
}

View File

@@ -6,12 +6,14 @@
export LC_ALL=C.UTF-8
if [[ $DOCKER_NAME_TAG == centos* ]]; then
export LC_ALL=en_US.utf8
fi
if [[ $QEMU_USER_CMD == qemu-s390* ]]; then
export LC_ALL=C
fi
if [ "$CI_OS_NAME" == "macos" ]; then
sudo -H pip3 install --upgrade pip
IN_GETOPT_BIN="/usr/local/opt/gnu-getopt/bin/getopt" ${CI_RETRY_EXE} pip3 install --user $PIP_PACKAGES
fi
@@ -34,12 +36,7 @@ if [ -z "$DANGER_RUN_CI_ON_HOST" ]; then
echo "Creating $DOCKER_NAME_TAG container to run in"
${CI_RETRY_EXE} docker pull "$DOCKER_NAME_TAG"
if [ -n "${RESTART_CI_DOCKER_BEFORE_RUN}" ] ; then
echo "Restart docker before run to stop and clear all containers started with --rm"
systemctl restart docker
fi
DOCKER_ID=$(docker run $DOCKER_ADMIN --rm --interactive --detach --tty \
DOCKER_ID=$(docker run $DOCKER_ADMIN -idt \
--mount type=bind,src=$BASE_ROOT_DIR,dst=/ro_base,readonly \
--mount type=bind,src=$CCACHE_DIR,dst=$CCACHE_DIR \
--mount type=bind,src=$DEPENDS_DIR,dst=$DEPENDS_DIR \
@@ -62,15 +59,12 @@ if [ -n "$DPKG_ADD_ARCH" ]; then
DOCKER_EXEC dpkg --add-architecture "$DPKG_ADD_ARCH"
fi
if [[ $DOCKER_NAME_TAG == *centos* ]]; then
${CI_RETRY_EXE} DOCKER_EXEC dnf -y install epel-release
${CI_RETRY_EXE} DOCKER_EXEC dnf -y --allowerasing install $DOCKER_PACKAGES $PACKAGES
if [[ $DOCKER_NAME_TAG == centos* ]]; then
${CI_RETRY_EXE} DOCKER_EXEC yum -y install epel-release
${CI_RETRY_EXE} DOCKER_EXEC yum -y install $DOCKER_PACKAGES $PACKAGES
elif [ "$CI_USE_APT_INSTALL" != "no" ]; then
${CI_RETRY_EXE} DOCKER_EXEC apt-get update
${CI_RETRY_EXE} DOCKER_EXEC apt-get install --no-install-recommends --no-upgrade -y $PACKAGES $DOCKER_PACKAGES
if [ -n "$PIP_PACKAGES" ]; then
${CI_RETRY_EXE} pip3 install --user $PIP_PACKAGES
fi
fi
if [ "$CI_OS_NAME" == "macos" ]; then
@@ -84,14 +78,11 @@ fi
DOCKER_EXEC echo "Free disk space:"
DOCKER_EXEC df -h
if [ "$RUN_FUZZ_TESTS" = "true" ] || [ "$RUN_UNIT_TESTS" = "true" ] || [ "$RUN_UNIT_TESTS_SEQUENTIAL" = "true" ]; then
if [ ! -d ${DIR_QA_ASSETS} ]; then
DOCKER_EXEC git clone --depth=1 https://github.com/bitcoin-core/qa-assets ${DIR_QA_ASSETS}
fi
export DIR_FUZZ_IN=${DIR_QA_ASSETS}/fuzz_seed_corpus/
export DIR_UNIT_TEST_DATA=${DIR_QA_ASSETS}/unit_test_data/
if [ ! -d ${DIR_QA_ASSETS} ]; then
DOCKER_EXEC git clone --depth=1 https://github.com/bitcoin-core/qa-assets ${DIR_QA_ASSETS}
fi
export DIR_FUZZ_IN=${DIR_QA_ASSETS}/fuzz_seed_corpus/
export DIR_UNIT_TEST_DATA=${DIR_QA_ASSETS}/unit_test_data/
DOCKER_EXEC mkdir -p "${BASE_SCRATCH_DIR}/sanitizer-output/"
@@ -99,7 +90,7 @@ if [[ ${USE_MEMORY_SANITIZER} == "true" ]]; then
DOCKER_EXEC "update-alternatives --install /usr/bin/clang++ clang++ \$(which clang++-9) 100"
DOCKER_EXEC "update-alternatives --install /usr/bin/clang clang \$(which clang-9) 100"
DOCKER_EXEC "mkdir -p ${BASE_SCRATCH_DIR}/msan/build/"
DOCKER_EXEC "git clone --depth=1 https://github.com/llvm/llvm-project -b llvmorg-12.0.0 ${BASE_SCRATCH_DIR}/msan/llvm-project"
DOCKER_EXEC "git clone --depth=1 https://github.com/llvm/llvm-project -b llvmorg-10.0.0 ${BASE_SCRATCH_DIR}/msan/llvm-project"
DOCKER_EXEC "cd ${BASE_SCRATCH_DIR}/msan/build/ && cmake -DLLVM_ENABLE_PROJECTS='libcxx;libcxxabi' -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_SANITIZER=Memory -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_TARGETS_TO_BUILD=X86 ../llvm-project/llvm/"
DOCKER_EXEC "cd ${BASE_SCRATCH_DIR}/msan/build/ && make $MAKEJOBS cxx"
fi

View File

@@ -19,16 +19,7 @@ OSX_SDK_BASENAME="Xcode-${XCODE_VERSION}-${XCODE_BUILD_ID}-extracted-SDK-with-li
OSX_SDK_PATH="${DEPENDS_DIR}/sdk-sources/${OSX_SDK_BASENAME}"
if [ -n "$XCODE_VERSION" ] && [ ! -f "$OSX_SDK_PATH" ]; then
DOCKER_EXEC curl --location --fail "${SDK_URL}/${OSX_SDK_BASENAME}" -o "$OSX_SDK_PATH"
fi
if [ -n "$ANDROID_TOOLS_URL" ]; then
ANDROID_TOOLS_PATH=$DEPENDS_DIR/sdk-sources/android-tools.zip
DOCKER_EXEC curl --location --fail "${ANDROID_TOOLS_URL}" -o "$ANDROID_TOOLS_PATH"
DOCKER_EXEC mkdir -p "${ANDROID_HOME}/cmdline-tools"
DOCKER_EXEC unzip -o "$ANDROID_TOOLS_PATH" -d "${ANDROID_HOME}/cmdline-tools"
DOCKER_EXEC "yes | ${ANDROID_HOME}/cmdline-tools/tools/bin/sdkmanager --install \"build-tools;${ANDROID_BUILD_TOOLS_VERSION}\" \"platform-tools\" \"platforms;android-${ANDROID_API_LEVEL}\" \"ndk;${ANDROID_NDK_VERSION}\""
curl --location --fail "${SDK_URL}/${OSX_SDK_BASENAME}" -o "$OSX_SDK_PATH"
fi
if [[ ${USE_MEMORY_SANITIZER} == "true" ]]; then
@@ -45,16 +36,18 @@ if [[ $HOST = *-mingw32 ]]; then
DOCKER_EXEC update-alternatives --set $HOST-g++ \$\(which $HOST-g++-posix\)
fi
if [ -z "$NO_DEPENDS" ]; then
if [[ $DOCKER_NAME_TAG == *centos* ]]; then
if [[ $DOCKER_NAME_TAG == centos* ]]; then
# CentOS has problems building the depends if the config shell is not explicitly set
# (i.e. for libevent a Makefile with an empty SHELL variable is generated, leading to
# an error as the first command is executed)
SHELL_OPTS="LC_ALL=en_US.UTF-8 CONFIG_SHELL=/bin/dash"
SHELL_OPTS="CONFIG_SHELL=/bin/bash"
else
SHELL_OPTS="CONFIG_SHELL="
fi
DOCKER_EXEC $SHELL_OPTS make $MAKEJOBS -C depends HOST=$HOST $DEP_OPTS
fi
if [ -n "$PREVIOUS_RELEASES_TO_DOWNLOAD" ]; then
BEGIN_FOLD previous-versions
DOCKER_EXEC test/get_previous_releases.py -b -t "$PREVIOUS_RELEASES_DIR" "${PREVIOUS_RELEASES_TO_DOWNLOAD}"
END_FOLD
fi

View File

@@ -6,36 +6,36 @@
export LC_ALL=C.UTF-8
if [ -n "$ANDROID_TOOLS_URL" ]; then
DOCKER_EXEC make distclean || true
DOCKER_EXEC ./autogen.sh
DOCKER_EXEC ./configure $BITCOIN_CONFIG --prefix=$DEPENDS_DIR/aarch64-linux-android || ( (DOCKER_EXEC cat config.log) && false)
DOCKER_EXEC "cd src/qt && make $MAKEJOBS && ANDROID_HOME=${ANDROID_HOME} ANDROID_NDK_HOME=${ANDROID_NDK_HOME} make apk"
exit 0
fi
BITCOIN_CONFIG_ALL="--enable-suppress-external-warnings --disable-dependency-tracking --prefix=$DEPENDS_DIR/$HOST --bindir=$BASE_OUTDIR/bin --libdir=$BASE_OUTDIR/lib"
if [ -z "$NO_WERROR" ]; then
BITCOIN_CONFIG_ALL="${BITCOIN_CONFIG_ALL} --enable-werror"
fi
DOCKER_EXEC "ccache --zero-stats --max-size=$CCACHE_SIZE"
BEGIN_FOLD autogen
if [ -n "$CONFIG_SHELL" ]; then
DOCKER_EXEC "$CONFIG_SHELL" -c "./autogen.sh"
else
DOCKER_EXEC ./autogen.sh
fi
END_FOLD
DOCKER_EXEC mkdir -p "${BASE_BUILD_DIR}"
export P_CI_DIR="${BASE_BUILD_DIR}"
BEGIN_FOLD configure
DOCKER_EXEC "${BASE_ROOT_DIR}/configure" --cache-file=config.cache $BITCOIN_CONFIG_ALL $BITCOIN_CONFIG || ( (DOCKER_EXEC cat config.log) && false)
END_FOLD
BEGIN_FOLD distdir
DOCKER_EXEC make distdir VERSION=$HOST
END_FOLD
export P_CI_DIR="${BASE_BUILD_DIR}/bitcoin-$HOST"
BEGIN_FOLD configure
DOCKER_EXEC ./configure --cache-file=../config.cache $BITCOIN_CONFIG_ALL $BITCOIN_CONFIG || ( (DOCKER_EXEC cat config.log) && false)
END_FOLD
set -o errtrace
trap 'DOCKER_EXEC "cat ${BASE_SCRATCH_DIR}/sanitizer-output/* 2> /dev/null"' ERR
@@ -48,8 +48,12 @@ if [[ ${USE_MEMORY_SANITIZER} == "true" ]]; then
DOCKER_EXEC 'grep -v HAVE_SYS_GETRANDOM src/config/bitcoin-config.h > src/config/bitcoin-config.h.tmp && mv src/config/bitcoin-config.h.tmp src/config/bitcoin-config.h'
fi
BEGIN_FOLD build
DOCKER_EXEC make $MAKEJOBS $GOAL || ( echo "Build failure. Verbose build follows." && DOCKER_EXEC make $GOAL V=1 ; false )
END_FOLD
BEGIN_FOLD cache_stats
DOCKER_EXEC "ccache --version | head -n 1 && ccache --show-stats"
DOCKER_EXEC du -sh "${DEPENDS_DIR}"/*/
DOCKER_EXEC du -sh "${PREVIOUS_RELEASES_DIR}"
END_FOLD

View File

@@ -7,39 +7,55 @@
export LC_ALL=C.UTF-8
if [[ $HOST = *-mingw32 ]]; then
BEGIN_FOLD wrap-wine
# Generate all binaries, so that they can be wrapped
DOCKER_EXEC make $MAKEJOBS -C src/secp256k1 VERBOSE=1
DOCKER_EXEC make $MAKEJOBS -C src/univalue VERBOSE=1
DOCKER_EXEC "${BASE_ROOT_DIR}/ci/test/wrap-wine.sh"
END_FOLD
fi
if [ -n "$QEMU_USER_CMD" ]; then
BEGIN_FOLD wrap-qemu
# Generate all binaries, so that they can be wrapped
DOCKER_EXEC make $MAKEJOBS -C src/secp256k1 VERBOSE=1
DOCKER_EXEC make $MAKEJOBS -C src/univalue VERBOSE=1
DOCKER_EXEC "${BASE_ROOT_DIR}/ci/test/wrap-qemu.sh"
END_FOLD
fi
if [ -n "$USE_VALGRIND" ]; then
BEGIN_FOLD wrap-valgrind
DOCKER_EXEC "${BASE_ROOT_DIR}/ci/test/wrap-valgrind.sh"
END_FOLD
fi
if [ "$RUN_UNIT_TESTS" = "true" ]; then
BEGIN_FOLD unit-tests
DOCKER_EXEC ${TEST_RUNNER_ENV} DIR_UNIT_TEST_DATA=${DIR_UNIT_TEST_DATA} LD_LIBRARY_PATH=$DEPENDS_DIR/$HOST/lib make $MAKEJOBS check VERBOSE=1
END_FOLD
fi
if [ "$RUN_UNIT_TESTS_SEQUENTIAL" = "true" ]; then
BEGIN_FOLD unit-tests-seq
DOCKER_EXEC ${TEST_RUNNER_ENV} DIR_UNIT_TEST_DATA=${DIR_UNIT_TEST_DATA} LD_LIBRARY_PATH=$DEPENDS_DIR/$HOST/lib "${BASE_BUILD_DIR}/bitcoin-*/src/test/test_bitcoin*" --catch_system_errors=no -l test_suite
END_FOLD
fi
if [ "$RUN_FUNCTIONAL_TESTS" = "true" ]; then
BEGIN_FOLD functional-tests
DOCKER_EXEC LD_LIBRARY_PATH=$DEPENDS_DIR/$HOST/lib ${TEST_RUNNER_ENV} test/functional/test_runner.py --ci $MAKEJOBS --tmpdirprefix "${BASE_SCRATCH_DIR}/test_runner/" --ansi --combinedlogslen=4000 --timeout-factor=${TEST_RUNNER_TIMEOUT_FACTOR} ${TEST_RUNNER_EXTRA} --quiet --failfast
END_FOLD
fi
if [ "$RUN_SECURITY_TESTS" = "true" ]; then
BEGIN_FOLD security-tests
DOCKER_EXEC make test-security-check
END_FOLD
fi
if [ "$RUN_FUZZ_TESTS" = "true" ]; then
BEGIN_FOLD fuzz-tests
DOCKER_EXEC LD_LIBRARY_PATH=$DEPENDS_DIR/$HOST/lib test/fuzz/test_runner.py ${FUZZ_TESTS_CONFIG} $MAKEJOBS -l DEBUG ${DIR_FUZZ_IN}
END_FOLD
fi

View File

@@ -1,6 +1,6 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2020 The Bitcoin Core developers
# Copyright (c) 2018-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.

View File

@@ -13,7 +13,7 @@ for b_name in {"${BASE_OUTDIR}/bin"/*,src/secp256k1/*tests,src/univalue/{no_nul,
echo "Wrap $b ..."
mv "$b" "${b}_orig"
echo '#!/usr/bin/env bash' > "$b"
echo "( wine \"${b}_orig\" \"\$@\" ) || ( sleep 1 && wine \"${b}_orig\" \"\$@\" )" >> "$b"
echo "wine64 \"${b}_orig\" \"\$@\"" >> "$b"
chmod +x "$b"
fi
done

View File

@@ -1,12 +1,13 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Copyright (c) 2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
set -o errexit; source ./ci/test/00_setup_env.sh
set -o errexit; source ./ci/test/03_before_install.sh
set -o errexit; source ./ci/test/04_install.sh
set -o errexit; source ./ci/test/05_before_script.sh
set -o errexit; source ./ci/test/06_script_a.sh

View File

@@ -1,13 +1,14 @@
AC_PREREQ([2.69])
define(_CLIENT_VERSION_MAJOR, 22)
define(_CLIENT_VERSION_MINOR, 1)
define(_CLIENT_VERSION_MAJOR, 0)
define(_CLIENT_VERSION_MINOR, 21)
define(_CLIENT_VERSION_REVISION, 2)
define(_CLIENT_VERSION_BUILD, 0)
define(_CLIENT_VERSION_RC, 0)
define(_CLIENT_VERSION_IS_RELEASE, true)
define(_COPYRIGHT_YEAR, 2021)
define(_COPYRIGHT_YEAR, 2020)
define(_COPYRIGHT_HOLDERS,[The %s developers])
define(_COPYRIGHT_HOLDERS_SUBSTITUTION,[[Bitcoin Core]])
AC_INIT([Bitcoin Core],m4_join([.], _CLIENT_VERSION_MAJOR, _CLIENT_VERSION_MINOR, _CLIENT_VERSION_BUILD)m4_if(_CLIENT_VERSION_RC, [0], [], [rc]_CLIENT_VERSION_RC),[https://github.com/bitcoin/bitcoin/issues],[bitcoin],[https://bitcoincore.org/])
AC_INIT([Bitcoin Core],m4_join([.], _CLIENT_VERSION_MAJOR, _CLIENT_VERSION_MINOR, _CLIENT_VERSION_REVISION, m4_if(_CLIENT_VERSION_BUILD, [0], [], _CLIENT_VERSION_BUILD))m4_if(_CLIENT_VERSION_RC, [0], [], [rc]_CLIENT_VERSION_RC),[https://github.com/bitcoin/bitcoin/issues],[bitcoin],[https://bitcoincore.org/])
AC_CONFIG_SRCDIR([src/validation.cpp])
AC_CONFIG_HEADERS([src/config/bitcoin-config.h])
AC_CONFIG_AUX_DIR([build-aux])
@@ -23,11 +24,7 @@ BITCOIN_DAEMON_NAME=bitcoind
BITCOIN_GUI_NAME=bitcoin-qt
BITCOIN_CLI_NAME=bitcoin-cli
BITCOIN_TX_NAME=bitcoin-tx
BITCOIN_UTIL_NAME=bitcoin-util
BITCOIN_WALLET_TOOL_NAME=bitcoin-wallet
dnl Multi Process
BITCOIN_MP_NODE_NAME=bitcoin-node
BITCOIN_MP_GUI_NAME=bitcoin-gui
dnl Unless the user specified ARFLAGS, force it to be cr
AC_ARG_VAR(ARFLAGS, [Flags for the archiver, defaults to <cr> if not set])
@@ -71,8 +68,18 @@ case $host in
;;
esac
dnl Require C++17 compiler (no GNU extensions)
AX_CXX_COMPILE_STDCXX([17], [noext], [mandatory])
AC_ARG_ENABLE([c++17],
[AS_HELP_STRING([--enable-c++17],
[enable compilation in c++17 mode (disabled by default)])],
[use_cxx17=$enableval],
[use_cxx17=no])
dnl Require C++11 or C++17 compiler (no GNU extensions)
if test "x$use_cxx17" = xyes -o "x$enable_fuzz" = xyes ; then
AX_CXX_COMPILE_STDCXX([17], [noext], [mandatory])
else
AX_CXX_COMPILE_STDCXX([11], [noext], [mandatory])
fi
dnl Check if -latomic is required for <std::atomic>
CHECK_ATOMIC
@@ -100,13 +107,14 @@ AC_PATH_TOOL(STRIP, strip)
AC_PATH_TOOL(GCOV, gcov)
AC_PATH_TOOL(LLVM_COV, llvm-cov)
AC_PATH_PROG(LCOV, lcov)
dnl Python 3.6 is specified in .python-version and should be used if available, see doc/dependencies.md
AC_PATH_PROGS([PYTHON], [python3.6 python3.7 python3.8 python3.9 python3 python])
dnl Python 3.5 is specified in .python-version and should be used if available, see doc/dependencies.md
AC_PATH_PROGS([PYTHON], [python3.5 python3.6 python3.7 python3.8 python3 python])
AC_PATH_PROG(GENHTML, genhtml)
AC_PATH_PROG([GIT], [git])
AC_PATH_PROG(CCACHE,ccache)
AC_PATH_PROG(XGETTEXT,xgettext)
AC_PATH_PROG(HEXDUMP,hexdump)
AC_PATH_TOOL(READELF, readelf)
AC_PATH_TOOL(CPPFILT, c++filt)
AC_PATH_TOOL(OBJCOPY, objcopy)
AC_PATH_PROG(DOXYGEN, doxygen)
@@ -118,7 +126,7 @@ AC_ARG_ENABLE([wallet],
[AS_HELP_STRING([--disable-wallet],
[disable wallet (enabled by default)])],
[enable_wallet=$enableval],
[enable_wallet=auto])
[enable_wallet=yes])
AC_ARG_WITH([sqlite],
[AS_HELP_STRING([--with-sqlite=yes|no|auto],
@@ -126,18 +134,6 @@ AC_ARG_WITH([sqlite],
[use_sqlite=$withval],
[use_sqlite=auto])
AC_ARG_WITH([bdb],
[AS_HELP_STRING([--without-bdb],
[disable bdb wallet support (default is enabled if wallet is enabled)])],
[use_bdb=$withval],
[use_bdb=auto])
AC_ARG_ENABLE([ebpf],
[AS_HELP_STRING([--enable-ebpf],
[enable eBPF tracing (default is yes if sys/sdt.h is found)])],
[use_ebpf=$enableval],
[use_ebpf=yes])
AC_ARG_WITH([miniupnpc],
[AS_HELP_STRING([--with-miniupnpc],
[enable UPNP (default is yes if libminiupnpc is found)])],
@@ -150,18 +146,6 @@ AC_ARG_ENABLE([upnp-default],
[use_upnp_default=$enableval],
[use_upnp_default=no])
AC_ARG_WITH([natpmp],
[AS_HELP_STRING([--with-natpmp],
[enable NAT-PMP (default is yes if libnatpmp is found)])],
[use_natpmp=$withval],
[use_natpmp=auto])
AC_ARG_ENABLE([natpmp-default],
[AS_HELP_STRING([--enable-natpmp-default],
[if NAT-PMP is enabled, turn it on at startup (default is no)])],
[use_natpmp_default=$enableval],
[use_natpmp_default=no])
AC_ARG_ENABLE(tests,
AS_HELP_STRING([--disable-tests],[do not compile tests (default is to compile)]),
[use_tests=$enableval],
@@ -184,16 +168,10 @@ AC_ARG_ENABLE([extended-functional-tests],
AC_ARG_ENABLE([fuzz],
AS_HELP_STRING([--enable-fuzz],
[build for fuzzing (default no). enabling this will disable all other targets and override --{enable,disable}-fuzz-binary]),
[enable building of fuzz targets (default no). enabling this will disable all other targets]),
[enable_fuzz=$enableval],
[enable_fuzz=no])
AC_ARG_ENABLE([fuzz-binary],
AS_HELP_STRING([--enable-fuzz-binary],
[enable building of fuzz binary (default yes).]),
[enable_fuzz_binary=$enableval],
[enable_fuzz_binary=yes])
AC_ARG_WITH([qrencode],
[AS_HELP_STRING([--with-qrencode],
[enable QR code support (default is yes if qt is enabled and libqrencode is found)])],
@@ -318,6 +296,13 @@ AC_ARG_ENABLE([gprof],
[enable_gprof=$enableval],
[enable_gprof=no])
dnl Pass compiler & linker flags that make builds deterministic
AC_ARG_ENABLE([determinism],
[AS_HELP_STRING([--enable-determinism],
[Enable compilation flags that make builds deterministic (default is no)])],
[enable_determinism=$enableval],
[enable_determinism=no])
dnl Turn warnings into errors
AC_ARG_ENABLE([werror],
[AS_HELP_STRING([--enable-werror],
@@ -325,11 +310,6 @@ AC_ARG_ENABLE([werror],
[enable_werror=$enableval],
[enable_werror=no])
AC_ARG_ENABLE([external-signer],
[AS_HELP_STRING([--enable-external-signer],[compile external signer support (default is yes, requires Boost::Process)])],
[use_external_signer=$enableval],
[use_external_signer=yes])
AC_LANG_PUSH([C++])
dnl Check for a flag to turn compiler warnings into errors. This is helpful for checks which may
@@ -373,7 +353,6 @@ if test "x$enable_debug" = xyes; then
AX_CHECK_PREPROC_FLAG([-DDEBUG],[[DEBUG_CPPFLAGS="$DEBUG_CPPFLAGS -DDEBUG"]],,[[$CXXFLAG_WERROR]])
AX_CHECK_PREPROC_FLAG([-DDEBUG_LOCKORDER],[[DEBUG_CPPFLAGS="$DEBUG_CPPFLAGS -DDEBUG_LOCKORDER"]],,[[$CXXFLAG_WERROR]])
AX_CHECK_PREPROC_FLAG([-DABORT_ON_FAILED_ASSUME],[[DEBUG_CPPFLAGS="$DEBUG_CPPFLAGS -DABORT_ON_FAILED_ASSUME"]],,[[$CXXFLAG_WERROR]])
AX_CHECK_COMPILE_FLAG([-ftrapv],[DEBUG_CXXFLAGS="$DEBUG_CXXFLAGS -ftrapv"],,[[$CXXFLAG_WERROR]])
fi
@@ -425,12 +404,6 @@ if test "x$enable_werror" = "xyes"; then
AX_CHECK_COMPILE_FLAG([-Werror=suggest-override],[ERROR_CXXFLAGS="$ERROR_CXXFLAGS -Werror=suggest-override"],,[[$CXXFLAG_WERROR]],
[AC_LANG_SOURCE([[struct A { virtual void f(); }; struct B : A { void f() final; };]])])
AX_CHECK_COMPILE_FLAG([-Werror=unreachable-code-loop-increment],[ERROR_CXXFLAGS="$ERROR_CXXFLAGS -Werror=unreachable-code-loop-increment"],,[[$CXXFLAG_WERROR]])
AX_CHECK_COMPILE_FLAG([-Werror=mismatched-tags], [ERROR_CXXFLAGS="$ERROR_CXXFLAGS -Werror=mismatched-tags"], [], [$CXXFLAG_WERROR])
AX_CHECK_COMPILE_FLAG([-Werror=implicit-fallthrough], [ERROR_CXXFLAGS="$ERROR_CXXFLAGS -Werror=implicit-fallthrough"], [], [$CXXFLAG_WERROR])
if test x$suppress_external_warnings != xno ; then
AX_CHECK_COMPILE_FLAG([-Werror=documentation],[ERROR_CXXFLAGS="$ERROR_CXXFLAGS -Werror=documentation"],,[[$CXXFLAG_WERROR]])
fi
fi
if test "x$CXXFLAGS_overridden" = "xno"; then
@@ -446,7 +419,6 @@ if test "x$CXXFLAGS_overridden" = "xno"; then
AX_CHECK_COMPILE_FLAG([-Wrange-loop-analysis],[WARN_CXXFLAGS="$WARN_CXXFLAGS -Wrange-loop-analysis"],,[[$CXXFLAG_WERROR]])
AX_CHECK_COMPILE_FLAG([-Wredundant-decls],[WARN_CXXFLAGS="$WARN_CXXFLAGS -Wredundant-decls"],,[[$CXXFLAG_WERROR]])
AX_CHECK_COMPILE_FLAG([-Wunused-variable],[WARN_CXXFLAGS="$WARN_CXXFLAGS -Wunused-variable"],,[[$CXXFLAG_WERROR]])
AX_CHECK_COMPILE_FLAG([-Wunused-member-function],[WARN_CXXFLAGS="$WARN_CXXFLAGS -Wunused-member-function"],,[[$CXXFLAG_WERROR]])
AX_CHECK_COMPILE_FLAG([-Wdate-time],[WARN_CXXFLAGS="$WARN_CXXFLAGS -Wdate-time"],,[[$CXXFLAG_WERROR]])
AX_CHECK_COMPILE_FLAG([-Wconditional-uninitialized],[WARN_CXXFLAGS="$WARN_CXXFLAGS -Wconditional-uninitialized"],,[[$CXXFLAG_WERROR]])
AX_CHECK_COMPILE_FLAG([-Wsign-compare],[WARN_CXXFLAGS="$WARN_CXXFLAGS -Wsign-compare"],,[[$CXXFLAG_WERROR]])
@@ -457,11 +429,6 @@ if test "x$CXXFLAGS_overridden" = "xno"; then
AX_CHECK_COMPILE_FLAG([-Wsuggest-override],[WARN_CXXFLAGS="$WARN_CXXFLAGS -Wsuggest-override"],,[[$CXXFLAG_WERROR]],
[AC_LANG_SOURCE([[struct A { virtual void f(); }; struct B : A { void f() final; };]])])
AX_CHECK_COMPILE_FLAG([-Wunreachable-code-loop-increment],[WARN_CXXFLAGS="$WARN_CXXFLAGS -Wunreachable-code-loop-increment"],,[[$CXXFLAG_WERROR]])
AX_CHECK_COMPILE_FLAG([-Wimplicit-fallthrough], [WARN_CXXFLAGS="$WARN_CXXFLAGS -Wimplicit-fallthrough"], [], [$CXXFLAG_WERROR])
if test x$suppress_external_warnings != xno ; then
AX_CHECK_COMPILE_FLAG([-Wdocumentation],[WARN_CXXFLAGS="$WARN_CXXFLAGS -Wdocumentation"],,[[$CXXFLAG_WERROR]])
fi
dnl Some compilers (gcc) ignore unknown -Wno-* options, but warn about all
dnl unknown options if any other warning is produced. Test the -Wfoo case, and
@@ -469,9 +436,9 @@ if test "x$CXXFLAGS_overridden" = "xno"; then
AX_CHECK_COMPILE_FLAG([-Wunused-parameter],[NOWARN_CXXFLAGS="$NOWARN_CXXFLAGS -Wno-unused-parameter"],,[[$CXXFLAG_WERROR]])
AX_CHECK_COMPILE_FLAG([-Wself-assign],[NOWARN_CXXFLAGS="$NOWARN_CXXFLAGS -Wno-self-assign"],,[[$CXXFLAG_WERROR]])
AX_CHECK_COMPILE_FLAG([-Wunused-local-typedef],[NOWARN_CXXFLAGS="$NOWARN_CXXFLAGS -Wno-unused-local-typedef"],,[[$CXXFLAG_WERROR]])
if test x$suppress_external_warnings != xyes ; then
AX_CHECK_COMPILE_FLAG([-Wdeprecated-copy],[NOWARN_CXXFLAGS="$NOWARN_CXXFLAGS -Wno-deprecated-copy"],,[[$CXXFLAG_WERROR]])
fi
AX_CHECK_COMPILE_FLAG([-Wdeprecated-register],[NOWARN_CXXFLAGS="$NOWARN_CXXFLAGS -Wno-deprecated-register"],,[[$CXXFLAG_WERROR]])
AX_CHECK_COMPILE_FLAG([-Wimplicit-fallthrough],[NOWARN_CXXFLAGS="$NOWARN_CXXFLAGS -Wno-implicit-fallthrough"],,[[$CXXFLAG_WERROR]])
AX_CHECK_COMPILE_FLAG([-Wdeprecated-copy],[NOWARN_CXXFLAGS="$NOWARN_CXXFLAGS -Wno-deprecated-copy"],,[[$CXXFLAG_WERROR]])
fi
dnl Don't allow extended (non-ASCII) symbols in identifiers. This is easier for code review.
@@ -568,17 +535,13 @@ AX_CHECK_COMPILE_FLAG([-march=armv8-a+crc+crypto],[[ARM_CRC_CXXFLAGS="-march=arm
TEMP_CXXFLAGS="$CXXFLAGS"
CXXFLAGS="$CXXFLAGS $ARM_CRC_CXXFLAGS"
AC_MSG_CHECKING(for AArch64 CRC32 intrinsics)
AC_MSG_CHECKING(for ARM CRC32 intrinsics)
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#include <arm_acle.h>
#include <arm_neon.h>
]],[[
#ifdef __aarch64__
__crc32cb(0, 0); __crc32ch(0, 0); __crc32cw(0, 0); __crc32cd(0, 0);
vmull_p64(0, 0);
#else
#error "crc32c library does not support hardware acceleration on 32-bit ARM"
#endif
]])],
[ AC_MSG_RESULT(yes); enable_arm_crc=yes; ],
[ AC_MSG_RESULT(no)]
@@ -591,7 +554,7 @@ CPPFLAGS="$CPPFLAGS -DHAVE_BUILD_INFO -D__STDC_FORMAT_MACROS"
AC_ARG_WITH([utils],
[AS_HELP_STRING([--with-utils],
[build bitcoin-cli bitcoin-tx bitcoin-util bitcoin-wallet (default=yes)])],
[build bitcoin-cli bitcoin-tx bitcoin-wallet (default=yes)])],
[build_bitcoin_utils=$withval],
[build_bitcoin_utils=yes])
@@ -613,12 +576,6 @@ AC_ARG_ENABLE([util-wallet],
[build_bitcoin_wallet=$enableval],
[build_bitcoin_wallet=$build_bitcoin_utils])
AC_ARG_ENABLE([util-util],
[AS_HELP_STRING([--enable-util-util],
[build bitcoin-util])],
[build_bitcoin_util=$enableval],
[build_bitcoin_util=$build_bitcoin_utils])
AC_ARG_WITH([libs],
[AS_HELP_STRING([--with-libs],
[build libraries (default=yes)])],
@@ -663,7 +620,7 @@ case $host in
AC_MSG_ERROR("windres not found")
fi
CPPFLAGS="$CPPFLAGS -D_MT -DWIN32 -D_WINDOWS -D_WIN32_WINNT=0x0601 -D_WIN32_IE=0x0501 -DWIN32_LEAN_AND_MEAN"
CPPFLAGS="$CPPFLAGS -D_MT -DWIN32 -D_WINDOWS -DBOOST_THREAD_USE_LIB -D_WIN32_WINNT=0x0601 -D_WIN32_IE=0x0501 -DWIN32_LEAN_AND_MEAN"
dnl libtool insists upon adding -nostdlib and a list of objects/libs to link against.
dnl That breaks our ability to build dll's with static libgcc/libstdc++/libssp. Override
@@ -696,6 +653,10 @@ case $host in
BDB_LIBS="-L$bdb_prefix/lib -ldb_cxx-4.8"
fi
if test "x$use_sqlite" != xno && $BREW list --versions sqlite3 >/dev/null; then
export PKG_CONFIG_PATH="$($BREW --prefix sqlite3 2>/dev/null)/lib/pkgconfig:$PKG_CONFIG_PATH"
fi
if $BREW list --versions qt5 >/dev/null; then
export PKG_CONFIG_PATH="$($BREW --prefix qt5 2>/dev/null)/lib/pkgconfig:$PKG_CONFIG_PATH"
fi
@@ -709,8 +670,7 @@ case $host in
AC_PATH_TOOL([DSYMUTIL], [dsymutil], dsymutil)
AC_PATH_TOOL([INSTALLNAMETOOL], [install_name_tool], install_name_tool)
AC_PATH_TOOL([OTOOL], [otool], otool)
AC_PATH_PROGS([XORRISOFS], [xorrisofs], xorrisofs)
AC_PATH_PROGS([DMG], [dmg], dmg)
AC_PATH_PROGS([GENISOIMAGE], [genisoimage mkisofs],genisoimage)
AC_PATH_PROGS([RSVG_CONVERT], [rsvg-convert rsvg],rsvg-convert)
AC_PATH_PROGS([IMAGEMAGICK_CONVERT], [convert],convert)
AC_PATH_PROGS([TIFFCP], [tiffcp],tiffcp)
@@ -731,21 +691,6 @@ case $host in
*android*)
dnl make sure android stays above linux for hosts like *linux-android*
TARGET_OS=android
case $host in
*x86_64*)
ANDROID_ARCH=x86_64
;;
*aarch64*)
ANDROID_ARCH=arm64-v8a
;;
*armv7a*)
ANDROID_ARCH=armeabi-v7a
;;
*i686*)
ANDROID_ARCH=i686
;;
*) AC_MSG_ERROR("Could not determine Android arch") ;;
esac
;;
*linux*)
TARGET_OS=linux
@@ -802,9 +747,6 @@ if test x$use_lcov_branch != xno; then
AC_SUBST(LCOV_OPTS, "$LCOV_OPTS --rc lcov_branch_coverage=1")
fi
dnl Check for __int128
AC_CHECK_TYPES([__int128])
dnl Check for endianness
AC_C_BIGENDIAN
@@ -830,14 +772,13 @@ if test x$ac_cv_sys_large_files != x &&
CPPFLAGS="$CPPFLAGS -D_LARGE_FILES=$ac_cv_sys_large_files"
fi
AX_GCC_FUNC_ATTRIBUTE([visibility])
AX_GCC_FUNC_ATTRIBUTE([dllexport])
AX_GCC_FUNC_ATTRIBUTE([dllimport])
if test x$use_glibc_compat != xno; then
AX_CHECK_LINK_FLAG([[-Wl,--wrap=__divmoddi4]], [COMPAT_LDFLAGS="$COMPAT_LDFLAGS -Wl,--wrap=__divmoddi4"])
AX_CHECK_LINK_FLAG([[-Wl,--wrap=log2f]], [COMPAT_LDFLAGS="$COMPAT_LDFLAGS -Wl,--wrap=log2f"])
case $host in
powerpc64* | ppc64*)
AX_CHECK_LINK_FLAG([[-Wl,--no-tls-get-addr-optimize]], [COMPAT_LDFLAGS="$COMPAT_LDFLAGS -Wl,--no-tls-get-addr-optimize"])
;;
esac
else
AC_SEARCH_LIBS([clock_gettime],[rt])
fi
@@ -873,21 +814,13 @@ if test x$use_hardening != xno; then
AX_CHECK_COMPILE_FLAG([-Wstack-protector],[HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -Wstack-protector"])
AX_CHECK_COMPILE_FLAG([-fstack-protector-all],[HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -fstack-protector-all"])
dnl -fcf-protection used with Clang 7 causes ld to emit warnings:
dnl ld: error: ... <corrupt x86 feature size: 0x8>
dnl Use CHECK_LINK_FLAG & --fatal-warnings to ensure we won't use the flag in this case.
AX_CHECK_LINK_FLAG([-fcf-protection=full],[HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -fcf-protection=full"],, [[$LDFLAG_WERROR]])
case $host in
*mingw*)
dnl stack-clash-protection doesn't currently work, and likely should just be skipped for Windows.
dnl See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90458 for more details.
;;
*)
AX_CHECK_COMPILE_FLAG([-fstack-clash-protection], [HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -fstack-clash-protection"], [], [$CXXFLAG_WERROR])
;;
esac
AX_CHECK_COMPILE_FLAG([-fcf-protection=full],[HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -fcf-protection=full"])
dnl stack-clash-protection does not work properly when building for Windows.
dnl We use the test case from https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90458
dnl to determine if it can be enabled.
AX_CHECK_COMPILE_FLAG([-fstack-clash-protection],[HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -fstack-clash-protection"],[],["-O0"],
[AC_LANG_SOURCE([[class D {public: unsigned char buf[32768];}; int main() {D d; return 0;}]])])
dnl When enable_debug is yes, all optimizations are disabled.
dnl However, FORTIFY_SOURCE requires that there is some level of optimization, otherwise it does nothing and just creates a compiler warning.
@@ -901,7 +834,6 @@ if test x$use_hardening != xno; then
])
fi
AX_CHECK_LINK_FLAG([[-Wl,--enable-reloc-section]], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -Wl,--enable-reloc-section"],, [[$LDFLAG_WERROR]])
AX_CHECK_LINK_FLAG([[-Wl,--dynamicbase]], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -Wl,--dynamicbase"],, [[$LDFLAG_WERROR]])
AX_CHECK_LINK_FLAG([[-Wl,--nxcompat]], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -Wl,--nxcompat"],, [[$LDFLAG_WERROR]])
AX_CHECK_LINK_FLAG([[-Wl,--high-entropy-va]], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -Wl,--high-entropy-va"],, [[$LDFLAG_WERROR]])
@@ -926,6 +858,12 @@ if test x$TARGET_OS = xdarwin; then
AX_CHECK_LINK_FLAG([[-Wl,-bind_at_load]], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -Wl,-bind_at_load"],, [[$LDFLAG_WERROR]])
fi
if test x$enable_determinism = xyes; then
if test x$TARGET_OS = xwindows; then
AX_CHECK_LINK_FLAG([[-Wl,--no-insert-timestamp]], [LDFLAGS="$LDFLAGS -Wl,--no-insert-timestamp"],, [[$LDFLAG_WERROR]])
fi
fi
AC_CHECK_HEADERS([endian.h sys/endian.h byteswap.h stdio.h stdlib.h unistd.h strings.h sys/types.h sys/stat.h sys/select.h sys/prctl.h sys/sysctl.h vm/vm_param.h sys/vmmeter.h sys/resources.h])
AC_CHECK_DECLS([getifaddrs, freeifaddrs],[CHECK_SOCKET],,
@@ -934,11 +872,8 @@ AC_CHECK_DECLS([getifaddrs, freeifaddrs],[CHECK_SOCKET],,
)
AC_CHECK_DECLS([strnlen])
dnl These are used for daemonization in bitcoind
AC_CHECK_DECLS([fork])
AC_CHECK_DECLS([setsid])
AC_CHECK_DECLS([pipe2])
dnl Check for daemon(3), unrelated to --with-daemon (although used by it)
AC_CHECK_DECLS([daemon])
AC_CHECK_DECLS([le16toh, le32toh, le64toh, htole16, htole32, htole64, be16toh, be32toh, be64toh, htobe16, htobe32, htobe64],,,
[#if HAVE_ENDIAN_H
@@ -1000,13 +935,13 @@ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
[ AC_MSG_RESULT(no)]
)
AC_MSG_CHECKING([for default visibility attribute])
AC_COMPILE_IFELSE([AC_LANG_SOURCE([
int foo(void) __attribute__((visibility("default")));
AC_MSG_CHECKING([for visibility attribute])
AC_LINK_IFELSE([AC_LANG_SOURCE([
int foo_def( void ) __attribute__((visibility("default")));
int main(){}
])],
[
AC_DEFINE(HAVE_DEFAULT_VISIBILITY_ATTRIBUTE,1,[Define if the visibility attribute is supported.])
AC_DEFINE(HAVE_VISIBILITY_ATTRIBUTE,1,[Define if the visibility attribute is supported.])
AC_MSG_RESULT(yes)
],
[
@@ -1017,18 +952,6 @@ AC_COMPILE_IFELSE([AC_LANG_SOURCE([
]
)
AC_MSG_CHECKING([for dllexport attribute])
AC_COMPILE_IFELSE([AC_LANG_SOURCE([
__declspec(dllexport) int foo(void);
int main(){}
])],
[
AC_DEFINE(HAVE_DLLEXPORT_ATTRIBUTE,1,[Define if the dllexport attribute is supported.])
AC_MSG_RESULT(yes)
],
[AC_MSG_RESULT(no)]
)
dnl thread_local is currently disabled when building with glibc back compat.
dnl Our minimum supported glibc is 2.17, however support for thread_local
dnl did not arrive in glibc until 2.18.
@@ -1164,7 +1087,6 @@ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <fcntl.h>]],
[ AC_MSG_RESULT(yes); HAVE_O_CLOEXEC=1 ],
[ AC_MSG_RESULT(no); HAVE_O_CLOEXEC=0 ]
)
AC_DEFINE_UNQUOTED([HAVE_O_CLOEXEC], [$HAVE_O_CLOEXEC], [Define to 1 if O_CLOEXEC flag is available.])
dnl crc32c platform checks
AC_MSG_CHECKING(for __builtin_prefetch)
@@ -1195,23 +1117,27 @@ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
]], [[
getauxval(AT_HWCAP);
]])],
[ AC_MSG_RESULT(yes); HAVE_STRONG_GETAUXVAL=1; AC_DEFINE(HAVE_STRONG_GETAUXVAL, 1, [Define this symbol to build code that uses getauxval)]) ],
[ AC_MSG_RESULT(yes); HAVE_STRONG_GETAUXVAL=1 ],
[ AC_MSG_RESULT(no); HAVE_STRONG_GETAUXVAL=0 ]
)
AC_MSG_CHECKING(for weak getauxval support in the compiler)
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#ifdef __linux__
unsigned long getauxval(unsigned long type) __attribute__((weak));
#define AT_HWCAP 16
#endif
]], [[
getauxval(AT_HWCAP);
]])],
[ AC_MSG_RESULT(yes); HAVE_WEAK_GETAUXVAL=1; AC_DEFINE(HAVE_WEAK_GETAUXVAL, 1, [Define this symbol to build code that uses getauxval (weak linking)]) ],
[ AC_MSG_RESULT(yes); HAVE_WEAK_GETAUXVAL=1 ],
[ AC_MSG_RESULT(no); HAVE_WEAK_GETAUXVAL=0 ]
)
dnl Check for reduced exports
if test x$use_reduce_exports = xyes; then
AX_CHECK_COMPILE_FLAG([-fvisibility=hidden],[RE_CXXFLAGS="-fvisibility=hidden"],
[AC_MSG_ERROR([Cannot set default symbol visibility. Use --disable-reduce-exports.])])
fi
AC_MSG_CHECKING([for std::system])
AC_LINK_IFELSE(
[ AC_LANG_PROGRAM(
@@ -1256,11 +1182,10 @@ AC_DEFUN([SUPPRESS_WARNINGS],
dnl enable-fuzz should disable all other targets
if test "x$enable_fuzz" = "xyes"; then
AC_MSG_WARN(enable-fuzz will disable all other targets and force --enable-fuzz-binary=yes)
AC_MSG_WARN(enable-fuzz will disable all other targets)
build_bitcoin_utils=no
build_bitcoin_cli=no
build_bitcoin_tx=no
build_bitcoin_util=no
build_bitcoin_wallet=no
build_bitcoind=no
build_bitcoin_libs=no
@@ -1269,34 +1194,27 @@ if test "x$enable_fuzz" = "xyes"; then
bitcoin_enable_qt_dbus=no
enable_wallet=no
use_bench=no
use_external_signer=no
use_upnp=no
use_natpmp=no
use_zmq=no
enable_fuzz_binary=yes
AX_CHECK_PREPROC_FLAG([-DABORT_ON_FAILED_ASSUME],[[DEBUG_CPPFLAGS="$DEBUG_CPPFLAGS -DABORT_ON_FAILED_ASSUME"]],,[[$CXXFLAG_WERROR]])
AC_MSG_CHECKING([whether main function is needed for fuzz binary])
AC_MSG_CHECKING([whether main function is needed])
AX_CHECK_LINK_FLAG(
[[-fsanitize=$use_sanitizers]],
[AC_MSG_RESULT([no])],
[AC_MSG_RESULT([yes])
CPPFLAGS="$CPPFLAGS -DPROVIDE_FUZZ_MAIN_FUNCTION"],
CPPFLAGS="$CPPFLAGS -DPROVIDE_MAIN_FUNCTION"],
[],
[AC_LANG_PROGRAM([[
#include <cstdint>
#include <cstddef>
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { return 0; }
/* comment to remove the main function ...
]],[[
*/ int not_main() {
]])])
/* unterminated comment to remove the main function ...
]],[[]])])
else
BITCOIN_QT_INIT
dnl sets $bitcoin_enable_qt, $bitcoin_enable_qt_test, $bitcoin_enable_qt_dbus
BITCOIN_QT_CONFIGURE([5.9.5])
BITCOIN_QT_CONFIGURE([5.5.1])
dnl Keep a copy of the original $QT_INCLUDES and use it when invoking qt's moc
QT_INCLUDES_UNSUPPRESSED=$QT_INCLUDES
@@ -1305,17 +1223,13 @@ else
QT_DBUS_INCLUDES=SUPPRESS_WARNINGS($QT_DBUS_INCLUDES)
QT_TEST_INCLUDES=SUPPRESS_WARNINGS($QT_TEST_INCLUDES)
fi
CPPFLAGS="$CPPFLAGS -DPROVIDE_FUZZ_MAIN_FUNCTION"
fi
if test x$enable_wallet != xno; then
dnl Check for libdb_cxx only if wallet enabled
if test "x$use_bdb" != "xno"; then
BITCOIN_FIND_BDB48
if test x$suppress_external_warnings != xno ; then
BITCOIN_FIND_BDB48
if test x$suppress_external_warnings != xno ; then
BDB_CPPFLAGS=SUPPRESS_WARNINGS($BDB_CPPFLAGS)
fi
fi
dnl Check for sqlite3
@@ -1337,32 +1251,12 @@ if test x$enable_wallet != xno; then
fi
fi
AC_MSG_RESULT([$use_sqlite])
dnl Disable wallet if both --without-bdb and --without-sqlite
if test "x$use_bdb$use_sqlite" = "xnono"; then
if test "x$enable_wallet" = "xyes"; then
AC_MSG_ERROR([wallet functionality requested but no BDB or SQLite support available.])
fi
enable_wallet=no
fi
fi
if test x$use_ebpf != xno; then
AC_MSG_CHECKING([whether eBPF tracepoints are supported])
AC_COMPILE_IFELSE([
AC_LANG_PROGRAM(
[#include <sys/sdt.h>],
[DTRACE_PROBE("context", "event");]
)],
[AC_MSG_RESULT(yes); have_sdt=yes; AC_DEFINE([ENABLE_TRACING], [1], [Define to 1 to enable eBPF user static defined tracepoints])],
[AC_MSG_RESULT(no); have_sdt=no;]
)
fi
dnl Check for libminiupnpc (optional)
if test x$use_upnp != xno; then
AC_CHECK_HEADERS(
[miniupnpc/miniupnpc.h miniupnpc/upnpcommands.h miniupnpc/upnperrors.h],
[miniupnpc/miniwget.h miniupnpc/miniupnpc.h miniupnpc/upnpcommands.h miniupnpc/upnperrors.h],
[AC_CHECK_LIB([miniupnpc], [upnpDiscover], [MINIUPNPC_LIBS=-lminiupnpc], [have_miniupnpc=no])],
[have_miniupnpc=no]
)
@@ -1388,46 +1282,44 @@ if test x$have_miniupnpc != xno; then
fi
fi
dnl Check for libnatpmp (optional).
if test "x$use_natpmp" != xno; then
AC_CHECK_HEADERS([natpmp.h],
[AC_CHECK_LIB([natpmp], [initnatpmp], [NATPMP_LIBS=-lnatpmp], [have_natpmp=no])],
[have_natpmp=no])
fi
if test x$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoind$bitcoin_enable_qt$use_tests$use_bench = xnonononononono; then
use_boost=no
use_boost=no
else
use_boost=yes
use_boost=yes
fi
if test x$use_boost = xyes; then
dnl Check for Boost headers
AX_BOOST_BASE([1.64.0],[],[AC_MSG_ERROR([Boost is not available!])])
if test x$want_boost = xno; then
dnl Minimum required Boost version
define(MINIMUM_REQUIRED_BOOST, 1.58.0)
dnl Check for Boost libs
AX_BOOST_BASE([MINIMUM_REQUIRED_BOOST])
if test x$want_boost = xno; then
AC_MSG_ERROR([[only libbitcoinconsensus can be built without boost]])
fi
AX_BOOST_SYSTEM
AX_BOOST_FILESYSTEM
fi
AX_BOOST_SYSTEM
AX_BOOST_FILESYSTEM
AX_BOOST_THREAD
if test x$suppress_external_warnings != xno; then
dnl Opt-in to boost-process
AS_IF([ test x$with_boost_process != x ], [ AX_BOOST_PROCESS ], [ ax_cv_boost_process=no ] )
if test x$suppress_external_warnings != xno; then
BOOST_CPPFLAGS=SUPPRESS_WARNINGS($BOOST_CPPFLAGS)
fi
BOOST_LIBS="$BOOST_LDFLAGS $BOOST_SYSTEM_LIB $BOOST_FILESYSTEM_LIB"
fi
if test "x$use_external_signer" != xno; then
AC_DEFINE([ENABLE_EXTERNAL_SIGNER],,[define if external signer support is enabled])
fi
AM_CONDITIONAL([ENABLE_EXTERNAL_SIGNER], [test "x$use_external_signer" = "xyes"])
dnl Boost 1.56 through 1.62 allow using std::atomic instead of its own atomic
dnl counter implementations. In 1.63 and later the std::atomic approach is default.
m4_pattern_allow(DBOOST_AC_USE_STD_ATOMIC) dnl otherwise it's treated like a macro
BOOST_CPPFLAGS="-DBOOST_SP_USE_STD_ATOMIC -DBOOST_AC_USE_STD_ATOMIC $BOOST_CPPFLAGS"
BOOST_LIBS="$BOOST_LDFLAGS $BOOST_SYSTEM_LIB $BOOST_FILESYSTEM_LIB $BOOST_THREAD_LIB"
fi
dnl Check for reduced exports
if test x$use_reduce_exports = xyes; then
AX_CHECK_COMPILE_FLAG([-fvisibility=hidden],[CXXFLAGS="$CXXFLAGS -fvisibility=hidden"],
[AC_MSG_ERROR([Cannot set hidden symbol visibility. Use --disable-reduce-exports.])],[[$CXXFLAG_WERROR]])
AX_CHECK_LINK_FLAG([[-Wl,--exclude-libs,ALL]],[RELDFLAGS="-Wl,--exclude-libs,ALL"],,[[$LDFLAG_WERROR]])
CXXFLAGS="$CXXFLAGS $RE_CXXFLAGS"
AX_CHECK_LINK_FLAG([[-Wl,--exclude-libs,ALL]], [RELDFLAGS="-Wl,--exclude-libs,ALL"],, [[$LDFLAG_WERROR]])
fi
if test x$use_tests = xyes; then
@@ -1438,25 +1330,25 @@ if test x$use_tests = xyes; then
if test x$use_boost = xyes; then
AX_BOOST_UNIT_TEST_FRAMEWORK
AX_BOOST_UNIT_TEST_FRAMEWORK
dnl Determine if -DBOOST_TEST_DYN_LINK is needed
AC_MSG_CHECKING([for dynamic linked boost test])
TEMP_LIBS="$LIBS"
LIBS="$LIBS $BOOST_LDFLAGS $BOOST_UNIT_TEST_FRAMEWORK_LIB"
TEMP_CPPFLAGS="$CPPFLAGS"
CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS"
AC_LINK_IFELSE([AC_LANG_SOURCE([
#define BOOST_TEST_DYN_LINK
#define BOOST_TEST_MAIN
#include <boost/test/unit_test.hpp>
dnl Determine if -DBOOST_TEST_DYN_LINK is needed
AC_MSG_CHECKING([for dynamic linked boost test])
TEMP_LIBS="$LIBS"
LIBS="$LIBS $BOOST_LDFLAGS $BOOST_UNIT_TEST_FRAMEWORK_LIB"
TEMP_CPPFLAGS="$CPPFLAGS"
CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS"
AC_LINK_IFELSE([AC_LANG_SOURCE([
#define BOOST_TEST_DYN_LINK
#define BOOST_TEST_MAIN
#include <boost/test/unit_test.hpp>
])],
[AC_MSG_RESULT(yes)]
[TESTDEFS="$TESTDEFS -DBOOST_TEST_DYN_LINK"],
[AC_MSG_RESULT(no)])
LIBS="$TEMP_LIBS"
CPPFLAGS="$TEMP_CPPFLAGS"
])],
[AC_MSG_RESULT(yes)]
[TESTDEFS="$TESTDEFS -DBOOST_TEST_DYN_LINK"],
[AC_MSG_RESULT(no)])
LIBS="$TEMP_LIBS"
CPPFLAGS="$TEMP_CPPFLAGS"
fi
fi
@@ -1468,10 +1360,6 @@ if test x$build_bitcoin_cli$build_bitcoind$bitcoin_enable_qt$use_tests$use_bench
if test x$TARGET_OS != xwindows; then
PKG_CHECK_MODULES([EVENT_PTHREADS], [libevent_pthreads >= 2.0.21],, [AC_MSG_ERROR([libevent_pthreads version 2.0.21 or greater not found.])])
fi
if test x$suppress_external_warnings != xno; then
EVENT_CFLAGS=SUPPRESS_WARNINGS($EVENT_CFLAGS)
fi
fi
dnl QR Code encoding library check
@@ -1504,7 +1392,7 @@ fi
dnl univalue check
need_bundled_univalue=yes
if test x$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoin_util$build_bitcoind$bitcoin_enable_qt$use_tests$use_bench = xnononononononono; then
if test x$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoind$bitcoin_enable_qt$use_tests$use_bench = xnonononononono; then
need_bundled_univalue=no
else
if test x$system_univalue != xno; then
@@ -1587,10 +1475,6 @@ AC_MSG_CHECKING([whether to build bitcoin-wallet])
AM_CONDITIONAL([BUILD_BITCOIN_WALLET], [test x$build_bitcoin_wallet = xyes])
AC_MSG_RESULT($build_bitcoin_wallet)
AC_MSG_CHECKING([whether to build bitcoin-util])
AM_CONDITIONAL([BUILD_BITCOIN_UTIL], [test x$build_bitcoin_util = xyes])
AC_MSG_RESULT($build_bitcoin_util)
AC_MSG_CHECKING([whether to build libraries])
AM_CONDITIONAL([BUILD_BITCOIN_LIBS], [test x$build_bitcoin_libs = xyes])
if test x$build_bitcoin_libs = xyes; then
@@ -1615,10 +1499,6 @@ if test "x$use_ccache" != "xno"; then
CXX="$ac_cv_path_CCACHE $CXX"
fi
AC_MSG_RESULT($use_ccache)
if test "x$use_ccache" = "xyes"; then
AX_CHECK_COMPILE_FLAG([-fdebug-prefix-map=A=B],[DEBUG_CXXFLAGS="$DEBUG_CXXFLAGS -fdebug-prefix-map=\$(abs_srcdir)=."],,[[$CXXFLAG_WERROR]])
AX_CHECK_PREPROC_FLAG([-fmacro-prefix-map=A=B],[DEBUG_CPPFLAGS="$DEBUG_CPPFLAGS -fmacro-prefix-map=\$(abs_srcdir)=."],,[[$CXXFLAG_WERROR]])
fi
fi
dnl enable wallet
@@ -1626,7 +1506,6 @@ AC_MSG_CHECKING([if wallet should be enabled])
if test x$enable_wallet != xno; then
AC_MSG_RESULT(yes)
AC_DEFINE_UNQUOTED([ENABLE_WALLET],[1],[Define to 1 to enable wallet functions])
enable_wallet=yes
else
AC_MSG_RESULT(no)
@@ -1660,34 +1539,6 @@ else
fi
fi
dnl Enable NAT-PMP support.
AC_MSG_CHECKING([whether to build with support for NAT-PMP])
if test "x$have_natpmp" = xno; then
if test "x$use_natpmp" = xyes; then
AC_MSG_ERROR([NAT-PMP requested but cannot be built. Use --without-natpmp])
fi
AC_MSG_RESULT([no])
use_natpmp=no
else
if test "x$use_natpmp" != xno; then
AC_MSG_RESULT([yes])
AC_MSG_CHECKING([whether to build with NAT-PMP enabled by default])
use_natpmp=yes
natpmp_setting=0
if test "x$use_natpmp_default" != xno; then
use_natpmp_default=yes
natpmp_setting=1
fi
AC_MSG_RESULT($use_natpmp_default)
AC_DEFINE_UNQUOTED([USE_NATPMP], [$natpmp_setting], [NAT-PMP support not compiled if undefined, otherwise value (0 or 1) determines default state])
if test x$TARGET_OS = xwindows; then
NATPMP_CPPFLAGS="-DSTATICLIB -DNATPMP_STATICLIB"
fi
else
AC_MSG_RESULT([no])
fi
fi
dnl these are only used when qt is enabled
BUILD_TEST_QT=""
if test x$bitcoin_enable_qt != xno; then
@@ -1730,11 +1581,7 @@ AM_CONDITIONAL([ENABLE_ZMQ], [test "x$use_zmq" = "xyes"])
AC_MSG_CHECKING([whether to build test_bitcoin])
if test x$use_tests = xyes; then
if test "x$enable_fuzz" = "xyes"; then
AC_MSG_RESULT([no, because fuzzing is enabled])
else
AC_MSG_RESULT([yes])
fi
AC_MSG_RESULT([yes])
BUILD_TEST="yes"
else
AC_MSG_RESULT([no])
@@ -1758,11 +1605,8 @@ AM_CONDITIONAL([TARGET_LINUX], [test x$TARGET_OS = xlinux])
AM_CONDITIONAL([TARGET_WINDOWS], [test x$TARGET_OS = xwindows])
AM_CONDITIONAL([ENABLE_WALLET],[test x$enable_wallet = xyes])
AM_CONDITIONAL([USE_SQLITE], [test "x$use_sqlite" = "xyes"])
AM_CONDITIONAL([USE_BDB], [test "x$use_bdb" = "xyes"])
AM_CONDITIONAL([ENABLE_TRACING],[test x$have_sdt = xyes])
AM_CONDITIONAL([ENABLE_TESTS],[test x$BUILD_TEST = xyes])
AM_CONDITIONAL([ENABLE_FUZZ],[test x$enable_fuzz = xyes])
AM_CONDITIONAL([ENABLE_FUZZ_BINARY],[test x$enable_fuzz_binary = xyes])
AM_CONDITIONAL([ENABLE_QT],[test x$bitcoin_enable_qt = xyes])
AM_CONDITIONAL([ENABLE_QT_TESTS],[test x$BUILD_TEST_QT = xyes])
AM_CONDITIONAL([ENABLE_BENCH],[test x$use_bench = xyes])
@@ -1778,11 +1622,10 @@ AM_CONDITIONAL([ENABLE_SHANI],[test x$enable_shani = xyes])
AM_CONDITIONAL([ENABLE_ARM_CRC],[test x$enable_arm_crc = xyes])
AM_CONDITIONAL([USE_ASM],[test x$use_asm = xyes])
AM_CONDITIONAL([WORDS_BIGENDIAN],[test x$ac_cv_c_bigendian = xyes])
AM_CONDITIONAL([USE_NATPMP],[test x$use_natpmp = xyes])
AM_CONDITIONAL([USE_UPNP],[test x$use_upnp = xyes])
AC_DEFINE(CLIENT_VERSION_MAJOR, _CLIENT_VERSION_MAJOR, [Major version])
AC_DEFINE(CLIENT_VERSION_MINOR, _CLIENT_VERSION_MINOR, [Minor version])
AC_DEFINE(CLIENT_VERSION_REVISION, _CLIENT_VERSION_REVISION, [Build revision])
AC_DEFINE(CLIENT_VERSION_BUILD, _CLIENT_VERSION_BUILD, [Version Build])
AC_DEFINE(CLIENT_VERSION_IS_RELEASE, _CLIENT_VERSION_IS_RELEASE, [Version is release])
AC_DEFINE(COPYRIGHT_YEAR, _COPYRIGHT_YEAR, [Copyright year])
@@ -1792,6 +1635,7 @@ define(_COPYRIGHT_HOLDERS_FINAL, [patsubst(_COPYRIGHT_HOLDERS, [%s], [_COPYRIGHT
AC_DEFINE(COPYRIGHT_HOLDERS_FINAL, "_COPYRIGHT_HOLDERS_FINAL", [Copyright holder(s)])
AC_SUBST(CLIENT_VERSION_MAJOR, _CLIENT_VERSION_MAJOR)
AC_SUBST(CLIENT_VERSION_MINOR, _CLIENT_VERSION_MINOR)
AC_SUBST(CLIENT_VERSION_REVISION, _CLIENT_VERSION_REVISION)
AC_SUBST(CLIENT_VERSION_BUILD, _CLIENT_VERSION_BUILD)
AC_SUBST(CLIENT_VERSION_IS_RELEASE, _CLIENT_VERSION_IS_RELEASE)
AC_SUBST(COPYRIGHT_YEAR, _COPYRIGHT_YEAR)
@@ -1802,10 +1646,7 @@ AC_SUBST(BITCOIN_DAEMON_NAME)
AC_SUBST(BITCOIN_GUI_NAME)
AC_SUBST(BITCOIN_CLI_NAME)
AC_SUBST(BITCOIN_TX_NAME)
AC_SUBST(BITCOIN_UTIL_NAME)
AC_SUBST(BITCOIN_WALLET_TOOL_NAME)
AC_SUBST(BITCOIN_MP_NODE_NAME)
AC_SUBST(BITCOIN_MP_GUI_NAME)
AC_SUBST(RELDFLAGS)
AC_SUBST(DEBUG_CPPFLAGS)
@@ -1830,8 +1671,6 @@ AC_SUBST(SHANI_CXXFLAGS)
AC_SUBST(ARM_CRC_CXXFLAGS)
AC_SUBST(LIBTOOL_APP_LDFLAGS)
AC_SUBST(USE_SQLITE)
AC_SUBST(USE_BDB)
AC_SUBST(ENABLE_EXTERNAL_SIGNER)
AC_SUBST(USE_UPNP)
AC_SUBST(USE_QRCODE)
AC_SUBST(BOOST_LIBS)
@@ -1839,8 +1678,6 @@ AC_SUBST(SQLITE_LIBS)
AC_SUBST(TESTDEFS)
AC_SUBST(MINIUPNPC_CPPFLAGS)
AC_SUBST(MINIUPNPC_LIBS)
AC_SUBST(NATPMP_CPPFLAGS)
AC_SUBST(NATPMP_LIBS)
AC_SUBST(EVENT_LIBS)
AC_SUBST(EVENT_PTHREADS_LIBS)
AC_SUBST(ZMQ_LIBS)
@@ -1853,14 +1690,11 @@ AC_SUBST(HAVE_BUILTIN_PREFETCH)
AC_SUBST(HAVE_MM_PREFETCH)
AC_SUBST(HAVE_STRONG_GETAUXVAL)
AC_SUBST(HAVE_WEAK_GETAUXVAL)
AC_SUBST(ANDROID_ARCH)
AC_CONFIG_FILES([Makefile src/Makefile doc/man/Makefile share/setup.nsi share/qt/Info.plist test/config.ini])
AC_CONFIG_FILES([contrib/devtools/split-debug.sh],[chmod +x contrib/devtools/split-debug.sh])
AM_COND_IF([HAVE_DOXYGEN], [AC_CONFIG_FILES([doc/Doxyfile])])
AC_CONFIG_LINKS([contrib/devtools/security-check.py:contrib/devtools/security-check.py])
AC_CONFIG_LINKS([contrib/devtools/symbol-check.py:contrib/devtools/symbol-check.py])
AC_CONFIG_LINKS([contrib/devtools/test-security-check.py:contrib/devtools/test-security-check.py])
AC_CONFIG_LINKS([contrib/devtools/test-symbol-check.py:contrib/devtools/test-symbol-check.py])
AC_CONFIG_LINKS([contrib/filter-lcov.py:contrib/filter-lcov.py])
AC_CONFIG_LINKS([test/functional/test_runner.py:test/functional/test_runner.py])
AC_CONFIG_LINKS([test/fuzz/test_runner.py:test/fuzz/test_runner.py])
@@ -1894,7 +1728,7 @@ if test x$need_bundled_univalue = xyes; then
AC_CONFIG_SUBDIRS([src/univalue])
fi
ac_configure_args="${ac_configure_args} --disable-shared --with-pic --enable-benchmark=no --enable-module-recovery --enable-module-schnorrsig --enable-experimental --disable-openssl-tests"
ac_configure_args="${ac_configure_args} --disable-shared --with-pic --enable-benchmark=no --with-bignum=no --enable-module-recovery --enable-module-schnorrsig --enable-experimental"
AC_CONFIG_SUBDIRS([src/secp256k1])
AC_OUTPUT
@@ -1909,43 +1743,37 @@ esac
echo
echo "Options used to compile and link:"
echo " external signer = $use_external_signer"
echo " multiprocess = $build_multiprocess"
echo " with libs = $build_bitcoin_libs"
echo " with wallet = $enable_wallet"
echo " boost process = $ax_cv_boost_process"
echo " multiprocess = $build_multiprocess"
echo " with wallet = $enable_wallet"
if test "x$enable_wallet" != "xno"; then
echo " with sqlite = $use_sqlite"
echo " with bdb = $use_bdb"
echo " with sqlite = $use_sqlite"
fi
echo " with gui / qt = $bitcoin_enable_qt"
echo " with gui / qt = $bitcoin_enable_qt"
if test x$bitcoin_enable_qt != xno; then
echo " with qr = $use_qr"
echo " with qr = $use_qr"
fi
echo " with zmq = $use_zmq"
if test x$enable_fuzz == xno; then
echo " with test = $use_tests"
else
echo " with test = not building test_bitcoin because fuzzing is enabled"
echo " with fuzz = $enable_fuzz"
echo " with zmq = $use_zmq"
echo " with test = $use_tests"
if test x$use_tests != xno; then
echo " with fuzz = $enable_fuzz"
fi
echo " with bench = $use_bench"
echo " with upnp = $use_upnp"
echo " with natpmp = $use_natpmp"
echo " use asm = $use_asm"
echo " ebpf tracing = $have_sdt"
echo " sanitizers = $use_sanitizers"
echo " debug enabled = $enable_debug"
echo " gprof enabled = $enable_gprof"
echo " werror = $enable_werror"
echo " with bench = $use_bench"
echo " with upnp = $use_upnp"
echo " use asm = $use_asm"
echo " sanitizers = $use_sanitizers"
echo " debug enabled = $enable_debug"
echo " gprof enabled = $enable_gprof"
echo " werror = $enable_werror"
echo
echo " target os = $TARGET_OS"
echo " build os = $build_os"
echo " target os = $TARGET_OS"
echo " build os = $build_os"
echo
echo " CC = $CC"
echo " CFLAGS = $PTHREAD_CFLAGS $CFLAGS"
echo " CPPFLAGS = $DEBUG_CPPFLAGS $HARDENED_CPPFLAGS $CPPFLAGS"
echo " CXX = $CXX"
echo " CXXFLAGS = $DEBUG_CXXFLAGS $HARDENED_CXXFLAGS $WARN_CXXFLAGS $NOWARN_CXXFLAGS $ERROR_CXXFLAGS $GPROF_CXXFLAGS $CXXFLAGS"
echo " LDFLAGS = $PTHREAD_LIBS $HARDENED_LDFLAGS $GPROF_LDFLAGS $LDFLAGS"
echo " ARFLAGS = $ARFLAGS"
echo " CC = $CC"
echo " CFLAGS = $PTHREAD_CFLAGS $CFLAGS"
echo " CPPFLAGS = $DEBUG_CPPFLAGS $HARDENED_CPPFLAGS $CPPFLAGS"
echo " CXX = $CXX"
echo " CXXFLAGS = $DEBUG_CXXFLAGS $HARDENED_CXXFLAGS $WARN_CXXFLAGS $NOWARN_CXXFLAGS $ERROR_CXXFLAGS $GPROF_CXXFLAGS $CXXFLAGS"
echo " LDFLAGS = $PTHREAD_LIBS $HARDENED_LDFLAGS $GPROF_LDFLAGS $LDFLAGS"
echo " ARFLAGS = $ARFLAGS"
echo

View File

@@ -29,8 +29,8 @@ All other packaging related files can be found in the [bitcoin-core/packaging](h
### [Gitian-descriptors](/contrib/gitian-descriptors) ###
Files used during the gitian build process. For more information about gitian, see the [the Bitcoin Core documentation repository](https://github.com/bitcoin-core/docs).
### [Builder keys](/contrib/builder-keys)
PGP keys used for signing Bitcoin Core [release](/doc/release-process.md) results.
### [Gitian-keys](/contrib/gitian-keys)
PGP keys used for signing Bitcoin Core [Gitian release](/doc/release-process.md) results.
### [MacDeploy](/contrib/macdeploy) ###
Scripts and notes for Mac builds.

22
contrib/bitcoin-qt.pro Normal file
View File

@@ -0,0 +1,22 @@
FORMS += \
../src/qt/forms/aboutdialog.ui \
../src/qt/forms/addressbookpage.ui \
../src/qt/forms/askpassphrasedialog.ui \
../src/qt/forms/coincontroldialog.ui \
../src/qt/forms/editaddressdialog.ui \
../src/qt/forms/helpmessagedialog.ui \
../src/qt/forms/intro.ui \
../src/qt/forms/openuridialog.ui \
../src/qt/forms/optionsdialog.ui \
../src/qt/forms/overviewpage.ui \
../src/qt/forms/receivecoinsdialog.ui \
../src/qt/forms/receiverequestdialog.ui \
../src/qt/forms/debugwindow.ui \
../src/qt/forms/sendcoinsdialog.ui \
../src/qt/forms/sendcoinsentry.ui \
../src/qt/forms/signverifymessagedialog.ui \
../src/qt/forms/transactiondescdialog.ui \
../src/qt/forms/createwalletdialog.ui
RESOURCES += \
../src/qt/bitcoin.qrc

View File

@@ -1,51 +0,0 @@
9D3CC86A72F8494342EA5FD10A41BDC3F4FAFF1C Aaron Clauson (sipsorcery)
617C90010B3BD370B0AC7D424BB42E31C79111B8 Akira Takizawa (akx20000)
E944AE667CF960B1004BC32FCA662BE18B877A60 Andreas Schildbach (aschildbach)
152812300785C96444D3334D17565732E08E5E41 Andrew Chow (achow101)
590B7292695AFFA5B672CBB2E13FC145CD3F4304 Antoine Poinsot (darosior)
0AD83877C1F0CD1EE9BD660AD7CC770B81FD22A8 Ben Carman (benthecarman)
912FD3228387123DC97E0E57D5566241A0295FA9 BtcDrak (btcdrak)
C519EBCF3B926298946783EFF6430754120EC2F4 Christian Decker (cdecker)
18AE2F798E0D239755DA4FD24B79F986CBDF8736 Chun Kuan Le (ken2812221)
101598DC823C1B5F9A6624ABA5E0907A0380E6C3 CoinForensics (CoinForensics)
F20F56EF6A067F70E8A5C99FFF95FAA971697405 centaur (centaur)
C060A6635913D98A3587D7DB1C2491FFEB0EF770 Cory Fields (cfields)
BF6273FAEF7CC0BA1F562E50989F6B3048A116B5 Dev Random (devrandom)
6D3170C1DC2C6FD0AEEBCA6743811D1A26623924 Douglas Roark (droark)
1C6621605EC50319C463D56C7F81D87985D61612 Emanuele Cisbani (cisba)
9A1689B60D1B3CCE9262307A2F40A9BF167FBA47 Erik Mossberg (erkmos)
D35176BE9264832E4ACA8986BF0792FBE95DC863 fivepiece (fivepiece)
6F993B250557E7B016ADE5713BDCDA2D87A881D9 Fuzzbawls (Fuzzbawls)
01CDF4627A3B88AAE4A571C87588242FBE38D3A8 Gavin Andresen (gavinandresen)
D1DBF2C4B96F2DEBF4C16654410108112E7EA81F Hennadii Stepanov (hebasto)
A2FD494D0021AA9B4FA58F759102B7AE654A4A5A Ilyas Ridhuan (IlyasRidhuan)
D3F22A3A4C366C2DCB66D3722DA9C5A7FA81EA35 Jarol Rodriguez (jarolrod)
7480909378D544EA6B6DCEB7535B12980BB8A4D3 Jeffri H Frontz (jhfrontz)
D3CC177286005BB8FF673294C5242A1AB3936517 jl2012 (jl2012)
82921A4B88FD454B7EB8CE3C796C4109063D4EAF Jon Atack (jonatack)
32EE5C4C3FA15CCADB46ABE529D4BCB6416F53EC Jonas Schnelli (jonasschnelli)
4B4E840451149DD7FB0D633477DFAB5C3108B9A8 Jorge Timon (jtimon)
C42AFF7C61B3E44A1454CD3557AF762DB3353322 Karl-Johan Alm (kallewoof)
30DE693AE0DE9E37B3E7EB6BBFF0F67810C1EED1 Lisa Neigut (niftynei)
E463A93F5F3117EEDE6C7316BD02942421F4889F Luke Dashjr (luke-jr)
B8B3F1C0E58C15DB6A81D30C3648A882F4316B9B Marco Falke (marco)
07DF3E57A548CCFB7530709189BBB8663E2E65CE Matt Corallo (BlueMatt)
CA03882CB1FC067B5D3ACFE4D300116E1C875A3D MeshCollider (meshcollider)
E777299FC265DD04793070EB944D35F9AC3DB76A Michael Ford (fanquake)
AD5764F4ADCE1B99BDFD179E12335A271D4D62EC Michael Tidwell (miketwenty1)
9692B91BBF0E8D34DFD33B1882C5C009628ECF0C Michagogo (michagogo)
C57E4B42223FDE851D4F69DD28DF2724F241D8EE midnightmagic (midnightmagic)
F4FC70F07310028424EFC20A8E4256593F177720 Oliver Gugger (guggero, Oliver Gugger)
D62A803E27E7F43486035ADBBCD04D8E9CCCAC2A Paul Rabahy (prab)
37EC7D7B0A217CDB4B4E007E7FAB114267E4FA04 Peter Todd (petertodd)
D762373D24904A3E42F33B08B9A408E71DAAC974 Pieter Wuille [Location: Leuven, Belgium] (sipa)
133EAC179436F14A5CF1B794860FEB804E669320 Pieter Wuille (sipa)
6A8F9C266528E25AEB1D7731C2371D91CB716EA7 Sebastian Falbesoner (theStack)
A8FC55F3B04BA3146F3492E79303B33A305224CB Sebastian Kung (TheCharlatan)
ED9BDF7AD6A55E232E84524257FF9BDBCC301009 Sjors Provoost (sjors)
867345026B6763E8B07EE73AB6737117397F5C4F Stephan Oeste (Emzy)
9EDAFF80E080659604F4A76B2EBB056FD847F8A7 Stephan Oeste (Emzy)
6DEEF79B050C4072509B743F8C275BC595448867 Tomas Kanocz (KanoczTomas)
AEC1884398647C47413C1C3FB1179EB7347DC10D Warren Togami (wtogami)
79D00BAC68B56D422F945A8F8E3A8F3247DBCBBF Willy Ko (willyko)
71A3B16735405025D447E8F274810B012346C9A6 Wladimir J. van der Laan (laanwj)

View File

@@ -1,11 +1,11 @@
Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: Bitcoin
Upstream-Contact: Satoshi Nakamoto <satoshin@gmx.com>
irc://#bitcoin-core-dev@libera.chat
irc://#bitcoin@freenode.net
Source: https://github.com/bitcoin/bitcoin
Files: *
Copyright: 2009-2021, Bitcoin Core Developers
Copyright: 2009-2020, Bitcoin Core Developers
License: Expat
Comment: The Bitcoin Core Developers encompasses the current developers listed on bitcoin.org,
as well as the numerous contributors to the project.
@@ -87,10 +87,6 @@ Files: src/qt/res/icons/proxy.png
Copyright: Cristian Mircea Messel
License: public-domain
Files: src/qt/res/fonts/RobotoMono-Bold.ttf
License: Apache-2.0
Comment: Site: https://fonts.google.com/specimen/Roboto+Mono
License: Expat
Permission is hereby granted, free of charge, to any person obtaining a
@@ -148,14 +144,3 @@ Comment:
License: public-domain
This work is in the public domain.
License: Apache-2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -7,8 +7,7 @@ clang-format-diff.py
A script to format unified git diffs according to [.clang-format](../../src/.clang-format).
Requires `clang-format`, installed e.g. via `brew install clang-format` on macOS,
or `sudo apt install clang-format` on Debian/Ubuntu.
Requires `clang-format`, installed e.g. via `brew install clang-format` on macOS.
For instance, to format the last commit with 0 lines of context,
the script should be called from the git root folder as follows.

View File

@@ -1,11 +1,10 @@
#!/usr/bin/env python3
# Copyright (c) 2018-2020 The Bitcoin Core developers
# Copyright (c) 2018-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
import sys
import re
from typing import Dict, List, Set
MAPPING = {
'core_read.cpp': 'core_io.cpp',
@@ -33,7 +32,7 @@ def module_name(path):
return None
files = dict()
deps: Dict[str, Set[str]] = dict()
deps = dict()
RE = re.compile("^#include <(.*)>")
@@ -60,12 +59,12 @@ for arg in sorted(files.keys()):
deps[module].add(included_module)
# Loop to find the shortest (remaining) circular dependency
have_cycle: bool = False
have_cycle = False
while True:
shortest_cycle = None
for module in sorted(deps.keys()):
# Build the transitive closure of dependencies of module
closure: Dict[str, List[str]] = dict()
closure = dict()
for dep in deps[module]:
closure[dep] = []
while True:

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# Copyright (c) 2016-2020 The Bitcoin Core developers
# Copyright (c) 2016-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@@ -14,27 +14,10 @@ BITCOIND=${BITCOIND:-$BINDIR/bitcoind}
BITCOINCLI=${BITCOINCLI:-$BINDIR/bitcoin-cli}
BITCOINTX=${BITCOINTX:-$BINDIR/bitcoin-tx}
WALLET_TOOL=${WALLET_TOOL:-$BINDIR/bitcoin-wallet}
BITCOINUTIL=${BITCOINQT:-$BINDIR/bitcoin-util}
BITCOINQT=${BITCOINQT:-$BINDIR/qt/bitcoin-qt}
[ ! -x $BITCOIND ] && echo "$BITCOIND not found or not executable." && exit 1
# Don't allow man pages to be generated for binaries built from a dirty tree
DIRTY=""
for cmd in $BITCOIND $BITCOINCLI $BITCOINTX $WALLET_TOOL $BITCOINUTIL $BITCOINQT; do
VERSION_OUTPUT=$($cmd --version)
if [[ $VERSION_OUTPUT == *"dirty"* ]]; then
DIRTY="${DIRTY}${cmd}\n"
fi
done
if [ -n "$DIRTY" ]
then
echo -e "WARNING: the following binaries were built from a dirty tree:\n"
echo -e $DIRTY
echo "man pages generated from dirty binaries should NOT be committed."
echo "To properly generate man pages, please commit your changes to the above binaries, rebuild them, then run this script again."
fi
# The autodetected version git tag can screw up manpage output a little bit
read -r -a BTCVER <<< "$($BITCOINCLI --version | head -n1 | awk -F'[ -]' '{ print $6, $7 }')"
@@ -44,9 +27,10 @@ read -r -a BTCVER <<< "$($BITCOINCLI --version | head -n1 | awk -F'[ -]' '{ prin
echo "[COPYRIGHT]" > footer.h2m
$BITCOIND --version | sed -n '1!p' >> footer.h2m
for cmd in $BITCOIND $BITCOINCLI $BITCOINTX $WALLET_TOOL $BITCOINUTIL $BITCOINQT; do
for cmd in $BITCOIND $BITCOINCLI $BITCOINTX $WALLET_TOOL $BITCOINQT; do
cmdname="${cmd##*/}"
help2man -N --version-string=${BTCVER[0]} --include=footer.h2m -o ${MANDIR}/${cmdname}.1 ${cmd}
sed -i "s/\\\-${BTCVER[1]}//g" ${MANDIR}/${cmdname}.1
done
rm -f footer.h2m

View File

@@ -1,323 +0,0 @@
#!/usr/bin/env python3
# Copyright (c) 2020 Wladimir J. van der Laan
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
'''
Compact, self-contained ELF implementation for bitcoin-core security checks.
'''
import struct
import types
from typing import Dict, List, Optional, Union, Tuple
# you can find all these values in elf.h
EI_NIDENT = 16
# Byte indices in e_ident
EI_CLASS = 4 # ELFCLASSxx
EI_DATA = 5 # ELFDATAxxxx
ELFCLASS32 = 1 # 32-bit
ELFCLASS64 = 2 # 64-bit
ELFDATA2LSB = 1 # little endian
ELFDATA2MSB = 2 # big endian
# relevant values for e_machine
EM_386 = 3
EM_PPC64 = 21
EM_ARM = 40
EM_AARCH64 = 183
EM_X86_64 = 62
EM_RISCV = 243
# relevant values for e_type
ET_DYN = 3
# relevant values for sh_type
SHT_PROGBITS = 1
SHT_STRTAB = 3
SHT_DYNAMIC = 6
SHT_DYNSYM = 11
SHT_GNU_verneed = 0x6ffffffe
SHT_GNU_versym = 0x6fffffff
# relevant values for p_type
PT_LOAD = 1
PT_GNU_STACK = 0x6474e551
PT_GNU_RELRO = 0x6474e552
# relevant values for p_flags
PF_X = (1 << 0)
PF_W = (1 << 1)
PF_R = (1 << 2)
# relevant values for d_tag
DT_NEEDED = 1
DT_FLAGS = 30
# relevant values of `d_un.d_val' in the DT_FLAGS entry
DF_BIND_NOW = 0x00000008
# relevant d_tags with string payload
STRING_TAGS = {DT_NEEDED}
# rrlevant values for ST_BIND subfield of st_info (symbol binding)
STB_LOCAL = 0
class ELFRecord(types.SimpleNamespace):
'''Unified parsing for ELF records.'''
def __init__(self, data: bytes, offset: int, eh: 'ELFHeader', total_size: Optional[int]) -> None:
hdr_struct = self.STRUCT[eh.ei_class][0][eh.ei_data]
if total_size is not None and hdr_struct.size > total_size:
raise ValueError(f'{self.__class__.__name__} header size too small ({total_size} < {hdr_struct.size})')
for field, value in zip(self.STRUCT[eh.ei_class][1], hdr_struct.unpack(data[offset:offset + hdr_struct.size])):
setattr(self, field, value)
def BiStruct(chars: str) -> Dict[int, struct.Struct]:
'''Compile a struct parser for both endians.'''
return {
ELFDATA2LSB: struct.Struct('<' + chars),
ELFDATA2MSB: struct.Struct('>' + chars),
}
class ELFHeader(ELFRecord):
FIELDS = ['e_type', 'e_machine', 'e_version', 'e_entry', 'e_phoff', 'e_shoff', 'e_flags', 'e_ehsize', 'e_phentsize', 'e_phnum', 'e_shentsize', 'e_shnum', 'e_shstrndx']
STRUCT = {
ELFCLASS32: (BiStruct('HHIIIIIHHHHHH'), FIELDS),
ELFCLASS64: (BiStruct('HHIQQQIHHHHHH'), FIELDS),
}
def __init__(self, data: bytes, offset: int) -> None:
self.e_ident = data[offset:offset + EI_NIDENT]
if self.e_ident[0:4] != b'\x7fELF':
raise ValueError('invalid ELF magic')
self.ei_class = self.e_ident[EI_CLASS]
self.ei_data = self.e_ident[EI_DATA]
super().__init__(data, offset + EI_NIDENT, self, None)
def __repr__(self) -> str:
return f'Header(e_ident={self.e_ident!r}, e_type={self.e_type}, e_machine={self.e_machine}, e_version={self.e_version}, e_entry={self.e_entry}, e_phoff={self.e_phoff}, e_shoff={self.e_shoff}, e_flags={self.e_flags}, e_ehsize={self.e_ehsize}, e_phentsize={self.e_phentsize}, e_phnum={self.e_phnum}, e_shentsize={self.e_shentsize}, e_shnum={self.e_shnum}, e_shstrndx={self.e_shstrndx})'
class Section(ELFRecord):
name: Optional[bytes] = None
FIELDS = ['sh_name', 'sh_type', 'sh_flags', 'sh_addr', 'sh_offset', 'sh_size', 'sh_link', 'sh_info', 'sh_addralign', 'sh_entsize']
STRUCT = {
ELFCLASS32: (BiStruct('IIIIIIIIII'), FIELDS),
ELFCLASS64: (BiStruct('IIQQQQIIQQ'), FIELDS),
}
def __init__(self, data: bytes, offset: int, eh: ELFHeader) -> None:
super().__init__(data, offset, eh, eh.e_shentsize)
self._data = data
def __repr__(self) -> str:
return f'Section(sh_name={self.sh_name}({self.name!r}), sh_type=0x{self.sh_type:x}, sh_flags={self.sh_flags}, sh_addr=0x{self.sh_addr:x}, sh_offset=0x{self.sh_offset:x}, sh_size={self.sh_size}, sh_link={self.sh_link}, sh_info={self.sh_info}, sh_addralign={self.sh_addralign}, sh_entsize={self.sh_entsize})'
def contents(self) -> bytes:
'''Return section contents.'''
return self._data[self.sh_offset:self.sh_offset + self.sh_size]
class ProgramHeader(ELFRecord):
STRUCT = {
# different ELF classes have the same fields, but in a different order to optimize space versus alignment
ELFCLASS32: (BiStruct('IIIIIIII'), ['p_type', 'p_offset', 'p_vaddr', 'p_paddr', 'p_filesz', 'p_memsz', 'p_flags', 'p_align']),
ELFCLASS64: (BiStruct('IIQQQQQQ'), ['p_type', 'p_flags', 'p_offset', 'p_vaddr', 'p_paddr', 'p_filesz', 'p_memsz', 'p_align']),
}
def __init__(self, data: bytes, offset: int, eh: ELFHeader) -> None:
super().__init__(data, offset, eh, eh.e_phentsize)
def __repr__(self) -> str:
return f'ProgramHeader(p_type={self.p_type}, p_offset={self.p_offset}, p_vaddr={self.p_vaddr}, p_paddr={self.p_paddr}, p_filesz={self.p_filesz}, p_memsz={self.p_memsz}, p_flags={self.p_flags}, p_align={self.p_align})'
class Symbol(ELFRecord):
STRUCT = {
# different ELF classes have the same fields, but in a different order to optimize space versus alignment
ELFCLASS32: (BiStruct('IIIBBH'), ['st_name', 'st_value', 'st_size', 'st_info', 'st_other', 'st_shndx']),
ELFCLASS64: (BiStruct('IBBHQQ'), ['st_name', 'st_info', 'st_other', 'st_shndx', 'st_value', 'st_size']),
}
def __init__(self, data: bytes, offset: int, eh: ELFHeader, symtab: Section, strings: bytes, version: Optional[bytes]) -> None:
super().__init__(data, offset, eh, symtab.sh_entsize)
self.name = _lookup_string(strings, self.st_name)
self.version = version
def __repr__(self) -> str:
return f'Symbol(st_name={self.st_name}({self.name!r}), st_value={self.st_value}, st_size={self.st_size}, st_info={self.st_info}, st_other={self.st_other}, st_shndx={self.st_shndx}, version={self.version!r})'
@property
def is_import(self) -> bool:
'''Returns whether the symbol is an imported symbol.'''
return self.st_bind != STB_LOCAL and self.st_shndx == 0
@property
def is_export(self) -> bool:
'''Returns whether the symbol is an exported symbol.'''
return self.st_bind != STB_LOCAL and self.st_shndx != 0
@property
def st_bind(self) -> int:
'''Returns STB_*.'''
return self.st_info >> 4
class Verneed(ELFRecord):
DEF = (BiStruct('HHIII'), ['vn_version', 'vn_cnt', 'vn_file', 'vn_aux', 'vn_next'])
STRUCT = { ELFCLASS32: DEF, ELFCLASS64: DEF }
def __init__(self, data: bytes, offset: int, eh: ELFHeader) -> None:
super().__init__(data, offset, eh, None)
def __repr__(self) -> str:
return f'Verneed(vn_version={self.vn_version}, vn_cnt={self.vn_cnt}, vn_file={self.vn_file}, vn_aux={self.vn_aux}, vn_next={self.vn_next})'
class Vernaux(ELFRecord):
DEF = (BiStruct('IHHII'), ['vna_hash', 'vna_flags', 'vna_other', 'vna_name', 'vna_next'])
STRUCT = { ELFCLASS32: DEF, ELFCLASS64: DEF }
def __init__(self, data: bytes, offset: int, eh: ELFHeader, strings: bytes) -> None:
super().__init__(data, offset, eh, None)
self.name = _lookup_string(strings, self.vna_name)
def __repr__(self) -> str:
return f'Veraux(vna_hash={self.vna_hash}, vna_flags={self.vna_flags}, vna_other={self.vna_other}, vna_name={self.vna_name}({self.name!r}), vna_next={self.vna_next})'
class DynTag(ELFRecord):
STRUCT = {
ELFCLASS32: (BiStruct('II'), ['d_tag', 'd_val']),
ELFCLASS64: (BiStruct('QQ'), ['d_tag', 'd_val']),
}
def __init__(self, data: bytes, offset: int, eh: ELFHeader, section: Section) -> None:
super().__init__(data, offset, eh, section.sh_entsize)
def __repr__(self) -> str:
return f'DynTag(d_tag={self.d_tag}, d_val={self.d_val})'
def _lookup_string(data: bytes, index: int) -> bytes:
'''Look up string by offset in ELF string table.'''
endx = data.find(b'\x00', index)
assert endx != -1
return data[index:endx]
VERSYM_S = BiStruct('H') # .gnu_version section has a single 16-bit integer per symbol in the linked section
def _parse_symbol_table(section: Section, strings: bytes, eh: ELFHeader, versym: bytes, verneed: Dict[int, bytes]) -> List[Symbol]:
'''Parse symbol table, return a list of symbols.'''
data = section.contents()
symbols = []
versym_iter = (verneed.get(v[0]) for v in VERSYM_S[eh.ei_data].iter_unpack(versym))
for ofs, version in zip(range(0, len(data), section.sh_entsize), versym_iter):
symbols.append(Symbol(data, ofs, eh, section, strings, version))
return symbols
def _parse_verneed(section: Section, strings: bytes, eh: ELFHeader) -> Dict[int, bytes]:
'''Parse .gnu.version_r section, return a dictionary of {versym: 'GLIBC_...'}.'''
data = section.contents()
ofs = 0
result = {}
while True:
verneed = Verneed(data, ofs, eh)
aofs = ofs + verneed.vn_aux
while True:
vernaux = Vernaux(data, aofs, eh, strings)
result[vernaux.vna_other] = vernaux.name
if not vernaux.vna_next:
break
aofs += vernaux.vna_next
if not verneed.vn_next:
break
ofs += verneed.vn_next
return result
def _parse_dyn_tags(section: Section, strings: bytes, eh: ELFHeader) -> List[Tuple[int, Union[bytes, int]]]:
'''Parse dynamic tags. Return array of tuples.'''
data = section.contents()
ofs = 0
result = []
for ofs in range(0, len(data), section.sh_entsize):
tag = DynTag(data, ofs, eh, section)
val = _lookup_string(strings, tag.d_val) if tag.d_tag in STRING_TAGS else tag.d_val
result.append((tag.d_tag, val))
return result
class ELFFile:
sections: List[Section]
program_headers: List[ProgramHeader]
dyn_symbols: List[Symbol]
dyn_tags: List[Tuple[int, Union[bytes, int]]]
def __init__(self, data: bytes) -> None:
self.data = data
self.hdr = ELFHeader(self.data, 0)
self._load_sections()
self._load_program_headers()
self._load_dyn_symbols()
self._load_dyn_tags()
self._section_to_segment_mapping()
def _load_sections(self) -> None:
self.sections = []
for idx in range(self.hdr.e_shnum):
offset = self.hdr.e_shoff + idx * self.hdr.e_shentsize
self.sections.append(Section(self.data, offset, self.hdr))
shstr = self.sections[self.hdr.e_shstrndx].contents()
for section in self.sections:
section.name = _lookup_string(shstr, section.sh_name)
def _load_program_headers(self) -> None:
self.program_headers = []
for idx in range(self.hdr.e_phnum):
offset = self.hdr.e_phoff + idx * self.hdr.e_phentsize
self.program_headers.append(ProgramHeader(self.data, offset, self.hdr))
def _load_dyn_symbols(self) -> None:
# first, load 'verneed' section
verneed = None
for section in self.sections:
if section.sh_type == SHT_GNU_verneed:
strtab = self.sections[section.sh_link].contents() # associated string table
assert verneed is None # only one section of this kind please
verneed = _parse_verneed(section, strtab, self.hdr)
assert verneed is not None
# then, correlate GNU versym sections with dynamic symbol sections
versym = {}
for section in self.sections:
if section.sh_type == SHT_GNU_versym:
versym[section.sh_link] = section
# finally, load dynsym sections
self.dyn_symbols = []
for idx, section in enumerate(self.sections):
if section.sh_type == SHT_DYNSYM: # find dynamic symbol tables
strtab_data = self.sections[section.sh_link].contents() # associated string table
versym_data = versym[idx].contents() # associated symbol version table
self.dyn_symbols += _parse_symbol_table(section, strtab_data, self.hdr, versym_data, verneed)
def _load_dyn_tags(self) -> None:
self.dyn_tags = []
for idx, section in enumerate(self.sections):
if section.sh_type == SHT_DYNAMIC: # find dynamic tag tables
strtab = self.sections[section.sh_link].contents() # associated string table
self.dyn_tags += _parse_dyn_tags(section, strtab, self.hdr)
def _section_to_segment_mapping(self) -> None:
for ph in self.program_headers:
ph.sections = []
for section in self.sections:
if ph.p_vaddr <= section.sh_addr < (ph.p_vaddr + ph.p_memsz):
ph.sections.append(section)
def query_dyn_tags(self, tag_in: int) -> List[Union[int, bytes]]:
'''Return the values of all dyn tags with the specified tag.'''
return [val for (tag, val) in self.dyn_tags if tag == tag_in]
def load(filename: str) -> ELFFile:
with open(filename, 'rb') as f:
data = f.read()
return ELFFile(data)

View File

@@ -6,31 +6,95 @@
Perform basic security checks on a series of executables.
Exit status will be 0 if successful, and the program will be silent.
Otherwise the exit status will be 1 and it will log which executables failed which checks.
Needs `readelf` (for ELF), `objdump` (for PE) and `otool` (for MACHO).
'''
import subprocess
import sys
import os
from typing import List, Optional
import lief
import pixie
READELF_CMD = os.getenv('READELF', '/usr/bin/readelf')
OBJDUMP_CMD = os.getenv('OBJDUMP', '/usr/bin/objdump')
OTOOL_CMD = os.getenv('OTOOL', '/usr/bin/otool')
def run_command(command) -> str:
p = subprocess.run(command, stdout=subprocess.PIPE, check=True, universal_newlines=True)
return p.stdout
def check_ELF_PIE(executable) -> bool:
'''
Check for position independent executable (PIE), allowing for address space randomization.
'''
elf = pixie.load(executable)
return elf.hdr.e_type == pixie.ET_DYN
stdout = run_command([READELF_CMD, '-h', '-W', executable])
ok = False
for line in stdout.splitlines():
tokens = line.split()
if len(line)>=2 and tokens[0] == 'Type:' and tokens[1] == 'DYN':
ok = True
return ok
def get_ELF_program_headers(executable):
'''Return type and flags for ELF program headers'''
stdout = run_command([READELF_CMD, '-l', '-W', executable])
in_headers = False
headers = []
for line in stdout.splitlines():
if line.startswith('Program Headers:'):
in_headers = True
count = 0
if line == '':
in_headers = False
if in_headers:
if count == 1: # header line
header = [x.strip() for x in line.split()]
ofs_typ = header.index('Type')
ofs_flags = header.index('Flg')
# assert readelf output is what we expect
if ofs_typ == -1 or ofs_flags == -1:
raise ValueError('Cannot parse elfread -lW output')
elif count > 1:
splitline = [x.strip() for x in line.split()]
typ = splitline[ofs_typ]
if not typ.startswith('[R'): # skip [Requesting ...]
splitline = [x.strip() for x in line.split()]
flags = splitline[ofs_flags]
# check for 'R', ' E'
if splitline[ofs_flags + 1] == 'E':
flags += ' E'
headers.append((typ, flags, []))
count += 1
if line.startswith(' Section to Segment mapping:'):
in_mapping = True
count = 0
if line == '':
in_mapping = False
if in_mapping:
if count == 1: # header line
ofs_segment = line.find('Segment')
ofs_sections = line.find('Sections...')
if ofs_segment == -1 or ofs_sections == -1:
raise ValueError('Cannot parse elfread -lW output')
elif count > 1:
segment = int(line[ofs_segment:ofs_sections].strip())
sections = line[ofs_sections:].strip().split()
headers[segment][2].extend(sections)
count += 1
return headers
def check_ELF_NX(executable) -> bool:
'''
Check that no sections are writable and executable (including the stack)
'''
elf = pixie.load(executable)
have_wx = False
have_gnu_stack = False
for ph in elf.program_headers:
if ph.p_type == pixie.PT_GNU_STACK:
for (typ, flags, _) in get_ELF_program_headers(executable):
if typ == 'GNU_STACK':
have_gnu_stack = True
if (ph.p_flags & pixie.PF_W) != 0 and (ph.p_flags & pixie.PF_X) != 0: # section is both writable and executable
if 'W' in flags and 'E' in flags: # section is both writable and executable
have_wx = True
return have_gnu_stack and not have_wx
@@ -40,34 +104,35 @@ def check_ELF_RELRO(executable) -> bool:
GNU_RELRO program header must exist
Dynamic section must have BIND_NOW flag
'''
elf = pixie.load(executable)
have_gnu_relro = False
for ph in elf.program_headers:
# Note: not checking p_flags == PF_R: here as linkers set the permission differently
for (typ, flags, _) in get_ELF_program_headers(executable):
# Note: not checking flags == 'R': here as linkers set the permission differently
# This does not affect security: the permission flags of the GNU_RELRO program
# header are ignored, the PT_LOAD header determines the effective permissions.
# However, the dynamic linker need to write to this area so these are RW.
# Glibc itself takes care of mprotecting this area R after relocations are finished.
# See also https://marc.info/?l=binutils&m=1498883354122353
if ph.p_type == pixie.PT_GNU_RELRO:
if typ == 'GNU_RELRO':
have_gnu_relro = True
have_bindnow = False
for flags in elf.query_dyn_tags(pixie.DT_FLAGS):
assert isinstance(flags, int)
if flags & pixie.DF_BIND_NOW:
have_bindnow = True
stdout = run_command([READELF_CMD, '-d', '-W', executable])
for line in stdout.splitlines():
tokens = line.split()
if len(tokens)>1 and tokens[1] == '(BIND_NOW)' or (len(tokens)>2 and tokens[1] == '(FLAGS)' and 'BIND_NOW' in tokens[2:]):
have_bindnow = True
return have_gnu_relro and have_bindnow
def check_ELF_Canary(executable) -> bool:
'''
Check for use of stack canary
'''
elf = pixie.load(executable)
stdout = run_command([READELF_CMD, '--dyn-syms', '-W', executable])
ok = False
for symbol in elf.dyn_symbols:
if symbol.name == b'__stack_chk_fail':
for line in stdout.splitlines():
if '__stack_chk_fail' in line:
ok = True
return ok
@@ -77,55 +142,48 @@ def check_ELF_separate_code(executable):
based on their permissions. This checks for missing -Wl,-z,separate-code
and potentially other problems.
'''
elf = pixie.load(executable)
R = pixie.PF_R
W = pixie.PF_W
E = pixie.PF_X
EXPECTED_FLAGS = {
# Read + execute
b'.init': R | E,
b'.plt': R | E,
b'.plt.got': R | E,
b'.plt.sec': R | E,
b'.text': R | E,
b'.fini': R | E,
'.init': 'R E',
'.plt': 'R E',
'.plt.got': 'R E',
'.plt.sec': 'R E',
'.text': 'R E',
'.fini': 'R E',
# Read-only data
b'.interp': R,
b'.note.gnu.property': R,
b'.note.gnu.build-id': R,
b'.note.ABI-tag': R,
b'.gnu.hash': R,
b'.dynsym': R,
b'.dynstr': R,
b'.gnu.version': R,
b'.gnu.version_r': R,
b'.rela.dyn': R,
b'.rela.plt': R,
b'.rodata': R,
b'.eh_frame_hdr': R,
b'.eh_frame': R,
b'.qtmetadata': R,
b'.gcc_except_table': R,
b'.stapsdt.base': R,
'.interp': 'R',
'.note.gnu.property': 'R',
'.note.gnu.build-id': 'R',
'.note.ABI-tag': 'R',
'.gnu.hash': 'R',
'.dynsym': 'R',
'.dynstr': 'R',
'.gnu.version': 'R',
'.gnu.version_r': 'R',
'.rela.dyn': 'R',
'.rela.plt': 'R',
'.rodata': 'R',
'.eh_frame_hdr': 'R',
'.eh_frame': 'R',
'.qtmetadata': 'R',
'.gcc_except_table': 'R',
'.stapsdt.base': 'R',
# Writable data
b'.init_array': R | W,
b'.fini_array': R | W,
b'.dynamic': R | W,
b'.got': R | W,
b'.data': R | W,
b'.bss': R | W,
'.init_array': 'RW',
'.fini_array': 'RW',
'.dynamic': 'RW',
'.got': 'RW',
'.data': 'RW',
'.bss': 'RW',
}
if elf.hdr.e_machine == pixie.EM_PPC64:
# .plt is RW on ppc64 even with separate-code
EXPECTED_FLAGS[b'.plt'] = R | W
# For all LOAD program headers get mapping to the list of sections,
# and for each section, remember the flags of the associated program header.
flags_per_section = {}
for ph in elf.program_headers:
if ph.p_type == pixie.PT_LOAD:
for section in ph.sections:
assert(section.name not in flags_per_section)
flags_per_section[section.name] = ph.p_flags
for (typ, flags, sections) in get_ELF_program_headers(executable):
if typ == 'LOAD':
for section in sections:
assert(section not in flags_per_section)
flags_per_section[section] = flags
# Spot-check ELF LOAD program header flags per section
# If these sections exist, check them against the expected R/W/E flags
for (section, flags) in flags_per_section.items():
@@ -134,72 +192,112 @@ def check_ELF_separate_code(executable):
return False
return True
def get_PE_dll_characteristics(executable) -> int:
'''Get PE DllCharacteristics bits'''
stdout = run_command([OBJDUMP_CMD, '-x', executable])
bits = 0
for line in stdout.splitlines():
tokens = line.split()
if len(tokens)>=2 and tokens[0] == 'DllCharacteristics':
bits = int(tokens[1],16)
return bits
IMAGE_DLL_CHARACTERISTICS_HIGH_ENTROPY_VA = 0x0020
IMAGE_DLL_CHARACTERISTICS_DYNAMIC_BASE = 0x0040
IMAGE_DLL_CHARACTERISTICS_NX_COMPAT = 0x0100
def check_PE_DYNAMIC_BASE(executable) -> bool:
'''PIE: DllCharacteristics bit 0x40 signifies dynamicbase (ASLR)'''
binary = lief.parse(executable)
return lief.PE.DLL_CHARACTERISTICS.DYNAMIC_BASE in binary.optional_header.dll_characteristics_lists
bits = get_PE_dll_characteristics(executable)
return (bits & IMAGE_DLL_CHARACTERISTICS_DYNAMIC_BASE) == IMAGE_DLL_CHARACTERISTICS_DYNAMIC_BASE
# Must support high-entropy 64-bit address space layout randomization
# in addition to DYNAMIC_BASE to have secure ASLR.
def check_PE_HIGH_ENTROPY_VA(executable) -> bool:
'''PIE: DllCharacteristics bit 0x20 signifies high-entropy ASLR'''
binary = lief.parse(executable)
return lief.PE.DLL_CHARACTERISTICS.HIGH_ENTROPY_VA in binary.optional_header.dll_characteristics_lists
bits = get_PE_dll_characteristics(executable)
return (bits & IMAGE_DLL_CHARACTERISTICS_HIGH_ENTROPY_VA) == IMAGE_DLL_CHARACTERISTICS_HIGH_ENTROPY_VA
def check_PE_RELOC_SECTION(executable) -> bool:
'''Check for a reloc section. This is required for functional ASLR.'''
binary = lief.parse(executable)
return binary.has_relocations
stdout = run_command([OBJDUMP_CMD, '-h', executable])
for line in stdout.splitlines():
if '.reloc' in line:
return True
return False
def check_PE_NX(executable) -> bool:
'''NX: DllCharacteristics bit 0x100 signifies nxcompat (DEP)'''
bits = get_PE_dll_characteristics(executable)
return (bits & IMAGE_DLL_CHARACTERISTICS_NX_COMPAT) == IMAGE_DLL_CHARACTERISTICS_NX_COMPAT
def get_MACHO_executable_flags(executable) -> List[str]:
stdout = run_command([OTOOL_CMD, '-vh', executable])
flags = []
for line in stdout.splitlines():
tokens = line.split()
# filter first two header lines
if 'magic' in tokens or 'Mach' in tokens:
continue
# filter ncmds and sizeofcmds values
flags += [t for t in tokens if not t.isdigit()]
return flags
def check_MACHO_PIE(executable) -> bool:
'''
Check for position independent executable (PIE), allowing for address space randomization.
'''
flags = get_MACHO_executable_flags(executable)
if 'PIE' in flags:
return True
return False
def check_MACHO_NOUNDEFS(executable) -> bool:
'''
Check for no undefined references.
'''
binary = lief.parse(executable)
return binary.header.has(lief.MachO.HEADER_FLAGS.NOUNDEFS)
flags = get_MACHO_executable_flags(executable)
if 'NOUNDEFS' in flags:
return True
return False
def check_MACHO_NX(executable) -> bool:
'''
Check for no stack execution
'''
flags = get_MACHO_executable_flags(executable)
if 'ALLOW_STACK_EXECUTION' in flags:
return False
return True
def check_MACHO_LAZY_BINDINGS(executable) -> bool:
'''
Check for no lazy bindings.
We don't use or check for MH_BINDATLOAD. See #18295.
'''
binary = lief.parse(executable)
return binary.dyld_info.lazy_bind == (0,0)
stdout = run_command([OTOOL_CMD, '-l', executable])
for line in stdout.splitlines():
tokens = line.split()
if 'lazy_bind_off' in tokens or 'lazy_bind_size' in tokens:
if tokens[1] != '0':
return False
return True
def check_MACHO_Canary(executable) -> bool:
'''
Check for use of stack canary
'''
binary = lief.parse(executable)
return binary.has_symbol('___stack_chk_fail')
def check_PIE(executable) -> bool:
'''
Check for position independent executable (PIE),
allowing for address space randomization.
'''
binary = lief.parse(executable)
return binary.is_pie
def check_NX(executable) -> bool:
'''
Check for no stack execution
'''
binary = lief.parse(executable)
return binary.has_nx
def check_control_flow(executable) -> bool:
'''
Check for control flow instrumentation
'''
binary = lief.parse(executable)
content = binary.get_content_from_virtual_address(binary.entrypoint, 4, lief.Binary.VA_TYPES.AUTO)
if content == [243, 15, 30, 250]: # endbr64
return True
return False
stdout = run_command([OTOOL_CMD, '-Iv', executable])
ok = False
for line in stdout.splitlines():
if '___stack_chk_fail' in line:
ok = True
return ok
CHECKS = {
'ELF': [
@@ -210,19 +308,17 @@ CHECKS = {
('separate_code', check_ELF_separate_code),
],
'PE': [
('PIE', check_PIE),
('DYNAMIC_BASE', check_PE_DYNAMIC_BASE),
('HIGH_ENTROPY_VA', check_PE_HIGH_ENTROPY_VA),
('NX', check_NX),
('NX', check_PE_NX),
('RELOC_SECTION', check_PE_RELOC_SECTION)
],
'MACHO': [
('PIE', check_PIE),
('PIE', check_MACHO_PIE),
('NOUNDEFS', check_MACHO_NOUNDEFS),
('NX', check_NX),
('NX', check_MACHO_NX),
('LAZY_BINDINGS', check_MACHO_LAZY_BINDINGS),
('Canary', check_MACHO_Canary),
('CONTROL_FLOW', check_control_flow),
('Canary', check_MACHO_Canary)
]
}
@@ -238,24 +334,24 @@ def identify_executable(executable) -> Optional[str]:
return None
if __name__ == '__main__':
retval: int = 0
retval = 0
for filename in sys.argv[1:]:
try:
etype = identify_executable(filename)
if etype is None:
print(f'{filename}: unknown format')
print('%s: unknown format' % filename)
retval = 1
continue
failed: List[str] = []
failed = []
for (name, func) in CHECKS[etype]:
if not func(filename):
failed.append(name)
if failed:
print(f'{filename}: failed {" ".join(failed)}')
print('%s: failed %s' % (filename, ' '.join(failed)))
retval = 1
except IOError:
print(f'{filename}: cannot open')
print('%s: cannot open' % filename)
retval = 1
sys.exit(retval)

View File

@@ -3,21 +3,18 @@
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
'''
A script to check that release executables only contain certain symbols
and are only linked against allowed libraries.
A script to check that the executables produced by gitian only contain
certain symbols and are only linked against allowed libraries.
Example usage:
find ../path/to/binaries -type f -executable | xargs python3 contrib/devtools/symbol-check.py
find ../gitian-builder/build -type f -executable | xargs python3 contrib/devtools/symbol-check.py
'''
import subprocess
import re
import sys
from typing import List, Optional
import lief
import pixie
from utils import determine_wellknown_cmd
import os
from typing import List, Optional, Tuple
# Debian 8 (Jessie) EOL: 2020. https://wiki.debian.org/DebianReleases#Production_Releases
#
@@ -42,16 +39,8 @@ from utils import determine_wellknown_cmd
#
MAX_VERSIONS = {
'GCC': (4,8,0),
'GLIBC': {
pixie.EM_386: (2,17),
pixie.EM_X86_64: (2,17),
pixie.EM_ARM: (2,17),
pixie.EM_AARCH64:(2,17),
pixie.EM_PPC64: (2,17),
pixie.EM_RISCV: (2,27),
},
'LIBATOMIC': (1,0),
'V': (0,5,0), # xkb (bitcoin-qt only)
'GLIBC': (2,17),
'LIBATOMIC': (1,0)
}
# See here for a description of _IO_stdin_used:
# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=634261#109
@@ -61,6 +50,10 @@ IGNORE_EXPORTS = {
'_edata', '_end', '__end__', '_init', '__bss_start', '__bss_start__', '_bss_end__', '__bss_end__', '_fini', '_IO_stdin_used', 'stdin', 'stdout', 'stderr',
'environ', '_environ', '__environ',
}
READELF_CMD = os.getenv('READELF', '/usr/bin/readelf')
CPPFILT_CMD = os.getenv('CPPFILT', '/usr/bin/c++filt')
OBJDUMP_CMD = os.getenv('OBJDUMP', '/usr/bin/objdump')
OTOOL_CMD = os.getenv('OTOOL', '/usr/bin/otool')
# Allowed NEEDED libraries
ELF_ALLOWED_LIBRARIES = {
@@ -75,17 +68,20 @@ ELF_ALLOWED_LIBRARIES = {
'ld-linux.so.2', # 32-bit dynamic linker
'ld-linux-aarch64.so.1', # 64-bit ARM dynamic linker
'ld-linux-armhf.so.3', # 32-bit ARM dynamic linker
'ld64.so.1', # POWER64 ABIv1 dynamic linker
'ld64.so.2', # POWER64 ABIv2 dynamic linker
'ld-linux-riscv64-lp64d.so.1', # 64-bit RISC-V dynamic linker
# bitcoin-qt only
'libxcb.so.1', # part of X11
'libxkbcommon.so.0', # keyboard keymapping
'libxkbcommon-x11.so.0', # keyboard keymapping
'libfontconfig.so.1', # font support
'libfreetype.so.6', # font parsing
'libdl.so.2' # programming interface to dynamic linker
}
ARCH_MIN_GLIBC_VER = {
'80386': (2,1),
'X86-64': (2,2,5),
'ARM': (2,4),
'AArch64':(2,17),
'RISC-V': (2,27)
}
MACHO_ALLOWED_LIBRARIES = {
# bitcoind and bitcoin-qt
@@ -99,15 +95,10 @@ MACHO_ALLOWED_LIBRARIES = {
'CoreGraphics', # 2D rendering
'CoreServices', # operating system services
'CoreText', # interface for laying out text and handling fonts.
'CoreVideo', # video processing
'Foundation', # base layer functionality for apps/frameworks
'ImageIO', # read and write image file formats.
'IOKit', # user-space access to hardware devices and drivers.
'IOSurface', # cross process image/drawing buffers
'libobjc.A.dylib', # Objective-C runtime library
'Metal', # 3D graphics
'Security', # access control and authentication
'QuartzCore', # animation
}
PE_ALLOWED_LIBRARIES = {
@@ -122,15 +113,12 @@ PE_ALLOWED_LIBRARIES = {
'dwmapi.dll', # desktop window manager
'GDI32.dll', # graphics device interface
'IMM32.dll', # input method editor
'NETAPI32.dll',
'ole32.dll', # component object model
'OLEAUT32.dll', # OLE Automation API
'SHLWAPI.dll', # light weight shell API
'USERENV.dll',
'UxTheme.dll',
'VERSION.dll', # version checking
'WINMM.dll', # WinMM audio API
'WTSAPI32.dll',
}
class CPPFilt(object):
@@ -140,7 +128,7 @@ class CPPFilt(object):
Use a pipe to the 'c++filt' command.
'''
def __init__(self):
self.proc = subprocess.Popen(determine_wellknown_cmd('CPPFILT', 'c++filt'), stdin=subprocess.PIPE, stdout=subprocess.PIPE, universal_newlines=True)
self.proc = subprocess.Popen(CPPFILT_CMD, stdin=subprocess.PIPE, stdout=subprocess.PIPE, universal_newlines=True)
def __call__(self, mangled):
self.proc.stdin.write(mangled + '\n')
@@ -152,6 +140,29 @@ class CPPFilt(object):
self.proc.stdout.close()
self.proc.wait()
def read_symbols(executable, imports=True) -> List[Tuple[str, str, str]]:
'''
Parse an ELF executable and return a list of (symbol,version, arch) tuples
for dynamic, imported symbols.
'''
p = subprocess.Popen([READELF_CMD, '--dyn-syms', '-W', '-h', executable], stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE, universal_newlines=True)
(stdout, stderr) = p.communicate()
if p.returncode:
raise IOError('Could not read symbols for {}: {}'.format(executable, stderr.strip()))
syms = []
for line in stdout.splitlines():
line = line.split()
if 'Machine:' in line:
arch = line[-1]
if len(line)>7 and re.match('[0-9]+:$', line[0]):
(sym, _, version) = line[7].partition('@')
is_import = line[6] == 'UND'
if version.startswith('@'):
version = version[1:]
if is_import == imports:
syms.append((sym, version, arch))
return syms
def check_version(max_versions, version, arch) -> bool:
if '_' in version:
(lib, _, ver) = version.rpartition('_')
@@ -161,89 +172,92 @@ def check_version(max_versions, version, arch) -> bool:
ver = tuple([int(x) for x in ver.split('.')])
if not lib in max_versions:
return False
if isinstance(max_versions[lib], tuple):
return ver <= max_versions[lib]
else:
return ver <= max_versions[lib][arch]
return ver <= max_versions[lib] or lib == 'GLIBC' and ver <= ARCH_MIN_GLIBC_VER[arch]
def elf_read_libraries(filename) -> List[str]:
p = subprocess.Popen([READELF_CMD, '-d', '-W', filename], stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE, universal_newlines=True)
(stdout, stderr) = p.communicate()
if p.returncode:
raise IOError('Error opening file')
libraries = []
for line in stdout.splitlines():
tokens = line.split()
if len(tokens)>2 and tokens[1] == '(NEEDED)':
match = re.match(r'^Shared library: \[(.*)\]$', ' '.join(tokens[2:]))
if match:
libraries.append(match.group(1))
else:
raise ValueError('Unparseable (NEEDED) specification')
return libraries
def check_imported_symbols(filename) -> bool:
elf = pixie.load(filename)
cppfilt = CPPFilt()
ok: bool = True
for symbol in elf.dyn_symbols:
if not symbol.is_import:
continue
sym = symbol.name.decode()
version = symbol.version.decode() if symbol.version is not None else None
if version and not check_version(MAX_VERSIONS, version, elf.hdr.e_machine):
ok = True
for sym, version, arch in read_symbols(filename, True):
if version and not check_version(MAX_VERSIONS, version, arch):
print('{}: symbol {} from unsupported version {}'.format(filename, cppfilt(sym), version))
ok = False
return ok
def check_exported_symbols(filename) -> bool:
elf = pixie.load(filename)
cppfilt = CPPFilt()
ok: bool = True
for symbol in elf.dyn_symbols:
if not symbol.is_export:
continue
sym = symbol.name.decode()
if elf.hdr.e_machine == pixie.EM_RISCV or sym in IGNORE_EXPORTS:
ok = True
for sym,version,arch in read_symbols(filename, False):
if arch == 'RISC-V' or sym in IGNORE_EXPORTS:
continue
print('{}: export of symbol {} not allowed'.format(filename, cppfilt(sym)))
ok = False
return ok
def check_ELF_libraries(filename) -> bool:
ok: bool = True
elf = pixie.load(filename)
for library_name in elf.query_dyn_tags(pixie.DT_NEEDED):
assert(isinstance(library_name, bytes))
if library_name.decode() not in ELF_ALLOWED_LIBRARIES:
print('{}: NEEDED library {} is not allowed'.format(filename, library_name.decode()))
ok = True
for library_name in elf_read_libraries(filename):
if library_name not in ELF_ALLOWED_LIBRARIES:
print('{}: NEEDED library {} is not allowed'.format(filename, library_name))
ok = False
return ok
def macho_read_libraries(filename) -> List[str]:
p = subprocess.Popen([OTOOL_CMD, '-L', filename], stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE, universal_newlines=True)
(stdout, stderr) = p.communicate()
if p.returncode:
raise IOError('Error opening file')
libraries = []
for line in stdout.splitlines():
tokens = line.split()
if len(tokens) == 1: # skip executable name
continue
libraries.append(tokens[0].split('/')[-1])
return libraries
def check_MACHO_libraries(filename) -> bool:
ok: bool = True
binary = lief.parse(filename)
for dylib in binary.libraries:
split = dylib.name.split('/')
if split[-1] not in MACHO_ALLOWED_LIBRARIES:
print(f'{split[-1]} is not in ALLOWED_LIBRARIES!')
ok = True
for dylib in macho_read_libraries(filename):
if dylib not in MACHO_ALLOWED_LIBRARIES:
print('{} is not in ALLOWED_LIBRARIES!'.format(dylib))
ok = False
return ok
def check_MACHO_min_os(filename) -> bool:
binary = lief.parse(filename)
if binary.build_version.minos == [10,14,0]:
return True
return False
def check_MACHO_sdk(filename) -> bool:
binary = lief.parse(filename)
if binary.build_version.sdk == [10, 15, 6]:
return True
return False
def pe_read_libraries(filename) -> List[str]:
p = subprocess.Popen([OBJDUMP_CMD, '-x', filename], stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE, universal_newlines=True)
(stdout, stderr) = p.communicate()
if p.returncode:
raise IOError('Error opening file')
libraries = []
for line in stdout.splitlines():
if 'DLL Name:' in line:
tokens = line.split(': ')
libraries.append(tokens[1])
return libraries
def check_PE_libraries(filename) -> bool:
ok: bool = True
binary = lief.parse(filename)
for dylib in binary.libraries:
ok = True
for dylib in pe_read_libraries(filename):
if dylib not in PE_ALLOWED_LIBRARIES:
print(f'{dylib} is not in ALLOWED_LIBRARIES!')
print('{} is not in ALLOWED_LIBRARIES!'.format(dylib))
ok = False
return ok
def check_PE_subsystem_version(filename) -> bool:
binary = lief.parse(filename)
major: int = binary.optional_header.major_subsystem_version
minor: int = binary.optional_header.minor_subsystem_version
if major == 6 and minor == 1:
return True
return False
CHECKS = {
'ELF': [
('IMPORTED_SYMBOLS', check_imported_symbols),
@@ -251,13 +265,10 @@ CHECKS = {
('LIBRARY_DEPENDENCIES', check_ELF_libraries)
],
'MACHO': [
('DYNAMIC_LIBRARIES', check_MACHO_libraries),
('MIN_OS', check_MACHO_min_os),
('SDK', check_MACHO_sdk),
('DYNAMIC_LIBRARIES', check_MACHO_libraries)
],
'PE' : [
('DYNAMIC_LIBRARIES', check_PE_libraries),
('SUBSYSTEM_VERSION', check_PE_subsystem_version),
('DYNAMIC_LIBRARIES', check_PE_libraries)
]
}
@@ -273,23 +284,23 @@ def identify_executable(executable) -> Optional[str]:
return None
if __name__ == '__main__':
retval: int = 0
retval = 0
for filename in sys.argv[1:]:
try:
etype = identify_executable(filename)
if etype is None:
print(f'{filename}: unknown format')
print('{}: unknown format'.format(filename))
retval = 1
continue
failed: List[str] = []
failed = []
for (name, func) in CHECKS[etype]:
if not func(filename):
failed.append(name)
if failed:
print(f'{filename}: failed {" ".join(failed)}')
print('{}: failed {}'.format(filename, ' '.join(failed)))
retval = 1
except IOError:
print(f'{filename}: cannot open')
print('{}: cannot open'.format(filename))
retval = 1
sys.exit(retval)

View File

@@ -5,12 +5,9 @@
'''
Test script for security-check.py
'''
import os
import subprocess
import unittest
from utils import determine_wellknown_cmd
def write_testcode(filename):
with open(filename, 'w', encoding="utf8") as f:
f.write('''
@@ -22,12 +19,8 @@ def write_testcode(filename):
}
''')
def clean_files(source, executable):
os.remove(source)
os.remove(executable)
def call_security_check(cc, source, executable, options):
subprocess.run([*cc,source,'-o',executable] + options, check=True)
subprocess.run([cc,source,'-o',executable] + options, check=True)
p = subprocess.run(['./contrib/devtools/security-check.py',executable], stdout=subprocess.PIPE, universal_newlines=True)
return (p.returncode, p.stdout.rstrip())
@@ -35,7 +28,7 @@ class TestSecurityChecks(unittest.TestCase):
def test_ELF(self):
source = 'test1.c'
executable = 'test1'
cc = determine_wellknown_cmd('CC', 'gcc')
cc = 'gcc'
write_testcode(source)
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-zexecstack','-fno-stack-protector','-Wl,-znorelro','-no-pie','-fno-PIE', '-Wl,-z,separate-code']),
@@ -51,51 +44,42 @@ class TestSecurityChecks(unittest.TestCase):
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-znoexecstack','-fstack-protector-all','-Wl,-zrelro','-Wl,-z,now','-pie','-fPIE', '-Wl,-z,separate-code']),
(0, ''))
clean_files(source, executable)
def test_PE(self):
source = 'test1.c'
executable = 'test1.exe'
cc = determine_wellknown_cmd('CC', 'x86_64-w64-mingw32-gcc')
cc = 'x86_64-w64-mingw32-gcc'
write_testcode(source)
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,--no-nxcompat','-Wl,--disable-reloc-section','-Wl,--no-dynamicbase','-Wl,--no-high-entropy-va','-no-pie','-fno-PIE']),
(1, executable+': failed PIE DYNAMIC_BASE HIGH_ENTROPY_VA NX RELOC_SECTION'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,--nxcompat','-Wl,--disable-reloc-section','-Wl,--no-dynamicbase','-Wl,--no-high-entropy-va','-no-pie','-fno-PIE']),
(1, executable+': failed PIE DYNAMIC_BASE HIGH_ENTROPY_VA RELOC_SECTION'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,--nxcompat','-Wl,--enable-reloc-section','-Wl,--no-dynamicbase','-Wl,--no-high-entropy-va','-no-pie','-fno-PIE']),
(1, executable+': failed PIE DYNAMIC_BASE HIGH_ENTROPY_VA'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,--nxcompat','-Wl,--enable-reloc-section','-Wl,--no-dynamicbase','-Wl,--no-high-entropy-va','-pie','-fPIE']),
(1, executable+': failed PIE DYNAMIC_BASE HIGH_ENTROPY_VA')) # -pie -fPIE does nothing unless --dynamicbase is also supplied
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,--nxcompat','-Wl,--enable-reloc-section','-Wl,--dynamicbase','-Wl,--no-high-entropy-va','-pie','-fPIE']),
(1, executable+': failed HIGH_ENTROPY_VA'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,--nxcompat','-Wl,--enable-reloc-section','-Wl,--dynamicbase','-Wl,--high-entropy-va','-pie','-fPIE']),
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,--no-nxcompat','-Wl,--no-dynamicbase','-Wl,--no-high-entropy-va','-no-pie','-fno-PIE']),
(1, executable+': failed DYNAMIC_BASE HIGH_ENTROPY_VA NX RELOC_SECTION'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,--nxcompat','-Wl,--no-dynamicbase','-Wl,--no-high-entropy-va','-no-pie','-fno-PIE']),
(1, executable+': failed DYNAMIC_BASE HIGH_ENTROPY_VA RELOC_SECTION'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,--nxcompat','-Wl,--dynamicbase','-Wl,--no-high-entropy-va','-no-pie','-fno-PIE']),
(1, executable+': failed HIGH_ENTROPY_VA RELOC_SECTION'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,--nxcompat','-Wl,--dynamicbase','-Wl,--high-entropy-va','-no-pie','-fno-PIE']),
(1, executable+': failed RELOC_SECTION'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,--nxcompat','-Wl,--dynamicbase','-Wl,--high-entropy-va','-pie','-fPIE']),
(0, ''))
clean_files(source, executable)
def test_MACHO(self):
source = 'test1.c'
executable = 'test1'
cc = determine_wellknown_cmd('CC', 'clang')
cc = 'clang'
write_testcode(source)
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-flat_namespace','-Wl,-allow_stack_execute','-fno-stack-protector']),
(1, executable+': failed PIE NOUNDEFS NX LAZY_BINDINGS Canary CONTROL_FLOW'))
(1, executable+': failed PIE NOUNDEFS NX LAZY_BINDINGS Canary'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-flat_namespace','-Wl,-allow_stack_execute','-fstack-protector-all']),
(1, executable+': failed PIE NOUNDEFS NX LAZY_BINDINGS CONTROL_FLOW'))
(1, executable+': failed PIE NOUNDEFS NX LAZY_BINDINGS'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-flat_namespace','-fstack-protector-all']),
(1, executable+': failed PIE NOUNDEFS LAZY_BINDINGS CONTROL_FLOW'))
(1, executable+': failed PIE NOUNDEFS LAZY_BINDINGS'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-fstack-protector-all']),
(1, executable+': failed PIE LAZY_BINDINGS CONTROL_FLOW'))
(1, executable+': failed PIE LAZY_BINDINGS'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-bind_at_load','-fstack-protector-all']),
(1, executable+': failed PIE CONTROL_FLOW'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-bind_at_load','-fstack-protector-all', '-fcf-protection=full']),
(1, executable+': failed PIE'))
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-pie','-Wl,-bind_at_load','-fstack-protector-all', '-fcf-protection=full']),
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-pie','-Wl,-bind_at_load','-fstack-protector-all']),
(0, ''))
clean_files(source, executable)
if __name__ == '__main__':
unittest.main()

View File

@@ -1,194 +0,0 @@
#!/usr/bin/env python3
# Copyright (c) 2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
'''
Test script for symbol-check.py
'''
import os
import subprocess
from typing import List
import unittest
from utils import determine_wellknown_cmd
def call_symbol_check(cc: List[str], source, executable, options):
subprocess.run([*cc,source,'-o',executable] + options, check=True)
p = subprocess.run(['./contrib/devtools/symbol-check.py',executable], stdout=subprocess.PIPE, universal_newlines=True)
os.remove(source)
os.remove(executable)
return (p.returncode, p.stdout.rstrip())
def get_machine(cc: List[str]):
p = subprocess.run([*cc,'-dumpmachine'], stdout=subprocess.PIPE, universal_newlines=True)
return p.stdout.rstrip()
class TestSymbolChecks(unittest.TestCase):
def test_ELF(self):
source = 'test1.c'
executable = 'test1'
cc = determine_wellknown_cmd('CC', 'gcc')
# there's no way to do this test for RISC-V at the moment; we build for
# RISC-V in a glibc 2.27 envinonment and we allow all symbols from 2.27.
if 'riscv' in get_machine(cc):
self.skipTest("test not available for RISC-V")
# nextup was introduced in GLIBC 2.24, so is newer than our supported
# glibc (2.17), and available in our release build environment (2.24).
with open(source, 'w', encoding="utf8") as f:
f.write('''
#define _GNU_SOURCE
#include <math.h>
double nextup(double x);
int main()
{
nextup(3.14);
return 0;
}
''')
self.assertEqual(call_symbol_check(cc, source, executable, ['-lm']),
(1, executable + ': symbol nextup from unsupported version GLIBC_2.24\n' +
executable + ': failed IMPORTED_SYMBOLS'))
# -lutil is part of the libc6 package so a safe bet that it's installed
# it's also out of context enough that it's unlikely to ever become a real dependency
source = 'test2.c'
executable = 'test2'
with open(source, 'w', encoding="utf8") as f:
f.write('''
#include <utmp.h>
int main()
{
login(0);
return 0;
}
''')
self.assertEqual(call_symbol_check(cc, source, executable, ['-lutil']),
(1, executable + ': NEEDED library libutil.so.1 is not allowed\n' +
executable + ': failed LIBRARY_DEPENDENCIES'))
# finally, check a conforming file that simply uses a math function
source = 'test3.c'
executable = 'test3'
with open(source, 'w', encoding="utf8") as f:
f.write('''
#include <math.h>
int main()
{
return (int)pow(2.0, 4.0);
}
''')
self.assertEqual(call_symbol_check(cc, source, executable, ['-lm']),
(0, ''))
def test_MACHO(self):
source = 'test1.c'
executable = 'test1'
cc = determine_wellknown_cmd('CC', 'clang')
with open(source, 'w', encoding="utf8") as f:
f.write('''
#include <expat.h>
int main()
{
XML_ExpatVersion();
return 0;
}
''')
self.assertEqual(call_symbol_check(cc, source, executable, ['-lexpat', '-Wl,-platform_version','-Wl,macos', '-Wl,11.4', '-Wl,11.4']),
(1, 'libexpat.1.dylib is not in ALLOWED_LIBRARIES!\n' +
f'{executable}: failed DYNAMIC_LIBRARIES MIN_OS SDK'))
source = 'test2.c'
executable = 'test2'
with open(source, 'w', encoding="utf8") as f:
f.write('''
#include <CoreGraphics/CoreGraphics.h>
int main()
{
CGMainDisplayID();
return 0;
}
''')
self.assertEqual(call_symbol_check(cc, source, executable, ['-framework', 'CoreGraphics', '-Wl,-platform_version','-Wl,macos', '-Wl,11.4', '-Wl,11.4']),
(1, f'{executable}: failed MIN_OS SDK'))
source = 'test3.c'
executable = 'test3'
with open(source, 'w', encoding="utf8") as f:
f.write('''
int main()
{
return 0;
}
''')
self.assertEqual(call_symbol_check(cc, source, executable, ['-Wl,-platform_version','-Wl,macos', '-Wl,10.14', '-Wl,11.4']),
(1, f'{executable}: failed SDK'))
def test_PE(self):
source = 'test1.c'
executable = 'test1.exe'
cc = determine_wellknown_cmd('CC', 'x86_64-w64-mingw32-gcc')
with open(source, 'w', encoding="utf8") as f:
f.write('''
#include <pdh.h>
int main()
{
PdhConnectMachineA(NULL);
return 0;
}
''')
self.assertEqual(call_symbol_check(cc, source, executable, ['-lpdh', '-Wl,--major-subsystem-version', '-Wl,6', '-Wl,--minor-subsystem-version', '-Wl,1']),
(1, 'pdh.dll is not in ALLOWED_LIBRARIES!\n' +
executable + ': failed DYNAMIC_LIBRARIES'))
source = 'test2.c'
executable = 'test2.exe'
with open(source, 'w', encoding="utf8") as f:
f.write('''
int main()
{
return 0;
}
''')
self.assertEqual(call_symbol_check(cc, source, executable, ['-Wl,--major-subsystem-version', '-Wl,9', '-Wl,--minor-subsystem-version', '-Wl,9']),
(1, executable + ': failed SUBSYSTEM_VERSION'))
source = 'test3.c'
executable = 'test3.exe'
with open(source, 'w', encoding="utf8") as f:
f.write('''
#include <windows.h>
int main()
{
CoFreeUnusedLibrariesEx(0,0);
return 0;
}
''')
self.assertEqual(call_symbol_check(cc, source, executable, ['-lole32', '-Wl,--major-subsystem-version', '-Wl,6', '-Wl,--minor-subsystem-version', '-Wl,1']),
(0, ''))
if __name__ == '__main__':
unittest.main()

View File

@@ -1,22 +0,0 @@
#!/usr/bin/env python3
# Copyright (c) 2021 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
'''
Common utility functions
'''
import shutil
import sys
import os
from typing import List
def determine_wellknown_cmd(envvar, progname) -> List[str]:
maybe_env = os.getenv(envvar)
maybe_which = shutil.which(progname)
if maybe_env:
return maybe_env.split(' ') # Well-known vars are often meant to be word-split
elif maybe_which:
return [ maybe_which ]
else:
sys.exit(f"{progname} not found")

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2017-2020 The Bitcoin Core developers
# Copyright (c) 2017-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2018-2020 The Bitcoin Core developers
# Copyright (c) 2018-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@@ -13,16 +13,15 @@ def setup():
programs = ['ruby', 'git', 'make', 'wget', 'curl']
if args.kvm:
programs += ['apt-cacher-ng', 'python-vm-builder', 'qemu-kvm', 'qemu-utils']
elif args.docker:
if not os.path.isfile('/lib/systemd/system/docker.service'):
dockers = ['docker.io', 'docker-ce']
for i in dockers:
return_code = subprocess.call(['sudo', 'apt-get', 'install', '-qq', i])
if return_code == 0:
break
if return_code != 0:
print('Cannot find any way to install Docker.', file=sys.stderr)
sys.exit(1)
elif args.docker and not os.path.isfile('/lib/systemd/system/docker.service'):
dockers = ['docker.io', 'docker-ce']
for i in dockers:
return_code = subprocess.call(['sudo', 'apt-get', 'install', '-qq', i])
if return_code == 0:
break
if return_code != 0:
print('Cannot find any way to install Docker.', file=sys.stderr)
sys.exit(1)
else:
programs += ['apt-cacher-ng', 'lxc', 'debootstrap']
subprocess.check_call(['sudo', 'apt-get', 'install', '-qq'] + programs)
@@ -35,14 +34,14 @@ def setup():
if not os.path.isdir('bitcoin'):
subprocess.check_call(['git', 'clone', 'https://github.com/bitcoin/bitcoin.git'])
os.chdir('gitian-builder')
make_image_prog = ['bin/make-base-vm', '--suite', 'focal', '--arch', 'amd64']
make_image_prog = ['bin/make-base-vm', '--suite', 'bionic', '--arch', 'amd64']
if args.docker:
make_image_prog += ['--docker']
elif not args.kvm:
make_image_prog += ['--lxc', '--disksize', '13000']
make_image_prog += ['--lxc']
subprocess.check_call(make_image_prog)
os.chdir(workdir)
if args.is_focal and not args.kvm and not args.docker:
if args.is_bionic and not args.kvm and not args.docker:
subprocess.check_call(['sudo', 'sed', '-i', 's/lxcbr0/br0/', '/etc/default/lxc-net'])
print('Reboot is required')
sys.exit(0)
@@ -176,7 +175,7 @@ def main():
args = parser.parse_args()
workdir = os.getcwd()
args.is_focal = b'focal' in subprocess.check_output(['lsb_release', '-cs'])
args.is_bionic = b'bionic' in subprocess.check_output(['lsb_release', '-cs'])
if args.kvm and args.docker:
raise Exception('Error: cannot have both kvm and docker')
@@ -210,7 +209,7 @@ def main():
args.macos = 'm' in args.os
# Disable for MacOS if no SDK found
if args.macos and not os.path.isfile('gitian-builder/inputs/Xcode-12.1-12A7403-extracted-SDK-with-libcxx-headers.tar.gz'):
if args.macos and not os.path.isfile('gitian-builder/inputs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers.tar.gz'):
print('Cannot build for MacOS, SDK does not exist. Will build for other OSes')
args.macos = False

2
contrib/gitian-descriptors/assign_DISTNAME Normal file → Executable file
View File

@@ -4,7 +4,7 @@
#
# A helper script to be sourced into the gitian descriptors
if RECENT_TAG="$(git describe --exact-match HEAD 2> /dev/null)"; then
if RECENT_TAG="$(git describe --exact-match HEAD)"; then
VERSION="${RECENT_TAG#v}"
else
VERSION="$(git rev-parse --short=12 HEAD)"

View File

@@ -1,9 +1,9 @@
---
name: "bitcoin-core-linux-22"
name: "bitcoin-core-linux-0.21"
enable_cache: true
distro: "ubuntu"
suites:
- "focal"
- "bionic"
architectures:
- "amd64"
packages:
@@ -11,19 +11,15 @@ packages:
- "autoconf"
- "automake"
- "binutils"
- "bison"
- "bsdmainutils"
- "ca-certificates"
- "curl"
- "faketime"
- "g++-8"
- "gcc-8"
- "git"
- "libtool"
- "patch"
- "pkg-config"
- "python3"
- "python3-pip"
# Cross compilation HOSTS:
# - arm-linux-gnueabihf
- "binutils-arm-linux-gnueabihf"
@@ -31,12 +27,6 @@ packages:
# - aarch64-linux-gnu
- "binutils-aarch64-linux-gnu"
- "g++-8-aarch64-linux-gnu"
# - powerpc64-linux-gnu
- "binutils-powerpc64-linux-gnu"
- "g++-8-powerpc64-linux-gnu"
# - powerpc64le-linux-gnu
- "binutils-powerpc64le-linux-gnu"
- "g++-8-powerpc64le-linux-gnu"
# - riscv64-linux-gnu
- "binutils-riscv64-linux-gnu"
- "g++-8-riscv64-linux-gnu"
@@ -48,14 +38,16 @@ script: |
set -e -o pipefail
WRAP_DIR=$HOME/wrapped
HOSTS="x86_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu powerpc64-linux-gnu powerpc64le-linux-gnu riscv64-linux-gnu"
CONFIGFLAGS="--enable-glibc-back-compat --enable-reduce-exports --disable-bench --disable-gui-tests --disable-fuzz-binary"
HOSTS="x86_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu riscv64-linux-gnu"
CONFIGFLAGS="--enable-glibc-back-compat --enable-reduce-exports --disable-bench --disable-gui-tests"
FAKETIME_HOST_PROGS="gcc g++"
FAKETIME_PROGS="date ar ranlib nm"
HOST_CFLAGS="-O2 -g"
HOST_CXXFLAGS="-O2 -g"
HOST_LDFLAGS_BASE="-static-libstdc++ -Wl,-O2"
export QT_RCC_TEST=1
export QT_RCC_SOURCE_DATE_OVERRIDE=1
export TZ="UTC"
export BUILD_DIR="$PWD"
mkdir -p ${WRAP_DIR}
@@ -72,7 +64,7 @@ script: |
echo "REAL=\`which -a ${prog} | grep -v ${WRAP_DIR}/${prog} | head -1\`" >> ${WRAP_DIR}/${prog}
echo "export LD_PRELOAD='/usr/\$LIB/faketime/libfaketime.so.1'" >> ${WRAP_DIR}/${prog}
echo "export FAKETIME=\"$1\"" >> ${WRAP_DIR}/${prog}
echo "exec \"\$REAL\" \"\$@\"" >> $WRAP_DIR/${prog}
echo "\$REAL \$@" >> $WRAP_DIR/${prog}
chmod +x ${WRAP_DIR}/${prog}
done
}
@@ -86,21 +78,13 @@ script: |
echo "REAL=\`which -a ${i}-${prog}-8 | grep -v ${WRAP_DIR}/${i}-${prog} | head -1\`" >> ${WRAP_DIR}/${i}-${prog}
echo "export LD_PRELOAD='/usr/\$LIB/faketime/libfaketime.so.1'" >> ${WRAP_DIR}/${i}-${prog}
echo "export FAKETIME=\"$1\"" >> ${WRAP_DIR}/${i}-${prog}
if [ "${i:0:11}" = "powerpc64le" ]; then
echo "exec \"\$REAL\" -mcpu=power8 -mtune=power9 \"\$@\"" >> $WRAP_DIR/${i}-${prog}
elif [ "${i:0:9}" = "powerpc64" ]; then
echo "exec \"\$REAL\" -mcpu=970 -mtune=power9 \"\$@\"" >> $WRAP_DIR/${i}-${prog}
else
echo "exec \"\$REAL\" \"\$@\"" >> $WRAP_DIR/${i}-${prog}
fi
echo "\$REAL \"\$@\"" >> $WRAP_DIR/${i}-${prog}
chmod +x ${WRAP_DIR}/${i}-${prog}
fi
done
done
}
pip3 install lief==0.11.5
# Faketime for depends so intermediate results are comparable
export PATH_orig=${PATH}
create_global_faketime_wrappers "2000-01-01 12:00:00"
@@ -111,7 +95,7 @@ script: |
BASEPREFIX="${PWD}/depends"
# Build dependencies for each host
for i in $HOSTS; do
make ${MAKEOPTS} -C ${BASEPREFIX} HOST="${i}" CC=${i}-gcc-8 CXX=${i}-g++-8
make ${MAKEOPTS} -C ${BASEPREFIX} HOST="${i}"
done
# Faketime for binaries
@@ -134,7 +118,7 @@ script: |
# Extract the git archive into a dir for each host and build
for i in ${HOSTS}; do
export PATH=${BASEPREFIX}/${i}/native/bin:${ORIGPATH}
if [ "${i}" = "powerpc64-linux-gnu" ]; then
if [ "${i}" = "riscv64-linux-gnu" ]; then
# Workaround for https://bugs.launchpad.net/ubuntu/+source/gcc-8-cross-ports/+bug/1853740
# TODO: remove this when no longer needed
HOST_LDFLAGS="${HOST_LDFLAGS_BASE} -Wl,-z,noexecstack"
@@ -148,7 +132,7 @@ script: |
tar --strip-components=1 -xf "${GIT_ARCHIVE}"
./autogen.sh
CONFIG_SITE=${BASEPREFIX}/${i}/share/config.site ./configure --prefix=/ --disable-ccache --disable-maintainer-mode --disable-dependency-tracking ${CONFIGFLAGS} CFLAGS="${HOST_CFLAGS}" CXXFLAGS="${HOST_CXXFLAGS}" LDFLAGS="${HOST_LDFLAGS}" CC=${i}-gcc-8 CXX=${i}-g++-8
CONFIG_SITE=${BASEPREFIX}/${i}/share/config.site ./configure --prefix=/ --disable-ccache --disable-maintainer-mode --disable-dependency-tracking ${CONFIGFLAGS} CFLAGS="${HOST_CFLAGS}" CXXFLAGS="${HOST_CXXFLAGS}" LDFLAGS="${HOST_LDFLAGS}"
make ${MAKEOPTS}
make ${MAKEOPTS} -C src check-security
make ${MAKEOPTS} -C src check-symbols

View File

@@ -2,12 +2,11 @@
name: "bitcoin-dmg-signer"
distro: "ubuntu"
suites:
- "focal"
- "bionic"
architectures:
- "amd64"
packages:
- "faketime"
- "xorriso"
- "python3-pip"
remotes:
- "url": "https://github.com/bitcoin-core/bitcoin-detached-sigs.git"
@@ -23,7 +22,7 @@ script: |
WRAP_DIR=$HOME/wrapped
mkdir -p ${WRAP_DIR}
export PATH="$PWD":$PATH
FAKETIME_PROGS="dmg xorrisofs"
FAKETIME_PROGS="dmg genisoimage"
# Create global faketime wrappers
for prog in ${FAKETIME_PROGS}; do
@@ -31,7 +30,7 @@ script: |
echo "REAL=\`which -a ${prog} | grep -v ${WRAP_DIR}/${prog} | head -1\`" >> ${WRAP_DIR}/${prog}
echo "export LD_PRELOAD='/usr/\$LIB/faketime/libfaketime.so.1'" >> ${WRAP_DIR}/${prog}
echo "export FAKETIME=\"${REFERENCE_DATETIME}\"" >> ${WRAP_DIR}/${prog}
echo "exec \"\$REAL\" \"\$@\"" >> $WRAP_DIR/${prog}
echo "\$REAL \$@" >> $WRAP_DIR/${prog}
chmod +x ${WRAP_DIR}/${prog}
done
@@ -49,5 +48,5 @@ script: |
tar -xf ${UNSIGNED_TARBALL}
OSX_VOLNAME="$(cat osx_volname)"
./detached-sig-apply.sh ${UNSIGNED_APP} signature/osx/dist
${WRAP_DIR}/xorrisofs -D -l -V "${OSX_VOLNAME}" -no-pad -r -dir-mode 0755 -o uncompressed.dmg signed-app
${WRAP_DIR}/genisoimage -no-cache-inodes -D -l -probe -V "${OSX_VOLNAME}" -no-pad -r -dir-mode 0755 -apple -o uncompressed.dmg signed-app
${WRAP_DIR}/dmg dmg uncompressed.dmg ${OUTDIR}/${SIGNED}

View File

@@ -1,9 +1,9 @@
---
name: "bitcoin-core-osx-22"
name: "bitcoin-core-osx-0.21"
enable_cache: true
distro: "ubuntu"
suites:
- "focal"
- "bionic"
architectures:
- "amd64"
packages:
@@ -21,27 +21,29 @@ packages:
- "bsdmainutils"
- "cmake"
- "imagemagick"
- "libcap-dev"
- "libz-dev"
- "libbz2-dev"
- "python3"
- "python3-pip"
- "python3-dev"
- "python3-setuptools"
- "fonts-tuffy"
- "xorriso"
- "libtinfo5"
remotes:
- "url": "https://github.com/bitcoin/bitcoin.git"
"dir": "bitcoin"
files:
- "Xcode-12.1-12A7403-extracted-SDK-with-libcxx-headers.tar.gz"
- "Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers.tar.gz"
script: |
set -e -o pipefail
WRAP_DIR=$HOME/wrapped
HOSTS="x86_64-apple-darwin18"
CONFIGFLAGS="--enable-reduce-exports --disable-bench --disable-gui-tests --disable-fuzz-binary XORRISOFS=${WRAP_DIR}/xorrisofs DMG=${WRAP_DIR}/dmg"
HOSTS="x86_64-apple-darwin16"
CONFIGFLAGS="--enable-reduce-exports --disable-bench --disable-gui-tests GENISOIMAGE=$WRAP_DIR/genisoimage"
FAKETIME_HOST_PROGS=""
FAKETIME_PROGS="ar ranlib date dmg xorrisofs"
FAKETIME_PROGS="ar ranlib date dmg genisoimage"
export QT_RCC_TEST=1
export QT_RCC_SOURCE_DATE_OVERRIDE=1
export TZ="UTC"
export BUILD_DIR="$PWD"
mkdir -p ${WRAP_DIR}
@@ -60,7 +62,7 @@ script: |
echo "REAL=\`which -a ${prog} | grep -v ${WRAP_DIR}/${prog} | head -1\`" >> ${WRAP_DIR}/${prog}
echo "export LD_PRELOAD='/usr/\$LIB/faketime/libfaketime.so.1'" >> ${WRAP_DIR}/${prog}
echo "export FAKETIME=\"$1\"" >> ${WRAP_DIR}/${prog}
echo "exec \"\$REAL\" \"\$@\"" >> $WRAP_DIR/${prog}
echo "\$REAL \$@" >> $WRAP_DIR/${prog}
chmod +x ${WRAP_DIR}/${prog}
done
}
@@ -72,14 +74,12 @@ script: |
echo "REAL=\`which -a ${i}-${prog} | grep -v ${WRAP_DIR}/${i}-${prog} | head -1\`" >> ${WRAP_DIR}/${i}-${prog}
echo "export LD_PRELOAD='/usr/\$LIB/faketime/libfaketime.so.1'" >> ${WRAP_DIR}/${i}-${prog}
echo "export FAKETIME=\"$1\"" >> ${WRAP_DIR}/${i}-${prog}
echo "exec \"\$REAL\" \"\$@\"" >> $WRAP_DIR/${i}-${prog}
echo "\$REAL \$@" >> $WRAP_DIR/${i}-${prog}
chmod +x ${WRAP_DIR}/${i}-${prog}
done
done
}
pip3 install lief==0.11.5
# Faketime for depends so intermediate results are comparable
export PATH_orig=${PATH}
create_global_faketime_wrappers "2000-01-01 12:00:00"
@@ -90,7 +90,7 @@ script: |
BASEPREFIX="${PWD}/depends"
mkdir -p ${BASEPREFIX}/SDKs
tar -C ${BASEPREFIX}/SDKs -xf ${BUILD_DIR}/Xcode-12.1-12A7403-extracted-SDK-with-libcxx-headers.tar.gz
tar -C ${BASEPREFIX}/SDKs -xf ${BUILD_DIR}/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers.tar.gz
# Build dependencies for each host
for i in $HOSTS; do
@@ -132,17 +132,19 @@ script: |
make osx_volname
make deploydir
OSX_VOLNAME="$(cat osx_volname)"
mkdir -p unsigned-app-${i}
cp osx_volname unsigned-app-${i}/
cp contrib/macdeploy/detached-sig-apply.sh unsigned-app-${i}
cp contrib/macdeploy/detached-sig-create.sh unsigned-app-${i}
cp ${BASEPREFIX}/${i}/native/bin/dmg unsigned-app-${i}
cp ${BASEPREFIX}/${i}/native/bin/dmg ${BASEPREFIX}/${i}/native/bin/genisoimage unsigned-app-${i}
mv dist unsigned-app-${i}
pushd unsigned-app-${i}
find . | sort | tar --mtime="$REFERENCE_DATETIME" --no-recursion --mode='u+rw,go+r-w,a+X' --owner=0 --group=0 -c -T - | gzip -9n > ${OUTDIR}/${DISTNAME}-osx-unsigned.tar.gz
popd
make deploy OSX_DMG="${OUTDIR}/${DISTNAME}-osx-unsigned.dmg"
make deploy
${WRAP_DIR}/dmg dmg "${OSX_VOLNAME}.dmg" ${OUTDIR}/${DISTNAME}-osx-unsigned.dmg
cd installed
find . -name "lib*.la" -delete

View File

@@ -2,7 +2,7 @@
name: "bitcoin-win-signer"
distro: "ubuntu"
suites:
- "focal"
- "bionic"
architectures:
- "amd64"
packages:

View File

@@ -1,9 +1,9 @@
---
name: "bitcoin-core-win-22"
name: "bitcoin-core-win-0.21"
enable_cache: true
distro: "ubuntu"
suites:
- "focal"
- "bionic"
architectures:
- "amd64"
packages:
@@ -22,7 +22,6 @@ packages:
- "zip"
- "ca-certificates"
- "python3"
- "python3-pip"
remotes:
- "url": "https://github.com/bitcoin/bitcoin.git"
"dir": "bitcoin"
@@ -32,12 +31,14 @@ script: |
WRAP_DIR=$HOME/wrapped
HOSTS="x86_64-w64-mingw32"
CONFIGFLAGS="--enable-reduce-exports --disable-bench --disable-gui-tests --disable-fuzz-binary"
CONFIGFLAGS="--enable-reduce-exports --disable-bench --disable-gui-tests"
FAKETIME_HOST_PROGS="ar ranlib nm windres strip objcopy"
FAKETIME_PROGS="date makensis zip"
HOST_CFLAGS="-O2 -g -fno-ident"
HOST_CXXFLAGS="-O2 -g -fno-ident"
export QT_RCC_TEST=1
export QT_RCC_SOURCE_DATE_OVERRIDE=1
export TZ="UTC"
export BUILD_DIR="$PWD"
mkdir -p ${WRAP_DIR}
@@ -54,7 +55,7 @@ script: |
echo "REAL=\`which -a ${prog} | grep -v ${WRAP_DIR}/${prog} | head -1\`" >> ${WRAP_DIR}/${prog}
echo "export LD_PRELOAD='/usr/\$LIB/faketime/libfaketime.so.1'" >> ${WRAP_DIR}/${prog}
echo "export FAKETIME=\"$1\"" >> ${WRAP_DIR}/${prog}
echo "exec \"\$REAL\" \"\$@\"" >> $WRAP_DIR/${prog}
echo "\$REAL \$@" >> $WRAP_DIR/${prog}
chmod +x ${WRAP_DIR}/${prog}
done
}
@@ -66,7 +67,7 @@ script: |
echo "REAL=\`which -a ${i}-${prog} | grep -v ${WRAP_DIR}/${i}-${prog} | head -1\`" >> ${WRAP_DIR}/${i}-${prog}
echo "export LD_PRELOAD='/usr/\$LIB/faketime/libfaketime.so.1'" >> ${WRAP_DIR}/${i}-${prog}
echo "export FAKETIME=\"$1\"" >> ${WRAP_DIR}/${i}-${prog}
echo "exec \"\$REAL\" \"\$@\"" >> $WRAP_DIR/${i}-${prog}
echo "\$REAL \$@" >> $WRAP_DIR/${i}-${prog}
chmod +x ${WRAP_DIR}/${i}-${prog}
done
done
@@ -80,14 +81,12 @@ script: |
echo "REAL=\`which -a ${i}-${prog}-posix | grep -v ${WRAP_DIR}/${i}-${prog} | head -1\`" >> ${WRAP_DIR}/${i}-${prog}
echo "export LD_PRELOAD='/usr/\$LIB/faketime/libfaketime.so.1'" >> ${WRAP_DIR}/${i}-${prog}
echo "export FAKETIME=\"$1\"" >> ${WRAP_DIR}/${i}-${prog}
echo "exec \"\$REAL\" \"\$@\"" >> $WRAP_DIR/${i}-${prog}
echo "\$REAL \"\$@\"" >> $WRAP_DIR/${i}-${prog}
chmod +x ${WRAP_DIR}/${i}-${prog}
done
done
}
pip3 install lief==0.11.5
# Faketime for depends so intermediate results are comparable
export PATH_orig=${PATH}
create_global_faketime_wrappers "2000-01-01 12:00:00"

View File

@@ -1,10 +1,10 @@
## PGP keys of builders and Developers
## PGP keys of Gitian builders and Developers
The file `keys.txt` contains fingerprints of the public keys of builders and
active developers.
The file `keys.txt` contains fingerprints of the public keys of Gitian builders
and active developers.
The associated keys are mainly used to sign git commits or the build results
of Guix builds.
of Gitian builds.
The most recent version of each pgp key can be found on most pgp key servers.
@@ -16,12 +16,12 @@ To fetch the latest version of all pgp keys in your gpg homedir,
gpg --refresh-keys
```
To fetch keys of builders and active developers, feed the list of fingerprints
of the primary keys into gpg:
To fetch keys of Gitian builders and active developers, feed the list of
fingerprints of the primary keys into gpg:
```sh
while read fingerprint keyholder_name; do gpg --keyserver hkp://subset.pool.sks-keyservers.net --recv-keys ${fingerprint}; done < ./keys.txt
```
Add your key to the list if you provided Guix attestations for two major or
Add your key to the list if you provided Gitian signatures for two major or
minor releases of Bitcoin Core.

View File

@@ -0,0 +1,36 @@
9D3CC86A72F8494342EA5FD10A41BDC3F4FAFF1C Aaron Clauson (sipsorcery)
617C90010B3BD370B0AC7D424BB42E31C79111B8 Akira Takizawa
E944AE667CF960B1004BC32FCA662BE18B877A60 Andreas Schildbach
152812300785C96444D3334D17565732E08E5E41 Andrew Chow
912FD3228387123DC97E0E57D5566241A0295FA9 BtcDrak
C519EBCF3B926298946783EFF6430754120EC2F4 Christian Decker (cdecker)
F20F56EF6A067F70E8A5C99FFF95FAA971697405 centaur
C060A6635913D98A3587D7DB1C2491FFEB0EF770 Cory Fields
BF6273FAEF7CC0BA1F562E50989F6B3048A116B5 Dev Random
6D3170C1DC2C6FD0AEEBCA6743811D1A26623924 Douglas Roark
9A1689B60D1B3CCE9262307A2F40A9BF167FBA47 Erik Mossberg (erkmos)
D35176BE9264832E4ACA8986BF0792FBE95DC863 fivepiece
01CDF4627A3B88AAE4A571C87588242FBE38D3A8 Gavin Andresen
D1DBF2C4B96F2DEBF4C16654410108112E7EA81F Hennadii Stepanov (hebasto)
D3CC177286005BB8FF673294C5242A1AB3936517 jl2012
82921A4B88FD454B7EB8CE3C796C4109063D4EAF Jon Atack
32EE5C4C3FA15CCADB46ABE529D4BCB6416F53EC Jonas Schnelli
4B4E840451149DD7FB0D633477DFAB5C3108B9A8 Jorge Timon
C42AFF7C61B3E44A1454CD3557AF762DB3353322 Karl-Johan Alm (kallewoof)
E463A93F5F3117EEDE6C7316BD02942421F4889F Luke Dashjr
B8B3F1C0E58C15DB6A81D30C3648A882F4316B9B Marco Falke
07DF3E57A548CCFB7530709189BBB8663E2E65CE Matt Corallo (BlueMatt)
CA03882CB1FC067B5D3ACFE4D300116E1C875A3D MeshCollider
E777299FC265DD04793070EB944D35F9AC3DB76A Michael Ford
9692B91BBF0E8D34DFD33B1882C5C009628ECF0C Michagogo
77E72E69DA7EE0A148C06B21B34821D4944DE5F7 Nils Schneider
D62A803E27E7F43486035ADBBCD04D8E9CCCAC2A Paul Rabahy
37EC7D7B0A217CDB4B4E007E7FAB114267E4FA04 Peter Todd
D762373D24904A3E42F33B08B9A408E71DAAC974 Pieter Wuille (Location: Leuven, Belgium)
133EAC179436F14A5CF1B794860FEB804E669320 Pieter Wuille
A8FC55F3B04BA3146F3492E79303B33A305224CB Sebastian Kung (TheCharlatan)
ED9BDF7AD6A55E232E84524257FF9BDBCC301009 Sjors Provoost
9EDAFF80E080659604F4A76B2EBB056FD847F8A7 Stephan Oeste (Emzy)
AEC1884398647C47413C1C3FB1179EB7347DC10D Warren Togami
79D00BAC68B56D422F945A8F8E3A8F3247DBCBBF Willy Ko
71A3B16735405025D447E8F274810B012346C9A6 Wladimir J. van der Laan

View File

@@ -1,801 +0,0 @@
# Guix Installation and Setup
This only needs to be done once per machine. If you have already completed the
installation and setup, please proceed to [perform a build](./README.md).
Otherwise, you may choose from one of the following options to install Guix:
1. Using the official **shell installer script** [⤓ skip to section][install-script]
- Maintained by Guix developers
- Easiest (automatically performs *most* setup)
- Works on nearly all Linux distributions
- Only installs latest release
- Binary installation only, requires high level of trust
- Note: The script needs to be run as root, so it should be inspected before it's run
2. Using the official **binary tarball** [⤓ skip to section][install-bin-tarball]
- Maintained by Guix developers
- Normal difficulty (full manual setup required)
- Works on nearly all Linux distributions
- Installs any release
- Binary installation only, requires high level of trust
3. Using fanquake's **Docker image** [↗︎ external instructions][install-fanquake-docker]
- Maintained by fanquake
- Easy (automatically performs *some* setup)
- Works wherever Docker images work
- Installs any release
- Binary installation only, requires high level of trust
4. Using a **distribution-maintained package** [⤓ skip to section][install-distro-pkg]
- Maintained by distribution's Guix package maintainer
- Normal difficulty (manual setup required)
- Works only on distributions with Guix packaged, see: https://repology.org/project/guix/versions
- Installs a release decided on by package maintainer
- Source or binary installation depending on the distribution
5. Building **from source** [⤓ skip to section][install-source]
- Maintained by you
- Hard, but rewarding
- Can be made to work on most Linux distributions
- Installs any commit (more granular)
- Source installation, requires lower level of trust
## Options 1 and 2: Using the official shell installer script or binary tarball
The installation instructions for both the official shell installer script and
the binary tarballs can be found in the GNU Guix Manual's [Binary Installation
section](https://guix.gnu.org/manual/en/html_node/Binary-Installation.html).
Note that running through the binary tarball installation steps is largely
equivalent to manually performing what the shell installer script does.
Note that at the time of writing (July 5th, 2021), the shell installer script
automatically creates an `/etc/profile.d` entry which the binary tarball
installation instructions do not ask you to create. However, you will likely
need this entry for better desktop integration. Please see [this
section](#add-an-etcprofiled-entry) for instructions on how to add a
`/etc/profile.d/guix.sh` entry.
Regardless of which installation option you chose, the changes to
`/etc/profile.d` will not take effect until the next shell or desktop session,
so you should log out and log back in.
## Option 3: Using fanquake's Docker image
Please refer to fanquake's instructions
[here](https://github.com/fanquake/core-review/tree/master/guix).
Note that the `Dockerfile` is largely equivalent to running through the binary
tarball installation steps.
## Option 4: Using a distribution-maintained package
Note that this section is based on the distro packaging situation at the time of
writing (July 2021). Guix is expected to be more widely packaged over time. For
an up-to-date view on Guix's package status/version across distros, please see:
https://repology.org/project/guix/versions
### Debian 11 (Bullseye)/Ubuntu 21.04 (Hirsute Hippo)
Guix v1.2.0 is available as a distribution package starting in [Debian
11](https://packages.debian.org/bullseye/guix) and [Ubuntu
21.04](https://packages.ubuntu.com/hirsute/guix).
Note that if you intend on using Guix without using any substitutes (more
details [here][security-model]), v1.2.0 has a known problem when building GnuTLS
from source. Solutions and workarounds are documented
[here](#gnutls-test-suite-fail-status-request-revoked).
To install:
```sh
sudo apt install guix
```
For up-to-date information on Debian and Ubuntu's release history:
- [Debian release history](https://www.debian.org/releases/)
- [Ubuntu release history](https://ubuntu.com/about/release-cycle)
### Arch Linux
Guix is available in the AUR as
[`guix`](https://aur.archlinux.org/packages/guix/), please follow the
installation instructions in the Arch Linux Wiki ([live
link](https://wiki.archlinux.org/index.php/Guix#AUR_Package_Installation),
[2021/03/30
permalink](https://wiki.archlinux.org/index.php?title=Guix&oldid=637559#AUR_Package_Installation))
to install Guix.
At the time of writing (2021/03/30), the `check` phase will fail if the path to
guix's build directory is longer than 36 characters due to an anachronistic
character limit on the shebang line. Since the `check` phase happens after the
`build` phase, which may take quite a long time, it is recommended that users
either:
1. Skip the `check` phase
- For `makepkg`: `makepkg --nocheck ...`
- For `yay`: `yay --mflags="--nocheck" ...`
- For `paru`: `paru --nocheck ...`
2. Or, check their build directory's length beforehand
- For those building with `makepkg`: `pwd | wc -c`
## Option 5: Building from source
Building Guix from source is a rather involved process but a rewarding one for
those looking to minimize trust and maximize customizability (e.g. building a
particular commit of Guix). Previous experience with using autotools-style build
systems to build packages from source will be helpful. *hic sunt dracones.*
I strongly urge you to at least skim through the entire section once before you
start issuing commands, as it will save you a lot of unnecessary pain and
anguish.
### Installing common build tools
There are a few basic build tools that are required for most things we'll build,
so let's install them now:
Text transformation/i18n:
- `autopoint` (sometimes packaged in `gettext`)
- `help2man`
- `po4a`
- `texinfo`
Build system tools:
- `g++` w/ C++11 support
- `libtool`
- `autoconf`
- `automake`
- `pkg-config` (sometimes packaged as `pkgconf`)
- `make`
- `cmake`
Miscellaneous:
- `git`
- `gnupg`
- `python3`
### Building and Installing Guix's dependencies
In order to build Guix itself from source, we need to first make sure that the
necessary dependencies are installed and discoverable. The most up-to-date list
of Guix's dependencies is kept in the ["Requirements"
section](https://guix.gnu.org/manual/en/html_node/Requirements.html) of the Guix
Reference Manual.
Depending on your distribution, most or all of these dependencies may already be
packaged and installable without manually building and installing.
For reference, the graphic below outlines Guix v1.3.0's dependency graph:
![bootstrap map](https://user-images.githubusercontent.com/6399679/125064185-a9a59880-e0b0-11eb-82c1-9b8e5dc9950d.png)
#### Guile
##### Choosing a Guile version and sticking to it
One of the first things you need to decide is which Guile version you want to
use: Guile v2.2 or Guile v3.0. Unlike the python2 to python3 transition, Guile
v2.2 and Guile v3.0 are largely compatible, as evidenced by the fact that most
Guile packages and even [Guix
itself](https://guix.gnu.org/en/blog/2020/guile-3-and-guix/) support running on
both.
What is important here is that you **choose one**, and you **remain consistent**
with your choice throughout **all Guile-related packages**, no matter if they
are installed via the distribution's package manager or installed from source.
This is because the files for Guile packages are installed to directories which
are separated based on the Guile version.
###### Example: Checking that Ubuntu's `guile-git` is compatible with your chosen Guile version
On Ubuntu Focal:
```sh
$ apt show guile-git
Package: guile-git
...
Depends: guile-2.2, guile-bytestructures, libgit2-dev
...
```
As you can see, the package `guile-git` depends on `guile-2.2`, meaning that it
was likely built for Guile v2.2. This means that if you decided to use Guile
v3.0 on Ubuntu Focal, you would need to build guile-git from source instead of
using the distribution package.
On Ubuntu Hirsute:
```sh
$ apt show guile-git
Package: guile-git
...
Depends: guile-3.0 | guile-2.2, guile-bytestructures (>= 1.0.7-3~), libgit2-dev (>= 1.0)
...
```
In this case, `guile-git` depends on either `guile-3.0` or `guile-2.2`, meaning
that it would work no matter what Guile version you decided to use.
###### Corner case: Multiple versions of Guile on one system
It is recommended to only install one version of Guile, so that build systems do
not get confused about which Guile to use.
However, if you insist on having both Guile v2.2 and Guile v3.0 installed on
your system, then you need to **consistently** specify one of
`GUILE_EFFECTIVE_VERSION=3.0` or `GUILE_EFFECTIVE_VERSION=2.2` to all
`./configure` invocations for Guix and its dependencies.
##### Installing Guile
Guile is most likely already packaged for your distribution, so after you have
[chosen a Guile version](#choosing-a-guile-version-and-sticking-to-it), install
it via your distribution's package manager.
If your distribution splits packages into `-dev`-suffixed and
non-`-dev`-suffixed sub-packages (as is the case for Debian-derived
distributions), please make sure to install both. For example, to install Guile
v2.2 on Debian/Ubuntu:
```sh
apt install guile-2.2 guile-2.2-dev
```
#### Mixing distribution packages and source-built packages
At the time of writing, most distributions have _some_ of Guix's dependencies
packaged, but not all. This means that you may want to install the distribution
package for some dependencies, and manually build-from-source for others.
Distribution packages usually install to `/usr`, which is different from the
default `./configure` prefix of source-built packages: `/usr/local`.
This means that if you mix-and-match distribution packages and source-built
packages and do not specify exactly `--prefix=/usr` to `./configure` for
source-built packages, you will need to augment the `GUILE_LOAD_PATH` and
`GUILE_LOAD_COMPILED_PATH` environment variables so that Guile will look
under the right prefix and find your source-built packages.
For example, if you are using Guile v2.2, and have Guile packages in the
`/usr/local` prefix, either add the following lines to your `.profile` or
`.bash_profile` so that the environment variable is properly set for all future
shell logins, or paste the lines into a POSIX-style shell to temporarily modify
the environment variables of your current shell session.
```sh
# Help Guile v2.2.x find packages in /usr/local
export GUILE_LOAD_PATH="/usr/local/share/guile/site/2.2${GUILE_LOAD_PATH:+:}$GUILE_LOAD_PATH"
export GUILE_LOAD_COMPILED_PATH="/usr/local/lib/guile/2.2/site-ccache${GUILE_LOAD_COMPILED_PATH:+:}$GUILE_COMPILED_LOAD_PATH"
```
Note that these environment variables are used to check for packages during
`./configure`, so they should be set as soon as possible should you want to use
a prefix other than `/usr`.
#### Building and installing source-built packages
***IMPORTANT**: A few dependencies have non-obvious quirks/errata which are
documented in the sub-sections immediately below. Please read these sections
before proceeding to build and install these packages.*
Although you should always refer to the README or INSTALL files for the most
accurate information, most of these dependencies use autoconf-style build
systems (check if there's a `configure.ac` file), and will likely do the right
thing with the following:
Clone the repository and check out the latest release:
```sh
git clone <git-repo-of-dependency>/<dependency>.git
cd <dependency>
git tag -l # check for the latest release
git checkout <latest-release>
```
For autoconf-based build systems (if `./autogen.sh` or `configure.ac` exists at
the root of the repository):
```sh
./autogen.sh || autoreconf -vfi
./configure --prefix=<prefix>
make
sudo make install
```
For CMake-based build systems (if `CMakeLists.txt` exists at the root of the
repository):
```sh
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=<prefix>
sudo cmake --build . --target install
```
If you choose not to specify exactly `--prefix=/usr` to `./configure`, please
make sure you've carefully read the [previous section] on mixing distribution
packages and source-built packages.
##### Binding packages require `-dev`-suffixed packages
Relevant for:
- Everyone
When building bindings, the `-dev`-suffixed version of the original package
needs to be installed. For example, building `Guile-zlib` on Debian-derived
distributions requires that `zlib1g-dev` is installed.
When using bindings, the `-dev`-suffixed version of the original package still
needs to be installed. This is particularly problematic when distribution
packages are mispackaged like `guile-sqlite3` is in Ubuntu Focal such that
installing `guile-sqlite3` does not automatically install `libsqlite3-dev` as a
dependency.
Below is a list of relevant Guile bindings and their corresponding `-dev`
packages in Debian at the time of writing.
| Guile binding package | -dev Debian package |
|-----------------------|---------------------|
| guile-gcrypt | libgcrypt-dev |
| guile-git | libgit2-dev |
| guile-lzlib | liblz-dev |
| guile-ssh | libssh-dev |
| guile-sqlite3 | libsqlite3-dev |
| guile-zlib | zlib1g-dev |
##### `guile-git` actually depends on `libgit2 >= 1.1`
Relevant for:
- Those building `guile-git` from source against `libgit2 < 1.1`
- Those installing `guile-git` from their distribution where `guile-git` is
built against `libgit2 < 1.1`
As of v0.4.0, `guile-git` claims to only require `libgit2 >= 0.28.0`, however,
it actually requires `libgit2 >= 1.1`, otherwise, it will be confused by a
reference of `origin/keyring`: instead of interpreting the reference as "the
'keyring' branch of the 'origin' remote", the reference is interpreted as "the
branch literally named 'origin/keyring'"
This is especially notable because Ubuntu Focal packages `libgit2 v0.28.4`, and
`guile-git` is built against it.
Should you be in this situation, you need to build both `libgit2 v1.1.x` and
`guile-git` from source.
Source: http://logs.guix.gnu.org/guix/2020-11-12.log#232527
##### `{scheme,guile}-bytestructures` v1.0.8 and v1.0.9 are broken for Guile v2.2
Relevant for:
- Those building `{scheme,guile}-bytestructures` from source against Guile v2.2
Commit
[707eea3](https://github.com/TaylanUB/scheme-bytestructures/commit/707eea3a85e1e375e86702229ebf73d496377669)
introduced a regression for Guile v2.2 and was first included in v1.0.8, this
was later corrected in commit
[ec9a721](https://github.com/TaylanUB/scheme-bytestructures/commit/ec9a721957c17bcda13148f8faa5f06934431ff7)
and included in v1.1.0.
TL;DR If you decided to use Guile v2.2, do not use `{scheme,guile}-bytestructures` v1.0.8 or v1.0.9.
### Building and Installing Guix itself
Start by cloning Guix:
```
git clone https://git.savannah.gnu.org/git/guix.git
cd guix
```
You will likely want to build the latest release, however, if the latest release
when you're reading this is still 1.2.0 then you may want to use 95aca29 instead
to avoid a problem in the GnuTLS test suite.
```
git branch -a -l 'origin/version-*' # check for the latest release
git checkout <latest-release>
```
Bootstrap the build system:
```
./bootstrap
```
Configure with the recommended `--localstatedir` flag:
```
./configure --localstatedir=/var
```
Note: If you intend to hack on Guix in the future, you will need to supply the
same `--localstatedir=` flag for all future Guix `./configure` invocations. See
the last paragraph of this
[section](https://guix.gnu.org/manual/en/html_node/Requirements.html) for more
details.
Build Guix (this will take a while):
```
make -j$(nproc)
```
Install Guix:
```
sudo make install
```
### Post-"build from source" Setup
#### Creating and starting a `guix-daemon-original` service with a fixed `argv[0]`
At this point, guix will be installed to `${bindir}`, which is likely
`/usr/local/bin` if you did not override directory variables at
`./configure`-time. More information on standard Automake directory variables
can be found
[here](https://www.gnu.org/software/automake/manual/html_node/Standard-Directory-Variables.html).
However, the Guix init scripts and service configurations for Upstart, systemd,
SysV, and OpenRC are installed (in `${libdir}`) to launch
`${localstatedir}/guix/profiles/per-user/root/current-guix/bin/guix-daemon`,
which does not yet exist, and will only exist after [`root` performs their first
`guix pull`](#guix-pull-as-root).
We need to create a `-original` version of these init scripts that's pointed to
the binaries we just built and `make install`'ed in `${bindir}` (normally,
`/usr/local/bin`).
Example for `systemd`, run as `root`:
```sh
# Create guix-daemon-original.service by modifying guix-daemon.service
libdir=# set according to your PREFIX (default is /usr/local/lib)
bindir="$(dirname $(command -v guix-daemon))"
sed -E -e "s|/\S*/guix/profiles/per-user/root/current-guix/bin/guix-daemon|${bindir}/guix-daemon|" "${libdir}"/systemd/system/guix-daemon.service > /etc/systemd/system/guix-daemon-original.service
chmod 664 /etc/systemd/system/guix-daemon-original.service
# Make systemd recognize the new service
systemctl daemon-reload
# Make sure that the non-working guix-daemon.service is stopped and disabled
systemctl stop guix-daemon
systemctl disable guix-daemon
# Make sure that the working guix-daemon-original.service is started and enabled
systemctl enable guix-daemon-original
systemctl start guix-daemon-original
```
#### Creating `guix-daemon` users / groups
Please see the [relevant
section](https://guix.gnu.org/manual/en/html_node/Build-Environment-Setup.html)
in the Guix Reference Manual for more details.
## Optional setup
At this point, you are set up to [use Guix to build Bitcoin
Core](./README.md#usage). However, if you want to polish your setup a bit and
make it "what Guix intended", then read the next few subsections.
### Add an `/etc/profile.d` entry
This section definitely does not apply to you if you installed Guix using:
1. The shell installer script
2. fanquake's Docker image
3. Debian's `guix` package
#### Background
Although Guix knows how to update itself and its packages, it does so in a
non-invasive way (it does not modify `/usr/local/bin/guix`).
Instead, it does the following:
- After a `guix pull`, it updates
`/var/guix/profiles/per-user/$USER/current-guix`, and creates a symlink
targeting this directory at `$HOME/.config/guix/current`
- After a `guix install`, it updates
`/var/guix/profiles/per-user/$USER/guix-profile`, and creates a symlink
targeting this directory at `$HOME/.guix-profile`
Therefore, in order for these operations to affect your shell/desktop sessions
(and for the principle of least astonishment to hold), their corresponding
directories have to be added to well-known environment variables like `$PATH`,
`$INFOPATH`, `$XDG_DATA_DIRS`, etc.
In other words, if `$HOME/.config/guix/current/bin` does not exist in your
`$PATH`, a `guix pull` will have no effect on what `guix` you are using. Same
goes for `$HOME/.guix-profile/bin`, `guix install`, and installed packages.
Helpfully, after a `guix pull` or `guix install`, a message will be printed like
so:
```
hint: Consider setting the necessary environment variables by running:
GUIX_PROFILE="$HOME/.guix-profile"
. "$GUIX_PROFILE/etc/profile"
Alternately, see `guix package --search-paths -p "$HOME/.guix-profile"'.
```
However, this is somewhat tedious to do for both `guix pull` and `guix install`
for each user on the system that wants to properly use `guix`. I recommend that
you instead add an entry to `/etc/profile.d` instead. This is done by default
when installing the Debian package later than 1.2.0-4 and when using the shell
script installer.
#### Instructions
Create `/etc/profile.d/guix.sh` with the following content:
```sh
# _GUIX_PROFILE: `guix pull` profile
_GUIX_PROFILE="$HOME/.config/guix/current"
if [ -L $_GUIX_PROFILE ]; then
export PATH="$_GUIX_PROFILE/bin${PATH:+:}$PATH"
# Export INFOPATH so that the updated info pages can be found
# and read by both /usr/bin/info and/or $GUIX_PROFILE/bin/info
# When INFOPATH is unset, add a trailing colon so that Emacs
# searches 'Info-default-directory-list'.
export INFOPATH="$_GUIX_PROFILE/share/info:$INFOPATH"
fi
# GUIX_PROFILE: User's default profile
GUIX_PROFILE="$HOME/.guix-profile"
[ -L $GUIX_PROFILE ] || return
GUIX_LOCPATH="$GUIX_PROFILE/lib/locale"
export GUIX_PROFILE GUIX_LOCPATH
[ -f "$GUIX_PROFILE/etc/profile" ] && . "$GUIX_PROFILE/etc/profile"
# set XDG_DATA_DIRS to include Guix installations
export XDG_DATA_DIRS="$GUIX_PROFILE/share:${XDG_DATA_DIRS:-/usr/local/share/:/usr/share/}"
```
Please note that this will not take effect until the next shell or desktop
session (log out and log back in).
### `guix pull` as root
Before you do this, you need to read the section on [choosing your security
model][security-model] and adjust `guix` and `guix-daemon` flags according to
your choice, as invoking `guix pull` may pull substitutes from substitute
servers (which you may not want).
As mentioned in a previous section, Guix expects
`${localstatedir}/guix/profiles/per-user/root/current-guix` to be populated with
`root`'s Guix profile, `guix pull`-ed and built by some former version of Guix.
However, this is not the case when we build from source. Therefore, we need to
perform a `guix pull` as `root`:
```sh
sudo --login guix pull --branch=version-<latest-release-version>
# or
sudo --login guix pull --commit=<particular-commit>
```
`guix pull` is quite a long process (especially if you're using
`--no-substitute`). If you encounter build problems, please refer to the
[troubleshooting section](#troubleshooting).
Note that running a bare `guix pull` with no commit or branch specified will
pull the latest commit on Guix's master branch, which is likely fine, but not
recommended.
If you installed Guix from source, you may get an error like the following:
```sh
error: while creating symlink '/root/.config/guix/current' No such file or directory
```
To resolve this, simply:
```
sudo mkdir -p /root/.config/guix
```
Then try the `guix pull` command again.
After the `guix pull` finishes successfully,
`${localstatedir}/guix/profiles/per-user/root/current-guix` should be populated.
#### Using the newly-pulled `guix` by restarting the daemon
Depending on how you installed Guix, you should now make sure that your init
scripts and service configurations point to the newly-pulled `guix-daemon`.
##### If you built Guix from source
If you followed the instructions for [fixing argv\[0\]][fix-argv0], you can now
do the following:
```sh
systemctl stop guix-daemon-original
systemctl disable guix-daemon-original
systemctl enable guix-daemon
systemctl start guix-daemon
```
##### If you installed Guix via the Debian/Ubuntu distribution packages
You will need to create a `guix-daemon-latest` service which points to the new
`guix` rather than a pinned one.
```sh
# Create guix-daemon-latest.service by modifying guix-daemon.service
sed -E -e "s|/usr/bin/guix-daemon|/var/guix/profiles/per-user/root/current-guix/bin/guix-daemon|" /etc/systemd/system/guix-daemon.service > /lib/systemd/system/guix-daemon-latest.service
chmod 664 /lib/systemd/system/guix-daemon-latest.service
# Make systemd recognize the new service
systemctl daemon-reload
# Make sure that the old guix-daemon.service is stopped and disabled
systemctl stop guix-daemon
systemctl disable guix-daemon
# Make sure that the new guix-daemon-latest.service is started and enabled
systemctl enable guix-daemon-latest
systemctl start guix-daemon-latest
```
##### If you installed Guix via lantw44's Arch Linux AUR package
At the time of writing (July 5th, 2021) the systemd unit for "updated Guix" is
`guix-daemon-latest.service`, therefore, you should do the following:
```sh
systemctl stop guix-daemon
systemctl disable guix-daemon
systemctl enable guix-daemon-latest
systemctl start guix-daemon-latest
```
##### Otherwise...
Simply do:
```sh
systemctl restart guix-daemon
```
### Checking everything
If you followed all the steps above to make your Guix setup "prim and proper,"
you can check that you did everything properly by running through this
checklist.
1. `/etc/profile.d/guix.sh` should exist and be sourced at each shell login
2. `guix describe` should not print `guix describe: error: failed to determine
origin`, but rather something like:
```
Generation 38 Feb 22 2021 16:39:31 (current)
guix f350df4
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: version-1.2.0
commit: f350df405fbcd5b9e27e6b6aa500da7f101f41e7
```
3. `guix-daemon` should be running from `${localstatedir}/guix/profiles/per-user/root/current-guix`
# Troubleshooting
## Derivation failed to build
When you see a build failure like below:
```
building /gnu/store/...-foo-3.6.12.drv...
/ 'check' phasenote: keeping build directory `/tmp/guix-build-foo-3.6.12.drv-0'
builder for `/gnu/store/...-foo-3.6.12.drv' failed with exit code 1
build of /gnu/store/...-foo-3.6.12.drv failed
View build log at '/var/log/guix/drvs/../...-foo-3.6.12.drv.bz2'.
cannot build derivation `/gnu/store/...-qux-7.69.1.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/...-bar-3.16.5.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/...-baz-2.0.5.drv': 1 dependencies couldn't be built
guix time-machine: error: build of `/gnu/store/...-baz-2.0.5.drv' failed
```
It means that `guix` failed to build a package named `foo`, which was a
dependency of `qux`, `bar`, and `baz`. Importantly, note that the last "failed"
line is not necessarily the root cause, the first "failed" line is.
Most of the time, the build failure is due to a spurious test failure or the
package's build system/test suite breaking when running multi-threaded. To
rebuild _just_ this derivation in a single-threaded fashion (please don't forget
to add other `guix` flags like `--no-substitutes` as appropriate):
```sh
$ guix build --cores=1 /gnu/store/...-foo-3.6.12.drv
```
If the single-threaded rebuild did not succeed, you may need to dig deeper.
You may view `foo`'s build logs in `less` like so (please replace paths with the
path you see in the build failure output):
```sh
$ bzcat /var/log/guix/drvs/../...-foo-3.6.12.drv.bz2 | less
```
`foo`'s build directory is also preserved and available at
`/tmp/guix-build-foo-3.6.12.drv-0`. However, if you fail to build `foo` multiple
times, it may be `/tmp/...drv-1` or `/tmp/...drv-2`. Always consult the build
failure output for the most accurate, up-to-date information.
### python(-minimal): [Errno 84] Invalid or incomplete multibyte or wide character
This error occurs when your `$TMPDIR` (default: /tmp) exists on a filesystem
which rejects characters not present in the UTF-8 character code set. An example
is ZFS with the utf8only=on option set.
More information: https://bugs.python.org/issue37584
### GnuTLS: test-suite FAIL: status-request-revoked
*The derivation is likely identified by: `/gnu/store/vhphki5sg9xkdhh2pbc8gi6vhpfzryf0-gnutls-3.6.12.drv`*
This unfortunate error is most common for non-substitute builders who installed
Guix v1.2.0. The problem stems from the fact that one of GnuTLS's tests uses a
hardcoded certificate which expired on 2020-10-24.
What's more unfortunate is that this GnuTLS derivation is somewhat special in
Guix's dependency graph and is not affected by the package transformation flags
like `--without-tests=`.
The easiest solution for those encountering this problem is to install a newer
version of Guix. However, there are ways to work around this issue:
#### Workaround 1: Using substitutes for this single derivation
If you've authorized the official Guix build farm's key (more info
[here](./README.md#step-1-authorize-the-signing-keys)), then you can use
substitutes just for this single derivation by invoking the following:
```sh
guix build --substitute-urls="https://ci.guix.gnu.org" /gnu/store/vhphki5sg9xkdhh2pbc8gi6vhpfzryf0-gnutls-3.6.12.drv
```
See [this section](./README.md#removing-authorized-keys) for instructions on how
to remove authorized keys if you don't want to keep the build farm's key
authorized.
#### Workaround 2: Temporarily setting the system clock back
This workaround was described [here](https://issues.guix.gnu.org/44559#5).
Basically:
1. Turn off networking
2. Turn off NTP
3. Set system time to 2020-10-01
4. guix build --no-substitutes /gnu/store/vhphki5sg9xkdhh2pbc8gi6vhpfzryf0-gnutls-3.6.12.drv
5. Set system time back to accurate current time
6. Turn NTP back on
7. Turn networking back on
### coreutils: FAIL: tests/tail-2/inotify-dir-recreate
The inotify-dir-create test fails on "remote" filesystems such as overlayfs
(Docker's default filesystem) due to the filesystem being mistakenly recognized
as non-remote.
A relatively easy workaround to this is to make sure that a somewhat traditional
filesystem is mounted at `/tmp` (where `guix-daemon` performs its builds). For
Docker users, this might mean [using a volume][docker/volumes], [binding
mounting][docker/bind-mnt] from host, or (for those with enough RAM and swap)
[mounting a tmpfs][docker/tmpfs] using the `--tmpfs` flag.
Please see the following links for more details:
- An upstream coreutils bug has been filed: [debbugs#47940](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47940)
- A Guix bug detailing the underlying problem has been filed: [guix-issues#47935](https://issues.guix.gnu.org/47935)
- A commit to skip this test in Guix has been merged into the core-updates branch:
[savannah/guix@6ba1058](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=6ba1058df0c4ce5611c2367531ae5c3cdc729ab4)
[install-script]: #options-1-and-2-using-the-official-shell-installer-script-or-binary-tarball
[install-bin-tarball]: #options-1-and-2-using-the-official-shell-installer-script-or-binary-tarball
[install-fanquake-docker]: #option-3-using-fanquakes-docker-image
[install-distro-pkg]: #option-4-using-a-distribution-maintained-package
[install-source]: #option-5-building-from-source
[fix-argv0]: #creating-and-starting-a-guix-daemon-original-service-with-a-fixed-argv0
[security-model]: ./README.md#choosing-your-security-model
[docker/volumes]: https://docs.docker.com/storage/volumes/
[docker/bind-mnt]: https://docs.docker.com/storage/bind-mounts/
[docker/tmpfs]: https://docs.docker.com/storage/tmpfs/

View File

@@ -9,222 +9,114 @@ downloads.
We achieve bootstrappability by using Guix as a functional package manager.
# Requirements
## Requirements
Conservatively, you will need an x86_64 machine with:
Conservatively, a x86_64 machine with:
- 16GB of free disk space on the partition that /gnu/store will reside in
- 8GB of free disk space **per platform triple** you're planning on building
(see the `HOSTS` [environment variable description][env-vars-list])
- 4GB of free disk space on the partition that /gnu/store will reside in
- 24GB of free disk space on the partition that the Bitcoin Core git repository
resides in
# Installation and Setup
> Note: these requirements are slightly less onerous than those of Gitian builds
If you don't have Guix installed and set up, please follow the instructions in
[INSTALL.md](./INSTALL.md)
## Setup
# Usage
### Installing Guix
If you haven't considered your security model yet, please read [the relevant
section](#choosing-your-security-model) before proceeding to perform a build.
If you're just testing this out, you can use the
[Dockerfile][fanquake/guix-docker] for convenience. It automatically speeds up
your builds by [using substitutes](#speeding-up-builds-with-substitute-servers).
If you don't want this behaviour, refer to the [next
section](#choosing-your-security-model).
## Making the Xcode SDK available for macOS cross-compilation
Otherwise, follow the [Guix installation guide][guix/bin-install].
In order to perform a build for macOS (which is included in the default set of
platform triples to build), you'll need to extract the macOS SDK tarball using
tools found in the [`macdeploy` directory](../macdeploy/README.md).
> Note: For those who like to keep their filesystems clean, Guix is designed to
> be very standalone and _will not_ conflict with your system's package
> manager/existing setup. It _only_ touches `/var/guix`, `/gnu`, and
> `~/.config/guix`.
You can then either point to the SDK using the `SDK_PATH` environment variable:
### Choosing your security model
Guix allows us to achieve better binary security by using our CPU time to build
everything from scratch. However, it doesn't sacrifice user choice in pursuit of
this: users can decide whether or not to bootstrap and to use substitutes.
After installation, you may want to consider [adding substitute
servers](#speeding-up-builds-with-substitute-servers) to speed up your build if
that fits your security model (say, if you're just testing that this works).
This is skippable if you're using the [Dockerfile][fanquake/guix-docker].
If you prefer not to use any substitutes, make sure to set
`ADDITIONAL_GUIX_ENVIRONMENT_FLAGS` like the following snippet. The first build
will take a while, but the resulting packages will be cached for future builds.
```sh
# Extract the SDK tarball to /path/to/parent/dir/of/extracted/SDK/Xcode-<foo>-<bar>-extracted-SDK-with-libcxx-headers
tar -C /path/to/parent/dir/of/extracted/SDK -xaf /path/to/Xcode-<foo>-<bar>-extracted-SDK-with-libcxx-headers.tar.gz
# Indicate where to locate the SDK tarball
export SDK_PATH=/path/to/parent/dir/of/extracted/SDK
export ADDITIONAL_GUIX_ENVIRONMENT_FLAGS='--no-substitutes'
```
or extract it into `depends/SDKs`:
Likewise, to perform a bootstrapped build (takes even longer):
```sh
mkdir -p depends/SDKs
tar -C depends/SDKs -xaf /path/to/SDK/tarball
export ADDITIONAL_GUIX_ENVIRONMENT_FLAGS='--bootstrap --no-substitutes'
```
## Building
### Using a version of Guix with `guix time-machine` capabilities
*The author highly recommends at least reading over the [common usage patterns
and examples](#common-guix-build-invocation-patterns-and-examples) section below
before starting a build. For a full list of customization options, see the
[recognized environment variables][env-vars-list] section.*
> Note: This entire section can be skipped if you are already using a version of
> Guix that has [the `guix time-machine` command][guix/time-machine].
To build Bitcoin Core reproducibly with all default options, invoke the
following from the top of a clean repository:
Once Guix is installed, if it doesn't have the `guix time-machine` command, pull
the latest `guix`.
```sh
./contrib/guix/guix-build
guix pull --max-jobs=4 # change number of jobs accordingly
```
## Codesigning build outputs
Make sure that you are using your current profile. (You are prompted to do this
at the end of the `guix pull`)
The `guix-codesign` command attaches codesignatures (produced by codesigners) to
existing non-codesigned outputs. Please see the [release process
documentation](/doc/release-process.md) for more context.
It respects many of the same environment variable flags as `guix-build`, with 2
crucial differences:
1. Since only Windows and macOS build outputs require codesigning, the `HOSTS`
environment variable will have a sane default value of `x86_64-w64-mingw32
x86_64-apple-darwin18` instead of all the platforms.
2. The `guix-codesign` command ***requires*** a `DETACHED_SIGS_REPO` flag.
* _**DETACHED_SIGS_REPO**_
Set the directory where detached codesignatures can be found for the current
Bitcoin Core version being built.
_REQUIRED environment variable_
An invocation with all default options would look like:
```
env DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign
```bash
export PATH="${HOME}/.config/guix/current/bin${PATH:+:}$PATH"
```
## Cleaning intermediate work directories
## Usage
By default, `guix-build` leaves all intermediate files or "work directories"
(e.g. `depends/work`, `guix-build-*/distsrc-*`) intact at the end of a build so
that they are available to the user (to aid in debugging, etc.). However, these
directories usually take up a large amount of disk space. Therefore, a
`guix-clean` convenience script is provided which cleans the current `git`
worktree to save disk space:
### As a Development Environment
```
./contrib/guix/guix-clean
```
## Attesting to build outputs
Much like how Gitian build outputs are attested to in a `gitian.sigs`
repository, Guix build outputs are attested to in the [`guix.sigs`
repository](https://github.com/bitcoin-core/guix.sigs).
After you've cloned the `guix.sigs` repository, to attest to the current
worktree's commit/tag:
```
env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=<gpg-key-name> ./contrib/guix/guix-attest
```
See `./contrib/guix/guix-attest --help` for more information on the various ways
`guix-attest` can be invoked.
## Verifying build output attestations
After at least one other signer has uploaded their signatures to the `guix.sigs`
repository:
```
git -C <path/to/guix.sigs> pull
env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify
```
## Common `guix-build` invocation patterns and examples
### Keeping caches and SDKs outside of the worktree
If you perform a lot of builds and have a bunch of worktrees, you may find it
more efficient to keep the depends tree's download cache, build cache, and SDKs
outside of the worktrees to avoid duplicate downloads and unnecessary builds. To
help with this situation, the `guix-build` script honours the `SOURCES_PATH`,
`BASE_CACHE`, and `SDK_PATH` environment variables and will pass them on to the
depends tree so that you can do something like:
For a Bitcoin Core depends development environment, simply invoke
```sh
env SOURCES_PATH="$HOME/depends-SOURCES_PATH" BASE_CACHE="$HOME/depends-BASE_CACHE" SDK_PATH="$HOME/macOS-SDKs" ./contrib/guix/guix-build
guix environment --manifest=contrib/guix/manifest.scm
```
Note that the paths that these environment variables point to **must be
directories**, and **NOT symlinks to directories**.
And you'll land back in your shell with all the build dependencies required for
a `depends` build injected into your environment.
See the [recognized environment variables][env-vars-list] section for more
details.
### As a Tool for Deterministic Builds
### Building a subset of platform triples
Sometimes you only want to build a subset of the supported platform triples, in
which case you can override the default list by setting the space-separated
`HOSTS` environment variable:
From the top of a clean Bitcoin Core repository:
```sh
env HOSTS='x86_64-w64-mingw32 x86_64-apple-darwin18' ./contrib/guix/guix-build
./contrib/guix/guix-build.sh
```
See the [recognized environment variables][env-vars-list] section for more
details.
### Controlling the number of threads used by `guix` build commands
Depending on your system's RAM capacity, you may want to decrease the number of
threads used to decrease RAM usage or vice versa.
By default, the scripts under `./contrib/guix` will invoke all `guix` build
commands with `--cores="$JOBS"`. Note that `$JOBS` defaults to `$(nproc)` if not
specified. However, astute manual readers will also notice that `guix` build
commands also accept a `--max-jobs=` flag (which defaults to 1 if unspecified).
Here is the difference between `--cores=` and `--max-jobs=`:
> Note: When I say "derivation," think "package"
`--cores=`
- controls the number of CPU cores to build each derivation. This is the value
passed to `make`'s `--jobs=` flag.
`--max-jobs=`
- controls how many derivations can be built in parallel
- defaults to 1
Therefore, the default is for `guix` build commands to build one derivation at a
time, utilizing `$JOBS` threads.
Specifying the `$JOBS` environment variable will only modify `--cores=`, but you
can also modify the value for `--max-jobs=` by specifying
`$ADDITIONAL_GUIX_COMMON_FLAGS`. For example, if you have a LOT of memory, you
may want to set:
After the build finishes successfully (check the status code please), compare
hashes:
```sh
export ADDITIONAL_GUIX_COMMON_FLAGS='--max-jobs=8'
find output/ -type f -print0 | sort -z | xargs -r0 sha256sum
```
Which allows for a maximum of 8 derivations to be built at the same time, each
utilizing `$JOBS` threads.
Or, if you'd like to avoid spurious build failures caused by issues with
parallelism within a single package, but would still like to build multiple
packages when the dependency graph allows for it, you may want to try:
```sh
export JOBS=1 ADDITIONAL_GUIX_COMMON_FLAGS='--max-jobs=8'
```
See the [recognized environment variables][env-vars-list] section for more
details.
## Recognized environment variables
#### Recognized environment variables
* _**HOSTS**_
Override the space-separated list of platform triples for which to perform a
bootstrappable build.
bootstrappable build. _(defaults to "x86\_64-linux-gnu
arm-linux-gnueabihf aarch64-linux-gnu riscv64-linux-gnu")_
_(defaults to "x86\_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu
riscv64-linux-gnu powerpc64-linux-gnu powerpc64le-linux-gnu
x86\_64-w64-mingw32 x86\_64-apple-darwin18")_
> Windows and OS X platform triplet support are WIP.
* _**SOURCES_PATH**_
@@ -232,48 +124,18 @@ details.
depends tree. Setting this to the same directory across multiple builds of the
depends tree can eliminate unnecessary redownloading of package sources.
The path that this environment variable points to **must be a directory**, and
**NOT a symlink to a directory**.
* _**MAX_JOBS**_
* _**BASE_CACHE**_
Set the depends tree cache for built packages. This is passed through to the
depends tree. Setting this to the same directory across multiple builds of the
depends tree can eliminate unnecessary building of packages.
The path that this environment variable points to **must be a directory**, and
**NOT a symlink to a directory**.
* _**SDK_PATH**_
Set the path where _extracted_ SDKs can be found. This is passed through to
the depends tree. Note that this is should be set to the _parent_ directory of
the actual SDK (e.g. `SDK_PATH=$HOME/Downloads/macOS-SDKs` instead of
`$HOME/Downloads/macOS-SDKs/Xcode-12.1-12A7403-extracted-SDK-with-libcxx-headers`).
The path that this environment variable points to **must be a directory**, and
**NOT a symlink to a directory**.
* _**JOBS**_
Override the number of jobs to run simultaneously, you might want to do so on
a memory-limited machine. This may be passed to:
- `guix` build commands as in `guix environment --cores="$JOBS"`
- `make` as in `make --jobs="$JOBS"`
- `xargs` as in `xargs -P"$JOBS"`
See [here](#controlling-the-number-of-threads-used-by-guix-build-commands) for
more details.
_(defaults to the value of `nproc` outside the container)_
Override the maximum number of jobs to run simultaneously, you might want to
do so on a memory-limited machine. This may be passed to `make` as in `make
--jobs="$MAX_JOBS"` or `xargs` as in `xargs -P"$MAX_JOBS"`. _(defaults to the
value of `nproc` outside the container)_
* _**SOURCE_DATE_EPOCH**_
Override the reference UNIX timestamp used for bit-for-bit reproducibility,
the variable name conforms to [standard][r12e/source-date-epoch].
_(defaults to the output of `$(git log --format=%at -1)`)_
the variable name conforms to [standard][r12e/source-date-epoch]. _(defaults
to the output of `$(git log --format=%at -1)`)_
* _**V**_
@@ -285,187 +147,74 @@ details.
string) is interpreted the same way as not setting `V` at all, and that `V=0`
has the same effect as `V=1`.
* _**SUBSTITUTE_URLS**_
A whitespace-delimited list of URLs from which to download pre-built packages.
A URL is only used if its signing key is authorized (refer to the [substitute
servers section](#option-1-building-with-substitutes) for more details).
* _**ADDITIONAL_GUIX_COMMON_FLAGS**_
Additional flags to be passed to all `guix` commands.
* _**ADDITIONAL_GUIX_TIMEMACHINE_FLAGS**_
Additional flags to be passed to `guix time-machine`.
* _**ADDITIONAL_GUIX_ENVIRONMENT_FLAGS**_
Additional flags to be passed to the invocation of `guix environment` inside
`guix time-machine`.
Additional flags to be passed to `guix environment`. For a fully-bootstrapped
build, set this to `--bootstrap --no-substitutes` (refer to the [security
model section](#choosing-your-security-model) for more details). Note that a
fully-bootstrapped build will take quite a long time on the first run.
# Choosing your security model
## Tips and Tricks
No matter how you installed Guix, you need to decide on your security model for
building packages with Guix.
### Speeding up builds with substitute servers
Guix allows us to achieve better binary security by using our CPU time to build
everything from scratch. However, it doesn't sacrifice user choice in pursuit of
this: users can decide whether or not to use **substitutes** (pre-built
packages).
_This whole section is automatically done in the convenience
[Dockerfiles][fanquake/guix-docker]_
## Option 1: Building with substitutes
For those who are used to life in the fast _(and trustful)_ lane, you can use
[substitute servers][guix/substitutes] to enable binary downloads of packages.
### Step 1: Authorize the signing keys
> For those who only want to use substitutes from the official Guix build farm
> and have authorized the build farm's signing key during Guix's installation,
> you don't need to do anything.
Depending on the installation procedure you followed, you may have already
authorized the Guix build farm key. In particular, the official shell installer
script asks you if you want the key installed, and the debian distribution
package authorized the key during installation.
#### Authorize the signing keys
You can check the current list of authorized keys at `/etc/guix/acl`.
At the time of writing, a `/etc/guix/acl` with just the Guix build farm key
authorized looks something like:
```lisp
(acl
(entry
(public-key
(ecc
(curve Ed25519)
(q #8D156F295D24B0D9A86FA5741A840FF2D24F60F7B6C4134814AD55625971B394#)
)
)
(tag
(guix import)
)
)
)
```
If you've determined that the official Guix build farm key hasn't been
authorized, and you would like to authorize it, run the following as root:
For the official Guix build farm at https://ci.guix.gnu.org, run as root:
```
guix archive --authorize < /var/guix/profiles/per-user/root/current-guix/share/guix/ci.guix.gnu.org.pub
guix archive --authorize < ~root/.config/guix/current/share/guix/ci.guix.gnu.org.pub
```
If
`/var/guix/profiles/per-user/root/current-guix/share/guix/ci.guix.gnu.org.pub`
doesn't exist, try:
```sh
guix archive --authorize < <PREFIX>/share/guix/ci.guix.gnu.org.pub
```
Where `<PREFIX>` is likely:
- `/usr` if you installed from a distribution package
- `/usr/local` if you installed Guix from source and didn't supply any
prefix-modifying flags to Guix's `./configure`
For dongcarl's substitute server at https://guix.carldong.io, run as root:
```sh
wget -qO- 'https://guix.carldong.io/signing-key.pub' | guix archive --authorize
```
#### Removing authorized keys
#### Use the substitute servers
To remove previously authorized keys, simply edit `/etc/guix/acl` and remove the
`(entry (public-key ...))` entry.
The official Guix build farm at https://ci.guix.gnu.org is automatically used
unless the `--no-substitutes` flag is supplied.
### Step 2: Specify the substitute servers
This can be overridden for all `guix` invocations by passing the
`--substitute-urls` option to your invocation of `guix-daemon`. This can also be
overridden on a call-by-call basis by passing the same `--substitute-urls`
option to client tools such at `guix environment`.
Once its key is authorized, the official Guix build farm at
https://ci.guix.gnu.org is automatically used unless the `--no-substitutes` flag
is supplied. This default list of substitute servers is overridable both on a
`guix-daemon` level and when you invoke `guix` commands. See examples below for
the various ways of adding dongcarl's substitute server after having [authorized
his signing key](#authorize-the-signing-keys).
To use dongcarl's substitute server for Bitcoin Core builds after having
[authorized his signing key](#authorize-the-signing-keys):
Change the **default list** of substitute servers by starting `guix-daemon` with
the `--substitute-urls` option (you will likely need to edit your init script):
```sh
guix-daemon <cmd> --substitute-urls='https://guix.carldong.io https://ci.guix.gnu.org'
```
export ADDITIONAL_GUIX_ENVIRONMENT_FLAGS='--substitute-urls="https://guix.carldong.io https://ci.guix.gnu.org"'
```
Override the default list of substitute servers by passing the
`--substitute-urls` option for invocations of `guix` commands:
## FAQ
```sh
guix <cmd> --substitute-urls='https://guix.carldong.io https://ci.guix.gnu.org'
```
### How can I trust the binary installation?
For scripts under `./contrib/guix`, set the `SUBSTITUTE_URLS` environment
variable:
As mentioned at the bottom of [this manual page][guix/bin-install]:
```sh
export SUBSTITUTE_URLS='https://guix.carldong.io https://ci.guix.gnu.org'
```
> The binary installation tarballs can be (re)produced and verified simply by
> running the following command in the Guix source tree:
>
> make guix-binary.x86_64-linux.tar.xz
## Option 2: Disabling substitutes on an ad-hoc basis
### When will Guix be packaged in debian?
If you prefer not to use any substitutes, make sure to supply `--no-substitutes`
like in the following snippet. The first build will take a while, but the
resulting packages will be cached for future builds.
For direct invocations of `guix`:
```sh
guix <cmd> --no-substitutes
```
For the scripts under `./contrib/guix/`:
```sh
export ADDITIONAL_GUIX_COMMON_FLAGS='--no-substitutes'
```
## Option 3: Disabling substitutes by default
`guix-daemon` accepts a `--no-substitutes` flag, which will make sure that,
unless otherwise overridden by a command line invocation, no substitutes will be
used.
If you start `guix-daemon` using an init script, you can edit said script to
supply this flag.
# Purging/Uninstalling Guix
In the extraordinarily rare case where you messed up your Guix installation in
an irreversible way, you may want to completely purge Guix from your system and
start over.
1. Uninstall Guix itself according to the way you installed it (e.g. `sudo apt
purge guix` for Ubuntu packaging, `sudo make uninstall` for a build from source).
2. Remove all build users and groups
You may check for relevant users and groups using:
```
getent passwd | grep guix
getent group | grep guix
```
Then, you may remove users and groups using:
```
sudo userdel <user>
sudo groupdel <group>
```
3. Remove all possible Guix-related directories
- `/var/guix/`
- `/var/log/guix/`
- `/gnu/`
- `/etc/guix/`
- `/home/*/.config/guix/`
- `/home/*/.cache/guix/`
- `/home/*/.guix-profile/`
- `/root/.config/guix/`
- `/root/.cache/guix/`
- `/root/.guix-profile/`
Vagrant Cascadian has been making good progress on this
[here][debian/guix-package]. We have all the pieces needed to put up an APT
repository and will likely put one up soon.
[b17e]: http://bootstrappable.org/
[r12e/source-date-epoch]: https://reproducible-builds.org/docs/source-date-epoch/
@@ -477,8 +226,5 @@ start over.
[guix/substitute-server-auth]: https://www.gnu.org/software/guix/manual/en/html_node/Substitute-Server-Authorization.html
[guix/time-machine]: https://guix.gnu.org/manual/en/html_node/Invoking-guix-time_002dmachine.html
[debian/guix-bullseye]: https://packages.debian.org/bullseye/guix
[ubuntu/guix-hirsute]: https://packages.ubuntu.com/hirsute/guix
[debian/guix-package]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=850644
[fanquake/guix-docker]: https://github.com/fanquake/core-review/tree/master/guix
[env-vars-list]: #recognized-environment-variables

View File

@@ -1,255 +0,0 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
# Source the common prelude, which:
# 1. Checks if we're at the top directory of the Bitcoin Core repository
# 2. Defines a few common functions and variables
#
# shellcheck source=libexec/prelude.bash
source "$(dirname "${BASH_SOURCE[0]}")/libexec/prelude.bash"
###################
## Sanity Checks ##
###################
################
# Required non-builtin commands should be invokable
################
check_tools cat env basename mkdir diff sort
if [ -z "$NO_SIGN" ]; then
check_tools gpg
fi
################
# Required env vars should be non-empty
################
cmd_usage() {
cat <<EOF
Synopsis:
env GUIX_SIGS_REPO=<path/to/guix.sigs> \\
SIGNER=GPG_KEY_NAME[=SIGNER_NAME] \\
[ NO_SIGN=1 ]
./contrib/guix/guix-attest
Example w/o overriding signing name:
env GUIX_SIGS_REPO=/home/achow101/guix.sigs \\
SIGNER=achow101 \\
./contrib/guix/guix-attest
Example overriding signing name:
env GUIX_SIGS_REPO=/home/dongcarl/guix.sigs \\
SIGNER=0x96AB007F1A7ED999=dongcarl \\
./contrib/guix/guix-attest
Example w/o signing, just creating SHA256SUMS:
env GUIX_SIGS_REPO=/home/achow101/guix.sigs \\
SIGNER=achow101 \\
NO_SIGN=1 \\
./contrib/guix/guix-attest
EOF
}
if [ -z "$GUIX_SIGS_REPO" ] || [ -z "$SIGNER" ]; then
cmd_usage
exit 1
fi
################
# GUIX_SIGS_REPO should exist as a directory
################
if [ ! -d "$GUIX_SIGS_REPO" ]; then
cat << EOF
ERR: The specified GUIX_SIGS_REPO is not an existent directory:
'$GUIX_SIGS_REPO'
Hint: Please clone the guix.sigs repository and point to it with the
GUIX_SIGS_REPO environment variable.
EOF
cmd_usage
exit 1
fi
################
# The key specified in SIGNER should be usable
################
IFS='=' read -r gpg_key_name signer_name <<< "$SIGNER"
if [ -z "${signer_name}" ]; then
signer_name="$gpg_key_name"
fi
if [ -z "$NO_SIGN" ] && ! gpg --dry-run --list-secret-keys "${gpg_key_name}" >/dev/null 2>&1; then
echo "ERR: GPG can't seem to find any key named '${gpg_key_name}'"
exit 1
fi
################
# We should be able to find at least one output
################
echo "Looking for build output SHA256SUMS fragments in ${OUTDIR_BASE}"
shopt -s nullglob
sha256sum_fragments=( "$OUTDIR_BASE"/*/SHA256SUMS.part ) # This expands to an array of directories...
shopt -u nullglob
noncodesigned_fragments=()
codesigned_fragments=()
if (( ${#sha256sum_fragments[@]} )); then
echo "Found build output SHA256SUMS fragments:"
for outdir in "${sha256sum_fragments[@]}"; do
echo " '$outdir'"
case "$outdir" in
"$OUTDIR_BASE"/*-codesigned/SHA256SUMS.part)
codesigned_fragments+=("$outdir")
;;
*)
noncodesigned_fragments+=("$outdir")
;;
esac
done
echo
else
echo "ERR: Could not find any build output SHA256SUMS fragments in ${OUTDIR_BASE}"
exit 1
fi
##############
## Attest ##
##############
# Usage: out_name $outdir
#
# HOST: The output directory being attested
#
out_name() {
basename "$(dirname "$1")"
}
shasum_already_exists() {
cat <<EOF
--
ERR: An ${1} file already exists for '${VERSION}' and attests
differently. You likely previously attested to a partial build (e.g. one
where you specified the HOST environment variable).
See the diff above for more context.
Hint: You may wish to remove the existing attestations and their signatures by
invoking:
rm '${PWD}/${1}'{,.asc}
Then try running this script again.
EOF
}
echo "Attesting to build outputs for version: '${VERSION}'"
echo ""
# Given a SHA256SUMS file as stdin that has lines like:
# 0ba536819b221a91d3d42e978be016aac918f40984754d74058aa0c921cd3ea6 a/b/d/c/d/s/bitcoin-22.0rc2-riscv64-linux-gnu.tar.gz
# ...
#
# Replace each line's file name with its basename:
# 0ba536819b221a91d3d42e978be016aac918f40984754d74058aa0c921cd3ea6 bitcoin-22.0rc2-riscv64-linux-gnu.tar.gz
# ...
#
basenameify_SHA256SUMS() {
sed -E 's@(^[[:xdigit:]]{64}[[:space:]]+).+/([^/]+$)@\1\2@'
}
outsigdir="$GUIX_SIGS_REPO/$VERSION/$signer_name"
mkdir -p "$outsigdir"
(
cd "$outsigdir"
temp_noncodesigned="$(mktemp)"
trap 'rm -rf -- "$temp_noncodesigned"' EXIT
if (( ${#noncodesigned_fragments[@]} )); then
cat "${noncodesigned_fragments[@]}" \
| sort -u \
| sort -k2 \
| basenameify_SHA256SUMS \
> "$temp_noncodesigned"
if [ -e noncodesigned.SHA256SUMS ]; then
# The SHA256SUMS already exists, make sure it's exactly what we
# expect, error out if not
if diff -u noncodesigned.SHA256SUMS "$temp_noncodesigned"; then
echo "A noncodesigned.SHA256SUMS file already exists for '${VERSION}' and is up-to-date."
else
shasum_already_exists noncodesigned.SHA256SUMS
exit 1
fi
else
mv "$temp_noncodesigned" noncodesigned.SHA256SUMS
fi
else
echo "ERR: No noncodesigned outputs found for '${VERSION}', exiting..."
exit 1
fi
temp_all="$(mktemp)"
trap 'rm -rf -- "$temp_all"' EXIT
if (( ${#codesigned_fragments[@]} )); then
# Note: all.SHA256SUMS attests to all of $sha256sum_fragments, but is
# not needed if there are no $codesigned_fragments
cat "${sha256sum_fragments[@]}" \
| sort -u \
| sort -k2 \
| basenameify_SHA256SUMS \
> "$temp_all"
if [ -e all.SHA256SUMS ]; then
# The SHA256SUMS already exists, make sure it's exactly what we
# expect, error out if not
if diff -u all.SHA256SUMS "$temp_all"; then
echo "An all.SHA256SUMS file already exists for '${VERSION}' and is up-to-date."
else
shasum_already_exists all.SHA256SUMS
exit 1
fi
else
mv "$temp_all" all.SHA256SUMS
fi
else
# It is fine to have the codesigned outputs be missing (perhaps the
# detached codesigs have not been published yet), just print a log
# message instead of erroring out
echo "INFO: No codesigned outputs found for '${VERSION}', skipping..."
fi
if [ -z "$NO_SIGN" ]; then
echo "Signing SHA256SUMS to produce SHA256SUMS.asc"
for i in *.SHA256SUMS; do
if [ ! -e "$i".asc ]; then
gpg --detach-sign \
--digest-algo sha256 \
--local-user "$gpg_key_name" \
--armor \
--output "$i".asc "$i"
else
echo "Signature already there"
fi
done
else
echo "Not signing SHA256SUMS as \$NO_SIGN is not empty"
fi
echo ""
)

View File

@@ -1,477 +0,0 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
# Source the common prelude, which:
# 1. Checks if we're at the top directory of the Bitcoin Core repository
# 2. Defines a few common functions and variables
#
# shellcheck source=libexec/prelude.bash
source "$(dirname "${BASH_SOURCE[0]}")/libexec/prelude.bash"
###################
## SANITY CHECKS ##
###################
################
# Required non-builtin commands should be invocable
################
check_tools cat mkdir make getent curl git guix
################
# GUIX_BUILD_OPTIONS should be empty
################
#
# GUIX_BUILD_OPTIONS is an environment variable recognized by guix commands that
# can perform builds. This seems like what we want instead of
# ADDITIONAL_GUIX_COMMON_FLAGS, but the value of GUIX_BUILD_OPTIONS is actually
# _appended_ to normal command-line options. Meaning that they will take
# precedence over the command-specific ADDITIONAL_GUIX_<CMD>_FLAGS.
#
# This seems like a poor user experience. Thus we check for GUIX_BUILD_OPTIONS's
# existence here and direct users of this script to use our (more flexible)
# custom environment variables.
if [ -n "$GUIX_BUILD_OPTIONS" ]; then
cat << EOF
Error: Environment variable GUIX_BUILD_OPTIONS is not empty:
'$GUIX_BUILD_OPTIONS'
Unfortunately this script is incompatible with GUIX_BUILD_OPTIONS, please unset
GUIX_BUILD_OPTIONS and use ADDITIONAL_GUIX_COMMON_FLAGS to set build options
across guix commands or ADDITIONAL_GUIX_<CMD>_FLAGS to set build options for a
specific guix command.
See contrib/guix/README.md for more details.
EOF
exit 1
fi
################
# The git worktree should not be dirty
################
if ! git diff-index --quiet HEAD -- && [ -z "$FORCE_DIRTY_WORKTREE" ]; then
cat << EOF
ERR: The current git worktree is dirty, which may lead to broken builds.
Aborting...
Hint: To make your git worktree clean, You may want to:
1. Commit your changes,
2. Stash your changes, or
3. Set the 'FORCE_DIRTY_WORKTREE' environment variable if you insist on
using a dirty worktree
EOF
exit 1
fi
mkdir -p "$VERSION_BASE"
################
# Build directories should not exist
################
# Default to building for all supported HOSTs (overridable by environment)
export HOSTS="${HOSTS:-x86_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu riscv64-linux-gnu powerpc64-linux-gnu powerpc64le-linux-gnu
x86_64-w64-mingw32
x86_64-apple-darwin18}"
# Usage: distsrc_for_host HOST
#
# HOST: The current platform triple we're building for
#
distsrc_for_host() {
echo "${DISTSRC_BASE}/distsrc-${VERSION}-${1}"
}
# Accumulate a list of build directories that already exist...
hosts_distsrc_exists=""
for host in $HOSTS; do
if [ -e "$(distsrc_for_host "$host")" ]; then
hosts_distsrc_exists+=" ${host}"
fi
done
if [ -n "$hosts_distsrc_exists" ]; then
# ...so that we can print them out nicely in an error message
cat << EOF
ERR: Build directories for this commit already exist for the following platform
triples you're attempting to build, probably because of previous builds.
Please remove, or otherwise deal with them prior to starting another build.
Aborting...
Hint: To blow everything away, you may want to use:
$ ./contrib/guix/guix-clean
Specifically, this will remove all files without an entry in the index,
excluding the SDK directory, the depends download cache, the depends built
packages cache, the garbage collector roots for Guix environments, and the
output directory.
EOF
for host in $hosts_distsrc_exists; do
echo " ${host} '$(distsrc_for_host "$host")'"
done
exit 1
else
mkdir -p "$DISTSRC_BASE"
fi
################
# When building for darwin, the macOS SDK should exists
################
for host in $HOSTS; do
case "$host" in
*darwin*)
OSX_SDK="$(make -C "${PWD}/depends" --no-print-directory HOST="$host" print-OSX_SDK | sed 's@^[^=]\+=@@g')"
if [ -e "$OSX_SDK" ]; then
echo "Found macOS SDK at '${OSX_SDK}', using..."
else
echo "macOS SDK does not exist at '${OSX_SDK}', please place the extracted, untarred SDK there to perform darwin builds, exiting..."
exit 1
fi
;;
esac
done
################
# VERSION_BASE should have enough space
################
avail_KiB="$(df -Pk "$VERSION_BASE" | sed 1d | tr -s ' ' | cut -d' ' -f4)"
total_required_KiB=0
for host in $HOSTS; do
case "$host" in
*darwin*) required_KiB=440000 ;;
*mingw*) required_KiB=7600000 ;;
*) required_KiB=6400000 ;;
esac
total_required_KiB=$((total_required_KiB+required_KiB))
done
if (( total_required_KiB > avail_KiB )); then
total_required_GiB=$((total_required_KiB / 1048576))
avail_GiB=$((avail_KiB / 1048576))
echo "Performing a Bitcoin Core Guix build for the selected HOSTS requires ${total_required_GiB} GiB, however, only ${avail_GiB} GiB is available. Please free up some disk space before performing the build."
exit 1
fi
################
# Check that we can connect to the guix-daemon
################
cat << EOF
Checking that we can connect to the guix-daemon...
Hint: If this hangs, you may want to try turning your guix-daemon off and on
again.
EOF
if ! guix gc --list-failures > /dev/null; then
cat << EOF
ERR: Failed to connect to the guix-daemon, please ensure that one is running and
reachable.
EOF
exit 1
fi
# Developer note: we could use `guix repl` for this check and run:
#
# (import (guix store)) (close-connection (open-connection))
#
# However, the internal API is likely to change more than the CLI invocation
################
# Services database must have basic entries
################
if ! getent services http https ftp > /dev/null 2>&1; then
cat << EOF
ERR: Your system's C library can not find service database entries for at least
one of the following services: http, https, ftp.
Hint: Most likely, /etc/services does not exist yet (common for docker images
and minimal distros), or you don't have permissions to access it.
If /etc/services does not exist yet, you may want to install the
appropriate package for your distro which provides it.
On Debian/Ubuntu: netbase
On Arch Linux: iana-etc
For more information, see: getent(1), services(5)
EOF
fi
#########
# SETUP #
#########
# Determine the maximum number of jobs to run simultaneously (overridable by
# environment)
JOBS="${JOBS:-$(nproc)}"
# Usage: host_to_commonname HOST
#
# HOST: The current platform triple we're building for
#
host_to_commonname() {
case "$1" in
*darwin*) echo osx ;;
*mingw*) echo win ;;
*linux*) echo linux ;;
*) exit 1 ;;
esac
}
# Determine the reference time used for determinism (overridable by environment)
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(git -c log.showSignature=false log --format=%at -1)}"
# Execute "$@" in a pinned, possibly older version of Guix, for reproducibility
# across time.
time-machine() {
# shellcheck disable=SC2086
guix time-machine --url=https://git.savannah.gnu.org/git/guix.git \
--commit=aa34d4d28dfe25ba47d5800d05000fb7221788c0 \
--cores="$JOBS" \
--keep-failed \
--fallback \
${SUBSTITUTE_URLS:+--substitute-urls="$SUBSTITUTE_URLS"} \
${ADDITIONAL_GUIX_COMMON_FLAGS} ${ADDITIONAL_GUIX_TIMEMACHINE_FLAGS} \
-- "$@"
}
# Precious directories are those which should not be cleaned between successive
# guix builds
depends_precious_dir_names='SOURCES_PATH BASE_CACHE SDK_PATH'
precious_dir_names="${depends_precious_dir_names} OUTDIR_BASE PROFILES_BASE"
# Usage: contains IFS-SEPARATED-LIST ITEM
contains() {
for i in ${1}; do
if [ "$i" = "${2}" ]; then
return 0 # Found!
fi
done
return 1
}
# If the user explicitly specified a precious directory, create it so we
# can map it into the container
for precious_dir_name in $precious_dir_names; do
precious_dir_path="${!precious_dir_name}"
if [ -n "$precious_dir_path" ]; then
if [ ! -e "$precious_dir_path" ]; then
mkdir -p "$precious_dir_path"
elif [ -L "$precious_dir_path" ]; then
echo "ERR: ${precious_dir_name} cannot be a symbolic link"
exit 1
elif [ ! -d "$precious_dir_path" ]; then
echo "ERR: ${precious_dir_name} must be a directory"
exit 1
fi
fi
done
mkdir -p "$VAR_BASE"
# Record the _effective_ values of precious directories such that guix-clean can
# avoid clobbering them if appropriate.
#
# shellcheck disable=SC2046,SC2086
{
# Get depends precious dir definitions from depends
make -C "${PWD}/depends" \
--no-print-directory \
-- $(printf "print-%s\n" $depends_precious_dir_names)
# Get remaining precious dir definitions from the environment
for precious_dir_name in $precious_dir_names; do
precious_dir_path="${!precious_dir_name}"
if ! contains "$depends_precious_dir_names" "$precious_dir_name"; then
echo "${precious_dir_name}=${precious_dir_path}"
fi
done
} > "${VAR_BASE}/precious_dirs"
# Make sure an output directory exists for our builds
OUTDIR_BASE="${OUTDIR_BASE:-${VERSION_BASE}/output}"
mkdir -p "$OUTDIR_BASE"
# Download the depends sources now as we won't have internet access in the build
# container
for host in $HOSTS; do
make -C "${PWD}/depends" -j"$JOBS" download-"$(host_to_commonname "$host")" ${V:+V=1} ${SOURCES_PATH:+SOURCES_PATH="$SOURCES_PATH"}
done
# Usage: outdir_for_host HOST SUFFIX
#
# HOST: The current platform triple we're building for
#
outdir_for_host() {
echo "${OUTDIR_BASE}/${1}${2:+-${2}}"
}
# Usage: profiledir_for_host HOST SUFFIX
#
# HOST: The current platform triple we're building for
#
profiledir_for_host() {
echo "${PROFILES_BASE}/${1}${2:+-${2}}"
}
#########
# BUILD #
#########
# Function to be called when building for host ${1} and the user interrupts the
# build
int_trap() {
cat << EOF
** INT received while building ${1}, you may want to clean up the relevant
work directories (e.g. distsrc-*) before rebuilding
Hint: To blow everything away, you may want to use:
$ ./contrib/guix/guix-clean
Specifically, this will remove all files without an entry in the index,
excluding the SDK directory, the depends download cache, the depends built
packages cache, the garbage collector roots for Guix environments, and the
output directory.
EOF
}
# Deterministically build Bitcoin Core
# shellcheck disable=SC2153
for host in $HOSTS; do
# Display proper warning when the user interrupts the build
trap 'int_trap ${host}' INT
(
# Required for 'contrib/guix/manifest.scm' to output the right manifest
# for the particular $HOST we're building for
export HOST="$host"
# shellcheck disable=SC2030
cat << EOF
INFO: Building ${VERSION:?not set} for platform triple ${HOST:?not set}:
...using reference timestamp: ${SOURCE_DATE_EPOCH:?not set}
...running at most ${JOBS:?not set} jobs
...from worktree directory: '${PWD}'
...bind-mounted in container to: '/bitcoin'
...in build directory: '$(distsrc_for_host "$HOST")'
...bind-mounted in container to: '$(DISTSRC_BASE=/distsrc-base && distsrc_for_host "$HOST")'
...outputting in: '$(outdir_for_host "$HOST")'
...bind-mounted in container to: '$(OUTDIR_BASE=/outdir-base && outdir_for_host "$HOST")'
EOF
# Run the build script 'contrib/guix/libexec/build.sh' in the build
# container specified by 'contrib/guix/manifest.scm'.
#
# Explanation of `guix environment` flags:
#
# --container run command within an isolated container
#
# Running in an isolated container minimizes build-time differences
# between machines and improves reproducibility
#
# --pure unset existing environment variables
#
# Same rationale as --container
#
# --no-cwd do not share current working directory with an
# isolated container
#
# When --container is specified, the default behavior is to share
# the current working directory with the isolated container at the
# same exact path (e.g. mapping '/home/satoshi/bitcoin/' to
# '/home/satoshi/bitcoin/'). This means that the $PWD inside the
# container becomes a source of irreproducibility. --no-cwd disables
# this behaviour.
#
# --share=SPEC for containers, share writable host file system
# according to SPEC
#
# --share="$PWD"=/bitcoin
#
# maps our current working directory to /bitcoin
# inside the isolated container, which we later cd
# into.
#
# While we don't want to map our current working directory to the
# same exact path (as this introduces irreproducibility), we do want
# it to be at a _fixed_ path _somewhere_ inside the isolated
# container so that we have something to build. '/bitcoin' was
# chosen arbitrarily.
#
# ${SOURCES_PATH:+--share="$SOURCES_PATH"}
#
# make the downloaded depends sources path available
# inside the isolated container
#
# The isolated container has no network access as it's in a
# different network namespace from the main machine, so we have to
# make the downloaded depends sources available to it. The sources
# should have been downloaded prior to this invocation.
#
# --keep-failed keep build tree of failed builds
#
# When builds of the Guix environment itself (not Bitcoin Core)
# fail, it is useful for the build tree to be kept for debugging
# purposes.
#
# ${SUBSTITUTE_URLS:+--substitute-urls="$SUBSTITUTE_URLS"}
#
# fetch substitute from SUBSTITUTE_URLS if they are
# authorized
#
# Depending on the user's security model, it may be desirable to use
# substitutes (pre-built packages) from servers that the user trusts.
# Please read the README.md in the same directory as this file for
# more information.
#
# shellcheck disable=SC2086,SC2031
time-machine environment --manifest="${PWD}/contrib/guix/manifest.scm" \
--container \
--pure \
--no-cwd \
--share="$PWD"=/bitcoin \
--share="$DISTSRC_BASE"=/distsrc-base \
--share="$OUTDIR_BASE"=/outdir-base \
--expose="$(git rev-parse --git-common-dir)" \
${SOURCES_PATH:+--share="$SOURCES_PATH"} \
${BASE_CACHE:+--share="$BASE_CACHE"} \
${SDK_PATH:+--share="$SDK_PATH"} \
--cores="$JOBS" \
--keep-failed \
--fallback \
--link-profile \
--root="$(profiledir_for_host "${HOST}")" \
${SUBSTITUTE_URLS:+--substitute-urls="$SUBSTITUTE_URLS"} \
${ADDITIONAL_GUIX_COMMON_FLAGS} ${ADDITIONAL_GUIX_ENVIRONMENT_FLAGS} \
-- env HOST="$host" \
DISTNAME="$DISTNAME" \
JOBS="$JOBS" \
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:?unable to determine value}" \
${V:+V=1} \
${SOURCES_PATH:+SOURCES_PATH="$SOURCES_PATH"} \
${BASE_CACHE:+BASE_CACHE="$BASE_CACHE"} \
${SDK_PATH:+SDK_PATH="$SDK_PATH"} \
DISTSRC="$(DISTSRC_BASE=/distsrc-base && distsrc_for_host "$HOST")" \
OUTDIR="$(OUTDIR_BASE=/outdir-base && outdir_for_host "$HOST")" \
DIST_ARCHIVE_BASE=/outdir-base/dist-archive \
bash -c "cd /bitcoin && bash contrib/guix/libexec/build.sh"
)
done

119
contrib/guix/guix-build.sh Executable file
View File

@@ -0,0 +1,119 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
# Determine the maximum number of jobs to run simultaneously (overridable by
# environment)
MAX_JOBS="${MAX_JOBS:-$(nproc)}"
# Download the depends sources now as we won't have internet access in the build
# container
make -C "${PWD}/depends" -j"$MAX_JOBS" download ${V:+V=1} ${SOURCES_PATH:+SOURCES_PATH="$SOURCES_PATH"}
# Determine the reference time used for determinism (overridable by environment)
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(git log --format=%at -1)}"
# Execute "$@" in a pinned, possibly older version of Guix, for reproducibility
# across time.
time-machine() {
guix time-machine --url=https://github.com/dongcarl/guix.git \
--commit=b066c25026f21fb57677aa34692a5034338e7ee3 \
-- "$@"
}
# Function to be called when building for host ${1} and the user interrupts the
# build
int_trap() {
cat << EOF
** INT received while building ${1}, you may want to clean up the relevant
output, deploy, and distsrc-* directories before rebuilding
Hint: To blow everything away, you may want to use:
$ git clean -xdff --exclude='/depends/SDKs/*'
Specifically, this will remove all files without an entry in the index,
excluding the SDK directory. Practically speaking, this means that all ignored
and untracked files and directories will be wiped, allowing you to start anew.
EOF
}
# Deterministically build Bitcoin Core for HOSTs (overridable by environment)
# shellcheck disable=SC2153
for host in ${HOSTS=x86_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu riscv64-linux-gnu x86_64-w64-mingw32}; do
# Display proper warning when the user interrupts the build
trap 'int_trap ${host}' INT
(
# Required for 'contrib/guix/manifest.scm' to output the right manifest
# for the particular $HOST we're building for
export HOST="$host"
# Run the build script 'contrib/guix/libexec/build.sh' in the build
# container specified by 'contrib/guix/manifest.scm'.
#
# Explanation of `guix environment` flags:
#
# --container run command within an isolated container
#
# Running in an isolated container minimizes build-time differences
# between machines and improves reproducibility
#
# --pure unset existing environment variables
#
# Same rationale as --container
#
# --no-cwd do not share current working directory with an
# isolated container
#
# When --container is specified, the default behavior is to share
# the current working directory with the isolated container at the
# same exact path (e.g. mapping '/home/satoshi/bitcoin/' to
# '/home/satoshi/bitcoin/'). This means that the $PWD inside the
# container becomes a source of irreproducibility. --no-cwd disables
# this behaviour.
#
# --share=SPEC for containers, share writable host file system
# according to SPEC
#
# --share="$PWD"=/bitcoin
#
# maps our current working directory to /bitcoin
# inside the isolated container, which we later cd
# into.
#
# While we don't want to map our current working directory to the
# same exact path (as this introduces irreproducibility), we do want
# it to be at a _fixed_ path _somewhere_ inside the isolated
# container so that we have something to build. '/bitcoin' was
# chosen arbitrarily.
#
# ${SOURCES_PATH:+--share="$SOURCES_PATH"}
#
# make the downloaded depends sources path available
# inside the isolated container
#
# The isolated container has no network access as it's in a
# different network namespace from the main machine, so we have to
# make the downloaded depends sources available to it. The sources
# should have been downloaded prior to this invocation.
#
# shellcheck disable=SC2086
time-machine environment --manifest="${PWD}/contrib/guix/manifest.scm" \
--container \
--pure \
--no-cwd \
--share="$PWD"=/bitcoin \
--expose="$(git rev-parse --git-common-dir)" \
${SOURCES_PATH:+--share="$SOURCES_PATH"} \
${ADDITIONAL_GUIX_ENVIRONMENT_FLAGS} \
-- env HOST="$host" \
MAX_JOBS="$MAX_JOBS" \
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:?unable to determine value}" \
${V:+V=1} \
${SOURCES_PATH:+SOURCES_PATH="$SOURCES_PATH"} \
bash -c "cd /bitcoin && bash contrib/guix/libexec/build.sh"
)
done

View File

@@ -1,83 +0,0 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
# Source the common prelude, which:
# 1. Checks if we're at the top directory of the Bitcoin Core repository
# 2. Defines a few common functions and variables
#
# shellcheck source=libexec/prelude.bash
source "$(dirname "${BASH_SOURCE[0]}")/libexec/prelude.bash"
###################
## Sanity Checks ##
###################
################
# Required non-builtin commands should be invokable
################
check_tools cat mkdir make git guix
#############
## Clean ##
#############
# Usage: under_dir MAYBE_PARENT MAYBE_CHILD
#
# If MAYBE_CHILD is a subdirectory of MAYBE_PARENT, print the relative path
# from MAYBE_PARENT to MAYBE_CHILD. Otherwise, return 1 as the error code.
#
# NOTE: This does not perform any symlink-resolving or path canonicalization.
#
under_dir() {
local path_residue
path_residue="${2##${1}}"
if [ -z "$path_residue" ] || [ "$path_residue" = "$2" ]; then
return 1
else
echo "$path_residue"
fi
}
# Usage: dir_under_git_root MAYBE_CHILD
#
# If MAYBE_CHILD is under the current git repository and exists, print the
# relative path from the git repository's top-level directory to MAYBE_CHILD,
# otherwise, exit with an error code.
#
dir_under_git_root() {
local rv
rv="$(under_dir "$(git_root)" "$1")"
[ -n "$rv" ] && echo "$rv"
}
shopt -s nullglob
found_precious_dirs_files=( "${version_base_prefix}"*/"${var_base_basename}/precious_dirs" ) # This expands to an array of directories...
shopt -u nullglob
exclude_flags=()
for precious_dirs_file in "${found_precious_dirs_files[@]}"; do
# Make sure the precious directories (e.g. SOURCES_PATH, BASE_CACHE, SDK_PATH)
# are excluded from git-clean
echo "Found precious_dirs file: '${precious_dirs_file}'"
# Exclude the precious_dirs file itself
if dirs_file_exclude_fragment=$(dir_under_git_root "$(dirname "$precious_dirs_file")"); then
exclude_flags+=( --exclude="${dirs_file_exclude_fragment}/precious_dirs" )
fi
# Read each 'name=dir' pair from the precious_dirs file
while IFS='=' read -r name dir; do
# Add an exclusion flag if the precious directory is under the git root.
if under=$(dir_under_git_root "$dir"); then
echo "Avoiding ${name}: ${under}"
exclude_flags+=( --exclude="$under" )
fi
done < "$precious_dirs_file"
done
git clean -xdff "${exclude_flags[@]}"

View File

@@ -1,392 +0,0 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
# Source the common prelude, which:
# 1. Checks if we're at the top directory of the Bitcoin Core repository
# 2. Defines a few common functions and variables
#
# shellcheck source=libexec/prelude.bash
source "$(dirname "${BASH_SOURCE[0]}")/libexec/prelude.bash"
###################
## SANITY CHECKS ##
###################
################
# Required non-builtin commands should be invocable
################
check_tools cat mkdir git guix
################
# Required env vars should be non-empty
################
cmd_usage() {
cat <<EOF
Synopsis:
env DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> \\
./contrib/guix/guix-codesign
EOF
}
if [ -z "$DETACHED_SIGS_REPO" ]; then
cmd_usage
exit 1
fi
################
# GUIX_BUILD_OPTIONS should be empty
################
#
# GUIX_BUILD_OPTIONS is an environment variable recognized by guix commands that
# can perform builds. This seems like what we want instead of
# ADDITIONAL_GUIX_COMMON_FLAGS, but the value of GUIX_BUILD_OPTIONS is actually
# _appended_ to normal command-line options. Meaning that they will take
# precedence over the command-specific ADDITIONAL_GUIX_<CMD>_FLAGS.
#
# This seems like a poor user experience. Thus we check for GUIX_BUILD_OPTIONS's
# existence here and direct users of this script to use our (more flexible)
# custom environment variables.
if [ -n "$GUIX_BUILD_OPTIONS" ]; then
cat << EOF
Error: Environment variable GUIX_BUILD_OPTIONS is not empty:
'$GUIX_BUILD_OPTIONS'
Unfortunately this script is incompatible with GUIX_BUILD_OPTIONS, please unset
GUIX_BUILD_OPTIONS and use ADDITIONAL_GUIX_COMMON_FLAGS to set build options
across guix commands or ADDITIONAL_GUIX_<CMD>_FLAGS to set build options for a
specific guix command.
See contrib/guix/README.md for more details.
EOF
exit 1
fi
################
# The codesignature git worktree should not be dirty
################
if ! git -C "$DETACHED_SIGS_REPO" diff-index --quiet HEAD -- && [ -z "$FORCE_DIRTY_WORKTREE" ]; then
cat << EOF
ERR: The DETACHED CODESIGNATURE git worktree is dirty, which may lead to broken builds.
Aborting...
Hint: To make your git worktree clean, You may want to:
1. Commit your changes,
2. Stash your changes, or
3. Set the 'FORCE_DIRTY_WORKTREE' environment variable if you insist on
using a dirty worktree
EOF
exit 1
fi
################
# Build directories should not exist
################
# Default to building for all supported HOSTs (overridable by environment)
export HOSTS="${HOSTS:-x86_64-w64-mingw32 x86_64-apple-darwin18}"
# Usage: distsrc_for_host HOST
#
# HOST: The current platform triple we're building for
#
distsrc_for_host() {
echo "${DISTSRC_BASE}/distsrc-${VERSION}-${1}-codesigned"
}
# Accumulate a list of build directories that already exist...
hosts_distsrc_exists=""
for host in $HOSTS; do
if [ -e "$(distsrc_for_host "$host")" ]; then
hosts_distsrc_exists+=" ${host}"
fi
done
if [ -n "$hosts_distsrc_exists" ]; then
# ...so that we can print them out nicely in an error message
cat << EOF
ERR: Build directories for this commit already exist for the following platform
triples you're attempting to build, probably because of previous builds.
Please remove, or otherwise deal with them prior to starting another build.
Aborting...
Hint: To blow everything away, you may want to use:
$ ./contrib/guix/guix-clean
Specifically, this will remove all files without an entry in the index,
excluding the SDK directory, the depends download cache, the depends built
packages cache, the garbage collector roots for Guix environments, and the
output directory.
EOF
for host in $hosts_distsrc_exists; do
echo " ${host} '$(distsrc_for_host "$host")'"
done
exit 1
else
mkdir -p "$DISTSRC_BASE"
fi
################
# Unsigned tarballs SHOULD exist
################
# Usage: outdir_for_host HOST SUFFIX
#
# HOST: The current platform triple we're building for
#
outdir_for_host() {
echo "${OUTDIR_BASE}/${1}${2:+-${2}}"
}
unsigned_tarball_for_host() {
case "$1" in
*mingw*)
echo "$(outdir_for_host "$1")/${DISTNAME}-win-unsigned.tar.gz"
;;
*darwin*)
echo "$(outdir_for_host "$1")/${DISTNAME}-osx-unsigned.tar.gz"
;;
*)
exit 1
;;
esac
}
# Accumulate a list of build directories that already exist...
hosts_unsigned_tarball_missing=""
for host in $HOSTS; do
if [ ! -e "$(unsigned_tarball_for_host "$host")" ]; then
hosts_unsigned_tarball_missing+=" ${host}"
fi
done
if [ -n "$hosts_unsigned_tarball_missing" ]; then
# ...so that we can print them out nicely in an error message
cat << EOF
ERR: Unsigned tarballs do not exist
...
EOF
for host in $hosts_unsigned_tarball_missing; do
echo " ${host} '$(unsigned_tarball_for_host "$host")'"
done
exit 1
fi
################
# Check that we can connect to the guix-daemon
################
cat << EOF
Checking that we can connect to the guix-daemon...
Hint: If this hangs, you may want to try turning your guix-daemon off and on
again.
EOF
if ! guix gc --list-failures > /dev/null; then
cat << EOF
ERR: Failed to connect to the guix-daemon, please ensure that one is running and
reachable.
EOF
exit 1
fi
# Developer note: we could use `guix repl` for this check and run:
#
# (import (guix store)) (close-connection (open-connection))
#
# However, the internal API is likely to change more than the CLI invocation
#########
# SETUP #
#########
# Determine the maximum number of jobs to run simultaneously (overridable by
# environment)
JOBS="${JOBS:-$(nproc)}"
# Determine the reference time used for determinism (overridable by environment)
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(git -c log.showSignature=false log --format=%at -1)}"
# Execute "$@" in a pinned, possibly older version of Guix, for reproducibility
# across time.
time-machine() {
# shellcheck disable=SC2086
guix time-machine --url=https://git.savannah.gnu.org/git/guix.git \
--commit=aa34d4d28dfe25ba47d5800d05000fb7221788c0 \
--cores="$JOBS" \
--keep-failed \
--fallback \
${SUBSTITUTE_URLS:+--substitute-urls="$SUBSTITUTE_URLS"} \
${ADDITIONAL_GUIX_COMMON_FLAGS} ${ADDITIONAL_GUIX_TIMEMACHINE_FLAGS} \
-- "$@"
}
# Make sure an output directory exists for our builds
OUTDIR_BASE="${OUTDIR_BASE:-${VERSION_BASE}/output}"
mkdir -p "$OUTDIR_BASE"
# Usage: profiledir_for_host HOST SUFFIX
#
# HOST: The current platform triple we're building for
#
profiledir_for_host() {
echo "${PROFILES_BASE}/${1}${2:+-${2}}"
}
#########
# BUILD #
#########
# Function to be called when codesigning for host ${1} and the user interrupts
# the codesign
int_trap() {
cat << EOF
** INT received while codesigning ${1}, you may want to clean up the relevant
work directories (e.g. distsrc-*) before recodesigning
Hint: To blow everything away, you may want to use:
$ ./contrib/guix/guix-clean
Specifically, this will remove all files without an entry in the index,
excluding the SDK directory, the depends download cache, the depends built
packages cache, the garbage collector roots for Guix environments, and the
output directory.
EOF
}
# Deterministically build Bitcoin Core
# shellcheck disable=SC2153
for host in $HOSTS; do
# Display proper warning when the user interrupts the build
trap 'int_trap ${host}' INT
(
# Required for 'contrib/guix/manifest.scm' to output the right manifest
# for the particular $HOST we're building for
export HOST="$host"
# shellcheck disable=SC2030
cat << EOF
INFO: Codesigning ${VERSION:?not set} for platform triple ${HOST:?not set}:
...using reference timestamp: ${SOURCE_DATE_EPOCH:?not set}
...from worktree directory: '${PWD}'
...bind-mounted in container to: '/bitcoin'
...in build directory: '$(distsrc_for_host "$HOST")'
...bind-mounted in container to: '$(DISTSRC_BASE=/distsrc-base && distsrc_for_host "$HOST")'
...outputting in: '$(outdir_for_host "$HOST" codesigned)'
...bind-mounted in container to: '$(OUTDIR_BASE=/outdir-base && outdir_for_host "$HOST" codesigned)'
...using detached signatures in: '${DETACHED_SIGS_REPO:?not set}'
...bind-mounted in container to: '/detached-sigs'
EOF
# Run the build script 'contrib/guix/libexec/build.sh' in the build
# container specified by 'contrib/guix/manifest.scm'.
#
# Explanation of `guix environment` flags:
#
# --container run command within an isolated container
#
# Running in an isolated container minimizes build-time differences
# between machines and improves reproducibility
#
# --pure unset existing environment variables
#
# Same rationale as --container
#
# --no-cwd do not share current working directory with an
# isolated container
#
# When --container is specified, the default behavior is to share
# the current working directory with the isolated container at the
# same exact path (e.g. mapping '/home/satoshi/bitcoin/' to
# '/home/satoshi/bitcoin/'). This means that the $PWD inside the
# container becomes a source of irreproducibility. --no-cwd disables
# this behaviour.
#
# --share=SPEC for containers, share writable host file system
# according to SPEC
#
# --share="$PWD"=/bitcoin
#
# maps our current working directory to /bitcoin
# inside the isolated container, which we later cd
# into.
#
# While we don't want to map our current working directory to the
# same exact path (as this introduces irreproducibility), we do want
# it to be at a _fixed_ path _somewhere_ inside the isolated
# container so that we have something to build. '/bitcoin' was
# chosen arbitrarily.
#
# ${SOURCES_PATH:+--share="$SOURCES_PATH"}
#
# make the downloaded depends sources path available
# inside the isolated container
#
# The isolated container has no network access as it's in a
# different network namespace from the main machine, so we have to
# make the downloaded depends sources available to it. The sources
# should have been downloaded prior to this invocation.
#
# ${SUBSTITUTE_URLS:+--substitute-urls="$SUBSTITUTE_URLS"}
#
# fetch substitute from SUBSTITUTE_URLS if they are
# authorized
#
# Depending on the user's security model, it may be desirable to use
# substitutes (pre-built packages) from servers that the user trusts.
# Please read the README.md in the same directory as this file for
# more information.
#
# shellcheck disable=SC2086,SC2031
time-machine environment --manifest="${PWD}/contrib/guix/manifest.scm" \
--container \
--pure \
--no-cwd \
--share="$PWD"=/bitcoin \
--share="$DISTSRC_BASE"=/distsrc-base \
--share="$OUTDIR_BASE"=/outdir-base \
--share="$DETACHED_SIGS_REPO"=/detached-sigs \
--expose="$(git rev-parse --git-common-dir)" \
--expose="$(git -C "$DETACHED_SIGS_REPO" rev-parse --git-common-dir)" \
${SOURCES_PATH:+--share="$SOURCES_PATH"} \
--cores="$JOBS" \
--keep-failed \
--fallback \
--link-profile \
--root="$(profiledir_for_host "${HOST}" codesigned)" \
${SUBSTITUTE_URLS:+--substitute-urls="$SUBSTITUTE_URLS"} \
${ADDITIONAL_GUIX_COMMON_FLAGS} ${ADDITIONAL_GUIX_ENVIRONMENT_FLAGS} \
-- env HOST="$host" \
DISTNAME="$DISTNAME" \
JOBS="$JOBS" \
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:?unable to determine value}" \
${V:+V=1} \
${SOURCES_PATH:+SOURCES_PATH="$SOURCES_PATH"} \
DISTSRC="$(DISTSRC_BASE=/distsrc-base && distsrc_for_host "$HOST")" \
OUTDIR="$(OUTDIR_BASE=/outdir-base && outdir_for_host "$HOST" codesigned)" \
DIST_ARCHIVE_BASE=/outdir-base/dist-archive \
DETACHED_SIGS_REPO=/detached-sigs \
UNSIGNED_TARBALL="$(OUTDIR_BASE=/outdir-base && unsigned_tarball_for_host "$HOST")" \
bash -c "cd /bitcoin && bash contrib/guix/libexec/codesign.sh"
)
done

View File

@@ -1,174 +0,0 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
# Source the common prelude, which:
# 1. Checks if we're at the top directory of the Bitcoin Core repository
# 2. Defines a few common functions and variables
#
# shellcheck source=libexec/prelude.bash
source "$(dirname "${BASH_SOURCE[0]}")/libexec/prelude.bash"
###################
## Sanity Checks ##
###################
################
# Required non-builtin commands should be invokable
################
check_tools cat diff gpg
################
# Required env vars should be non-empty
################
cmd_usage() {
cat <<EOF
Synopsis:
env GUIX_SIGS_REPO=<path/to/guix.sigs> [ SIGNER=<signer> ] ./contrib/guix/guix-verify
Example overriding signer's manifest to use as base
env GUIX_SIGS_REPO=/home/dongcarl/guix.sigs SIGNER=achow101 ./contrib/guix/guix-verify
EOF
}
if [ -z "$GUIX_SIGS_REPO" ]; then
cmd_usage
exit 1
fi
################
# GUIX_SIGS_REPO should exist as a directory
################
if [ ! -d "$GUIX_SIGS_REPO" ]; then
cat << EOF
ERR: The specified GUIX_SIGS_REPO is not an existent directory:
'$GUIX_SIGS_REPO'
Hint: Please clone the guix.sigs repository and point to it with the
GUIX_SIGS_REPO environment variable.
EOF
cmd_usage
exit 1
fi
##############
## Verify ##
##############
OUTSIGDIR_BASE="${GUIX_SIGS_REPO}/${VERSION}"
echo "Looking for signature directories in '${OUTSIGDIR_BASE}'"
echo ""
# Usage: verify compare_manifest current_manifest
verify() {
local compare_manifest="$1"
local current_manifest="$2"
if ! gpg --quiet --batch --verify "$current_manifest".asc "$current_manifest" 1>&2; then
echo "ERR: Failed to verify GPG signature in '${current_manifest}'"
echo ""
echo "Hint: Either the signature is invalid or the public key is missing"
echo ""
failure=1
elif ! diff --report-identical "$compare_manifest" "$current_manifest" 1>&2; then
echo "ERR: The SHA256SUMS attestation in these two directories differ:"
echo " '${compare_manifest}'"
echo " '${current_manifest}'"
echo ""
failure=1
else
echo "Verified: '${current_manifest}'"
echo ""
fi
}
shopt -s nullglob
all_noncodesigned=( "$OUTSIGDIR_BASE"/*/noncodesigned.SHA256SUMS )
shopt -u nullglob
echo "--------------------"
echo ""
if (( ${#all_noncodesigned[@]} )); then
compare_noncodesigned="${all_noncodesigned[0]}"
if [[ -n "$SIGNER" ]]; then
signer_noncodesigned="$OUTSIGDIR_BASE/$SIGNER/noncodesigned.SHA256SUMS"
if [[ -f "$signer_noncodesigned" ]]; then
echo "Using $SIGNER's manifest as the base to compare against"
compare_noncodesigned="$signer_noncodesigned"
else
echo "Unable to find $SIGNER's manifest, using the first one found"
fi
else
echo "No SIGNER provided, using the first manifest found"
fi
for current_manifest in "${all_noncodesigned[@]}"; do
verify "$compare_noncodesigned" "$current_manifest"
done
echo "DONE: Checking output signatures for noncodesigned.SHA256SUMS"
echo ""
else
echo "WARN: No signature directories with noncodesigned.SHA256SUMS found"
echo ""
fi
shopt -s nullglob
all_all=( "$OUTSIGDIR_BASE"/*/all.SHA256SUMS )
shopt -u nullglob
echo "--------------------"
echo ""
if (( ${#all_all[@]} )); then
compare_all="${all_all[0]}"
if [[ -n "$SIGNER" ]]; then
signer_all="$OUTSIGDIR_BASE/$SIGNER/all.SHA256SUMS"
if [[ -f "$signer_all" ]]; then
echo "Using $SIGNER's manifest as the base to compare against"
compare_all="$signer_all"
else
echo "Unable to find $SIGNER's manifest, using the first one found"
fi
else
echo "No SIGNER provided, using the first manifest found"
fi
for current_manifest in "${all_all[@]}"; do
verify "$compare_all" "$current_manifest"
done
# Sanity check: there should be no entries that exist in
# noncodesigned.SHA256SUMS that doesn't exist in all.SHA256SUMS
if [[ "$(comm -23 <(sort "$compare_noncodesigned") <(sort "$compare_all") | wc -c)" -ne 0 ]]; then
echo "ERR: There are unique lines in noncodesigned.SHA256SUMS which"
echo " do not exist in all.SHA256SUMS, something went very wrong."
exit 1
fi
echo "DONE: Checking output signatures for all.SHA256SUMS"
echo ""
else
echo "WARN: No signature directories with all.SHA256SUMS found"
echo ""
fi
echo "===================="
echo ""
if (( ${#all_noncodesigned[@]} + ${#all_all[@]} == 0 )); then
echo "ERR: Unable to perform any verifications as no signature directories"
echo " were found"
echo ""
exit 1
fi
if [ -n "$failure" ]; then
exit 1
fi

305
contrib/guix/libexec/build.sh Executable file → Normal file
View File

@@ -3,16 +3,6 @@ export LC_ALL=C
set -e -o pipefail
export TZ=UTC
# Although Guix _does_ set umask when building its own packages (in our case,
# this is all packages in manifest.scm), it does not set it for `guix
# environment`. It does make sense for at least `guix environment --container`
# to set umask, so if that change gets merged upstream and we bump the
# time-machine to a commit which includes the aforementioned change, we can
# remove this line.
#
# This line should be placed before any commands which creates files.
umask 0022
if [ -n "$V" ]; then
# Print both unexpanded (-v) and expanded (-x) forms of commands as they are
# read from this file.
@@ -21,20 +11,9 @@ if [ -n "$V" ]; then
export VERBOSE="$V"
fi
# Check that required environment variables are set
cat << EOF
Required environment variables as seen inside the container:
DIST_ARCHIVE_BASE: ${DIST_ARCHIVE_BASE:?not set}
DISTNAME: ${DISTNAME:?not set}
HOST: ${HOST:?not set}
SOURCE_DATE_EPOCH: ${SOURCE_DATE_EPOCH:?not set}
JOBS: ${JOBS:?not set}
DISTSRC: ${DISTSRC:?not set}
OUTDIR: ${OUTDIR:?not set}
EOF
ACTUAL_OUTDIR="${OUTDIR}"
OUTDIR="${DISTSRC}/output"
# Check that environment variables assumed to be set by the environment are set
echo "Building for platform triple ${HOST:?not set} with reference timestamp ${SOURCE_DATE_EPOCH:?not set}..."
echo "At most ${MAX_JOBS:?not set} jobs will run at once..."
#####################
# Environment Setup #
@@ -44,6 +23,19 @@ OUTDIR="${DISTSRC}/output"
# $HOSTs after successfully building.
BASEPREFIX="${PWD}/depends"
# Setup an output directory for our build
OUTDIR="${OUTDIR:-${PWD}/output}"
[ -e "$OUTDIR" ] || mkdir -p "$OUTDIR"
# Setup the directory where our Bitcoin Core build for HOST will occur
DISTSRC="${DISTSRC:-${PWD}/distsrc-${HOST}}"
if [ -e "$DISTSRC" ]; then
echo "DISTSRC directory '${DISTSRC}' exists, probably because of previous builds... Aborting..."
exit 1
else
mkdir -p "$DISTSRC"
fi
# Given a package name and an output name, return the path of that output in our
# current guix environment
store_path() {
@@ -53,77 +45,37 @@ store_path() {
--expression='s|"[[:space:]]*$||'
}
# Set environment variables to point the NATIVE toolchain to the right
# includes/libs
NATIVE_GCC="$(store_path gcc-toolchain)"
NATIVE_GCC_STATIC="$(store_path gcc-toolchain static)"
unset LIBRARY_PATH
unset CPATH
unset C_INCLUDE_PATH
unset CPLUS_INCLUDE_PATH
unset OBJC_INCLUDE_PATH
unset OBJCPLUS_INCLUDE_PATH
export LIBRARY_PATH="${NATIVE_GCC}/lib:${NATIVE_GCC}/lib64:${NATIVE_GCC_STATIC}/lib:${NATIVE_GCC_STATIC}/lib64"
export C_INCLUDE_PATH="${NATIVE_GCC}/include"
export CPLUS_INCLUDE_PATH="${NATIVE_GCC}/include/c++:${NATIVE_GCC}/include"
export OBJC_INCLUDE_PATH="${NATIVE_GCC}/include"
export OBJCPLUS_INCLUDE_PATH="${NATIVE_GCC}/include/c++:${NATIVE_GCC}/include"
prepend_to_search_env_var() {
export "${1}=${2}${!1:+:}${!1}"
}
case "$HOST" in
*darwin*)
# When targeting darwin, zlib is required by native_libdmg-hfsplus.
zlib_store_path=$(store_path "zlib")
zlib_static_store_path=$(store_path "zlib" static)
prepend_to_search_env_var LIBRARY_PATH "${zlib_static_store_path}/lib:${zlib_store_path}/lib"
prepend_to_search_env_var C_INCLUDE_PATH "${zlib_store_path}/include"
prepend_to_search_env_var CPLUS_INCLUDE_PATH "${zlib_store_path}/include"
prepend_to_search_env_var OBJC_INCLUDE_PATH "${zlib_store_path}/include"
prepend_to_search_env_var OBJCPLUS_INCLUDE_PATH "${zlib_store_path}/include"
esac
# Set environment variables to point the CROSS toolchain to the right
# Set environment variables to point Guix's cross-toolchain to the right
# includes/libs for $HOST
case "$HOST" in
*mingw*)
# Determine output paths to use in CROSS_* environment variables
CROSS_GLIBC="$(store_path "mingw-w64-x86_64-winpthreads")"
CROSS_GCC="$(store_path "gcc-cross-${HOST}")"
CROSS_GCC_LIB_STORE="$(store_path "gcc-cross-${HOST}" lib)"
CROSS_GCC_LIBS=( "${CROSS_GCC_LIB_STORE}/lib/gcc/${HOST}"/* ) # This expands to an array of directories...
CROSS_GCC_LIBS=( "${CROSS_GCC}/lib/gcc/${HOST}"/* ) # This expands to an array of directories...
CROSS_GCC_LIB="${CROSS_GCC_LIBS[0]}" # ...we just want the first one (there should only be one)
# The search path ordering is generally:
# 1. gcc-related search paths
# 2. libc-related search paths
# 2. kernel-header-related search paths (not applicable to mingw-w64 hosts)
NATIVE_GCC="$(store_path gcc-glibc-2.27-toolchain)"
export LIBRARY_PATH="${NATIVE_GCC}/lib:${NATIVE_GCC}/lib64"
export CPATH="${NATIVE_GCC}/include"
export CROSS_C_INCLUDE_PATH="${CROSS_GCC_LIB}/include:${CROSS_GCC_LIB}/include-fixed:${CROSS_GLIBC}/include"
export CROSS_CPLUS_INCLUDE_PATH="${CROSS_GCC}/include/c++:${CROSS_GCC}/include/c++/${HOST}:${CROSS_GCC}/include/c++/backward:${CROSS_C_INCLUDE_PATH}"
export CROSS_LIBRARY_PATH="${CROSS_GCC_LIB_STORE}/lib:${CROSS_GCC}/${HOST}/lib:${CROSS_GCC_LIB}:${CROSS_GLIBC}/lib"
;;
*darwin*)
# The CROSS toolchain for darwin uses the SDK and ignores environment variables.
# See depends/hosts/darwin.mk for more details.
export CROSS_LIBRARY_PATH="${CROSS_GCC}/lib:${CROSS_GCC}/${HOST}/lib:${CROSS_GCC_LIB}:${CROSS_GLIBC}/lib"
;;
*linux*)
CROSS_GLIBC="$(store_path "glibc-cross-${HOST}")"
CROSS_GLIBC_STATIC="$(store_path "glibc-cross-${HOST}" static)"
CROSS_KERNEL="$(store_path "linux-libre-headers-cross-${HOST}")"
CROSS_GCC="$(store_path "gcc-cross-${HOST}")"
CROSS_GCC_LIB_STORE="$(store_path "gcc-cross-${HOST}" lib)"
CROSS_GCC_LIBS=( "${CROSS_GCC_LIB_STORE}/lib/gcc/${HOST}"/* ) # This expands to an array of directories...
CROSS_GCC_LIBS=( "${CROSS_GCC}/lib/gcc/${HOST}"/* ) # This expands to an array of directories...
CROSS_GCC_LIB="${CROSS_GCC_LIBS[0]}" # ...we just want the first one (there should only be one)
export CROSS_C_INCLUDE_PATH="${CROSS_GCC_LIB}/include:${CROSS_GCC_LIB}/include-fixed:${CROSS_GLIBC}/include:${CROSS_KERNEL}/include"
# NOTE: CROSS_C_INCLUDE_PATH is missing ${CROSS_GCC_LIB}/include-fixed, because
# the limits.h in it is missing a '#include_next <limits.h>'
export CROSS_C_INCLUDE_PATH="${CROSS_GCC_LIB}/include:${CROSS_GLIBC}/include:${CROSS_KERNEL}/include"
export CROSS_CPLUS_INCLUDE_PATH="${CROSS_GCC}/include/c++:${CROSS_GCC}/include/c++/${HOST}:${CROSS_GCC}/include/c++/backward:${CROSS_C_INCLUDE_PATH}"
export CROSS_LIBRARY_PATH="${CROSS_GCC_LIB_STORE}/lib:${CROSS_GCC}/${HOST}/lib:${CROSS_GCC_LIB}:${CROSS_GLIBC}/lib:${CROSS_GLIBC_STATIC}/lib"
export CROSS_LIBRARY_PATH="${CROSS_GCC}/lib:${CROSS_GCC}/${HOST}/lib:${CROSS_GCC_LIB}:${CROSS_GLIBC}/lib:${CROSS_GLIBC_STATIC}/lib"
;;
*)
exit 1 ;;
@@ -132,25 +84,14 @@ esac
# Sanity check CROSS_*_PATH directories
IFS=':' read -ra PATHS <<< "${CROSS_C_INCLUDE_PATH}:${CROSS_CPLUS_INCLUDE_PATH}:${CROSS_LIBRARY_PATH}"
for p in "${PATHS[@]}"; do
if [ -n "$p" ] && [ ! -d "$p" ]; then
if [ ! -d "$p" ]; then
echo "'$p' doesn't exist or isn't a directory... Aborting..."
exit 1
fi
done
# Disable Guix ld auto-rpath behavior
case "$HOST" in
*darwin*)
# The auto-rpath behavior is necessary for darwin builds as some native
# tools built by depends refer to and depend on Guix-built native
# libraries
#
# After the native packages in depends are built, the ld wrapper should
# no longer affect our build, as clang would instead reach for
# x86_64-apple-darwin18-ld from cctools
;;
*) export GUIX_LD_WRAPPER_DISABLE_RPATH=yes ;;
esac
export GUIX_LD_WRAPPER_DISABLE_RPATH=yes
# Make /usr/bin if it doesn't exist
[ -e /usr/bin ] || mkdir -p /usr/bin
@@ -164,43 +105,31 @@ case "$HOST" in
*linux*)
glibc_dynamic_linker=$(
case "$HOST" in
i686-linux-gnu) echo /lib/ld-linux.so.2 ;;
x86_64-linux-gnu) echo /lib64/ld-linux-x86-64.so.2 ;;
arm-linux-gnueabihf) echo /lib/ld-linux-armhf.so.3 ;;
aarch64-linux-gnu) echo /lib/ld-linux-aarch64.so.1 ;;
riscv64-linux-gnu) echo /lib/ld-linux-riscv64-lp64d.so.1 ;;
powerpc64-linux-gnu) echo /lib64/ld64.so.1;;
powerpc64le-linux-gnu) echo /lib64/ld64.so.2;;
*) exit 1 ;;
i686-linux-gnu) echo /lib/ld-linux.so.2 ;;
x86_64-linux-gnu) echo /lib64/ld-linux-x86-64.so.2 ;;
arm-linux-gnueabihf) echo /lib/ld-linux-armhf.so.3 ;;
aarch64-linux-gnu) echo /lib/ld-linux-aarch64.so.1 ;;
riscv64-linux-gnu) echo /lib/ld-linux-riscv64-lp64d.so.1 ;;
*) exit 1 ;;
esac
)
;;
esac
# Environment variables for determinism
export QT_RCC_TEST=1
export QT_RCC_SOURCE_DATE_OVERRIDE=1
export TAR_OPTIONS="--owner=0 --group=0 --numeric-owner --mtime='@${SOURCE_DATE_EPOCH}' --sort=name"
export TZ="UTC"
case "$HOST" in
*darwin*)
# cctools AR, unlike GNU binutils AR, does not have a deterministic mode
# or a configure flag to enable determinism by default, it only
# understands if this env-var is set or not. See:
#
# https://github.com/tpoechtrager/cctools-port/blob/55562e4073dea0fbfd0b20e0bf69ffe6390c7f97/cctools/ar/archive.c#L334
export ZERO_AR_DATE=yes
;;
esac
####################
# Depends Building #
####################
# Build the depends tree, overriding variables that assume multilib gcc
make -C depends --jobs="$JOBS" HOST="$HOST" \
make -C depends --jobs="$MAX_JOBS" HOST="$HOST" \
${V:+V=1} \
${SOURCES_PATH+SOURCES_PATH="$SOURCES_PATH"} \
${BASE_CACHE+BASE_CACHE="$BASE_CACHE"} \
${SDK_PATH+SDK_PATH="$SDK_PATH"} \
i686_linux_CC=i686-linux-gnu-gcc \
i686_linux_CXX=i686-linux-gnu-g++ \
i686_linux_AR=i686-linux-gnu-ar \
@@ -213,16 +142,18 @@ make -C depends --jobs="$JOBS" HOST="$HOST" \
x86_64_linux_RANLIB=x86_64-linux-gnu-ranlib \
x86_64_linux_NM=x86_64-linux-gnu-nm \
x86_64_linux_STRIP=x86_64-linux-gnu-strip \
qt_config_opts_i686_linux='-platform linux-g++ -xplatform bitcoin-linux-g++' \
qt_config_opts_x86_64_linux='-platform linux-g++ -xplatform bitcoin-linux-g++' \
FORCE_USE_SYSTEM_CLANG=1
qt_config_opts_i686_linux='-platform linux-g++ -xplatform bitcoin-linux-g++'
###########################
# Source Tarball Building #
###########################
GIT_ARCHIVE="${DIST_ARCHIVE_BASE}/${DISTNAME}.tar.gz"
# Define DISTNAME variable.
# shellcheck source=contrib/gitian-descriptors/assign_DISTNAME
source contrib/gitian-descriptors/assign_DISTNAME
GIT_ARCHIVE="${OUTDIR}/src/${DISTNAME}.tar.gz"
# Create the source tarball if not already there
if [ ! -e "$GIT_ARCHIVE" ]; then
@@ -230,16 +161,14 @@ if [ ! -e "$GIT_ARCHIVE" ]; then
git archive --prefix="${DISTNAME}/" --output="$GIT_ARCHIVE" HEAD
fi
mkdir -p "$OUTDIR"
###########################
# Binary Tarball Building #
###########################
# CONFIGFLAGS
CONFIGFLAGS="--enable-reduce-exports --disable-bench --disable-gui-tests --disable-fuzz-binary"
CONFIGFLAGS="--enable-reduce-exports --disable-bench --disable-gui-tests"
case "$HOST" in
*linux*) CONFIGFLAGS+=" --disable-threadlocal" ;;
*linux*) CONFIGFLAGS+=" --enable-glibc-back-compat" ;;
esac
# CFLAGS
@@ -247,36 +176,19 @@ HOST_CFLAGS="-O2 -g"
case "$HOST" in
*linux*) HOST_CFLAGS+=" -ffile-prefix-map=${PWD}=." ;;
*mingw*) HOST_CFLAGS+=" -fno-ident" ;;
*darwin*) unset HOST_CFLAGS ;;
esac
# CXXFLAGS
HOST_CXXFLAGS="$HOST_CFLAGS"
case "$HOST" in
arm-linux-gnueabihf) HOST_CXXFLAGS="${HOST_CXXFLAGS} -Wno-psabi" ;;
esac
# LDFLAGS
case "$HOST" in
*linux*) HOST_LDFLAGS="-Wl,--as-needed -Wl,--dynamic-linker=$glibc_dynamic_linker -static-libstdc++ -Wl,-O2" ;;
*mingw*) HOST_LDFLAGS="-Wl,--no-insert-timestamp" ;;
esac
# Using --no-tls-get-addr-optimize retains compatibility with glibc 2.17, by
# avoiding a PowerPC64 optimisation available in glibc 2.22 and later.
# https://sourceware.org/binutils/docs-2.35/ld/PowerPC64-ELF64.html
case "$HOST" in
*powerpc64*) HOST_LDFLAGS="${HOST_LDFLAGS} -Wl,--no-tls-get-addr-optimize" ;;
esac
case "$HOST" in
powerpc64-linux-*|riscv64-linux-*) HOST_LDFLAGS="${HOST_LDFLAGS} -Wl,-z,noexecstack" ;;
esac
# Make $HOST-specific native binaries from depends available in $PATH
export PATH="${BASEPREFIX}/${HOST}/native/bin:${PATH}"
mkdir -p "$DISTSRC"
(
cd "$DISTSRC"
@@ -293,23 +205,25 @@ mkdir -p "$DISTSRC"
--disable-maintainer-mode \
--disable-dependency-tracking \
${CONFIGFLAGS} \
${HOST_CFLAGS:+CFLAGS="${HOST_CFLAGS}"} \
${HOST_CXXFLAGS:+CXXFLAGS="${HOST_CXXFLAGS}"} \
CFLAGS="${HOST_CFLAGS}" \
CXXFLAGS="${HOST_CXXFLAGS}" \
${HOST_LDFLAGS:+LDFLAGS="${HOST_LDFLAGS}"}
sed -i.old 's/-lstdc++ //g' config.status libtool src/univalue/config.status src/univalue/libtool
# Build Bitcoin Core
make --jobs="$JOBS" ${V:+V=1}
make --jobs="$MAX_JOBS" ${V:+V=1}
# Check that symbol/security checks tools are sane.
make test-security-check ${V:+V=1}
# Perform basic security checks on a series of executables.
# Perform basic ELF security checks on a series of executables.
make -C src --jobs=1 check-security ${V:+V=1}
# Check that executables only contain allowed version symbols.
make -C src --jobs=1 check-symbols ${V:+V=1}
mkdir -p "$OUTDIR"
case "$HOST" in
*linux*|*mingw*)
# Check that executables only contain allowed gcc, glibc and libstdc++
# version symbols for Linux distro back-compatibility.
make -C src --jobs=1 check-symbols ${V:+V=1}
;;
esac
# Make the os-specific installers
case "$HOST" in
@@ -324,36 +238,8 @@ mkdir -p "$DISTSRC"
INSTALLPATH="${PWD}/installed/${DISTNAME}"
mkdir -p "${INSTALLPATH}"
# Install built Bitcoin Core to $INSTALLPATH
case "$HOST" in
*darwin*)
make install-strip DESTDIR="${INSTALLPATH}" ${V:+V=1}
;;
*)
make install DESTDIR="${INSTALLPATH}" ${V:+V=1}
;;
esac
make install DESTDIR="${INSTALLPATH}" ${V:+V=1}
case "$HOST" in
*darwin*)
make osx_volname ${V:+V=1}
make deploydir ${V:+V=1}
mkdir -p "unsigned-app-${HOST}"
cp --target-directory="unsigned-app-${HOST}" \
osx_volname \
contrib/macdeploy/detached-sig-{apply,create}.sh \
"${BASEPREFIX}/${HOST}"/native/bin/dmg
mv --target-directory="unsigned-app-${HOST}" dist
(
cd "unsigned-app-${HOST}"
find . -print0 \
| sort --zero-terminated \
| tar --create --no-recursion --mode='u+rw,go+r-w,a+X' --null --files-from=- \
| gzip -9n > "${OUTDIR}/${DISTNAME}-osx-unsigned.tar.gz" \
|| ( rm -f "${OUTDIR}/${DISTNAME}-osx-unsigned.tar.gz" && exit 1 )
)
make deploy ${V:+V=1} OSX_DMG="${OUTDIR}/${DISTNAME}-osx-unsigned.dmg"
;;
esac
(
cd installed
@@ -368,18 +254,13 @@ mkdir -p "$DISTSRC"
find . -name "lib*.a" -delete
# Prune pkg-config files
rm -rf "${DISTNAME}/lib/pkgconfig"
rm -r "${DISTNAME}/lib/pkgconfig"
case "$HOST" in
*darwin*) ;;
*)
# Split binaries and libraries from their debug symbols
{
find "${DISTNAME}/bin" -type f -executable -print0
find "${DISTNAME}/lib" -type f -print0
} | xargs -0 -n1 -P"$JOBS" -I{} "${DISTSRC}/contrib/devtools/split-debug.sh" {} {} {}.dbg
;;
esac
# Split binaries and libraries from their debug symbols
{
find "${DISTNAME}/bin" -type f -executable -print0
find "${DISTNAME}/lib" -type f -print0
} | xargs -0 -n1 -P"$MAX_JOBS" -I{} "${DISTSRC}/contrib/devtools/split-debug.sh" {} {} {}.dbg
case "$HOST" in
*mingw*)
@@ -419,44 +300,22 @@ mkdir -p "$DISTSRC"
| gzip -9n > "${OUTDIR}/${DISTNAME}-${HOST}-debug.tar.gz" \
|| ( rm -f "${OUTDIR}/${DISTNAME}-${HOST}-debug.tar.gz" && exit 1 )
;;
*darwin*)
find "${DISTNAME}" -print0 \
| sort --zero-terminated \
| tar --create --no-recursion --mode='u+rw,go+r-w,a+X' --null --files-from=- \
| gzip -9n > "${OUTDIR}/${DISTNAME}-${HOST//x86_64-apple-darwin18/osx64}.tar.gz" \
|| ( rm -f "${OUTDIR}/${DISTNAME}-${HOST//x86_64-apple-darwin18/osx64}.tar.gz" && exit 1 )
;;
esac
) # $DISTSRC/installed
case "$HOST" in
*mingw*)
cp -rf --target-directory=. contrib/windeploy
(
cd ./windeploy
mkdir -p unsigned
cp --target-directory=unsigned/ "${OUTDIR}/${DISTNAME}-win64-setup-unsigned.exe"
find . -print0 \
| sort --zero-terminated \
| tar --create --no-recursion --mode='u+rw,go+r-w,a+X' --null --files-from=- \
| gzip -9n > "${OUTDIR}/${DISTNAME}-win-unsigned.tar.gz" \
|| ( rm -f "${OUTDIR}/${DISTNAME}-win-unsigned.tar.gz" && exit 1 )
)
;;
esac
) # $DISTSRC
rm -rf "$ACTUAL_OUTDIR"
mv --no-target-directory "$OUTDIR" "$ACTUAL_OUTDIR" \
|| ( rm -rf "$ACTUAL_OUTDIR" && exit 1 )
(
cd /outdir-base
{
echo "$GIT_ARCHIVE"
find "$ACTUAL_OUTDIR" -type f
} | xargs realpath --relative-base="$PWD" \
| xargs sha256sum \
| sort -k2 \
| sponge "$ACTUAL_OUTDIR"/SHA256SUMS.part
)
)
case "$HOST" in
*mingw*)
cp -rf --target-directory=. contrib/windeploy
(
cd ./windeploy
mkdir unsigned
cp --target-directory=unsigned/ "${OUTDIR}/${DISTNAME}-win64-setup-unsigned.exe"
find . -print0 \
| sort --zero-terminated \
| tar --create --no-recursion --mode='u+rw,go+r-w,a+X' --null --files-from=- \
| gzip -9n > "${OUTDIR}/${DISTNAME}-win-unsigned.tar.gz" \
|| ( rm -f "${OUTDIR}/${DISTNAME}-win-unsigned.tar.gz" && exit 1 )
)
;;
esac

View File

@@ -1,113 +0,0 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
export TZ=UTC
# Although Guix _does_ set umask when building its own packages (in our case,
# this is all packages in manifest.scm), it does not set it for `guix
# environment`. It does make sense for at least `guix environment --container`
# to set umask, so if that change gets merged upstream and we bump the
# time-machine to a commit which includes the aforementioned change, we can
# remove this line.
#
# This line should be placed before any commands which creates files.
umask 0022
if [ -n "$V" ]; then
# Print both unexpanded (-v) and expanded (-x) forms of commands as they are
# read from this file.
set -vx
# Set VERBOSE for CMake-based builds
export VERBOSE="$V"
fi
# Check that required environment variables are set
cat << EOF
Required environment variables as seen inside the container:
UNSIGNED_TARBALL: ${UNSIGNED_TARBALL:?not set}
DETACHED_SIGS_REPO: ${DETACHED_SIGS_REPO:?not set}
DIST_ARCHIVE_BASE: ${DIST_ARCHIVE_BASE:?not set}
DISTNAME: ${DISTNAME:?not set}
HOST: ${HOST:?not set}
SOURCE_DATE_EPOCH: ${SOURCE_DATE_EPOCH:?not set}
DISTSRC: ${DISTSRC:?not set}
OUTDIR: ${OUTDIR:?not set}
EOF
ACTUAL_OUTDIR="${OUTDIR}"
OUTDIR="${DISTSRC}/output"
git_head_version() {
local recent_tag
if recent_tag="$(git -C "$1" describe --exact-match HEAD 2> /dev/null)"; then
echo "${recent_tag#v}"
else
git -C "$1" rev-parse --short=12 HEAD
fi
}
CODESIGNATURE_GIT_ARCHIVE="${DIST_ARCHIVE_BASE}/${DISTNAME}-codesignatures-$(git_head_version "$DETACHED_SIGS_REPO").tar.gz"
# Create the codesignature tarball if not already there
if [ ! -e "$CODESIGNATURE_GIT_ARCHIVE" ]; then
mkdir -p "$(dirname "$CODESIGNATURE_GIT_ARCHIVE")"
git -C "$DETACHED_SIGS_REPO" archive --output="$CODESIGNATURE_GIT_ARCHIVE" HEAD
fi
mkdir -p "$OUTDIR"
mkdir -p "$DISTSRC"
(
cd "$DISTSRC"
tar -xf "$UNSIGNED_TARBALL"
mkdir -p codesignatures
tar -C codesignatures -xf "$CODESIGNATURE_GIT_ARCHIVE"
case "$HOST" in
*mingw*)
find "$PWD" -name "*-unsigned.exe" | while read -r infile; do
infile_base="$(basename "$infile")"
# Codesigned *-unsigned.exe and output to OUTDIR
osslsigncode attach-signature \
-in "$infile" \
-out "${OUTDIR}/${infile_base/-unsigned}" \
-sigin codesignatures/win/"$infile_base".pem
done
;;
*darwin*)
# Apply detached codesignatures to dist/ (in-place)
signapple apply dist/Bitcoin-Qt.app codesignatures/osx/dist
# Make an uncompressed DMG from dist/
xorrisofs -D -l -V "$(< osx_volname)" -no-pad -r -dir-mode 0755 \
-o uncompressed.dmg \
dist \
-- -volume_date all_file_dates ="$SOURCE_DATE_EPOCH"
# Compress uncompressed.dmg and output to OUTDIR
./dmg dmg uncompressed.dmg "${OUTDIR}/${DISTNAME}-osx-signed.dmg"
;;
*)
exit 1
;;
esac
) # $DISTSRC
rm -rf "$ACTUAL_OUTDIR"
mv --no-target-directory "$OUTDIR" "$ACTUAL_OUTDIR" \
|| ( rm -rf "$ACTUAL_OUTDIR" && exit 1 )
(
cd /outdir-base
{
echo "$UNSIGNED_TARBALL"
echo "$CODESIGNATURE_GIT_ARCHIVE"
find "$ACTUAL_OUTDIR" -type f
} | xargs realpath --relative-base="$PWD" \
| xargs sha256sum \
| sort -k2 \
| sponge "$ACTUAL_OUTDIR"/SHA256SUMS.part
)

Some files were not shown because too many files have changed in this diff Show More