b42643c2537ffbe99d6d94fb1fb3b7f9d5234f93 doc: update init.cpp -conf help text (josibake)
970b9987ad5bcb72e581c40a7cdd408d94a48c81 doc: update devtools, release-process readmes (josibake)
50635d27b45d125b6264ac2abfbd6a1129c7228f build: include bitcoin.conf in build outputs (josibake)
6aac946f49aea243de1dc50631bb72f0186bbf58 doc: update bitcoin-conf.md (Josiah Baker)
1c7e820ded0846ef6ab4be9616b0de452336ef64 script: add script to generate example bitcoin.conf (josibake)
b483084d866c16d97a34251ae652bac94f85f61d doc: replace bitcoin.conf with placeholder file (josibake)
Pull request description:
create a script for parsing the output from `bitcoind --help` to create an example conf file for new users
## problem
per #10746 , `bitcoin.conf` not being put into the data directory during installation causes some confusion for users when running bitcoin. in the discussion on the issue, one proposed solution was to have an example config file and instruct users to `cp` it into their data directory after startup. in addition to #10746 , there have been other requests for a "skeleton config file" (https://github.com/bitcoin/bitcoin/issues/19641) to help users get started with configuring bitcoind.
the main issue with an example config file is that it creates a second source of truth regarding what options are available for configuring bitcoind. this means any changes to the options (including the addition or removal of options) would have to be updated for the command line and also updated in the example file.
this PR addresses this issue by providing a script to generate an example file directly from the `bitcoind --help` on-demand by running `contrib/devtools/gen-bitcoin-conf.sh`. this solution was originally proposed on #10746 and would also solve #19641 . this guarantees any changes made to the command-line options or the command-line options help would also be reflected in the example file after compiling and running the script.
the main purpose of this script is to generate a config file to be included with releases, same as `gen-manpages.sh`. this ensures every release also includes an up-to-date, full example config file for users to edit. the script is also available for users who compile from source for generating an example config for their compiled binary.
## special considerations
this removes the `bitcoin.conf` example file from the repo as it is now generated by this script. the original example file did contain extra text related to how to use certain options but going forward all option help docs should be moved into `init.cpp`
this also edits `init.cpp` to have the option help indicate that `-conf` is not usable from the config file. this is similar to how `-includeconf` 's help indicates it cannot be used from the command line
ACKs for top commit:
laanwj:
Tested and code review ACK b42643c2537ffbe99d6d94fb1fb3b7f9d5234f93
Tree-SHA512: 4546e0cef92aa1398da553294ce4712d02e616dd72dcbe0b921af474e54f24750464ec813661f1283802472d1e8774e634dd1cc26fbf1f13286d3e0406c02c09
copy over bitcoin.conf during the build process.
this means `contrib/devtools/gen-bitcoin-conf.sh` will need
to be run and the generated file committed during the release process.
this is the same process used for generating man pages for each release.
027aab663aaca32818051d456c3900326377281c test, contrib, refactor: use `with` when opening a file (brunoerg)
Pull request description:
When manipulating a file in Python without using `with()`, you have to close the file manually, so this PR does it in `get_block_hashes` (`contrib/linearize/linearize-data.py`).
Edit: this PR does it for all occurances that previously weren't using `with`.
ACKs for top commit:
laanwj:
Code review ACK 027aab663aaca32818051d456c3900326377281c
Tree-SHA512: 879400968e0013e8678ec16f1fe5d0963a73c1e0d442ca34802d885214f0783d2e9a9b500fc6be7c3b93560a367b6a3d685eee24d2f9ce53fddf064ea6feecf8
this ensures bitcoind option help is the source of truth and also
gives an example conf file for users to customize and copy to their
data directory.
closes#10746
9b0a13a2891641a3d12e525cee8ddddb1aa1bc73 tidy: Add include-what-you-use (fanquake)
74cd038e300bfbe2473295fc3b0c3a4f3e853a07 refactor: fix includes in src/init (fanquake)
c79ad935f0412bac3e19a6b925efdb390eb00bd9 refactor: fix includes in src/compat (fanquake)
Pull request description:
We recently added a [`clang-tidy` job](https://github.com/bitcoin/bitcoin/blob/master/ci/test/00_setup_env_native_tidy.sh) to the CI, which generates a compilation database. We can leverage that now existing database to begin running [include-what-you-use](https://include-what-you-use.org/) over the codebase.
This PR demonstrates using a mapping_file to indicate fixups / includes that may differ from IWYU suggestions. In this case, I've added some fixups for glibc includes that I've [upstreamed changes for](https://github.com/include-what-you-use/include-what-you-use/pull/1026):
```bash
# Fixups / upstreamed changes
[
{ include: [ "<bits/termios-c_lflag.h>", private, "<termios.h>", public ] },
{ include: [ "<bits/termios-struct.h>", private, "<termios.h>", public ] },
{ include: [ "<bits/termios-tcflow.h>", private, "<termios.h>", public ] },
]
```
The include "fixing" commits of this PR:
* Adds missing includes.
* Swaps C headers for their C++ counterparts.
* Removes the pointless / unmaintainable `//for abc, xyz` comments. When using IWYU, if anyone wants to see / generate those comments, to see why something is included, it is trivial to do so (IWYU outputs them by default). i.e:
```cpp
// The full include-list for compat/stdin.cpp:
#include <compat/stdin.h>
#include <poll.h> // for poll, pollfd, POLLIN
#include <termios.h> // for tcgetattr, tcsetattr
#include <unistd.h> // for isatty, STDIN_FILENO
```
TODO:
- [ ] Qt mapping_file. There is one in the IWYU repo, but it's for Qt 5.11. Needs testing.
- [ ] Boost mapping_file. There is one in the IWYU repo, but it's for Boost 1.75. Needs testing.
I'm not suggesting we turn this on the for entire codebase, or immediately go-nuts refactoring all includes. However I think our dependency includes are now slim enough, and our CI infrastructure in place such that we can start doing this in some capacity, and just automate away include fixups / refactorings etc.
ACKs for top commit:
MarcoFalke:
review ACK 9b0a13a2891641a3d12e525cee8ddddb1aa1bc73
jonatack:
ACK 9b0a13a2891641a3d12e525cee8ddddb1aa1bc73 reviewed changes and run CI output in https://cirrus-ci.com/task/4750910332076032
Tree-SHA512: 00beab5a5f2a6fc179abf08321a15391ecccaa91ab56f3c50c511e7b29a0d7c95d8bb43eac2c31489711086f6f77319d43d803cf8ea458e7cd234a780d9ae69e
914076ed5ffd644581600322bf911f13eb839951 guix: Improve error message about missed macOS SDK (Hennadii Stepanov)
Pull request description:
The error message now mentions another option for users to specify the path to the macOS SDK.
ACKs for top commit:
fanquake:
ACK 914076ed5ffd644581600322bf911f13eb839951
Tree-SHA512: e8b9a6c8aa984f02cdb4ad1eeb6c22510c0c74c47c831104ac410ef68b3bc40024a52400b5065a3e25603f8a040e72c8c464436cb9a0a018ca73b0d28e0d609d
This change makes naming of `install_name_tool` consistent across
the whole build system.
-BEGIN VERIFY SCRIPT-
sed --in-place --expression='s/INSTALLNAMETOOL/INSTALL_NAME_TOOL/g' $(git grep --files-with-matches 'INSTALLNAMETOOL')
-END VERIFY SCRIPT-
1dd8cbfbc6318308b5e75023568f818ef6a9c7e4 build: don't compress macOS DMG (fanquake)
Pull request description:
Skip compressing the macOS DMG, and drop related build steps and dependencies. Uncompressed the DMG increases from ~16mb to ~30mb, which compared to other software a user may download, (Firefox 125mb, VLC 52mb, Open Office 176mb), is still relatively small. When contrasted against the 100's of GB of blockchain data a node will download, an additional 15mb to get the release binary, isn't much additional overhead. Note that if / when we build with LTO enabled for releases, this size will shrink back down significantly again.
`native_libdmg-hfsplus` is not maintained, and I doubt the DMG creation feature will ever be fixed. If at some point `xorrisofs` supports compressing dmgs, we could enable that.
Guix Build on x86_64:
```bash
25b7c8bb7bc8ea014d43cebb844a842d2ac8d5a343039a820d24b649c9e6bc8a guix-build-1dd8cbfbc631/output/arm64-apple-darwin/SHA256SUMS.part
16beb5c52c9bf51b5ce9ef5a0d17c0038238a833383586a1b14acbca78533e4b guix-build-1dd8cbfbc631/output/arm64-apple-darwin/bitcoin-1dd8cbfbc631-arm64-apple-darwin-unsigned.dmg
d8f89a61a7448d6334dbb3639386a7b6340542393933f35421a9e6dfc724e455 guix-build-1dd8cbfbc631/output/arm64-apple-darwin/bitcoin-1dd8cbfbc631-arm64-apple-darwin-unsigned.tar.gz
11617dc261ef602433f5bb29956a40a9085dbc783f519f75fbe06e80970148d0 guix-build-1dd8cbfbc631/output/arm64-apple-darwin/bitcoin-1dd8cbfbc631-arm64-apple-darwin.tar.gz
aa8550d4a394d3161d14ec5e6012ed07354135afb022e905a1946785b4665664 guix-build-1dd8cbfbc631/output/dist-archive/bitcoin-1dd8cbfbc631.tar.gz
2b837f2f971a9738d0b7b8497f7ded740ef5e67c8baa7f30ca33e6b7d826eec8 guix-build-1dd8cbfbc631/output/x86_64-apple-darwin/SHA256SUMS.part
db972b2c06dbde5525a3f9e6ceb9c20a8120bc9a6f15e1d852a4bfac09d88569 guix-build-1dd8cbfbc631/output/x86_64-apple-darwin/bitcoin-1dd8cbfbc631-x86_64-apple-darwin-unsigned.dmg
50fe990c3f9923ee92195125faf6517396e7c1b017a8f4f7d52e991ebce52f0c guix-build-1dd8cbfbc631/output/x86_64-apple-darwin/bitcoin-1dd8cbfbc631-x86_64-apple-darwin-unsigned.tar.gz
1d9022b0ae46ead41046c40f82291ce363760660a3cd6e6ef6a5b1128b90faef guix-build-1dd8cbfbc631/output/x86_64-apple-darwin/bitcoin-1dd8cbfbc631-x86_64-apple-darwin.tar.gz
```
Guix Build on arm64:
```bash
```
ACKs for top commit:
Sjors:
re-tACK 1dd8cbfbc6318308b5e75023568f818ef6a9c7e4 on Intel macOS
laanwj:
Build system changes code review ACK 1dd8cbfbc6318308b5e75023568f818ef6a9c7e4, I don't know anything about MacOS application formats and their internals so do not have an opinion on the contents of this change.
jarolrod:
ACK 1dd8cbfbc6
Tree-SHA512: 04c5bf78f26a9877777093ec4c50c457107bef59d720839ea5e7d7e4f7961dfee9f86b40cf791524a9e60e9e77403a797e9fcdae3849b60b759f9f66cc31b6ab
ba30a5407e065e9d6dd037351e83f56a43f38f19 contrib: macdeploy: monkey-patch gen-sdk to be deterministic (Pavol Rusnak)
1868a17e5a46c96bfd2d3f28e876bbaa74d1417f contrib: macdeploy: make gen-sdk deterministic (Pavol Rusnak)
Pull request description:
This PR attempts to make `contrib/macdeploy/gen-sdk` deterministic
Can anyone with the `Xcode_12.2.xip` confirm that `gen-sdk` produces the same hash? => `e7ca56bc8804d16624fad68be2e71647747d6629cacaaa3de5fbfa7f444e9eae `
ACKs for top commit:
laanwj:
Tested ACK ba30a5407e065e9d6dd037351e83f56a43f38f19
jarolrod:
Tested ACK ba30a5407e
Tree-SHA512: 1638ceaf28e87ef0d21a1a71ef02989f75942b60a12f07236ac709bde96f08f39f816767e35a0fe68c26bf5978e63e74f5385be9d4b8f80a2e89b30f163f4526
107582039ac3cbfe072dc761f621c37c48aa2dc1 doc: Add gpg key import instructions for Windows (Dave Scotese)
Pull request description:
This is a single commit to replace the three commits from #23916
I propose this change so that Windows users can more easily import signers' keys.
ACKs for top commit:
sipsorcery:
tACK 107582039ac3cbfe072dc761f621c37c48aa2dc1.
Tree-SHA512: 7d4ec77ce10f751748c49f1453fa8baf0976b15af4f87dc27f4e2715ad73fbd7dc1f07fcf3e660d63a6b9eb895a5e4105774613d39a2328f73b92d9e6cff4ebd
I have some qualms with maintaining a suspicious hosts list as part as
the repository. But also, it's stale and irrelevant. I've checked the
entire list and none of them is connectable. Only one still appars in
`nodes_main.txt` but with low uptime and an old subversion string so it
wouldn't be picked in the first place.
c457fb144cc3f76db46d8167744f3865af371ed7 improve clarity and up max ipv6 ASNs (Baas)
Pull request description:
This PR attempts to address some of the areas of improvement raised in #17020 . Concretely, my proposed change is fairly minor but addresses the following changes to [`makeseeds.py`](https://github.com/bitcoin/bitcoin/blob/master/contrib/seeds/makeseeds.py):
- Increase max seeds per ASN for IPv6 to 10 as recommended [here](https://github.com/bitcoin/bitcoin/pull/16999#issuecomment-536999544), while keeping max seeds per ASN for IPv4 at 2.
- Bump `MIN_BLOCKS` to 730000.
- Improved script clarity: added function types and more docs to functions, added progress indicator when performing ASN lookup, and change string formatting to better align with [bitcoin python style guidelines](https://github.com/bitcoin/bitcoin/blob/master/test/functional/README.md#style-guidelines)
With the different ASN limits for IPv4 and IPv6, and the new minimum block requirement, the current stats look look like:
```
IPv4 IPv6 Onion Pass
470689 73238 0 Initial
470689 73238 0 Skip entries with invalid address
470689 73238 0 After removing duplicates
470688 73238 0 Skip entries from suspicious hosts
6098 1676 0 Enforce minimal number of blocks
5252 1443 0 Require service bit 1
3812 898 0 Require minimum uptime
3738 877 0 Require a known and recent user agent
3715 869 0 Filter out hosts with multiple bitcoin ports
512 512 0 Look up ASNs and limit results per ASN and per net
```
The new ASN max seeds of 10 allows for 512 IPv6 addresses to be included, up from the ~150 that was filtered by the previous version.
While there is more to do for #17020 , these changes I think are fairly isolated from the rest and should make it a bit easier for others to get up to speed with what the functions in the script do.
ACKs for top commit:
laanwj:
Concept and code review ACK c457fb144cc3f76db46d8167744f3865af371ed7
Tree-SHA512: 3ed67868443cc50544e23b27e2341758c3a8866997b0dba47b137032d5e1a13428855daaeed682626ed471542b44435635178d54848a2cd6fe73777679428032
65c49ac750ba39801b349d0a59c27471dfa9868e test: throw `ValueError` for invalid base58 checksum (Sebastian Falbesoner)
219d2c7ee1d35a353a96c55d4c411d43fe39548a contrib: testgen: use base58 methods from test framework (Sebastian Falbesoner)
605fecfb66ba51467b35a3f269116ec786aedd05 scripted-diff: rename `chars` to `b58chars` in test_framework.address (Sebastian Falbesoner)
11c63e374d058d3bde64a725068d29c874950b45 contrib: testgen: import OP_* constants from test framework (Sebastian Falbesoner)
7d755bb31cd58099cd97b604e04a6a4bb99cd2a9 contrib: testgen: avoid need for manually setting PYTHONPATH (Sebastian Falbesoner)
Pull request description:
This PR removes the redundant base58 implementation [contrib/testgen/base58.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/testgen/base58.py) for the test generation script `gen_key_io_test_vectors.py` and uses the one from the test framework instead. Additionally, three other cleanups/improvements are done:
- import script operator constants `OP_*` from test framework instead of manually defining them
- add Python path to test framework directly in the script (via `sys.path.append(...)`) instead of needing the caller to specify `PYTHONPATH=...` on the command line (the same approach is done for the signet miner and the message capture scripts)
- rename `chars` to `b58chars` in the test_framework.address module (is more explicit and makes the diff for the base58 replacement smaller)
ACKs for top commit:
laanwj:
Code review ACK 65c49ac750ba39801b349d0a59c27471dfa9868e
Tree-SHA512: 92e1534cc320cd56262bf455de7231c6ec821bfcd0ed58aa5718271ecec1a89df7951bf31527a2306db6398e7f2664d2ff8508200c28163c0b164d3f5aaf8b0e
af74e061c052fa8b260ee71bf048fe22028962ae guix: make it possible to override gpg binary (Pavol Rusnak)
Pull request description:
For example on Qubes OS one might want to use qubes-gpg-client-wrapper instead
Fixes https://github.com/bitcoin/bitcoin/issues/24346
ACKs for top commit:
laanwj:
Concept and code review ACK af74e061c052fa8b260ee71bf048fe22028962ae
Tree-SHA512: 9e56b5fab231f8908fff15c88fe5b356ac4a31a14a27ae2dd3b6e876f32628910a666a4e2da5bf7c5d159de66cf57652c94c81cdc3b1c3d39a23c23e2c77dd03
15893a07811e022bf9e774b831f8386df7ac52e5 supp: remove Boost Valgrind suppression (fanquake)
b0740fdcb84120b76b201e6d17859e9a398f93b4 ci: use DWARF-4 for Valgrind CI job (fanquake)
Pull request description:
[clang-14 defaults to using DWARF-5](https://releases.llvm.org/14.0.0/tools/clang/docs/ReleaseNotes.html#dwarf-support-in-clang), which breaks vlagrinds (3.18) ability
to parse debug info. Valgrind [claims to support DWARF-5](https://valgrind.org/docs/manual/dist.news.html) from version
3.18 onwards, but maybe that only works when compiling with GCC.
Explicitly use DWARF-4 for now. Note that from 11.0 [GCC also defaults to
using DWARF-5](https://www.gnu.org/software/gcc/gcc-11/changes.html).
Also remove a Boost related suppression.
Top commit has no ACKs.
Tree-SHA512: e4f476170ac5ccbb43d26e990b24753bda3985b2ac5c8a32e74d2d1d64d1b3a2d80a90fbab345f0a9e404eac7fbd783c20147379208e615d526657e8a57890ca
254a63e097def1bf7e157c72ea9e4fff1eeb7a28 contrib: refactor: replace `hex_switchEndian` in linearize scripts (Sebastian Falbesoner)
3f863cfff1ec902454d64f82b12eb09bdb1eee04 contrib: refactor: simplify block header string routine in linearize-data.py (Sebastian Falbesoner)
Pull request description:
This PR simplifies the linearization scripts `linearize-data.py` and `linearize-hashes.py` by replacing overly complicated cruft (block header hash string calculation, hex string reverse) with means of the Python3 standard library.
ACKs for top commit:
laanwj:
Code review ACK 254a63e097def1bf7e157c72ea9e4fff1eeb7a28
Tree-SHA512: 4a0e20e63bd11f23a190480b22dbc2f2a3070e2a4f3a01b8797f99bb5fc830185e91e6712c8ae97562f9a24a98aa4f19d52f02a3f5fcbe4c578ee88a41382d1d
97af6527885c3b2dac183350f6095c98f8980e49 guix: Drop code for the unsupported `i686-linux-gnu` host (Hennadii Stepanov)
Pull request description:
Now GUIX build for the `i686-linux-gnu` host is broken, and [there are no plans to re-add it](https://github.com/bitcoin/bitcoin/pull/24448).
ACKs for top commit:
fanquake:
ACK 97af6527885c3b2dac183350f6095c98f8980e49
Tree-SHA512: 968181aff65e607a7c1a1b06ac7dfd79f6e2ce49b3c4c3828def020e925769fdbab1859d37ea924ded7632405b30539ac3ec81ac714cb9a01a2f7d5c93301dd9
