2a1b85f3c54fdaa8d13aeaf7e1acab1afb98fd97 tx: Bump transifex slug to 020x (Wladimir J. van der Laan)
82dd8860bbe81567e4afb033d9026dc0e69b1574 qt: Periodical translations update (Wladimir J. van der Laan)
Pull request description:
Need to merge this so that translations for 0.20 can start.
Top commit has no ACKs.
Tree-SHA512: c78307eea3130b9bbc301f23790340e4c86a3f3877ab308bf67d9e1c9e977f7dda4b2cd83b96d9398b01dce616a6c962b98115123628bba35170eb0f301c4dd8
fa192952507dbe5f405abffce38b3556923ba271 test: Bump timeouts to avoid valgrind failures (MarcoFalke)
Pull request description:
This should fix ci timeout issues such as:
* https://travis-ci.org/github/bitcoin/bitcoin/jobs/661946972#L3109
* https://travis-ci.org/github/bitcoin/bitcoin/jobs/662521570#L4922
ACKs for top commit:
practicalswift:
ACK fa192952507dbe5f405abffce38b3556923ba271
Tree-SHA512: 150f804957575033a83fd47c68fd6adff2c11b110ec5803bd77f121256349805b595c39a3a5047738ec538d082ee38cebbcb6792c787ef22f56b8dd0d9618c1f
e6e622e5a0e22c2ac1b50b96af818e412d67ac54 Implement O(1) OP_IF/NOTIF/ELSE/ENDIF logic (Pieter Wuille)
d0e8f4d5d8ddaccb37f98b7989fb944081e41ab8 [refactor] interpreter: define interface for vfExec (Anthony Towns)
89fb241c54fc85befacfa3703d8e21bf3b8a76eb Benchmark script verification with 100 nested IFs (Pieter Wuille)
Pull request description:
While investigating what mechanisms are possible to maximize the per-opcode verification cost of scripts, I noticed that the logic for determining whether a particular opcode is to be executed is O(n) in the nesting depth. This issue was also pointed out by Sergio Demian Lerner in https://bitslog.wordpress.com/2017/04/17/new-quadratic-delays-in-bitcoin-scripts/, and this PR implements a variant of the O(1) algorithm suggested there.
This is not a problem currently, because even with a nesting depth of 100 (the maximum possible right now due to the 201 ops limit), the slowdown caused by this on my machine is around 70 ns per opcode (or 0.25 s per block) at worst, far lower than what is possible with other opcodes.
This PR mostly serves as a proof of concept that it's possible to avoid it, which may be relevant in discussions around increasing the opcode limits in future script versions. Without it, the execution time of scripts can grow quadratically with the nesting depth, which very quickly becomes unreasonable.
This improves upon #14245 by completely removing the `vfExec` vector.
ACKs for top commit:
jnewbery:
Code review ACK e6e622e5a0e22c2ac1b50b96af818e412d67ac54
MarcoFalke:
ACK e6e622e5a0e22c2ac1b50b96af818e412d67ac54 🐴
fjahr:
ACK e6e622e5a0e22c2ac1b50b96af818e412d67ac54
ajtowns:
ACK e6e622e5a0e22c2ac1b50b96af818e412d67ac54
laanwj:
concept and code review ACK e6e622e5a0e22c2ac1b50b96af818e412d67ac54
jonatack:
ACK e6e622e5a0e22c2ac1b50b96af818e412d67ac54 code review, build, benches, fuzzing
Tree-SHA512: 1dcfac3411ff04773de461959298a177f951cb5f706caa2734073bcec62224d7cd103767cfeef85cd129813e70c14c74fa8f1e38e4da70ec38a0f615aab1f7f7
09e25071f40c564af08a1386c39c4f2d8eb484b6 Cache parent xpub inside of BIP32PubkeyProvider (Andrew Chow)
deb791c7ba057a3765d09b12bf3e55547a5298e4 Only cache xpubs that have a hardened last step (Andrew Chow)
f76733eda5f4c161e9eb47c74b949582ab8f448a Cache the immediate derivation parent xpub (Andrew Chow)
58f54b686f663e4c46a2cf7a64560409007c7eb3 Add DescriptorCache* read_cache and DescriptorCache* write_cache to Expand and GetPubKey (Andrew Chow)
66c2cadc91d26074b89e5ada68350b5c8676efac Rename BIP32PubkeyProvider.m_extkey to m_root_extkey (Andrew Chow)
df55d44d0de2174ba74ed3a28bef5e83b0a51b47 Track the index of the key expression in PubkeyProvider (Andrew Chow)
474ea3b927ddc67e64ae78e08c20c9264817e84d Introduce DescriptorCache struct which caches xpubs (Andrew Chow)
Pull request description:
Improves the descriptor cache by changing it from a `std::vector<unsigned char>` to a newly introduced `DescriptorCache` class. Instead of serializing pubkeys and whatever else we would want to cache in a way that may not be backwards compatible, we instead create a `DescriptorCache` object and populate it. This object contains only an xpub cache. Since the only `PubkeyProvider` that used the cache is the `BIP32PubkeyProvider` we just have it store the xpubs instead of the pubkeys. This allows us to have both the parent xpub and the child xpubs in the same container. The map is keyed by `KeyOriginInfo`.
Sine we are caching `CExtPubKey`s in `DescriptorCache`, `BIP32PubKeyProviders` can use the cached parent xpubs to derive the children if unhardened derivation is used in the last step. This also means that we can still derive the keys for a `BIP32PubkeyProvider` that has hardened derivation steps. When combined with descriptor wallets, this should allow us to be able to import a descriptor with an `xprv` and hardened steps and still be able to derive from it. In that sense, this is an alternative to #18163
To test that this works, the tests have been updated to do an additional `Expand` at the `i + 1` position. This expansion is not cached. We then do an `ExpandFromCache` at `i + 1` and use the cache that was produced by the expansion at `i`. This way, we won't have the child xpubs for `i + 1` but we will have the parent xpubs. So this checks whether the parent xpubs are being stored and can be used to derive the child keys. Descriptors that have a hardened last step are skipped for this part of the test because that will always require private keys.
ACKs for top commit:
instagibbs:
code review re-re-ACK 09e25071f4
Sjors:
re-ACK 09e25071f40c564af08a1386c39c4f2d8eb484b6
Tree-SHA512: 95c8d0092274cdf115ce39f6d49dec767679abf3758d5b9e418afc308deca9dc6f67167980195bcc036cd9c09890bbbb39ec1dacffbfacdc03efd72a7e23b276
c8e24ddce31a8de6255b23c19d958c1cd44a8847 [REFACTOR] Abstract out script execution out of VerifyWitnessProgram() (Pieter Wuille)
Pull request description:
This is a refactoring cherry-picked out of #17977. As it touches consensus code, I don't think this would ordinarily meet the bar for review cost vs benefit. However, it simplifies the changes for Taproot significantly, and if it's going to be necessitated by inclusion of that code, I may as well give it some additional attention by PRing it independently.
ACKs for top commit:
fjahr:
Re-ACK c8e24ddce31a8de6255b23c19d958c1cd44a8847
theStack:
re-ACK c8e24ddce3
Empact:
Code Review Re-ACK c8e24ddce3
ajtowns:
ACK c8e24ddce31a8de6255b23c19d958c1cd44a8847
jnewbery:
ACK c8e24ddce31a8de6255b23c19d958c1cd44a8847
jonatack:
ACK c8e24dd
Tree-SHA512: 96c2aa5d2f9c7c802bcc008f5cde55b1dfedfaf42e34101331e6c0d594acdf6437661102dc939718f0877c20451336855dfbaa8aa8f57d9e722a7fa7329e3a46
a33cffbeabcc42137c4a66aa19b7dd1d300e6d73 util: HelpExampleRpc formatting fixup (Jon Atack)
Pull request description:
Minor visual fixup of the HelpExampleRpc template; conforms to the JSON-RPC spec as per https://www.jsonrpc.org/specification#examples. (I'm... somewhat embarassed to open such a minor change, but this is what is shown in all the CLI/RPC help docs.)
ACKs for top commit:
laanwj:
ACK a33cffbeabcc42137c4a66aa19b7dd1d300e6d73
Tree-SHA512: 8f1dee080c224742fff60a33fec6f5fb1d59c9fa51f3f2a67bf2e1837dbfa25f12a69e34518936588940013b0e61f55378b4f1a571c47c3cb081ca5b245e1091
faf6f156ffd1a8ed1aed047428d791a8c13c162b test: Add missing syncwithvalidationinterfacequeue (MarcoFalke)
Pull request description:
The wallet rebroadcast functionality learns about new blocks via the validation interface queue. To avoid test failures such as https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/31119387#L466 , we can sync with the queue before advancing the test.
ACKs for top commit:
jonatack:
ACK faf6f156 this makes sense; the fix was previously added to mempool_persist.py and wallet_zapwallettxes.py in #12217 and to wallet_balance.py in #16302. It is also used in src/test/validation_block_tests.cpp (processnewblock_signals_ordering) and src/bench/wallet_balance.cpp.
Tree-SHA512: d72fd4b597b669d8111007902b523e946712913cd6eea6f9a695b0f04ecbe2321d05019873af999a95b9e0aa0f5c140a17109b37503723e40c9eab24ec358eb7
fa2cf85e6f366d62ebf173bf0bd51af45806afe1 test: Fix race in p2p_segwit (MarcoFalke)
Pull request description:
Fixes#11696
Top commit has no ACKs.
Tree-SHA512: 09de07ea26236547586f5c373a0df2b68d84af5cfa8f40bd2ca9f951fc083c5f4b8a472a60668d99118bbd9f3942ec3d6a34f05944d47345acca41c95475bb27
0ae42a16c766a7ecb8711bfad6f22b8581ea0258 guix: Remove now-unnecessary gcc make flag (Carl Dong)
Pull request description:
```
Previously, Guix would produce a gcc which did not know to use the SSP
function from glibc, and required a gcc make flag for it to do so, in my
attempt to fix it upstream I realized that this is no longer the case.
This can be verified by performing a Guix build and doing
readelf -s ... | grep __stack_chk
to check that symbols are coming from glibc, and doing
readelf -d ... | grep NEEDED | grep ssp
to see that libssp.so is not being depended on
```
ACKs for top commit:
fanquake:
ACK 0ae42a16c766a7ecb8711bfad6f22b8581ea0258 - ran a Guix build (hashes below) and checked all the linux binaries:
Tree-SHA512: 701b91e7c323b12a29af9539cb2656d10ce0a93af573a02e57f0b7fea05a6e1819798536eadb24d0a17e7f35b503f5e863fee5e7409db1b8a3973c4375e49d4e
Previously, Guix would produce a gcc which did not know to use the SSP
function from glibc, and required a gcc make flag for it to do so, in my
attempt to fix it upstream I realized that this is no longer the case.
This can be verified by performing a Guix build and doing
readelf -s ... | grep __stack_chk
to check that symbols are coming from glibc, and doing
readelf -d ... | grep NEEDED | grep ssp
to see that libssp.so is not being depended on
111880aaf7e12a12f0797f1b19673e3d96328edd [test] Add coverage to estimaterawfee and estimatesmartfee (Ben Woosley)
Pull request description:
This adds light functional coverage to estimaterawfee - a subset of
the testing applied to estimatesmartfee, and argument validation
testing to both estimaterawfee and estimatesmartfee.
One valid estimatesmartfee signature test is commented out because it
fails currently.
Extracted from #12940
Top commit has no ACKs.
Tree-SHA512: 361a883457b28b2dc75081666e49d6dc6b5d76eed40d858abe2dd4f35ece152cf1f99c94480a91f42a896aa2a73cf55f57921316fe66970b2d7ba691a3b17e2d
8a2a652e6fab5eb8224beefcc07d9011b61865a8 Remove redundant type information from rpc docs (David O'Callaghan)
Pull request description:
Simple edit of the RPC calls to remove redundant text ("A json object/array ...") from the beginning of help.
Fixes: #18258
Top commit has no ACKs.
Tree-SHA512: cbbf760e0b7b4eda61c40b420ed77f5d878318e37b0eb13e63567212240b2c4ecc15d84030e98075e21c9ae9016539adfd201e5661ea824166a76d335180c32f
3e32499909ca8127baaa9b40ad113b25ee151bbd Change example addresses to bech32 (Yusuf Sahin HAMZA)
Pull request description:
This is a follow-up PR to #18197 that fixes RPCExamples.
Fixes#18185.
ACKs for top commit:
MarcoFalke:
ACK 3e32499909ca8127baaa9b40ad113b25ee151bbd
jonatack:
ACK 3e32499
Tree-SHA512: c7a6410ef8b6e169016c2c5eac3e6b9501caabd0e8a0871ec31e56bfc44589f056d3f5cb55b5a13bba36f6c15136c2352f883e30e4dcc0997ffd36b27f9173b9
ddc7e42d600a0cb3e763cda0dc04a1f2f34e9440 build: Set minimum Automake version to 1.13 (Hennadii Stepanov)
Pull request description:
This PR suggests to set the required minimum Automake version to `1.13` explicitly for the following reasons:
- it guarantees that [CVE-2012-3386](https://lists.gnu.org/archive/html/automake/2012-07/msg00023.html) has been fixed
- `AC_CONFIG_MACRO_DIR` macro support, which we already use; from the [release notes](https://lists.gnu.org/archive/html/automake/2012-12/msg00038.html):
> Improvements to aclocal and related rebuilds rules:
> - Autoconf-provided macros AC_CONFIG_MACRO_DIR and AC_CONFIG_MACRO_DIRS are now traced by aclocal, and can be used to declare the local m4 include directories. Formerly, one had to specify it with an explicit '-I' option to the 'aclocal' invocation.
- `AM_SILENT_RULES` macro support (since version `1.11`)
Automake `1.13` requires Autoconf `2.65` or greater. We already have `2.69` since #17769.
---
For reference, Automake `1.13` was released in [December of 2012](https://lists.gnu.org/archive/html/automake/2012-12/msg00038.html).
CentOS 7 uses Automake [`1.13.4`](https://centos.pkgs.org/7/centos-x86_64/automake-1.13.4-3.el7.noarch.rpm.html)
See the Automake docs for more info:
- [`AM_INIT_AUTOMAKE`](https://www.gnu.org/software/automake/manual/automake.html#Public-Macros)
- [List of Automake options](https://www.gnu.org/software/automake/manual/automake.html#List-of-Automake-options)
ACKs for top commit:
laanwj:
so also ACK ddc7e42d600a0cb3e763cda0dc04a1f2f34e9440
fanquake:
ACK ddc7e42d600a0cb3e763cda0dc04a1f2f34e9440 - I think adding a minimum required version here is fine. I'd be surprised if someone who is currently building Bitcoin Core was unable to after this change.
Tree-SHA512: a1f97864bc3a513450c03d041498f28e823e6f8cd9710d81df081435d72bd4b6cd2f3deb997dbf902f950215a859e48a2ee7ca1f8ebf4271778dd951ab78abf4
fab7d14ea5a4305317d66f35beb3225a07823d42 test: Check that wait_until returns if time point is in the past (MarcoFalke)
Pull request description:
Add an explicit regression test for the condvar bug (#18227), so that this doesn't happen again
ACKs for top commit:
laanwj:
ACK fab7d14ea5a4305317d66f35beb3225a07823d42
Tree-SHA512: 6ec0d0b3945cae87a001e367af34cca1953a8082b4a0d9f8a20d30acd1f36363e98035d4eb173ff786cf6692d352d41f960633415c46394af042eb44e3b5ad71
c1e07423083cd2a7e3f2b28f69a573ea1837af4d doc: Warn about wallet.dat re-use and backups (Albert)
Pull request description:
Following discussion in #18205, this PR adds a warning against re-use of the same wallet file on two different nodes, as that can cause problems due to race conditions between nodes (eg: both nodes using the same addresses at the same time for different things because they are not aware of the other node).
I've also included the rationale behind the warning but I've kept it short to make it clearer to users, not sure if I should have written a longer explanation instead.
Also, while this PR may help some users avoid problems, the changes are largely inconsequential, so feel free to close it if it's not worth the effort.
On an unrelated note, I've also set up [this site](https://corollari.github.io/bitcoin-core-docs/), which periodically pulls bitcoin core and turns its docs into a webpage. Browsing the docs can also be done locally or on github, so this doesn't add much value, but I personally find that more comfortable and it makes them more searchable.
Top commit has no ACKs.
Tree-SHA512: 5ce06026176917304932714470be8c3410d35698f925875b0955ecd3b1756ef52793feb469dd4bdac4921f1a24daf59001e9911f1f096f559fb28c250baae378
7ba962276e3e540a15eb0bdbbaaa101958229bd4 doc: Minor grammatical changes and flow improvements (Travin Keith)
Pull request description:
**Grammar:**
Line 49: There shouldn't be a period at the end of a phrase.
Lines 56, 57, 116, 137, and 177: Adding necessary commas
Lines 103 and 136: Run-on sentence issues fixed.
Line 176: Fixed punctuation and added necessary conjunction
Line 178: Singular noun when it should be plural
**Flow:**
Line 49: Adding "for" makes it more natural.
Line 54: Though it's not grammatically incorrect to end a sentence with a preposition in an informal document such as this, the word "followed" is much easier to understand anyway, especially for those who don't have English as their native language.
Top commit has no ACKs.
Tree-SHA512: 17fdb5fc6146e025f160e860dbcbdbfa07b7608b8cb611c3b9d4ed91c426100ef772915251bc1f6bacb3a62df57b72c2003fb72cb2c8542454638545985313da
155a11f897c7dfdc891587cc7ddd7c153cbc2a8f doc: Added running functional tests in valgrind (Elichai Turkel)
Pull request description:
Technically the notes only show an "example" of how to run valgrind with the suppression file,
but now that https://github.com/bitcoin/bitcoin/pull/17633 is merged then maybe this can encourage more people to run also the functional tests in valgrind
Top commit has no ACKs.
Tree-SHA512: b8417249b720d0ed5e10b732648f2e07e8889bfc7aa7e94192d1c049b4b7837971678d30c535f273c227848f1290cf11e14369fd6c1924b734f2e47e2af41401
2a6bce482c13cff37c1af00231265de4656a454b doc: Add a note about backporting (Carnhof Daki)
Pull request description:
See laanwj's comment in #17158https://github.com/bitcoin/bitcoin/pull/17158#issuecomment-542627090
Top commit has no ACKs.
Tree-SHA512: ac5248a796050ce1a5bd0718955f941f6a3c025e192599948f12566eb55296079404b999676b9a2c8fe10616fc8334698dfa415af0fb4db6c98038d52218af1f
9220a0fdd0f3dc2c8dd7cbeefac7d11106451b51 tests: Add one specialized ProcessMessage(...) fuzzing binary per message type for optimal results when using coverage-guided fuzzing (practicalswift)
fd1dae10b4a549ba9292d837235d59bd9eebbed3 tests: Add fuzzing harness for ProcessMessage(...) (practicalswift)
Pull request description:
Add fuzzing harness for `ProcessMessage(...)`. Enables high-level fuzzing of the P2P layer.
All code paths reachable from this fuzzer can be assumed to be reachable for an untrusted peer.
Seeded from thin air (an empty corpus) this fuzzer reaches roughly 20 000 lines of code.
To test this PR:
```
$ make distclean
$ ./autogen.sh
$ CC=clang CXX=clang++ ./configure --enable-fuzz \
--with-sanitizers=address,fuzzer,undefined
$ make
$ src/test/fuzz/process_message
…
```
Worth noting about this fuzzing harness:
* To achieve a reasonable number of executions per seconds the state of the fuzzer is unfortunately not entirely reset between `test_one_input` calls. The set-up (`FuzzingSetup` ctor) and tear-down (`~FuzzingSetup`) work is simply too costly to be run on every iteration. There is a trade-off to handle here between a.) achieving high executions/second and b.) giving the fuzzer a totally blank slate for each call. Please let me know if you have any suggestion on how to improve this situation while maintaining >1000 executions/second.
* To achieve optimal results when using coverage-guided fuzzing I've chosen to create one specialised fuzzing binary per message type (`process_message_addr`, `process_message_block`, `process_message_blocktxn `, etc.) and one general fuzzing binary (`process_message`) which handles all messages types. The latter general fuzzer can be seeded with inputs generated by the former specialised fuzzers.
Happy fuzzing friends!
ACKs for top commit:
MarcoFalke:
ACK 9220a0fdd0 🏊
Tree-SHA512: c314ef12b0db17b53cbf3abfb9ecc10ce420fb45b17c1db0b34cabe7c30e453947b3ae462020b0c9f30e2c67a7ef1df68826238687dc2479cd816f0addb530e5
4444edc2e6671d3f73de3725447130f73ecf0375 ci: Enable all functional tests in valgrind (MarcoFalke)
Pull request description:
The travis timeout for our repo has been bumped to 2h, so we can run all tests in valgrind now
ACKs for top commit:
practicalswift:
ACK 4444edc2e6671d3f73de3725447130f73ecf0375 -- regarding the three disabled cases (`feature_abortnode`, `feature_block` and `rpc_bind`): not a big deal since MSan will take care of those once #18288 is merged. More is more :)
Tree-SHA512: ea2f798112911b6d1f3d88cfcdf0a7cdb698687248343703d6fe55da144542c961c15d888bffb41672c10aa76765615cb7c7ff93d468bfad3c51f962f24e7abb
