Commit Graph

23800 Commits

Author SHA1 Message Date
6fb4bbfb0e Merge #18176: tests: Add fuzzing harness for CScript and CScriptNum operations
e37f53648e Make lifetime correctness easier to see (avoid reference lifetime extension) (practicalswift)
e7ddbd9893 tests: Add fuzzing harness for CScriptNum operations (practicalswift)
65a52a0024 tests: Add fuzzing harness for CScript operations (practicalswift)
eb7c50ca1f tests: Add common Consume* fuzzing functions (practicalswift)

Pull request description:

  Add fuzzing harness for `CScript` and `CScriptNum` operations.

  Test this PR using:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/script_ops
  …
  $ src/test/fuzz/scriptnum_ops
  …
  ```

ACKs for top commit:
  MarcoFalke:
    ACK e37f53648e 🦂

Tree-SHA512: 5165d918ffe3f1e3e85ab0e61d8b05934f682d324cf63ce188da5890899df2b5727aba9ed10c0437260ecff8055250e60c79d81d764bc740a7652d543a7c5fa3
2020-03-09 23:01:31 -04:00
e37f53648e Make lifetime correctness easier to see (avoid reference lifetime extension) 2020-03-09 20:39:48 +00:00
e7ddbd9893 tests: Add fuzzing harness for CScriptNum operations 2020-03-09 20:39:48 +00:00
ccb2c9e789 Merge #18306: test: add logging to wallet_listsinceblock.py
d484279a46 test: add logging to wallet_listsinceblock.py (Jon Atack)

Pull request description:

  This is the first commit from #17535.

Top commit has no ACKs.

Tree-SHA512: bb4f527a41bca3ffbf69e910311ce7f85dcc7a2be41350b3c653a27f4044f392b7e528f330e9691f497212469f6b16ce263230bb7a919548dd4e3e21cc72142f
2020-03-09 16:16:21 -04:00
d484279a46 test: add logging to wallet_listsinceblock.py 2020-03-09 21:12:03 +01:00
dcf2ccbfde Merge #18115: wallet: Pass in transactions and messages for signing instead of exporting the private keys
d2774c09cf Clear any input_errors for an input after it is signed (Andrew Chow)
dc174881ad Replace GetSigningProvider with GetSolvingProvider (Andrew Chow)
6a9c429084 Move direct calls to MessageSign into new SignMessage functions in CWallet and ScriptPubKeyMan (Andrew Chow)
82a30fade7 Move key and script filling and signing from CWallet::FillPSBT to ScriptPubKeyMan::FillPSBT (Andrew Chow)
3d70dd99f9 Move FillPSBT to be a member of CWallet (Andrew Chow)
a4af324d15 Use CWallet::SignTransaction in CreateTransaction and signrawtransactionwithwallet (Andrew Chow)
f37de92744 Implement CWallet::SignTransaction using ScriptPubKeyMan::SignTransaction (Andrew Chow)
d999dd588c Add SignTransaction function to ScriptPubKeyMan and LegacyScriptPubKeyMan (Andrew Chow)
2c52b59d0a Refactor rawtransaction's SignTransaction into generic SignTransaction function (Andrew Chow)

Pull request description:

  Following #17261, the way to sign transactions, PSBTs, and messages was to use `GetSigningProvider()` and get a `SigningProvider` containing the private keys. However this may not be feasible for future `ScriptPubKeyMan`s, such as for hardware wallets. Instead of exporting a `SigningProvider` containing private keys, we need to pass these things into the `ScriptPubKeyMan` (via `CWallet`) so that they can do whatever is needed internally to sign them. This is largely a refactor as the logic of processing transactions, PSBTs, and messages for is moved into `LegacyScriptPubKeyMan` and `CWallet` instead of being handled by the caller (e.g. `signrawtransaction`).

  To help with this, I've refactored the 3(!) implementations of a `SignTransaction()` function into one generic one. This function will be called by `signrawtransactionwithkey` and `LegacyScriptPubKeyMan::SignTransaction()`. `CWallet::CreateTransaction()` is changed to call `CWallet::SignTransaction()` which in turn, calls `LegacyScriptPubKeyMan::SignTransaction()`. Other `ScriptPubKeyMan`s may implement `SignTransaction()` differently.

  `FillPSBT()` is moved to be a member function of `CWallet` and the `psbtwallet.cpp/h` files removed. It is further split so that `CWallet` handles filling the UTXOs while the `ScriptPubKeyMan` handles adding keys, derivation paths, scripts, and signatures. In the end `LegacyScriptPubKeyMan::FillPSBT` still calls `SignPSBTInput`, but the `SigningProvider` is internal to `LegacyScriptPubKeyMan`. Other `ScriptPubKeyMan`s may do something different.

  A new `SignMessage()` function is added to both `CWallet` and `ScriptPubKeyMan`. Instead of having the caller (i.e. `signmessage` or the sign message dialog) get the private key, hash the message, and sign, `ScriptPubKeyMan` will now handle that (`CWallet` passes through to the `ScriptPubKeyMan`s as it does for many functions). This signing code is thus consolidated into `LegacyScriptPubKeyMan::SignMessage()`, though other `ScriptPubKeyMan`s may implement it differently. Additionally, a `SigningError` enum is introduced for the different errors that we expect to see from `SignMessage()`.

  Lastly, `GetSigningProvider()` is renamed to `GetPublicSigningProvider()`. It will now only provide pubkeys, key origins, and scripts. `LegacySigningProvider` has it's `GetKey` and `HaveKey` functions changed to only return false. Future implementations should return `HidingSigningProvider`s where private keys are hidden.

  Other things like `dumpprivkey` and `dumpwallet` are not changed because they directly need and access the `LegacyScriptPubKeyMan` so are not relevant to future changes.

ACKs for top commit:
  instagibbs:
    reACK d2774c09cf
  Sjors:
    re-utACK d2774c09cf
  meshcollider:
    re-utACK d2774c09cf

Tree-SHA512: 89c83e7e7e9315e283fae145a2264648a9d7f7ace8f3281cb3f44f0b013c988d67ba4fa9726e50c643c0ed921bdd269adaec984840d11acf4a681f3e8a582cc1
2020-03-10 09:02:12 +13:00
ffdde475b0 Merge #18303: doc: Add historical release notes for 0.19.1
46a1ee7732 doc: Add historical release notes for 0.19.1 (Wladimir J. van der Laan)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: 9ff578fbaec5694d63256d65cbe7e9d0ba99ac50803c8192f9b5e90a614dc89778bdb17b03f5f7725397f820e085e7d704deb1d6a58b18cd4e4fc91ea413b6ae
2020-03-09 20:54:19 +01:00
65a52a0024 tests: Add fuzzing harness for CScript operations 2020-03-09 19:24:50 +00:00
eb7c50ca1f tests: Add common Consume* fuzzing functions 2020-03-09 19:24:50 +00:00
5518eeec27 Merge #18047: tests: Add basic fuzzing harness for CNetAddr/CService/CSubNet related functions (netaddress.h)
6590395f60 tests: Remove FUZZERS_MISSING_CORPORA (practicalswift)
815c7a6793 tests: Add basic fuzzing harness for CNetAddr/CService/CSubNet related functions (netaddress.h) (practicalswift)

Pull request description:

  Add basic fuzzing harness for `CNetAddr`/`CService`/`CSubNet` related functions (`netaddress.h`).

  To test this PR:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/netaddress
  …
  ```

Top commit has no ACKs.

Tree-SHA512: 69dc0e391d56d5e9cdb818ac0ac4b69445d0195f714442a06cf662998e38b6e0bbaa635dce78df37ba797feed633e94abba4764b946c1716d392756e7809112d
2020-03-09 13:53:46 -04:00
c1e0742308 doc: Warn about wallet.dat re-use and backups
See discussion in #18205 and https://github.com/bitcoin/bitcoin/pull/18219#issuecomment-594871230.
2020-03-10 01:51:03 +08:00
46a1ee7732 doc: Add historical release notes for 0.19.1 2020-03-09 18:41:39 +01:00
6590395f60 tests: Remove FUZZERS_MISSING_CORPORA 2020-03-09 17:20:52 +00:00
be00ef2aca Merge #18292: fuzz: Add assert(script == decompressed_script)
fab0e5ba7f fuzz: Add assert(script == decompressed_script) (MarcoFalke)

Pull request description:

  Presumably an oversight in https://github.com/bitcoin/bitcoin/pull/17926#discussion_r389262151

ACKs for top commit:
  practicalswift:
    Tested ACK fab0e5ba7f

Tree-SHA512: 6dcec06169df497a540fd6ebbcd89f5db22257241b2bbe756de868742f9bc324b80d38dbababfa07e5f3a830aaae9fc6d168dcc2ca5d75da437bdf4dc4e0f370
2020-03-09 11:19:29 -04:00
815c7a6793 tests: Add basic fuzzing harness for CNetAddr/CService/CSubNet related functions (netaddress.h) 2020-03-09 15:16:36 +00:00
d2774c09cf Clear any input_errors for an input after it is signed
Make sure that there are no errors set for an input after it is signed.
This is useful for when there are multiple ScriptPubKeyMans. Some may
fail to sign, but one may be able to sign, and after it does, we don't
want there to be any more errors there.
2020-03-09 11:16:20 -04:00
dc174881ad Replace GetSigningProvider with GetSolvingProvider
Not all ScriptPubKeyMans will be able to provide private keys,
but pubkeys and scripts should be. So only provide public-only
SigningProviders, i.e. ones that can help with Solving.
2020-03-09 11:16:20 -04:00
6a9c429084 Move direct calls to MessageSign into new SignMessage functions in CWallet and ScriptPubKeyMan
Instead of getting a SigningProvider and then going to MessageSign,
have ScriptPubKeyMan handle the message signing internally.
2020-03-09 11:16:20 -04:00
82a30fade7 Move key and script filling and signing from CWallet::FillPSBT to ScriptPubKeyMan::FillPSBT
Instead of fetching a SigningProvider from ScriptPubKeyMan in order
to fill and sign the keys and scripts for a PSBT, just pass that
PSBT to a new FillPSBT function that does all that for us.
2020-03-09 11:16:20 -04:00
3d70dd99f9 Move FillPSBT to be a member of CWallet 2020-03-09 11:16:17 -04:00
aaaa055ff7 fuzz: Add option to merge input dir to test runner 2020-03-09 11:13:57 -04:00
fa4fa88d76 doc: Remove --disable-ccache from docs 2020-03-09 11:13:48 -04:00
31b2ce70fb Merge #18299: tests: Update FUZZERS_MISSING_CORPORA to enable regression fuzzing for all harnesses in master
bf06641819 tests: Reset FUZZERS_MISSING_CORPORA to enable regression fuzzing for more harnesses (practicalswift)

Pull request description:

  Update `FUZZERS_MISSING_CORPORA` to enable regression fuzzing for all harnesses in `master`.

  Note that this PR is expected to fail in Travis until https://github.com/bitcoin-core/qa-assets/pull/4 is merged :)

Top commit has no ACKs.

Tree-SHA512: 6b4d23726ca99c7ab9454087f0354e35d307e7e16b90741cdf7838265a07a631cf54d9533b4f1bf9a0fd900c5ed909756a35cab738284bf39b09ab1c2ed54368
2020-03-09 11:05:52 -04:00
bf06641819 tests: Reset FUZZERS_MISSING_CORPORA to enable regression fuzzing for more harnesses 2020-03-09 14:30:18 +00:00
6ddf435493 Merge #18274: rpc/wallet: initialize nFeeRequired to avoid using garbage value on failure
a652ba6293 rpc/wallet: initialize nFeeRequired to avoid using garbage value on failure (Karl-Johan Alm)

Pull request description:

  Initialize the `nFeeRequired` variable to avoid using an uninitialized value for errors happening before it is set to 0.

  Note: this originally fixed `nFeeRet` in `wallet.cpp`.

ACKs for top commit:
  promag:
    ACK a652ba6293.
  Sjors:
    utACK a652ba6293
  practicalswift:
    ACK a652ba6293 -- patch looks correct
  meshcollider:
    utACK a652ba6293

Tree-SHA512: 0d12f1ffd0851ed5ce6d109d2c87f55e8b1d57da297e684feeabb57229200c4078f029c55ca5aa5712bd18e26dda3ce538443dfe68a7a6d504428068f81fded0
2020-03-09 19:17:51 +08:00
a652ba6293 rpc/wallet: initialize nFeeRequired to avoid using garbage value on failure 2020-03-09 10:20:41 +09:00
a4af324d15 Use CWallet::SignTransaction in CreateTransaction and signrawtransactionwithwallet
Instead of duplicating signing code, just use the function we already
have.
2020-03-08 12:27:05 -04:00
f37de92744 Implement CWallet::SignTransaction using ScriptPubKeyMan::SignTransaction 2020-03-08 12:26:32 -04:00
d999dd588c Add SignTransaction function to ScriptPubKeyMan and LegacyScriptPubKeyMan 2020-03-08 12:26:32 -04:00
2c52b59d0a Refactor rawtransaction's SignTransaction into generic SignTransaction function 2020-03-08 12:26:27 -04:00
fab0e5ba7f fuzz: Add assert(script == decompressed_script) 2020-03-07 16:55:34 -05:00
5e12a61044 Merge #17926: tests: Add key_io fuzzing harness. Fuzz additional functions in existing fuzzing harnesses.
52fed696d2 tests: Fuzz additional functions in the script fuzzing harness (practicalswift)
5fc10f3cb5 tests: Fuzz additional functions in the transaction fuzzing harness (practicalswift)
1d324ce922 tests: Fuzz additional functions in the integer fuzzing harness (practicalswift)
4fe4de6364 tests: Fuzz additional functions in the hex fuzzing harness (practicalswift)
c7ea12d098 tests: Add key_io fuzzing harness (practicalswift)

Pull request description:

  Add `key_io` fuzzing harness.

  Fuzz additional functions in the `hex` fuzzing harness.

  Fuzz additional functions in the `integer` fuzzing harness.

  Fuzz additional functions in the `script` fuzzing harness.

  Fuzz additional functions in the `transaction` fuzzing harness.

  **How to test this PR**

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/key_io
  …
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 52fed696d2 🛫

Tree-SHA512: a57ab66c18d260c2e39d987cab9fa576f7a5520dc1ea7fd607d64d8e005e16558312ddb4c9f4d4f3147dc6194d8ae0b0fb86ed5e58ba6aef5383ea726463df97
2020-03-07 11:19:52 -05:00
ddc7e42d60 build: Set minimum Automake version to 1.13 2020-03-07 17:38:06 +02:00
09e25071f4 Cache parent xpub inside of BIP32PubkeyProvider
Optimize Expand by having BIP32PubkeyProvider also cache the parent
(or only) xpub within itself. Since Expand does not provide a read
cache, it is useful to internally cache this xpub to avoid re-deriving
the same xpub.
2020-03-07 10:13:47 -05:00
deb791c7ba Only cache xpubs that have a hardened last step
Also adds tests for this:
For ranged descriptors with unhardened derivation, we expect to
find parent keys in the cache but no child keys.

For descriptors containing an xpub but do not have unhardened derivation
(i.e. hardened derivation or single xpub with or without derivation),
we expect to find all of the keys in the cache, and the same
number of keys in the cache as in the SigningProvider.

For everything else (no xpub), nothing should be cached at all.
2020-03-07 10:13:47 -05:00
f76733eda5 Cache the immediate derivation parent xpub
If unhardened derivation is used, cache the immediate derivation
parent xpub and use it for unhardened derivation
2020-03-07 10:13:47 -05:00
58f54b686f Add DescriptorCache* read_cache and DescriptorCache* write_cache to Expand and GetPubKey
Have Expand, ExpandFromCache, and ExpandHelper take additional DescriptorCache
parameters. These are then passed into PubkeyProvider::GetPubKey which
also takes them as arguments.

Reading and writing to the cache is pushed down into GetPubKey. The old cache where
pubkeys are serialized to a vector is completely removed and instead xpubs are being
cached in DescriptorCache.
2020-03-07 10:13:47 -05:00
66c2cadc91 Rename BIP32PubkeyProvider.m_extkey to m_root_extkey
Renaming clarifies that m_extkey is actually the root
extkey that keys are derived from.
2020-03-07 10:13:47 -05:00
df55d44d0d Track the index of the key expression in PubkeyProvider 2020-03-07 10:13:47 -05:00
474ea3b927 Introduce DescriptorCache struct which caches xpubs 2020-03-07 10:13:43 -05:00
52fed696d2 tests: Fuzz additional functions in the script fuzzing harness 2020-03-07 14:35:49 +00:00
5fc10f3cb5 tests: Fuzz additional functions in the transaction fuzzing harness 2020-03-07 14:35:49 +00:00
fa70ccc6c4 scheduler: Use C++11 member initialization, add shutdown assert
"Initializing the members in the declaration makes it easy to spot
uninitialized ones".
https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#c-data-structures
2020-03-07 08:58:38 -05:00
1d324ce922 tests: Fuzz additional functions in the integer fuzzing harness 2020-03-07 13:40:19 +00:00
4fe4de6364 tests: Fuzz additional functions in the hex fuzzing harness 2020-03-07 13:39:25 +00:00
c7ea12d098 tests: Add key_io fuzzing harness 2020-03-07 13:39:25 +00:00
ad345909b2 doc: remove Boost Chrono from install docs 2020-03-07 08:34:31 +08:00
e21fa542b1 test: remove Boost Chrono installation from CI 2020-03-07 08:34:31 +08:00
bd37f2bc26 build: remove Boost Chrono detection from build system 2020-03-07 08:34:31 +08:00
1d0a87e712 build: remove chrono package from depends Boost 2020-03-07 08:34:30 +08:00