Merge #12257: [wallet] Use destination groups instead of coins in coin select

232f96f5c8 doc: Add release notes for -avoidpartialspends (Karl-Johan Alm)
e00b4699cc clean-up: Remove no longer used ivars from CInputCoin (Karl-Johan Alm)
43e04d13b1 wallet: Remove deprecated OutputEligibleForSpending (Karl-Johan Alm)
0128121101 test: Add basic testing for wallet groups (Karl-Johan Alm)
59d6f7b4e2 wallet: Switch to using output groups instead of coins in coin selection (Karl-Johan Alm)
87ebce25d6 wallet: Add output grouping (Karl-Johan Alm)
bb629cb9dc Add -avoidpartialspends and m_avoid_partial_spends (Karl-Johan Alm)
65b3eda458 wallet: Add input bytes to CInputCoin (Karl-Johan Alm)
a443d7a0ca moveonly: CoinElegibilityFilter into coinselection.h (Karl-Johan Alm)
173e18a289 utils: Add insert() convenience templates (Karl-Johan Alm)

Pull request description:

  This PR adds an optional (off by default) `-avoidpartialspends` flag, which changes coin select to use output groups rather than outputs, where each output group corresponds to all outputs with the same destination.

  It is a privacy improvement, as each time you spend some output, any other output that is publicly associated with the destination (address) will also be spent at the same time, at the cost of fee increase for cases where coin select without group restriction would find a more optimal set of coins (see example below).

  For regular use without address reuse, this PR should have no effect on the user experience whatsoever; it only affects users who, for some reason, have multiple outputs with the same destination (i.e. address reuse).

  Nodes with this turned off will still try to avoid partial spending, if the fee of the resulting transaction is not greater than the fee of the original transaction.

  Example: a node has four outputs linked to two addresses `A` and `B`:

  * 1.0 btc to `A`
  * 0.5 btc to `A`
  * 1.0 btc to `B`
  * 0.5 btc to `B`

  The node sends 0.2 btc to `C`. Without `-avoidpartialspends`, the following coin selection will occur:
  * 0.5 btc to `A` or `B` is picked
  * 0.2 btc is output to `C`
  * 0.3 - fee is output to (unique change address)

  With `-avoidpartialspends`, the following will instead happen:
  * Both of (0.5, 1.0) btc to `A` or `B` is picked (one or the other pair)
  * 0.2 btc is output to `C`
  * 1.3 - fee is output to (unique change address)

  As noted, the pro here is that, assuming nobody sends to the address after you spend from it, you will only ever use one address once. The con is that the transaction becomes slightly larger in this case, because it is overpicking outputs to adhere to the no partial spending rule.

  This complements #10386, in particular it addresses @luke-jr and @gmaxwell's concerns in https://github.com/bitcoin/bitcoin/pull/10386#issuecomment-300667926 and https://github.com/bitcoin/bitcoin/pull/10386#issuecomment-302361381.

  Together with `-avoidreuse`, this fully addresses the concerns in #10065 I believe.

Tree-SHA512: 24687a4490ba59cf4198ed90052944ff4996653a4257833bb52ed24d058b3e924800c9b3790aeb6be6385b653b49e304453e5d7ff960e64c682fc23bfc447621

test/functional/test_runner.py

@@ -105,6 +105,7 @@ BASE_SCRIPTS = [
     'rpc_users.py',
     'feature_proxy.py',
     'rpc_signrawtransaction.py',
+    'wallet_groups.py',
     'p2p_disconnect_ban.py',
     'rpc_decodescript.py',
     'rpc_blockchain.py',

test/functional/wallet_groups.py

@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+# Copyright (c) 2018 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test wallet group functionality."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import (
+    assert_equal,
+)
+
+def assert_approx(v, vexp, vspan=0.00001):
+    if v < vexp - vspan:
+        raise AssertionError("%s < [%s..%s]" % (str(v), str(vexp - vspan), str(vexp + vspan)))
+    if v > vexp + vspan:
+        raise AssertionError("%s > [%s..%s]" % (str(v), str(vexp - vspan), str(vexp + vspan)))
+
+class WalletGroupTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 3
+        self.extra_args = [[], [], ['-avoidpartialspends']]
+
+    def run_test (self):
+        # Mine some coins
+        self.nodes[0].generate(110)
+
+        # Get some addresses from the two nodes
+        addr1 = [self.nodes[1].getnewaddress() for i in range(3)]
+        addr2 = [self.nodes[2].getnewaddress() for i in range(3)]
+        addrs = addr1 + addr2
+
+        # Send 1 + 0.5 coin to each address
+        [self.nodes[0].sendtoaddress(addr, 1.0) for addr in addrs]
+        [self.nodes[0].sendtoaddress(addr, 0.5) for addr in addrs]
+
+        self.nodes[0].generate(1)
+        self.sync_all()
+
+        # For each node, send 0.2 coins back to 0;
+        # - node[1] should pick one 0.5 UTXO and leave the rest
+        # - node[2] should pick one (1.0 + 0.5) UTXO group corresponding to a
+        #   given address, and leave the rest
+        txid1 = self.nodes[1].sendtoaddress(self.nodes[0].getnewaddress(), 0.2)
+        tx1 = self.nodes[1].getrawtransaction(txid1, True)
+        # txid1 should have 1 input and 2 outputs
+        assert_equal(1, len(tx1["vin"]))
+        assert_equal(2, len(tx1["vout"]))
+        # one output should be 0.2, the other should be ~0.3
+        v = [vout["value"] for vout in tx1["vout"]]
+        v.sort()
+        assert_approx(v[0], 0.2)
+        assert_approx(v[1], 0.3, 0.0001)
+
+        txid2 = self.nodes[2].sendtoaddress(self.nodes[0].getnewaddress(), 0.2)
+        tx2 = self.nodes[2].getrawtransaction(txid2, True)
+        # txid2 should have 2 inputs and 2 outputs
+        assert_equal(2, len(tx2["vin"]))
+        assert_equal(2, len(tx2["vout"]))
+        # one output should be 0.2, the other should be ~1.3
+        v = [vout["value"] for vout in tx2["vout"]]
+        v.sort()
+        assert_approx(v[0], 0.2)
+        assert_approx(v[1], 1.3, 0.0001)
+
+if __name__ == '__main__':
+    WalletGroupTest().main ()