4c9666bd73645b94ae81be1faf7a0b8237dd6e99 Mention `mempoolfullrbf` in policy/mempool-replacements.md (Antoine Riard)
aae66ab43d794bdfaa2dade91760cc55861c9693 Update getmempoolinfo RPC with `mempoolfullrbf` (Antoine Riard)
3e27e317270fdc2dd02794fea9da016387699636 Introduce `mempoolfullrbf` node setting. (Antoine Riard)
Pull request description:
This is ready for review.
Recent discussions among LN devs have brought back on the surface concerns about the security of multi-party funded transactions against pinnings attacks and other mempool-based nuisances. The lack of full-rbf transaction-relay topology connected to miners open the way to cheap and naive DoS against multi-party funded transactions (e.g coinjoins, dual-funded channels, on-chain DLCs, ...) without solutions introducing an overhead cost or centralization vectors afaik . For more details, see [0].
This PR implements a simple `fullrbf` setting, where the node always allows transaction replacement, ignoring BIP125 opt-in flag. The default value of the setting stays **false**, therefore opt-in replacement is still the default Bitcoin Core replacement policy. Contrary to a previous proposal of mine and listening to feedbacks collected since then [1], I think this new setting simply offers more flexibility in a node transaction-relay policy suiting one's application requirements, without arguing a change of the default behavior.
I [posted](https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-June/020557.html) on the ML to invite operators with a bitcoin application sensitive to full-rbf (e.g dual-funded LN channels service providers) or mempool researchers to join a bootstrapped full-rbf activated peers network for experimentation and learning. If people have strong opinions against the existence of such full-rbf transaction-relay network, I'm proposing to express them on the future thread.
[0] https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-June/019074.html
Follow-up suggestions :
- soft-enable opt-in RBF in the wallet : https://github.com/bitcoin/bitcoin/pull/25353#issuecomment-1154918789
- p2p discovery and additional outbound connection to full-rbf peers : https://github.com/bitcoin/bitcoin/pull/25353#issuecomment-1156044401
- match the code between RPC, wallet and mempool about disregard of inherited signaling : #22698
ACKs for top commit:
instagibbs:
reACK 4c9666bd73
glozow:
ACK 4c9666bd73645b94ae81be1faf7a0b8237dd6e99, a few nits which are non-blocking.
w0xlt:
ACK 4c9666bd73
Tree-SHA512: 9e288bf22e06a9808804e58178444ef1830c3fdd42fd8a7cd7ffb101f8f586e08b000679be407d63ca76a56f7216227b368ff630c81f3fac3243db1a1202ab1c
This change eliminates memory usage spike when compiling with Visual
Studio 2022 (at least in Cirrus CI environment).
Easy to review using
`git diff --color-moved-ws=allow-indentation-change --color-moved=dimmed-zebra`
This change eliminates memory usage spike when compiling with Visual
Studio 2022 (at least in Cirrus CI environment).
Easy to review using
`git diff --color-moved-ws=allow-indentation-change --color-moved=dimmed-zebra`
This change eliminates memory usage spike when compiling with Visual
Studio 2022 (at least in Cirrus CI environment).
Easy to review using
`git diff --color-moved-ws=allow-indentation-change --color-moved=dimmed-zebra`
e734228d8585c0870c71ce8ba8c037f8cf8b249a Update GCSFilter benchmarks (Calvin Kim)
aee9a8140b3a58b744766f9e89572f1d953a808b Add GCSFilterDecodeSkipCheck benchmark (Patrick Strateman)
299023c1d9962628d158fac0306f8531506a0123 Add GCSFilterDecode and GCSBlockFilterGetHash benchmarks. (Patrick Strateman)
b0a53d50d9142bed51a8372eeb848816bfa94da8 Make sanity check in GCSFilter constructor optional (Patrick Strateman)
Pull request description:
This PR picks up the abandoned #19280
BlockFilterIndex was depending on `GolombRiceDecode()` during the filter decode to sanity check that the filter wasn't corrupt. However, we can check for corruption by ensuring that the encoded blockfilter's hash matches up with the one stored in the index database.
Benchmarks that were added in #19280 showed that checking the hash is much faster.
The benchmarks were changed to nanobench and the relevant benchmarks were like below, showing a clear win for the hash check method.
```
| ns/elem | elem/s | err% | ins/elem | bra/elem | miss% | total | benchmark
|--------------------:|--------------------:|--------:|----------------:|---------------:|--------:|----------:|:----------
| 531.40 | 1,881,819.43 | 0.3% | 3,527.01 | 411.00 | 0.2% | 0.01 | `DecodeCheckedGCSFilter`
| 258,220.50 | 3,872.66 | 0.1% | 2,990,092.00 | 586,706.00 | 1.7% | 0.01 | `DecodeGCSFilter`
| 13,036.77 | 76,706.09 | 0.3% | 64,238.24 | 513.04 | 0.2% | 0.01 | `BlockFilterGetHash`
```
ACKs for top commit:
mzumsande:
Code Review ACK e734228d8585c0870c71ce8ba8c037f8cf8b249a
theStack:
Code-review ACK e734228d8585c0870c71ce8ba8c037f8cf8b249a
stickies-v:
ACK e734228d8585c0870c71ce8ba8c037f8cf8b249a
ryanofsky:
Code review ACK e734228d8585c0870c71ce8ba8c037f8cf8b249a, with caveat that I mostly paid attention to the main code, not the changes to the benchmark. Only changes since last review were changes to the benchmark code.
Tree-SHA512: 02b86eab7b554e1a57a15b17a4d6d71faa91b556c637b0da29f0c9ee76597a110be8e3b4d0c158d4cab04af0623de18b764837be0ec2a72afcfe1ad9c78a83c6
0101d2bc3c3bcf698d6cc2a237a680fc52395987 [net] Move eviction logic to its own file (dergoegge)
c741d748d4d9836940b99091cc7be09c65efcb79 [net] Move ConnectionType to its own file (Cory Fields)
a3c27070396ab8c2941c437e8099547e8fc9c110 [net] Add connection type to NodeEvictionCandidate (dergoegge)
42aa5d5b6269d27af525d5001907558442e96023 [net] Add NoBan status to NodeEvictionCandidate (dergoegge)
Pull request description:
This PR splits of the first couple commits from #25268 that move the inbound eviction logic from `net.{h,cpp}` to `eviction.{h,cpp}`.
Please look at #25268 for motivation and conceptual review.
ACKs for top commit:
jnewbery:
utACK 0101d2bc3c3bcf698d6cc2a237a680fc52395987
theuni:
utACK 0101d2bc3c3bcf698d6cc2a237a680fc52395987. I quickly verified with `git --color-moved` that the move-only changes are indeed move-only.
Tree-SHA512: e0c345a698030e049cb22fe281b44503c04403c5be5a3750ca14bfcc603a162ac6bac9a39552472feb57c460102b7ca91430b8ad6268f2efccc49b5e8959331b
ebe106a754d2da567702e60185142d9e381bf1cd add glozow to trusted-keys (glozow)
Pull request description:
For maintaining mempool and policy areas of the codebase, as discussed yesterday's meeting: https://gnusha.org/bitcoin-core-dev/2022-06-30.log
ACKs for top commit:
Sjors:
ACK ebe106a754d2da567702e60185142d9e381bf1cd and congrats!
achow101:
ACK ebe106a754d2da567702e60185142d9e381bf1cd
theuni:
ACK ebe106a754d2da567702e60185142d9e381bf1cd
dergoegge:
ACK ebe106a754d2da567702e60185142d9e381bf1cd
sipa:
ACK ebe106a754d2da567702e60185142d9e381bf1cd, though relying on others to verify the PGP key.
laanwj:
ACK ebe106a754d2da567702e60185142d9e381bf1cd
josibake:
ACK ebe106a754 (i have not personally signed this key, but verified it matches other places glozow has posted her key)
Xekyo:
ACK ebe106a754d2da567702e60185142d9e381bf1cd.
brunoerg:
ACK ebe106a754d2da567702e60185142d9e381bf1cd
fanquake:
ACK ebe106a754d2da567702e60185142d9e381bf1cd
hebasto:
ACK ebe106a754d2da567702e60185142d9e381bf1cd, confirming my approval given at the IRC meeting.
Tree-SHA512: 215ff8872ea3fa9ca35b25e74a668e50c2e7be3ff653f8d6fd213ac878c33b90bba9defc59a000c644a9df1b06a826eb5d97a89b3c6cca24b5a87c6ab739b01b
5bff18bce5d9d8bd1bae0cb89facf73c829c947b guix: patch gcc 10 with pthreads to remap guix store paths (Andrew Chow)
Pull request description:
The only thing preventing windows from being cross architecture reproducible is a single guix store winpthreads path in the debug symbols. This can be removed by patching libgcc to use `-ffile-prefix-map` so that the debug symbol will be mapped to a fixed `/usr` instead of the guix store path which depends on the building architecture.
x86_64
```
2e585c4a66e930b5e273e89b8aeddc9c3bd1c8375b19d988a6fff64f0d49edfd guix-build-5bff18bce5d9/output/dist-archive/bitcoin-5bff18bce5d9.tar.gz
b9235dc1a8541e840231cfafd0d971bd5e8a3ea7d5331c4d7af9dbfdabc6905b guix-build-5bff18bce5d9/output/x86_64-w64-mingw32/SHA256SUMS.part
f82d861de60e22fc7dd731bef60a3e4399b5317eb16e41e92ded171490d1a578 guix-build-5bff18bce5d9/output/x86_64-w64-mingw32/bitcoin-5bff18bce5d9-win64-debug.zip
bfd59561c3cfce91b09d05b17cfc67cd70cb78eea39ea863119870260a8dbdec guix-build-5bff18bce5d9/output/x86_64-w64-mingw32/bitcoin-5bff18bce5d9-win64-setup-unsigned.exe
3d049d98c6add13b0eb4c7adcf0d3ae59d1eab09799292a2c900de0ad067912a guix-build-5bff18bce5d9/output/x86_64-w64-mingw32/bitcoin-5bff18bce5d9-win64-unsigned.tar.gz
7af4c34c47f349028ec1f4c2edea547bd9fa30d1c67977d482607a9c6bf2ddee guix-build-5bff18bce5d9/output/x86_64-w64-mingw32/bitcoin-5bff18bce5d9-win64.zip
```
arm64
```
2e585c4a66e930b5e273e89b8aeddc9c3bd1c8375b19d988a6fff64f0d49edfd guix-build-5bff18bce5d9/output/dist-archive/bitcoin-5bff18bce5d9.tar.gz
b9235dc1a8541e840231cfafd0d971bd5e8a3ea7d5331c4d7af9dbfdabc6905b guix-build-5bff18bce5d9/output/x86_64-w64-mingw32/SHA256SUMS.part
f82d861de60e22fc7dd731bef60a3e4399b5317eb16e41e92ded171490d1a578 guix-build-5bff18bce5d9/output/x86_64-w64-mingw32/bitcoin-5bff18bce5d9-win64-debug.zip
bfd59561c3cfce91b09d05b17cfc67cd70cb78eea39ea863119870260a8dbdec guix-build-5bff18bce5d9/output/x86_64-w64-mingw32/bitcoin-5bff18bce5d9-win64-setup-unsigned.exe
3d049d98c6add13b0eb4c7adcf0d3ae59d1eab09799292a2c900de0ad067912a guix-build-5bff18bce5d9/output/x86_64-w64-mingw32/bitcoin-5bff18bce5d9-win64-unsigned.tar.gz
7af4c34c47f349028ec1f4c2edea547bd9fa30d1c67977d482607a9c6bf2ddee guix-build-5bff18bce5d9/output/x86_64-w64-mingw32/bitcoin-5bff18bce5d9-win64.zip
```
ACKs for top commit:
fanquake:
ACK 5bff18bce5d9d8bd1bae0cb89facf73c829c947b
hebasto:
ACK 5bff18bce5d9d8bd1bae0cb89facf73c829c947b, I have reviewed the code and it looks OK. Confirming reproducibility for `x86_64` and `arm64` platforms.
Tree-SHA512: 7cc34e6348e4cab847a7b8745179fceced0f37d639cf2ae81748dd73820809ea8f5e049b5b3ce2b912528491967e33fafd56e75aa47714e09b41859091433c5d
fad690ba0a62dfd97ef22711b2344909fcd1fb73 test: Remove -acceptnonstdtxn=1 from feature_rbf.py (MacroFake)
fa5059b7dfba6d95c14a12f21d02b63fe29b3f2b test: Make the scriptPubKey of MiniWallet created txs mutable (MacroFake)
fa2924582726ee00b392bd0b5591e7d9770e1b90 test: Allow setting sequence per input in MiniWallet create_self_transfer_multi (MacroFake)
fac3800d2c962420303ab5c61b2f02f35b9f693a test: Allow amount_per_output in MiniWallet create_self_transfer_multi (MacroFake)
2222842ae73f85494797b14753bc18446e4817a2 test: Allow absolute fee in MiniWallet create_self_transfer (MacroFake)
Pull request description:
On the main network, nonstandard transactions are hardly relayed, so it seems odd that one of our policy test requires a policy setting opposite of the norm.
Surely it is also important to test that nonstandard transactions can be replaced. However, rbf code should not care about the standardness at all. Moreover, I think testing nonstandardness rbf is of lower priority than testing the stuff that actually happens in production.
ACKs for top commit:
theStack:
re-ACK fad690ba0a62dfd97ef22711b2344909fcd1fb73
jamesob:
ACK fad690ba0a62dfd97ef22711b2344909fcd1fb73 ([`jamesob/ackr/25522.1.MarcoFalke.test_remove_acceptnonstd`](https://github.com/jamesob/bitcoin/tree/ackr/25522.1.MarcoFalke.test_remove_acceptnonstd))
Tree-SHA512: e0a0c808bccdddf738fb6a84e5e5672d7c341bffd941c4f0c232112bfc68265fa81a2e42ddcab107d586358ffdb3dccc46bb5533d46999fd9ab024169dac0f78
6eb0909cb7d5883a258f76ad6cf2c989fc6f892f fuzz: add low-level target for txorphanage (chinggg)
Pull request description:
This adds a low-level fuzz target for orphan transaction handling by creating random transactions and calling all functions in `TxOrphanage`.
It cannot simulate real-world `orphan/unorphan` scenarios effectively since it does not maintain any state about the node and the chain. A high-level fuzz target which construct well-designed transaction graphs will be added later.
ACKs for top commit:
MarcoFalke:
review ACK 6eb0909cb7d5883a258f76ad6cf2c989fc6f892f 🐈
Tree-SHA512: b4d64f5941df77d13981f75ec170cef6ffabe782797c982ede7f34134be01dc0026dd7c0bee614bc1d64715e90a933d2a8c95974d402e32eaba8e24cc928299e
eac1099e0071bfe11fe664a8a490eee6bbc80c47 test: remove wallet dependency from mempool_updatefromblock.py (Ayush Sharma)
Pull request description:
This PR enables one of the non-wallet functional tests(`mempool_updatefromblock.py`) to be run with the wallet disabled.
ACKs for top commit:
aureleoules:
tACK eac1099e0071bfe11fe664a8a490eee6bbc80c47.
Tree-SHA512: 9734815f2d2e65e8813bd776cf1d847a55ba4181e218c5e7b066ec69a556261069214f675681d672f5d7b0ba8e06342c4a456619dcc005cbf5618a0527303b7f
NetBSD doc has not seen any meaningful contribution since 2018.
This PR intends to update the docs so that one can successfully build on
the latest NetBSD release. It also adds dependency information and
instructions to build the GUI.
This new node policy setting enables to accept replaced-by-fee
transaction without inspection of the replaceability signaling
as described in BIP125 "explicit signaling".
If turns on, the node mempool accepts transaction replacement
as described in `policy/mempool-replacements.md`.
The default setting value is `false`, implying opt-in RBF
is enforced.
140d942634f9f1bba191aafa948df57812c0f3fe wallet: don't add change fee to target if subtracting fees from output (S3RK)
Pull request description:
Change fee is payed by the recipient, so we don't need to add it to our target for coin selection.
ACKs for top commit:
achow101:
ACK 140d942634f9f1bba191aafa948df57812c0f3fe
ishaanam:
ACK 140d942634f9f1bba191aafa948df57812c0f3fe
furszy:
Code review ACK 140d9426
Tree-SHA512: b5efd0264c47ecee9204a3fd039bad24c69f9e614c6e1d9bb240ee5be6356b175aa074f3be123e6cfb8becd4d7bd1028eebe18801662cc69d19413d8d5a9dd5c
f1c16ed733f416a3bea878f1c21621dbd93b267c doc: remove note on arm cross-compilation from build-unix.md (Jarol Rodriguez)
Pull request description:
No reason to have this here with outdated information. We already point users to the depends readme, the doc cross builders should be pointed to, within this doc.
Master: [render](https://github.com/bitcoin/bitcoin/blob/master/doc/build-unix.md)
PR: [render](f1c16ed733/doc/build-unix.md)
ACKs for top commit:
laanwj:
ACK f1c16ed733f416a3bea878f1c21621dbd93b267c
Tree-SHA512: 395c1cd5dd9168cd60d2165a9d826a3cb438b84bbe3d26c18602e7f7b7961444f2f3f6504323b2bac15c943df8c2c734c8f642f40159e88a704ec4abc3d7eeaa
No reason to have this here with outdated information. We already point
users to the depends readme, the doc cross builders should be pointed to
, within this doc.
c0a5fceee9858afd24fe0bf655b7b30728e96e78 test: Add test for erase orphan tx conflicted by block (Hennadii Stepanov)
fa45bb21193ae0c220cfc224d5e3ea0e7f3ec988 test: Add test for erase orphan tx included by block (Hennadii Stepanov)
5c049780c8b310428cf72fb304bf0c1071742785 test: Add test for erase orphan tx from peer (Hennadii Stepanov)
Pull request description:
This PR adds test coverage for the following cases:
- erase orphan transactions when a peer is disconnected
- erase an orphan transaction when it is included in a new tip block
- erase an orphan transaction when it is conflicted with other transactions included in a new tip block
Found useful while working on #19374.
ACKs for top commit:
aureleoules:
tACK c0a5fceee9858afd24fe0bf655b7b30728e96e78 (`make check` and `test/functional/test_runner.py`).
kouloumos:
ACK c0a5fceee9858afd24fe0bf655b7b30728e96e78 with a nit per https://github.com/bitcoin/bitcoin/pull/19393#discussion_r899156623.
pg156:
Reviewed to c0a5fceee9. Concept ACK. Agree due to the lack of RPC calls to inspect orphan pool, using `assert_debug_log` to match strings in log is a reasonable way to test.
Tree-SHA512: 98f8deeee2d1c588c7e28a82e513d4a18655084198369db33fe2710458251eeaffed030626940072d7576f57fcbf7d856d761990129e2ca9e372d2ccbd86d07d
Instead of calling GetCachableAmount twice, which will result in
iterating through all the transaction txins/txouts and calling
GetDebit/GetCredit (which lock cs_wallet), just merge the filters and do
it once.
cccf691c24f9cbc4aedd1b36c1d9ba173910ceca contrib: dedup `get_witness_script` helper in signet miner (Sebastian Falbesoner)
Pull request description:
The helper `get_witness_script` is already available in the `blocktools` module of our test framework, i.e. there is no need to re-implement it in the signet miner script. Note that the cast from CScript to bytes is necessary for applying the `+=` operator on the scriptPubKey later, which would fail for CScript:
```
File "/home/honeybadger/bitcoin/contrib/signet/miner", line 132, in signet_txs
txs[0].vout[-1].scriptPubKey += CScriptOp.encode_op_pushdata(SIGNET_HEADER)
File "/home/honeybadger/bitcoin/test/functional/test_framework/script.py", line 460, in __add__
raise NotImplementedError
NotImplementedError
```
ACKs for top commit:
kallewoof:
ACK cccf691c24f9cbc4aedd1b36c1d9ba173910ceca
Tree-SHA512: 5965a9f27626e3dd2769a0436263fb646e9d4b67071505122c017f7b0050250e83f524135e57093870b8c64894d64762a51d2c3c68d52dd1e545f23d4734fecb
99f4785cad94657dcf349d00fdd6f1d44cac9bb0 Replace GetTime() with NodeClock in MaybeSendGetHeaders() (Suhas Daftuar)
abf5d16c24cb08b0451bdbd4d1de63a12930e8f5 Don't send getheaders message when another request is outstanding (Suhas Daftuar)
ffe87db247b19ffb8bfba329c5dd0be39ef5a53f Cleanup received_new_header calculation to use WITH_LOCK (Suhas Daftuar)
6d95cd3e7444ebaaabb64a76783ea3551530f1d7 Move peer state updates from headers message into separate function (Suhas Daftuar)
2b341db731793844f12944363186edea23eabdeb Move headers direct fetch to end of ProcessHeadersMessage (Suhas Daftuar)
29c45185223441943ab610e62937a118c7c3a5b2 Move headers-direct-fetch logic into own function (Suhas Daftuar)
bf8ea6df75749c27f753b562c4724b3f8d263ad4 Move additional headers fetching to own function (Suhas Daftuar)
9492e93bf9f4a841bf43ca4b593871c0863d5b63 Add helper function for checking header continuity (Suhas Daftuar)
7f2450871b3ea0b4d02d56bd2ca365fcc25cf90e Move handling of unconnecting headers into own function (Suhas Daftuar)
Pull request description:
Change `getheaders` messages so that we wait up to 2 minutes for a response to a prior `getheaders` message before issuing a new one.
Also change the handling of the `getheaders` message sent in response to a block INV, so that we no longer use the hashstop variable (including the hash stop will just mean that if our peer's headers chain is longer, then we won't learn
it, so there's no benefit to using hashstop).
Also, now respond to a `getheaders` during IBD with an empty headers message (rather than nothing) -- this better conforms to the intent of the new logic that it's better to not ignore a peer's `getheaders` message, even if you have nothing to give. This also avoids a lot of functional tests breaking.
This PR also reworks the headers processing logic to make it more readable.
ACKs for top commit:
ajtowns:
ACK 99f4785cad94657dcf349d00fdd6f1d44cac9bb0 ; code review, check over new logic of when to send getheaders messages
dergoegge:
Code review ACK 99f4785cad94657dcf349d00fdd6f1d44cac9bb0
mzumsande:
Code Review ACK 99f4785cad94657dcf349d00fdd6f1d44cac9bb0
sipa:
utACK 99f4785cad94657dcf349d00fdd6f1d44cac9bb0
w0xlt:
tACK 99f4785cad Good improvement in the code.
Tree-SHA512: b8a63f6f71ac83e292edc0200def7835ad8b06b2955dd34e3ea6fac85980fa6962efd31d689ef5ea121ff5477ec14aafa4bbe2d0db134c05f4a31a57a8ced365
31346a31962ab06d49cb45aee8acd92d8806538b [ci] apply cache size limit and print ccache statistics in "ARM64 Android APK" (sogoagain)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/25475
Modified `ci/test/06_script_a.sh` file to apply cache size limit and print ccahce statistics in "ARM64 Android APK" task.
Please feel free to give me any feedback. Thanks.
ACKs for top commit:
hebasto:
ACK 31346a31962ab06d49cb45aee8acd92d8806538b, my previous [comment](https://github.com/bitcoin/bitcoin/pull/25530#issuecomment-1173822177) can be considered as a suggestion for a follow up.
Tree-SHA512: 1204fe78f90a34f0c74f256309626c6bbba0848e5f7c632ee2ca96529dc7243eb7282d83bab6c960b0c9f6ee21a49528b40c45be1e3da5958e2db83f4c00a1d2
1770be72d5eb47cae565d4ac86de5bc16f94fdd6 test: pass `dustrelayfee=0` option for tests using dust (instead of `acceptnonstdtxn=1`) (Sebastian Falbesoner)
Pull request description:
By specifying the `dustrelayfee=0` option instead of the more generic `acceptnonstdtxn=1`, we can be more specific about what
part of the transaction is non-standard and can be sure that all other aspects follow the standard policy.
Note that for the test `feature_dbcrash.py`, the UTXO creation at the start of the test has to be split up to several txs to not exceed the tx standard size limit of 100k vbytes
4129c13754/src/policy/policy.h (L26-L27)
ACKs for top commit:
MarcoFalke:
review ACK 1770be72d5eb47cae565d4ac86de5bc16f94fdd6
Tree-SHA512: 5cb852a92883a7443ab7dc15b48efa76b5d1424b6b0da1fa6b075fbe9a83522e3ff60382d36c08d4b07143ed898c115614582474e37837332caaee73b0db0e47
