mirror of
https://github.com/bitcoin/bitcoin.git
synced 2025-12-11 13:13:49 +01:00
e7b0004b37
edbed31066chainparams: add signet assumeutxo param at height 160_000 (Sjors Provoost)b8cafe3871chainparams: add testnet assumeutxo param at height 2_500_000 (Sjors Provoost)99839bbfa7doc: add note about confusing HaveTxsDownloaded name (James O'Beirne)7ee46a755fcontrib: add script to demo/test assumeutxo (James O'Beirne)42cae39356test: add feature_assumeutxo functional test (James O'Beirne)0f64bac603rpc: add getchainstates (James O'Beirne)bb05857794refuse to activate a UTXO snapshot if mempool not empty (James O'Beirne)ce585a9a15rpc: add loadtxoutset (James O'Beirne)62ac519e71validation: do not activate snapshot if behind active chain (James O'Beirne)9511fb3616validation: assumeutxo: swap m_mempool on snapshot activation (James O'Beirne)7fcd21544ablockstorage: segment normal/assumedvalid blockfiles (James O'Beirne)4c3b8ca35cvalidation: populate nChainTx value for assumedvalid chainstates (James O'Beirne)49ef778158test: adjust chainstate tests to use recognized snapshot base (James O'Beirne)1019c39982validation: pruning for multiple chainstates (James O'Beirne)373cf91531validation: indexing changes for assumeutxo (James O'Beirne)1fffdd76a1net_processing: validationinterface: ignore some events for bg chain (James O'Beirne)fbe0a7d7cawallet: validationinterface: only handle active chain notifications (James O'Beirne)f073917a9evalidationinterface: only send zmq notifications for active (James O'Beirne)4d8f4dcb45validation: pass ChainstateRole for validationinterface calls (James O'Beirne)1e59acdf17validation: only call UpdatedBlockTip for active chainstate (James O'Beirne)c6af23c517validation: add ChainstateRole (James O'Beirne)9f2318c76cvalidation: MaybeRebalanceCaches when chain leaves IBD (James O'Beirne)434495a8c1chainparams: add blockhash to AssumeutxoData (James O'Beirne)c711ca186fassumeutxo: remove snapshot during -reindex{-chainstate} (James O'Beirne)c93ef43e4fbugfix: correct is_snapshot_cs in VerifyDB (James O'Beirne)b73d3bbd23net_processing: Request assumeutxo background chain blocks (Suhas Daftuar) Pull request description: - Background and FAQ: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal - Prior progress/project: https://github.com/bitcoin/bitcoin/projects/11 - Replaces https://github.com/bitcoin/bitcoin/pull/15606, which was closed due to Github slowness. Original description and commentary can be found there. --- This changeset finishes the first phase of the assumeutxo project. It makes UTXO snapshots loadable via RPC (`loadtxoutset`) and adds `assumeutxo` parameters to chainparams. It contains all the remaining changes necessary to both use an assumedvalid snapshot chainstate and do a full validation sync in the background. This may look like a lot to review, but note that - ~200 lines are a (non-essential) demo shell script - Many lines are functional test, documentation, and relatively dilute RPC code. So it shouldn't be as burdensome to review as the linecount might suggest. - **P2P**: minor changes are made to `init.cpp` and `net_processing.cpp` to make simultaneous IBD across multiple chainstates work. - **Pruning**: implement correct pruning behavior when using a background chainstate - **Blockfile separation**: to prevent "fragmentation" in blockfile storage, have background chainstates use separate blockfiles from active snapshot chainstates to avoid interleaving heights and impairing pruning. - **Indexing**: some `CValidationInterface` events are given with an additional parameter, ChainstateRole, and all indexers ignore events from ChainstateRole::ASSUMEDVALID so that indexation only happens sequentially. - Have `-reindex` properly wipe snapshot chainstates. - **RPC**: introduce RPC commands `loadtxoutset` and (hidden) `getchainstates`. - **Release docs & first assumeutxo commitment**: add notes and a particular assumeutxo hash value for first AU-enabled release. - This will complete the project and allow use of UTXO snapshots for faster node bootstrap. The next phase, if it were to be pursued, would be coming up with a way to distribute the UTXO snapshots over the P2P network. --- ### UTXO snapshots Create your own with `./contrib/devtools/utxo_snapshot.sh`, e.g. ```shell ./contrib/devtools/utxo_snapshot.sh 788000 utxo.dat ./src/bitcoin-cli -datadir=$(pwd)/testdata`) ``` or use the pre-generated ones listed below. - Testnet: **2'500'000** (Sjors): - torrent: `magnet:?xt=urn:btih:511e09f4bf853aefab00de5c070b1e031f0ecbe9&dn=utxo-testnet-2500000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969` - sha256: `79db4b025448cc0ac388d8589a28eab02de53055d181e34eb47391717aa16388` - Signet: **160'000** (Sjors): - torrent: `magnet:?xt=urn:btih:9da986cb27b3980ea7fd06b21e199b148d486880&dn=utxo-signet-160000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969` - sha256: `eeeca845385ba91e84ef58c09d38f98f246a24feadaad57fe1e5874f3f92ef8c` - Mainnet: **800'000** (Sjors): - Note: this needs the following commit cherry-picked in:24deb2022b- torrent: `magnet:?xt=urn:btih:50ee955bef37f5ec3e5b0df4cf0288af3d715a2e&dn=utxo-800000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969` ### Testing #### For fun (~5min) If you want to do a quick test, you can run `./contrib/devtools/test_utxo_snapshots.sh` and follow the instructions. This is mostly obviated by the functional tests, though. #### For real (longer) If you'd like to experience a real usage of assumeutxo, you can do that too. I've cut a new snapshot at height 788'000 (http://img.jameso.be/utxo-788000.dat - but you can do it yourself with `./contrib/devtools/utxo_snapshot.sh` if you want). Download that, and then create a datadir for testing: ```sh $ cd ~/src/bitcoin # or whatever # get the snapshot $ curl http://img.jameso.be/utxo-788000.dat > utxo-788000.dat # you'll want to do this if you like copy/pasting $ export AU_DATADIR=/home/${USER}/au-test # or wherever $ mkdir ${AU_DATADIR} $ vim ${AU_DATADIR}/bitcoin.conf dbcache=8000 # or, you know, something high blockfilterindex=1 coinstatsindex=1 prune=3000 logthreadnames=1 ``` Obtain this branch, build it, and then start bitcoind: ```sh $ git remote add jamesob https://github.com/jamesob/bitcoin $ git fetch jamesob assumeutxo $ git checkout jamesob/assumeutxo $ ./configure $conf_args && make # (whatever you like to do here) # start 'er up and watch the logs $ ./src/bitcoind -datadir=${AU_DATADIR} ``` Then, in some other window, load the snapshot ```sh $ ./src/bitcoin-cli -datadir=${AU_DATADIR} loadtxoutset $(pwd)/utxo-788000.dat ``` You'll see some log messages about headers retrieval and waiting to see the snapshot in the headers chain. Once you get the full headers chain, you'll spend a decent amount of time (~10min) loading the snapshot, checking it, and flushing it to disk. After all that happens, you should be syncing to tip in pretty short order, and you'll see the occasional `[background validation]` log message go by. In yet another window, you can check out chainstate status with ```sh $ ./src/bitcoin-cli -datadir=${AU_DATADIR} getchainstates ``` as well as usual favorites like `getblockchaininfo`. ACKs for top commit: achow101: ACKedbed31066Tree-SHA512: 6086fb9a38dc7df85fedc76b30084dd8154617a2a91e89a84fb41326d34ef8e7d7ea593107afba01369093bf8cc91770621d98f0ea42a5b3b99db868d2f14dc2
This folder contains lint scripts.
Running locally
To run linters locally with the same versions as the CI environment, use the included Dockerfile:
DOCKER_BUILDKIT=1 docker build -t bitcoin-linter --file "./ci/lint_imagefile" ./
docker run --rm -v $(pwd):/bitcoin -it bitcoin-linter
After building the container once, you can simply run the last command any time you want to lint.
check-doc.py
Check for missing documentation of command line options.
commit-script-check.sh
Verification of scripted diffs. Scripted diffs are only assumed to run on the latest LTS release of Ubuntu. Running them on other operating systems might require installing GNU tools, such as GNU sed.
git-subtree-check.sh
Run this script from the root of the repository to verify that a subtree matches the contents of the commit it claims to have been updated to.
Usage: test/lint/git-subtree-check.sh [-r] DIR [COMMIT]
test/lint/git-subtree-check.sh -?
DIRis the prefix within the repository to check.COMMITis the commit to check, if it is not provided, HEAD will be used.-rchecks that subtree commit is present in repository.
To do a full check with -r, make sure that you have fetched the upstream repository branch in which the subtree is
maintained:
- for
src/secp256k1: https://github.com/bitcoin-core/secp256k1.git (branch master) - for
src/leveldb: https://github.com/bitcoin-core/leveldb-subtree.git (branch bitcoin-fork) - for
src/crypto/ctaes: https://github.com/bitcoin-core/ctaes.git (branch master) - for
src/crc32c: https://github.com/bitcoin-core/crc32c-subtree.git (branch bitcoin-fork) - for
src/minisketch: https://github.com/sipa/minisketch.git (branch master)
To do so, add the upstream repository as remote:
git remote add --fetch secp256k1 https://github.com/bitcoin-core/secp256k1.git
all-lint.py
Calls other scripts with the lint- prefix.