64 Commits

Author SHA1 Message Date
merge-script
aa87e0b446
Merge bitcoin/bitcoin#31519: refactor: Use std::span over Span
ffff4a293ad878494e12f8f00108cc99ee2b713e bench: Update span-serialize comment (MarcoFalke)
fa4d6ec97bcb1790a7cd4363a13fda7c80c3dd90 refactor: Avoid false-positive gcc warning (MarcoFalke)
fa942332b40c97375af0722f32f7575bca3af819 scripted-diff: Bump copyright headers after std::span changes (MarcoFalke)
fa0c6b7179c062b7ca92d120455ce02a9f4e9e19 refactor: Remove unused Span alias (MarcoFalke)
fade0b5e5e6e80e3da1ab6448b6212244bafa5d3 scripted-diff: Use std::span over Span (MarcoFalke)
fadccc26c03db00a2be3f703aa7e5eec4312bd2e refactor: Make Span an alias of std::span (MarcoFalke)
fa27e36717ec18d64b7ff7bba71b8f0c202ba31d test: Fix broken span_tests (MarcoFalke)
fadf02ef8bf96ad5b3b8e34fd425b31b555f4371 refactor: Return std::span from MakeUCharSpan (MarcoFalke)
fa720b94be17fa9e7c91188710e6a04939ceab11 refactor: Return std::span from MakeByteSpan (MarcoFalke)

Pull request description:

  `Span` has some issues:

  * It does not support fixed-size spans, which are available through `std::span`.
  * It is confusing to have it available and in use at the same time with `std::span`.
  * It does not obey the standard library iterator build hardening flags. See https://github.com/bitcoin/bitcoin/issues/31272 for a discussion. For example, this allows to catch issues like the one fixed in commit fabeca3458b38a3d8930cb0cbc866388c3f120f1.

  Both types are type-safe and can even implicitly convert into each other in most contexts.

  However, exclusively using `std::span` seems less confusing, so do it here with a scripted-diff.

ACKs for top commit:
  l0rinc:
    reACK ffff4a293ad878494e12f8f00108cc99ee2b713e
  theuni:
    ACK ffff4a293ad878494e12f8f00108cc99ee2b713e.

Tree-SHA512: 9cc2f1f43551e2c07cc09f38b1f27d11e57e9e9bc0c6138c8fddd0cef54b91acd8b14711205ff949be874294a121910d0aceffe0e8914c4cff07f1e0e87ad5b8
2025-03-20 13:41:54 +08:00
MarcoFalke
fa942332b4
scripted-diff: Bump copyright headers after std::span changes
Historically, the headers have been bumped some time after a file has
been touched. Do it now to avoid having to touch them again in the
future for that reason.

-BEGIN VERIFY SCRIPT-
 sed -i --regexp-extended 's;( 20[0-2][0-9])(-20[0-2][0-9])? The Bitcoin Core developers;\1-present The Bitcoin Core developers;g' $( git show --pretty="" --name-only HEAD~1 )
-END VERIFY SCRIPT-
2025-03-12 19:46:54 +01:00
MarcoFalke
fade0b5e5e
scripted-diff: Use std::span over Span
-BEGIN VERIFY SCRIPT-

 ren() { sed -i "s!\<$1\>!$2!g" $( git grep -l "$1" -- "./src" ":(exclude)src/span.h" ":(exclude)src/leveldb/db/log_test.cc" ) ; }

 ren Span            std::span
 ren AsBytes         std::as_bytes
 ren AsWritableBytes std::as_writable_bytes

 sed -i 's!SpanPopBack(Span!SpanPopBack(std::span!g' ./src/span.h

-END VERIFY SCRIPT-
2025-03-12 19:45:37 +01:00
Lőrinc
ab2b67fce2 scripted-diff: modernize outdated trait patterns - values
See https://en.cppreference.com/w/cpp/types/is_enum for more details.

-BEGIN VERIFY SCRIPT-
sed -i -E 's/(std::[a-z_]+)(<[^<>]+>)::value\b/\1_v\2/g' $(git grep -l '::value' ./src ':(exclude)src/bench/nanobench.h' ':(exclude)src/minisketch' ':(exclude)src/span.h')
-END VERIFY SCRIPT-
2025-02-21 10:43:01 +01:00
MarcoFalke
fa72646f2b
move-only: Detail_CheckNumFormatSpecifiers and G_TRANSLATION_FUN
This is required for a future commit. Can be reviewed via the git
options --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space

Also move util::detail::Hex to a proper namespace instead of an inline
namespace so it doesn't conflict with the new util::detail namespace, and
won't create other problems for callers trying to use the inline namespaces.

Also fix a misleading comment in util_string_tests.cpp.

Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
2024-11-15 17:16:27 +01:00
l0rinc
5b74a849cf
util: Add consteval ""_hex[_v][_u8] literals
""_hex is a compile-time user-defined literal returning std::array<std::byte>, equivalent of ParseHex.

Variants:
- ""_hex_v returns std::vector<std::byte>
- ""_hex_u8 returns std::array<uint8_t>
- ""_hex_v_u8 returns std::vector<uint8_t> - Directly serializable as a size-prefixed OP_PUSH CScript payload using operator<<.

Also extracts from_hex into shared util::ConstevalHexDigit function.

Co-Authored-By: hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-Authored-By: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
Co-Authored-By: stickies-v <stickies-v@protonmail.com>
2024-08-28 19:09:51 +02:00
stickies-v
2e58fdb544
util: remove unused IsHexNumber
The relevant unit tests have been incorporated in
uint256_tests/from_user_hex in a previous commit.
2024-08-23 13:53:40 +01:00
Ryan Ofsky
4f74c59334 util: Move util/string.h functions to util namespace
There are no changes to behavior. Changes in this commit are all additions, and
are easiest to review using "git diff -U0 --word-diff-regex=." options.

Motivation for this change is to keep util functions with really generic names
like "Split" and "Join" out of the global namespace so it is easier to see
where these functions are defined, and so they don't interfere with function
overloading, especially since the util library is a dependency of the kernel
library and intended to be used with external code.
2024-05-16 10:16:08 -05:00
TheCharlatan
23cc8ddff4 util: move HexStr and HexDigit from util to crypto
Move HexStr and HexDigit functions from util to crypto. The crypto library does
not actually use these functions, but the consensus library does. The consensus
and util libraries not allowed to depend on each other, but are allowed to
depend on the cryto library, so the crypto library is a reasonable put these.

The consensus library uses HexStr and HexDigit in script.cpp, transaction.cpp,
and uint256.cpp.

The util library does not use HexStr but does use HexDigit in strencodings.cpp
to parse integers.
2024-05-16 17:16:08 +02:00
Jon Atack
bb91131d54 doc: remove out-of-date external link in src/util/strencodings.h 2023-08-24 15:51:25 -06:00
MarcoFalke
fa6b11a556
test: Throw error when -signetchallenge is non-hex 2023-05-25 19:24:05 +02:00
MarcoFalke
faab273e06
util: Return empty vector on invalid hex encoding 2023-02-27 13:39:55 +01:00
Hennadii Stepanov
306ccd4927
scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-

Commits of previous years:
- 2021: f47dda2c58b5d8d623e0e7ff4e74bc352dfa83d7
- 2020: fa0074e2d82928016a43ca408717154a1c70a4db
- 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
2022-12-24 23:49:50 +00:00
amadeuszpawlik
f8387c4234 Validate port value in SplitHostPort
Forward the validation of the port from `ParseUInt16(...)`.
Consider port 0 as invalid.
Add suitable test for the `SplitHostPort` function.
Add doxygen description to the `SplitHostPort` function.
2022-10-05 19:24:04 +02:00
fanquake
07f2c25d04
refactor: add most of src/util to iwyu
These files change infrequently, and not much header shuffling is required.

We don't add everything in src/util/ yet, because IWYU makes some
dubious suggestions, which I'm going to follow up with upstream.
2022-07-08 11:06:01 +01:00
Ben Woosley
71a8dbe5da
refactor: Remove defunct attributes.h includes
Since the removal of NODISCARD in 81d5af42f4dba5b68a597536cad7f61894dc22a3,
the only attributes def is LIFETIMEBOUND, and it's included in many more
places that it is used.

This removes all includes which do not have an associated use of LIFETIMEBOUND,
and adds it to the following files, due to their use of the same:
* src/validationinterface.h
* src/script/standard.h
2022-05-21 13:54:33 -05:00
MarcoFalke
fae1006019
util: Add ParseHex<std::byte>() helper 2022-04-27 19:53:17 +02:00
Pieter Wuille
e7d2fbda63 Use std::string_view throughout util strencodings/string 2022-04-27 14:13:39 +02:00
Pieter Wuille
8ffbd1412d Make DecodeBase{32,64} take string_view arguments 2022-04-27 14:12:55 +02:00
Pieter Wuille
1a72d62152 Generalize ConvertBits to permit transforming the input 2022-04-27 14:12:55 +02:00
Pieter Wuille
78f3ac51b7 Make DecodeBase{32,64} return optional instead of taking bool* 2022-04-27 14:12:55 +02:00
Pieter Wuille
a65931e3ce Make DecodeBase{32,64} always return vector, not string
Base32/base64 are mechanisms for encoding binary data. That they'd
decode to a string is just bizarre. The fact that they'd do that
based on the type of input arguments even more so.
2022-04-27 14:12:55 +02:00
Pieter Wuille
a4377a0843 Reject incorrect base64 in HTTP auth
In addition, to make sure that no call site ignores the invalid
decoding status, make the pf_invalid argument mandatory.
2022-04-27 14:12:55 +02:00
Pieter Wuille
d648b5120b Make SanitizeString use string_view 2022-04-27 14:12:55 +02:00
Pieter Wuille
963bc9b576 Make IsHexNumber use string_view 2022-04-27 14:12:55 +02:00
Pieter Wuille
40062997f2 Make IsHex use string_view 2022-04-27 14:12:55 +02:00
Pieter Wuille
c1d165a8c2 Make ParseHex use string_view 2022-04-27 14:12:55 +02:00
James O'Beirne
b5c9bb5cb9 util: Restore GetIntArg saturating behavior
The new locale-independent atoi64 method introduced in #20452 parses
large integer values higher than maximum representable value as 0
instead of the maximum value, which breaks backwards compatibility.
This commit restores compatibility and adds test coverage for this case
in terms of the related GetIntArg and strtoll functions.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2022-01-11 19:54:36 -05:00
Hennadii Stepanov
f47dda2c58
scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-

Commits of previous years:
* 2020: fa0074e2d82928016a43ca408717154a1c70a4db
* 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
2021-12-30 19:36:57 +02:00
MarcoFalke
fad6761cf7
Fix implicit integer sign changes in strencodings 2021-12-13 09:57:33 +01:00
MarcoFalke
9394964f6b
Merge bitcoin/bitcoin#23451: span: Add std::byte helpers
faa3ec2304051be7cfbe301cfbfbda3faf7514fc span: Add std::byte helpers (MarcoFalke)
fa18038f519db76befb9a7bd0b1540143bfeb12b refactor: Use ignore helper when unserializing an invalid pubkey (MarcoFalke)
fabe18d0b39b4b918bf60e3a313eaa36fb4067f2 Use value_type in CDataStream where possible (MarcoFalke)

Pull request description:

  This adds (currently unused) span std::byte helpers, so that they can be used in new code.

  The refactors are also required for https://github.com/bitcoin/bitcoin/pull/23438, but they are split up because the other pull doesn't compile with msvc right now.

  The third commit is not needed for the other pull, but still nice.

ACKs for top commit:
  klementtan:
    reACK  faa3ec2. Verified that all the new `std::byte` helper functions are tested.
  laanwj:
    Code review ACK faa3ec2304051be7cfbe301cfbfbda3faf7514fc

Tree-SHA512: b1f6af39f03ea4dfebf20d4a8538fa993a6104e7fc92ddf0c4606a7efc3ca9a8c1a4741d98a1418569c11bb9ce9258bf0c0c06d93d85ed7e208902a2db04e407
2021-11-24 11:04:37 +01:00
Douglas Chimento
21b58f430f
util: ParseByteUnits - Parse a string with suffix unit [k|K|m|M|g|G|t|T]
A convenience utility for human readable arguments/config e.g. -maxuploadtarget=500g
2021-11-17 12:47:30 +02:00
MarcoFalke
faa3ec2304
span: Add std::byte helpers
Also, add Span<std::byte> interface to strencondings.
2021-11-09 17:42:13 +01:00
MarcoFalke
fad55e79ca
doc: Fixup ToIntegral docs 2021-10-08 15:54:50 +02:00
MarcoFalke
42fedb4acd
Merge bitcoin/bitcoin#23156: refactor: Remove unused ParsePrechecks and ParseDouble
fa9d72a7947d2cff541794e21e0040c3c1d43b32 Remove unused ParseDouble and ParsePrechecks (MarcoFalke)
fa3cd2853530c86c261ac7266ffe4f1726fe9ce6 refactor: Remove unused ParsePrechecks from ParseIntegral (MarcoFalke)

Pull request description:

  All of the `ParsePrechecks` are already done by `ToIntegral`, so remove them from `ParseIntegral`.

  Also:
  * Remove redundant `{}`. See https://github.com/bitcoin/bitcoin/pull/20457#discussion_r720116866
  * Add missing failing c-string test case
  * Add missing failing test cases for non-int32_t integral types

ACKs for top commit:
  laanwj:
    Code review ACK fa9d72a7947d2cff541794e21e0040c3c1d43b32, good find on ParseDouble not being used at all, and testing for behavior of embedded NULL characters is always a good thing.
  practicalswift:
    cr ACK fa9d72a7947d2cff541794e21e0040c3c1d43b32

Tree-SHA512: 3d654dcaebbf312dd57e54241f9aa6d35b1d1d213c37e4c6b8b9a69bcbe8267a397474a8b86b57740fbdd8e3d03b4cdb6a189a9eb8e05cd38035dab195410aa7
2021-10-04 15:06:37 +02:00
MarcoFalke
fa9d72a794
Remove unused ParseDouble and ParsePrechecks 2021-10-04 09:46:17 +02:00
MarcoFalke
fa3cd28535
refactor: Remove unused ParsePrechecks from ParseIntegral
Also:
* Remove redundant {} from return statement
* Add missing failing c-string test case and "-" and "+" strings
* Add missing failing test cases for non-int32_t integral types
2021-10-01 18:05:33 +02:00
practicalswift
4343f114cc Replace use of locale dependent atoi(…) with locale-independent std::from_chars(…) (C++17)
test: Add test cases for LocaleIndependentAtoi

fuzz: Assert legacy atoi(s) == LocaleIndependentAtoi<int>(s)

fuzz: Assert legacy atoi64(s) == LocaleIndependentAtoi<int64_t>(s)
2021-09-30 14:21:17 +00:00
practicalswift
4747db8761 util: Introduce ToIntegral<T>(const std::string&) for locale independent parsing using std::from_chars(…) (C++17)
util: Avoid locale dependent functions strtol/strtoll/strtoul/strtoull in ParseInt32/ParseInt64/ParseUInt32/ParseUInt64

fuzz: Assert equivalence between new and old Parse{Int,Uint}{8,32,64} functions

test: Add unit tests for ToIntegral<T>(const std::string&)
2021-09-18 04:31:24 +00:00
Jon Atack
2875a764f7
util: add ParseUInt16(), use it in SplitHostPort() 2021-03-16 19:52:33 +01:00
Jon Atack
6423c8175f
p2p, refactor: pass and use uint16_t CService::port as uint16_t 2021-03-16 19:52:31 +01:00
Sebastian Falbesoner
e829c9afbf refactor: replace sizeof(a)/sizeof(a[0]) by std::size (C++17)
Removes the macro ARRAYLEN and also substitutes all other uses of the same
"sizeof(a)/sizeof(a[0])" pattern by std::size, available since C++17.
2021-01-31 17:35:16 +01:00
Sawyer Billings
e8640849c7
doc: Use https URLs where possible 2021-01-04 12:23:16 +08:00
practicalswift
4848e71107 scripted-diff: Use [[nodiscard]] (C++17) instead of NODISCARD
-BEGIN VERIFY SCRIPT-
sed -i "s/NODISCARD/[[nodiscard]]/g" $(git grep -l "NODISCARD" ":(exclude)src/bench/nanobench.h" ":(exclude)src/attributes.h")
-END VERIFY SCRIPT-
2020-11-26 09:05:59 +00:00
Vasil Dimov
7be6ff6187
net: recognize TORv3/I2P/CJDNS networks
Recognizing addresses from those networks allows us to accept and gossip
them, even though we don't know how to connect to them (yet).

Co-authored-by: eriknylund <erik@daychanged.com>
2020-09-21 10:13:34 +02:00
Sebastian Falbesoner
e2aa1a585a util: make EncodeBase64 consume Spans 2020-08-25 18:52:57 +02:00
Sebastian Falbesoner
2bc207190e util: make EncodeBase32 consume Spans 2020-08-25 18:52:51 +02:00
MarcoFalke
8d6224fefe
Merge #19628: net: change CNetAddr::ip to have flexible size
102867c587f5f7954232fb8ed8e85cda78bb4d32 net: change CNetAddr::ip to have flexible size (Vasil Dimov)
1ea57ad67406b3aaaef5254bc2fa7e4134f3a6df net: don't accept non-left-contiguous netmasks (Vasil Dimov)

Pull request description:

  (chopped off from #19031 to ease review)

  Before this change `CNetAddr::ip` was a fixed-size array of 16 bytes,
  not being able to store larger addresses (e.g. TORv3) and encoded
  smaller ones as 16-byte IPv6 addresses.

  Change its type to `prevector`, so that it can hold larger addresses and
  do not disguise non-IPv6 addresses as IPv6. So the IPv4 address
  `1.2.3.4` is now encoded as `01020304` instead of
  `00000000000000000000FFFF01020304`.

  Rename `CNetAddr::ip` to `CNetAddr::m_addr` because it is not an "IP" or
  "IP address" (TOR addresses are not IP addresses).

  In order to preserve backward compatibility with serialization (where
  e.g. `1.2.3.4` is serialized as `00000000000000000000FFFF01020304`)
  introduce `CNetAddr` dedicated legacy serialize/unserialize methods.

  Adjust `CSubNet` accordingly. Still use `CSubNet::netmask[]` of fixed 16
  bytes, but use the first 4 for IPv4 (not the last 4). Do not accept
  invalid netmasks that have 0-bits followed by 1-bits and only allow
  subnetting for IPv4 and IPv6.

  Co-authored-by: Carl Dong <contact@carldong.me>

ACKs for top commit:
  sipa:
    utACK 102867c587f5f7954232fb8ed8e85cda78bb4d32
  MarcoFalke:
    Concept ACK 102867c587f5f7954232fb8ed8e85cda78bb4d32
  ryanofsky:
    Code review ACK 102867c587f5f7954232fb8ed8e85cda78bb4d32. Just many suggested updates since last review. Thanks for following up on everything!
  jonatack:
    re-ACK 102867c587f5f7954232fb8ed8e85cda78bb4d32 diff review, code review, build/tests/running bitcoind with ipv4/ipv6/onion peers
  kallewoof:
    ACK 102867c587f5f7954232fb8ed8e85cda78bb4d32

Tree-SHA512: d60bf716cecf8d3e8146d2f90f897ebe956befb16f711a24cfe680024c5afc758fb9e4a0a22066b42f7630d52cf916318bedbcbc069ae07092d5250a11e8f762
2020-08-25 18:10:25 +02:00
Vasil Dimov
102867c587
net: change CNetAddr::ip to have flexible size
Before this change `CNetAddr::ip` was a fixed-size array of 16 bytes,
not being able to store larger addresses (e.g. TORv3) and encoded
smaller ones as 16-byte IPv6 addresses.

Change its type to `prevector`, so that it can hold larger addresses and
do not disguise non-IPv6 addresses as IPv6. So the IPv4 address
`1.2.3.4` is now encoded as `01020304` instead of
`00000000000000000000FFFF01020304`.

Rename `CNetAddr::ip` to `CNetAddr::m_addr` because it is not an "IP" or
"IP address" (TOR addresses are not IP addresses).

In order to preserve backward compatibility with serialization (where
e.g. `1.2.3.4` is serialized as `00000000000000000000FFFF01020304`)
introduce `CNetAddr` dedicated legacy serialize/unserialize methods.

Adjust `CSubNet` accordingly. Still use `CSubNet::netmask[]` of fixed 16
bytes, but use the first 4 for IPv4 (not the last 4). Only allow
subnetting for IPv4 and IPv6.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-08-24 21:50:59 +02:00
Sebastian Falbesoner
71e0f07e9c util: remove unused c-string variant of atoi64() 2020-08-17 17:56:59 +02:00