Commit Graph

19432 Commits

Author SHA1 Message Date
Elle Mouton
0f7c0ae9e7 graph/db: refactor to use sqldb.QueryConfig value
This commit is a pure refactor. Here all we are doing is removing the
old `pageSize` constant used in the SQLStore code and instead using the
value provided in the QueryConfig struct.
2025-07-31 15:33:08 +02:00
Elle Mouton
e276f1ec3e sqldb: add ExecutePaginatedQuery helper
In this commit we add the ExecutePaginatedQuery helper function which
can be used to execute cursor-based paginated queries.
2025-07-31 15:33:07 +02:00
Elle Mouton
a6248872cb sqldb: rename ExecutePagedQuery to ExecuteBatchQuery
We rename this helper along the config types & helper types for it
because the word "page" is used more often in the context of paging
through results using an offset and limit whereas this helper is
specifically used to split up the slice in queries of the form
"WHERE x in []slice". We do this rename so that there is mimimal
confusion in contexts where we use batching along with actual paging.

The config struct is also renamed to QueryConfig in preparation for it
holding more config options.
2025-07-31 15:32:53 +02:00
Elle
b5c290d90f Merge pull request #10116 from ellemouton/graphPerf4
[3] graph/db: batch-fetch channel & policy data
2025-07-31 10:03:25 +02:00
Elle Mouton
c1ba7a97a8 graph/db: increase default page size 2025-07-31 07:19:15 +02:00
Elle Mouton
a6b077064f graph/db: use batch loading for ForEachChannel 2025-07-31 07:19:15 +02:00
Elle Mouton
d5da637a38 graph/db: remove unused param 2025-07-31 07:19:14 +02:00
Elle Mouton
f39edae6e3 graph/db: use new batch helpers for edge loading
Use the new batch helpers to replace the existing logic for loading an
edge.
2025-07-31 07:19:14 +02:00
Elle Mouton
8ad5f633bc sqldb: add channel data batch queries
Also add the calling logic for these queries. This logic is not yet
used.
2025-07-31 07:19:14 +02:00
Elle
23dd01cb35 Merge pull request #10115 from ellemouton/graphPerf3
[2] graph/db: batch-fetch node data
2025-07-31 07:17:15 +02:00
Elle Mouton
6ca43173a1 sqldb: change default slices page size 2025-07-30 19:29:23 +02:00
Elle Mouton
36eb0e3ff6 graph/db: use batch loading for NodeUpdatesInHorizon 2025-07-30 19:29:22 +02:00
Elle Mouton
c8c86717f2 graph/db: add NodeUpdatesInHorizon benchmark 2025-07-30 19:29:22 +02:00
Elle Mouton
84b1670742 graph/db: use batch loading for ForEachNode 2025-07-30 19:29:22 +02:00
Elle Mouton
d05b918c7a graph/db: refactor buildNode to use batch fetching
Here, we add a new `buildNodeWithBatchData` helper method that can be
used to construct a `models.LightningNode` object using pre-fetched
batch data. The existing `buildNode` method is then adjusted to use this
new helper.
2025-07-30 19:29:22 +02:00
Elle Mouton
0dc0d320f8 graph/db+sqldb: queries and logic for batch fetching node data
In this commit, we add the queries that will be needed to batch-fetch
the data of a set of nodes. The logic for using these new queries is
also added but not used yet.
2025-07-30 19:29:21 +02:00
Elle
4389067989 Merge pull request #10113 from ellemouton/graphPerf2
[1] graph/db: add some SQL performance improvements
2025-07-30 19:27:21 +02:00
Elle Mouton
5a1184c664 graph/db+sqldb: remove LEFT JOIN for fetching node addresses
In this commit, we remove the LEFT JOIN query that was used for fetching
a nodes addresses. The reason it was used before was to ensure that we'd
get an empty address list if the node did exist but had no addresses.
This was for the purposes of the `AddrsForNode` method since it needs to
return false/true to indicate if the given node exists.
2025-07-30 18:11:01 +02:00
Elle Mouton
43db6683d2 graph/db+sqldb: only fetch what is needed for graph cache
Add a ListChannelsWithPoliciesForCachePaginated query that only fetches
the channel/policy fields that the cache requires.
2025-07-30 18:11:01 +02:00
Elle Mouton
3eed2f94c1 sqldb/sqlc: add index on graph_channels(version, id)
This new index greatly improves the speed of the
ListChannelsWithPoliciesPaginated query.
2025-07-30 18:11:01 +02:00
Oliver Gugger
f7efc15a9e Merge pull request #9625 from MPins/issue-8161
Add deletecanceledinvoice RPC call
2025-07-30 09:14:06 -06:00
Oliver Gugger
117035d95c Merge pull request #10109 from ellemouton/graphPerf1
graph/db: helper tests and some benchmarks for SQL
2025-07-30 09:10:45 -06:00
Elle Mouton
5800a3e054 graph/db: benchmark various graph read calls
This commit adds benchmark tests for the ForEachNode and ForEachChannel
DB calls which are called by DescribeGraph.
2025-07-30 14:06:33 +02:00
Elle Mouton
f5ce4a5656 graph/db: add BenchmarkCacheLoading
Add a benchmark test to test graph cache loading against various local
graph DBs.
2025-07-30 14:06:33 +02:00
Elle Mouton
45033b8c54 graph/db: add SQL migration progress logs 2025-07-30 14:06:33 +02:00
Elle Mouton
77fe1816f6 graph/db: add test helper for populating via migration
This commit adds a helper test that is similar to TestPopulateDBs except
for the fact that it uses the MigrateGraphToSQL function directly to
migrate a local kvdb-sql graph to a native SQL one.
2025-07-30 14:06:33 +02:00
Elle Mouton
afbe6b12ed graph/db: define various db connection helpers for incoming tests
This is a prep commit that just adds all the boilerplate connection
logic for connecting to various graph DBs. These will be used in
upcoming commits which will add tests and benchmarks.
2025-07-30 14:06:33 +02:00
MPins
4b521e1647 doc: release-notes-0.20.0 2025-07-29 13:55:23 -03:00
MPins
1a9f591b2b itest: add the test for DeleteCanceledInvoice 2025-07-29 13:55:17 -03:00
Olaoluwa Osuntokun
0eacd07c60 Merge pull request #10107 from ellemouton/fixLogLevelInheritance
go.mod: bump btclog v2 version
2025-07-29 12:29:52 -04:00
Elle Mouton
240966de90 graph/db+log: use v2 logger for graph subsystem 2025-07-29 11:34:47 +02:00
Oliver Gugger
ce10a9d82e Merge pull request #10114 from ellemouton/fixPolicySwop
graph/db: fix potential policy swop
2025-07-29 00:25:27 -06:00
Elle Mouton
f129b217cf go.mod: bump btclog v2 version 2025-07-29 07:40:43 +02:00
Oliver Gugger
13cec7daec Merge pull request #10083 from starius/reorg-notif-itest3
itest: RegisterSpendNtfn detects reorgs
2025-07-28 23:31:38 -06:00
Boris Nagaev
8723113d27 itest: RegisterSpendNtfn detects reorgs
Added itest "reorg_notifications" which tests that RegisterSpendNtfn behaves as
expected during a reorg. A reorg notification is produced after a reorg affects
the block which has produced a spending notification for this registration.
2025-07-28 20:18:09 -03:00
Elle Mouton
858c064ca2 graph/db: fix potential nil pointer derefs
Here, we fix two bugs that could lead to a nil pointer dereference.
Both are caused by refering to policies that may be nil at the
call-site.
2025-07-28 14:39:28 +02:00
MPins
c00939d0a6 lntest: add functions for DeleteCanceledInvoice itest 2025-07-25 21:21:58 -03:00
MPins
e6a420647c commands: add command to delete canceled invoice
Adds the deletecanceledinvoice by payment hash command.
2025-07-25 21:21:58 -03:00
MPins
3a7df1d7c8 rpcserver+invoices: add support for deleting canceled invoice
Adds server-side RPC functionality to delete canceled invoice,
support deleting specific canceled invoice from the database.
2025-07-25 21:21:51 -03:00
MPins
0146e374fd lnrpc: add the deletecanceledinvoice rpc call 2025-07-25 21:09:18 -03:00
Olaoluwa Osuntokun
2e36f9b8b9 Merge pull request #10096 from yyforyongyu/increase-gossip-default
Increase the default outgoing bandwidth
2025-07-25 15:34:01 -05:00
Oliver Gugger
f4173dbb83 Merge pull request #10090 from starius/up-deps
update golang dependencies
2025-07-25 13:25:33 -06:00
Boris Nagaev
d02520d2b9 go.mod: bump mapstructure/v2 to v2.3.0 2025-07-25 12:21:18 -03:00
Boris Nagaev
3ff11e0765 go.mod: bump runc to v1.1.14 2025-07-25 12:21:13 -03:00
Boris Nagaev
9e58a644aa go.mod: bump google.golang.org/protobuf to 1.33.0
Updated github.com/golang/protobuf to v1.5.4 to be compatible with
google.golang.org/protobuf.

Updated the pinned version in pinned_dep CI job.
2025-07-25 12:20:24 -03:00
Oliver Gugger
839370946e Merge pull request #10108 from yyforyongyu/fix-arb-deadlock
Fix arbitrator deadlock found in `ResolveContract`
2025-07-25 04:42:32 -06:00
yyforyongyu
d0c07d76d2 docs: update release notes 2025-07-25 17:45:02 +08:00
yyforyongyu
03bf1974a3 lncfg: update docs about msg-rate-bytes and msg-burst-bytes 2025-07-25 17:41:00 +08:00
yyforyongyu
cadf870dcb discovery: increase default msg rates to 1MB 2025-07-25 17:40:59 +08:00
yyforyongyu
19a6358a3e docs: add release notes 2025-07-25 17:13:44 +08:00