Commit Graph

144 Commits

Author SHA1 Message Date
Elle Mouton
d05b918c7a graph/db: refactor buildNode to use batch fetching
Here, we add a new `buildNodeWithBatchData` helper method that can be
used to construct a `models.LightningNode` object using pre-fetched
batch data. The existing `buildNode` method is then adjusted to use this
new helper.
2025-07-30 19:29:22 +02:00
Elle Mouton
0dc0d320f8 graph/db+sqldb: queries and logic for batch fetching node data
In this commit, we add the queries that will be needed to batch-fetch
the data of a set of nodes. The logic for using these new queries is
also added but not used yet.
2025-07-30 19:29:21 +02:00
Elle Mouton
5a1184c664 graph/db+sqldb: remove LEFT JOIN for fetching node addresses
In this commit, we remove the LEFT JOIN query that was used for fetching
a nodes addresses. The reason it was used before was to ensure that we'd
get an empty address list if the node did exist but had no addresses.
This was for the purposes of the `AddrsForNode` method since it needs to
return false/true to indicate if the given node exists.
2025-07-30 18:11:01 +02:00
Elle Mouton
43db6683d2 graph/db+sqldb: only fetch what is needed for graph cache
Add a ListChannelsWithPoliciesForCachePaginated query that only fetches
the channel/policy fields that the cache requires.
2025-07-30 18:11:01 +02:00
Elle Mouton
858c064ca2 graph/db: fix potential nil pointer derefs
Here, we fix two bugs that could lead to a nil pointer dereference.
Both are caused by refering to policies that may be nil at the
call-site.
2025-07-28 14:39:28 +02:00
Elle Mouton
ddc0e95eda graph/db+sqldb: delete channels in batches
Use the new `SLICES` directive to add a DeleteChannels query which takes
a set of DB channel IDs. Then replace all our calls to DeleteChannel
with a paginated call to DeleteChannels.
2025-07-22 17:17:23 +02:00
Elle Mouton
de6c030f29 graph/db: let DeleteChannelEdges use new wrapped SQL call
Update it to use the new wrapped version of
GetChannelsBySCIDWithPolicies to reduce the number of DB calls.
2025-07-22 17:17:13 +02:00
Elle Mouton
e269d57ffa sqldb+graph/db: use pagination for FetchChanInfos 2025-07-22 17:17:11 +02:00
Elle Mouton
88e9a21d63 sqldb+graph/db: update FilterKnownChanIDs to use pagination
Remove a TODO by making use of the new sqldb.ExecutePagedQuery to fetch
channels in batches rather than one by one.
2025-07-22 17:16:59 +02:00
Elle Mouton
f72c48b283 graph/db+sqldb: pass set of outpoints to SQL
This commit adds a new GetChannelsByOutpoints query which takes a slice
of outpoint strings. This lets us then update PruneGraph to use
paginated calls to GetChannelsByOutpoints instead of making one DB call
per outpoint.
2025-07-22 17:16:49 +02:00
Elle Mouton
2fa30e8735 graph+config: add sql pagination config to ChannelGraph 2025-07-22 17:16:41 +02:00
Elle Mouton
74b70a5108 sqldb+graph/db: prefix graph SQL objects with "graph_"
This makes it more clear what each table is for especially when viewed
with other invoice/payments tables.
2025-07-15 18:40:16 +02:00
Elle Mouton
83f6dc4983 multi: reset for ForEachNodeDirectedChannel 2025-07-15 11:25:10 +02:00
Elle Mouton
13d0bd5ef6 graph/db: let ForEachChannelCacheable take a reset param 2025-07-15 11:25:10 +02:00
Elle Mouton
b3f3766fe6 multi: pass reset to GraphSession 2025-07-15 11:25:09 +02:00
Elle Mouton
e17fd58a1d graph/db: pass reset to ForEachNodeCacheable 2025-07-15 11:25:09 +02:00
Elle Mouton
21ac7ef6f4 multi: add reset to ForEachNode 2025-07-15 11:25:09 +02:00
Elle Mouton
c32bf642d2 multi: pass reset to ForEachNodeCached 2025-07-15 11:23:27 +02:00
Elle Mouton
e5fbca8299 multi: let ForEachNodeChannel take a reset param 2025-07-15 11:23:27 +02:00
Elle Mouton
16126d7703 graph/db: expose reset in ForEachSourceNodeChannel 2025-07-15 11:23:27 +02:00
Elle Mouton
88261834e5 graph/db: expose reset in ForEachChannel
Add a `reset` call-back param to the V1Store ForEachChannel method.
2025-07-15 11:23:26 +02:00
Elle Mouton
e00f0a03fc graph/db: prep SQLStore for reset param
This commit just surfaces the locations in the SQLStore where we will
later pass reset params through.
2025-07-15 11:23:26 +02:00
Elle Mouton
03ede9ccef graph/db: migrate prune log
This commit expands the `MigrateGraphToSQL` to include migration of the
prune log.
2025-07-11 09:15:47 +02:00
Elle Mouton
3d5e852c8c graph/db: let ForEachNodeCached take a context 2025-07-08 15:11:02 +02:00
Elle Mouton
1e804a3d5f graph/db: let ForEachNode take a context 2025-07-08 15:11:02 +02:00
Elle Mouton
f05ef2db97 graph/db: let ForEachNodeCacheable take a context 2025-07-08 15:10:30 +02:00
Elle Mouton
046e106a17 graph/db: let ForEachChannel take a context 2025-07-08 15:10:30 +02:00
Elle Mouton
f8d3a3f3ba graph/db: let ForEachNodeChannel take a context 2025-07-08 15:10:30 +02:00
Elle Mouton
67c0f54e13 graph/db+server: let ForEachSourceNodeChannel take a context 2025-07-08 15:10:30 +02:00
Elle Mouton
2f2845dfc0 refactor+multi: use *lnwire.FeatureVector for ChannelEdgeInfo features
In this commit, we move the serialisation details of a channel's
features to the DB layer and change the `models` field to instead use a
more useful `*lnwire.FeatureVector` type.

This makes the features easier to work with and moves the serialisation
to where it is actually used.
2025-07-01 17:02:07 +02:00
Elle Mouton
92849388b8 graph/db: return channel DB info from insertChannel
In preparation for the kvdb->migration code, this commit updates
`insertChannel` to return the ID of the newly inserted channel along
with the IDs of the nodes that the channel links to.
2025-07-01 13:37:09 +02:00
Elle Mouton
4bde8e2d04 graph/db: refactor and clean-up
Refactor channelIDToBytes to return a slice instead of an 8 byte array
so that we dont need to use `[:]` everywhere.

Also make sure we are using this helper everywhere.
2025-07-01 13:37:09 +02:00
Elle Mouton
2310756307 graph/db: refactor source node fetching logic
Small refactor to introduce a re-usable getSourceNode function that we
can then make use of in the upcoming kvdb->SQL migration logic.
2025-07-01 13:37:07 +02:00
Oliver Gugger
500808fadd Merge pull request #10010 from ellemouton/sqlGraphUpdates
graph/db: various misc updates
2025-07-01 12:40:50 +02:00
Elle Mouton
e4137a3f14 graph/db: fix ChanUpdate message/channel flag bug
Here we start using the newly added message_flags and channel_flags
columns of the channel_policies table. The test added previoulsy to
demonstrate the bug is now updated to show that the bug has been fixed.
2025-07-01 10:12:54 +02:00
Elle Mouton
f1b7ccc6b2 sqldb+graph/db: prune graph nodes in a single query
Update the pruneGraphNodes routine to prune the graph nodes in a single
query instead of two.
2025-06-30 18:41:28 +02:00
Elle Mouton
714e528a3a graph/db: fix address fetching error 2025-06-30 18:40:25 +02:00
Elle Mouton
73eca1bf3d graph/db: update outdated comment 2025-06-30 18:40:13 +02:00
Elle Mouton
64110cf459 graph/db: let FetchChannelEdgesByID behave as promised
The comment of FetchChannelEdgesByID says that if the ErrZombieEdge
error is returned, then the ChannelEdgeInfo return parameter will also
still be populated and returned (ie, wont be nil). This commit updates
the SQLStore implementation of FetchChannelEdgesByID to do this. This is
needed to prevent nil dereference panics at any call-sites that depend
on the method working as it describes.
2025-06-30 11:04:35 +02:00
Elle Mouton
cc0be0bca2 graph/db: let IsZombieEdge return an error 2025-06-26 10:19:51 +02:00
Elle Mouton
b4b1c4bd8d graph/db: remove KVStore fall-back
Now that SQLStore fully implements V1Store, we no longer need the
KVStore fall-back.
2025-06-26 10:12:46 +02:00
Elle Mouton
daf6a08747 graph/db: fix log precision 2025-06-25 13:26:53 +02:00
Elle Mouton
81ce05fced graph/db: start lastID for pagination at -1
Just to be very sure that we don't miss a valid ID.
2025-06-25 13:26:53 +02:00
Elle Mouton
23c8f7099c graph/db: impl ForEachChannelCacheable 2025-06-25 13:26:52 +02:00
Elle Mouton
34cb6a9f27 graph/db: impl GraphSession 2025-06-25 13:26:52 +02:00
Elle Mouton
933ab3c6b7 graph/db+sqldb: impl PutClosedScid and IsClosedScid 2025-06-25 13:26:52 +02:00
Elle Mouton
d46552f5ad graph/db+sqldb: implement AddEdgeProof
And run `TestAddEdgeProof` against the SQL backends.
2025-06-25 13:26:51 +02:00
Elle Mouton
e875183c4f sqldb+graph/db: impl DisconnectBlockAtHeight
Which lets us run `TestDisconnectBlockAtHeight` and
`TestStressTestChannelGraphAPI` against our SQL backends.
2025-06-25 11:22:03 +02:00
Elle Mouton
9dd0361ed0 graph/db+sqldb: impl PruneGraph, PruneTip, ChannelView
Which lets us run `TestGraphPruning` and `TestBatchedAddChannelEdge`
against our SQL backends.
2025-06-25 11:22:00 +02:00
Elle Mouton
2da701cc4f graph/db+sqldb: impl PruneGraphNodes
Which lets us run `TestChannelEdgePruningUpdateIndexDeletion` against
our SQL backends.
2025-06-25 10:48:35 +02:00