In this commit, we refactor the ChanUpdatesInHorizon method to return
an iterator instead of a slice. This change significantly reduces
memory usage when dealing with large result sets by allowing callers to
process items incrementally rather than loading everything into memory
at once.
In this commit, we refactor the NodeUpdatesInHorizon method to return
an iterator instead of a slice. This change significantly reduces
memory usage when dealing with large result sets by allowing callers to
process items incrementally rather than loading everything into memory
at once.
The new implementation uses Go 1.23's iter.Seq type to provide a
standard iterator interface. The method now supports configurable batch
sizes through functional options, allowing fine-tuned control over
memory usage and performance characteristics.
Rather than reading all the entries from disk into memory (before this
commit, we did consult the cache for most entries, skipping the disk
hits), we now expose a chunked iterator instead.
We also make the process of filtering out public nodes first class. This
saves many newly created db transactions later.
Let the helper method only take the params it needs so that we dont need
to construct an entire models.ChannelEdgeInfo object to pass to it. This
will be useful later on.
Here we adjust the ForEachNodeCached graph DB method to pass in a node's
addresses into the provided call-back if requested. This will allow us
to improve the performance of node/channel iteration in the autopilot
subserver.
This commit passes no-op reset callbacks to various places where they
are needed in the KVStore. Upcoming commits will replace the no-ops by
adding reset params to the methods that require it.
In this commit, the `MigrateGraphToSQL` function is expanded to migrate
the channel and channe policy data. Both of these have the special case
where the kvdb store records may contain invalid TLV. If we encounter a
channel with invalid TLV, we skip it and its policies. If we encounter a
policy with invalid TLV, we skip it.
The `TestMigrateGraphToSQL` and `TestMigrationWithChannelDB` tests are
updated accordingly.
Here, we move TLV validation for the KVStore out of `updateEdgePolicy`
so that we can re-use `updateEdgePolicy` in our tests to write policies
with invalid TLV (since that was possible before the recently added TLV
sanity check) so that we can test that our SQL migration behaves
correctly for these cases.
In this commit, we move the serialisation details of a channel's
features to the DB layer and change the `models` field to instead use a
more useful `*lnwire.FeatureVector` type.
This makes the features easier to work with and moves the serialisation
to where it is actually used.
And delete the old MakeTestGraph function.
Here we need to update some of our unit tests to ensure that any updates
to existing nodes or channel policies have newer timestamps. This is so
that we don't violate our SQL DB constraints that prevent updates to a
node or channel policy record if the new update is not newer (has a
newer LastUpdate time) than the currently presisted record).
