Here we adjust the ForEachNodeCached graph DB method to pass in a node's
addresses into the provided call-back if requested. This will allow us
to improve the performance of node/channel iteration in the autopilot
subserver.
In the graph/db code, we should always expect to deal with potentially
nil ChannelEdgePolicy pointers and so we should always do a nil check
before making use of the struct.
Implement ForEachChannelCacheable which is like ForEachChannel but its
call-back takes the cached versions of channel info & policies. This is
then used during graph cache population. This will be useful once the
SQL implementation is added so that we can reduce the number of DB trips
on cache population.
Define a new CachedEdgeInfo type and let the graph cache's AddChannel
use this. This will let us later on (for the SQL impl of the graph db)
only load from the DB what we actually need for the graph cache.
Update the GraphCache.UpdatePolicy method to take a
`models.CachedEdgePolicy` instead of a `models.ChannelEdgePolicy`.
Doing this will allow us later on to only fetch the necessary info for
populating the CachedEdgePolicy when we are populating the cache via
UpdatePolicy.
In this commit, we add a `test_kvdb.go` file with a single definition of
the `NewTestDB` function. A new version of `MakeTestGraph` (called
`MakeTestGraphNew` is added which makes use of this `NewTestDB` function
to create the backing `V1Store` passed to the `ChannelGraph` for tests.
Later on, we will add new implementations of this method backed by
sqlite and postgres. When those are added, then build flags will
control which version of `NewTestDB` is called.
With this change, the only test call-site of `NewKVStore` is the new
`test_kvdb.go` file.
Use the new `V1Store` interface as a field in the `ChannelGraph` rather than
a pointer to the `KVStore` implementation in preparation for allowing a
SQL implementation of `V1Store` to be used by the `ChannelGraph`.
Note that two tests are adjusted in this commit to be skipped if the
V1Store is not the `KVStore` implementation since the tests are very
bbolt specific.
In this commit, we ensure that any topology update is forced to go via
the `handleTopologySubscriptions` handler so that client subscriptions
and updates are handled correctly and in the correct order.
This removes a bug that could result from a client missing a
notification about a channel being closed if the client is subscribed
and shortly after, `PruneGraph` is called which would notify all
subscribed clients and possibly do so before the client subscription has
actually been persisted.
We remove the mutex that was previously held between DB calls and calls
that update the graphCache. This is done so that the underlying DB calls
can make use of any batch requests which they currently cannot since the
mutex prevents multiple requests from calling the methods at once.
The reason the cacheMu was originally added here was during a code
refactor that moved the `graphCache` out of the `KVStore` and into the
`ChannelGraph` and the aim was then to have a best effort way of
ensuring that updates to the DB and updates to the graphCache were as
consistent/atomic as possible.
We plan to later on add an option for a remote graph source which will
be managed from the ChannelGraph. In such a set-up, a node would rely on
the remote graph source for graph updates instead of from gossip sync.
In this scenario, however, our topology subscription logic should still
notify clients of all updates and so it makes more sense to have the
logic as part of the ChannelGraph so that we can send updates we receive
from the remote graph.
We do this in preparation for moving channel cache population logic out
of the constructor and into the Start method. We also will later on
(when topology subscription is moved to the ChannelGraph), have a
goroutine that will need to be kicked off and stopped.
Here, we move the graph cache writes for AddLightningNode,
DeleteLightningNode, AddChannelEdge and MarkEdgeLive to the
ChannelGraph. Since these are writes, the cache is only updated if the
DB write is successful.
This commit moves the graph cache checks for FetchNodeFeatures,
ForEachNodeDirectedChannel, GraphSession and ForEachNodeCached from the
KVStore to the ChannelGraph. Since the ChannelGraph is currently just a
pass-through for any of the KVStore methods, all that needs to be done
for calls to go via the ChannelGraph instead directly to the KVStore is
for the ChannelGraph to go and implement those methods.
In this commit, we let the ChannelGraph be responsible for populating
the graphCache and then passing it to the KVStore. This is a first step
in moving the graphCache completely out of the KVStore layer.
And use this struct to pass NewChannelGraph anything it needs to be able
to init the KVStore that it houses. This will allow us to add
ChannelGraph specific options.
In this commit, we rename the existing ChannelGraph struct to KVStore to
better reflect its responsibilities as the CRUD layer. We then introduce
a new ChannelGraph struct which will eventually be the layer above the
CRUD layer in which we will handle cacheing and topology subscriptions.
For now, however, it houses only the KVStore. This means that all calls
to the KVStore will now go through this layer of indirection first. This
will allow us to slowly move the graph Cache management out of the
KVStore and into the new ChannelGraph layer.
We introduce the new ChannelGraph and rename the old one in the same
commit so that all existing call-sites don't need to change at all :)
Rename it to kv_store.go so that we can re-use the graph.go file name
later on. We will use it to house the _new_ ChannelGraph when the
existing ChannelGraph is renamed to more clearly reflect its
responsibilities as the CRUD layer.
This commit restricts the graph CRUD interface such that one can only
add a proof to a channel announcement and not update any other fields.
This pattern is better suited for SQL land too.
This commit fixes a bug that would check that the passed `edge` argument
of UpdateChannelEdge is nil but it should actually be checking if the
`edges` bucket is nil.
In this commit, we use the available kvdb `View` method directly for
starting a graph session instead of manually creating and commiting the
transaction. Note this changes behaviour since failed tx create/commit
will now error instead of just log.
