The `GraphSource` interface in the `autopilot` package is directly
implemented by the `graphdb.KVStore` and so we will eventually thread
contexts through to this interface. So in this commit, we start updating
the autopilot system to thread contexts through in preparation for
passing the context through to any calls made to the GraphSource.
Two context.TODOs are added here which will be addressed in follow up
commits.
This commit was previously split into the following parts to ease
review:
- 2d746f68: replace imports
- 4008f0fd: use ecdsa.Signature
- 849e33d1: remove btcec.S256()
- b8f6ebbd: use v2 library correctly
- fa80bca9: bump go modules
We rename the field Capacity on local channels to Balance, define a new
method ChannelInfo on the manager config that let us query the database
for our latest channels state. Using this we will use the current local
balance instead of the channel capacity when doing allocation
calculations.
This commit removes an extra filter on address availability which is not
needed as the scored nodes are a already prefiltered subset of the whole
graph where address availability has already been checked.
This commit adds a new signal to the autopilot agent, meant to signal
when any of the available heuristics has gotten an update.
We currently use this to trigger a new channel opening after the
external scores have been updated.
To make the autopilot able to account for fees, we let it use the
subtractFees option when opening channels.
This makes sure that each channel we attempt to open will eat at most
Amt out of our budget. Previously fees would eat into our funds in
addition, causing us to deplete our funds more than expected on each
channel opening.
This commit moves the logic querying the available heuristics out of the
autopilot agent and into the autopilot manager. This lets us query the
heuristic without the autopilot agent being active.
If called without the agent being active, the current set of channels
will be considered by the heuristics. If the agent is active also the
pending channels will be considered.
This commit fixes a regression in how we allocate funds to attempted
channels. We would earlier stay within the channel size limits, but we
wouldn't account for funds consumed by other channels being opened in
parallel.
We fix this by introducing a loop which greadily tries to distribute the
funds among the channels to open, and reduces the number of channels to
open in case not enough funds are available to satisfy the channel size
limits.
To ensure a call to ConnectToPeer doesn't block the agent from
shutting down, we'll launch it in a non-waitgrouped goroutine, that
will signal when a result is returned.
Since NodeScores no longer returns fully populated AttachmentDirectives,
we make this explicit by defining a new type NodeScore that includes a
subset of what the AttachmentDirective does.
We create a new type NodeScore which is a tuple (NodeID, score). The
weightedChoice and chooseN algorithms are altered to expect this type.
This is done in order to simplify the types we are using, since we were
only using a subset of the fields in AttachmentDirective.
Since we want to combine scores from multiple heuristics, things get
complicated if the heuristics report their own individual channel sizes.
Therefore we change the NodeScores interface slightly, letting the agent
specify the wanted channel size, and let the heuristic score the nodes
accordingly.
We let the agent call ChannelBudget on its constraints directly, and
not go through the heuristic. This is needed since when we want to have
multiple active heuristics concurrently, it won't make sense anymore to
ask each of the heuristics.
The mockConstraints are also updated to act as the mockHeuristic did
before, by making it possible to control the responses it gives by
sending them on the contained channels.
To decouple the autopilot heuristic from the constraints, we start by
abstracting them behind an interface to make them easier to mock. We
also rename them HeuristicConstraints->AgentConstraints to make it clear
that they are now constraints the agent must adhere to.
This commit makes the autopilot agent use the new NodeScores heuristic
API to select channel candiates, instead of the Select API. The result
will be similar, but instead of selecting a set of nodes to open
channels to, we get a score based results which can later be used
together with other heuristics to choose nodes to open channels to.
This commit also makes the existing autopilot agent tests compatible
with the new NodeScores API.
This commit modifies the autopilot agent to track
all pending connection requests, and forgo further
attempts if a connection is already present.
Previously, the agent would try and establish
hundreds of requests to a node, especially if the
connections were timing out and not returning.
This resulted in an OOM OMM when cranking up
maxchannels to 200, since there would be close
to 10k pending connections before the program was
terminated. The issue was compounded by periodic
batch timeouts, causing autopilot to try and
process thousands of triggers for failing
connections to the same peer.
With these fixes, autopilot will skip nodes that we
are trying to connect to during heuristic selection.
The CPU and memory utilization have been significantly
reduced as a result.
We do this to avoid a huge amount of goroutines piling up when autopilot
is trying to open many channels, as they will all block trying to send
the update on the stateUpdates channel. Now we instead send them on a
buffered channel, similar to what is done with the nodeUpdates.
We do this to avoid a huge amount of goroutines piling up on initial
graph sync, as they will all block trying to send the node update on the
stateUpdates channel. Now we instead make a new buffered channel
nodeUpdates, and just return immediately if there is already a signal in
the channel waiting to be processed.
