Commit Graph

8220 Commits

Author SHA1 Message Date
38e313a869 Merge pull request #3633 from joostjager/safe-migrations
channeldb: isolate migrations
2019-11-01 18:28:56 -07:00
bef1cd01c7 Merge pull request #3520 from halseth/autopilot-heuristic-update-trigger
[autopilot] Trigger channel opening on external score update
2019-11-01 17:24:22 -07:00
e0744a3983 Merge pull request #3650 from guggero/update-grpc-gateway
proto+lnrpc: update gRPC-REST gateway to v1.8.6
2019-11-01 09:03:48 +01:00
4592f87033 Merge pull request #3653 from Roasbeef/external-funding-chainfees
chainfees: create new chainfees package extracting fees from lnwallet
2019-10-31 17:44:00 -07:00
cc159decf5 Merge pull request #3654 from Roasbeef/external-funding-chantype-bits
multi: convert the existing channeldb.ChannelType uint8 into a bit field
2019-10-31 17:24:20 -07:00
777ed104a3 chainfee: create new chainfee package extracting fees from lnwallet
In this commit, we create a new chainfee package, that houses all fee
related functionality used within the codebase. The creation of this new
package furthers our long-term goal of extracting functionality from the
bloated `lnwallet` package into new distinct packages. Additionally,
this new packages resolves a class of import cycle that could arise if a
new package that was imported by something in `lnwallet` wanted to use
the existing fee related functions in the prior `lnwallet` package.
2019-10-31 16:41:57 -07:00
a73ee28e46 multi: convert the existing channeldb.ChannelType uint8 into a bit field
In this commit, we convert the existing `channeldb.ChannelType` type
into a _bit field_. This doesn't require us to change the current
serialization or interpretation or the type as it is, since all the
current defined values us a distinct bit. This PR lays the ground work
for any future changes that may introduce new channel types (like anchor
outputs), and also any changes that may modify the existing invariants
around channels (if we're the initiator, we always have the funding
transaction).
2019-10-31 16:34:37 -07:00
4486a06b1a migration_01_to_11: remove version checking for migration tests 2019-10-31 11:25:07 +01:00
5a8ecfcc49 autopilot/agent_test: add TestAgentHeuristicUpdateSignal
TestAgentHeuristicUpdateSignal tests that upon notification about a
heuristic update, the agent reconsults the heuristic.
2019-10-31 10:19:50 +01:00
7524265ab1 autopilot: trigger agent channel open on new scores
This commit adds a new signal to the autopilot agent, meant to signal
when any of the available heuristics has gotten an update.

We currently use this to trigger a new channel opening after the
external scores have been updated.
2019-10-31 10:17:31 +01:00
f0ba4be758 autopilot/manager: fix slice modifications
When appending to a slice, there is no guarantee the slice won't be
modified. So instead of appending to the global slice
availableHeuristics, we create a temporary local one.
2019-10-31 10:17:31 +01:00
df5baa4275 autopilot/agent: trace log reason nodes being skipped 2019-10-31 10:17:31 +01:00
b6c7ff8183 lnrpc: generate protos with new gateway version 2019-10-31 09:13:38 +01:00
b6b57d1c00 proto: update to grpc-gateway v1.8.6 2019-10-31 09:13:33 +01:00
fcf81ed8ff Merge pull request #3556 from joostjager/query-mc-prob
routerrpc: add queryprob rpc
2019-10-31 08:03:29 +01:00
60503d6c44 channeldb/migration_01_to_11: remove unused code 2019-10-31 07:18:36 +01:00
64e5d06cd1 Merge pull request #2598 from skwp/docker-caching
Add source after installing apk packages to avoid breaking docker cache
2019-10-30 15:47:57 +01:00
f5191440c5 channeldb: initialize migrations logger 2019-10-30 08:56:18 +01:00
a8f077a841 Merge pull request #3558 from joostjager/non-strict-pathfinding
routing: non-strict path finding
2019-10-30 08:42:28 +01:00
0fdfb4d76f Merge pull request #3646 from alrs/fix-routing-test-goroutine
routing: fix use of T.Fatal() in test goroutine
2019-10-29 20:00:30 -07:00
6254c53a4a routing: fix use of T.Fatal() in test goroutine 2019-10-29 13:21:53 -07:00
729c3a6bd6 routing: use unified policy for build route 2019-10-29 19:17:58 +01:00
fb57255b5c routerrpc: add mc probability query rpc
Probabilities are no longer returned for querymc calls. To still provide
some insight into the mission control internals, this commit adds a new
rpc that calculates a success probability estimate for a specific node
pair and amount.
2019-10-29 12:32:19 +01:00
04e7c98461 routerrpc: create pair data proto message
With a separate proto message, it becomes possible to also return the
pair data for a single pair. This prepares for the new mc probability
querying rpc.
2019-10-29 12:32:17 +01:00
3d8adaef95 routing: embed TimedPairResult in snapshot data 2019-10-29 12:32:15 +01:00
57911faa98 routing: export TimedPairResult
Prepares for this data structure being accessed in mission control rpc
implementations.
2019-10-29 12:32:13 +01:00
912a8201f9 routing: remove embedded struct from timedPairResult
This prepares for decoupling the result interpretation of a single
payment attempt from the information stored in mission control memory
on the history of a node pair. A planned follow-up where we store both
the last success and last failure requires this decoupling.
2019-10-29 12:32:11 +01:00
43449ca7a7 channeldb/migration_01_to_11: add references to untested migrations 2019-10-29 11:49:33 +01:00
c069bdd4c7 Merge pull request #3634 from halseth/funding-sync-sendmessage
fundingmanager: make SendToPeer sync
2019-10-28 17:47:03 -07:00
13b56d5849 Merge pull request #3632 from wpaulino/server-not-active-closechannel
rpcserver: ensure server has started before CloseChannel
2019-10-28 17:01:21 -04:00
1a0ab538d5 Merge pull request #3332 from carlaKC/chanfitness-trackchanneluptime
Chanfitness: Track peer uptime
2019-10-25 14:29:59 +02:00
31bf542276 lnrpc: Add experimential uptime and lifetime fields to list channels
This commit adds the total observed lifetime of a channel and the
totaluptime of its remote peer to the lnrpc channel struct. These
fields are marked as experimential because they are subject to
change.
2019-10-25 11:41:16 +02:00
a347237e7a routing: use unified policy for path finding
In this commit we change path finding to no longer consider all channels
between a pair of nodes individually. We assume that nodes forward
non-strict and when we attempt a connection between two nodes, we don't
want to try multiple channels because their policies may not be identical.
Having distinct policies for channel to the same peer is against the
recommendation in the spec, but it happens in the wild. Especially since
we recently changed the default cltv delta value.

What this commit introduces is a unified policy. This can be looked upon
as the greatest common denominator of all policies and should maximize
the probability of getting the payment forwarded.
2019-10-25 11:22:28 +02:00
6b391d04d0 routing/test: mark function as test helper 2019-10-25 11:22:26 +02:00
5ae4f0eae4 Merge pull request #3418 from champo/routing_is_fast_now
routing, channeldb: several optimizations for path finding
2019-10-25 10:54:43 +02:00
1e86589bee chanfitness: Add channel event store
This commit adds a channel event store to the channel fitness
package which is used to manage tracking of a node's channels.
It adds tracking for channel open/closed and peer online/offline
events for all channels that a node has open.

Events are consumed from channelNotifier and peerNotifier event
subscriptions. If either of these subscriptions is cancelled,
channel scoring stops, because both subscriptions are expected
to run until node shutdown.

Two functions are exposed to allow external callers to get uptime
information about a channel. GetLifespan returns the period over
which the channel has been monitored. GetUptime returns the channel's
uptime over a specified period. Callers can use these functions to
get the channel's remote peer uptime over its entire lifetime, or
a subset of that period.
2019-10-25 09:55:09 +02:00
744876003d chanfitness: Add channel event log structure
This commit adds a chanfitness package which will be used to track
channel health and performance metrics. It adds a channel event
structure which will be used to track channel opens/closes and peer
uptime.

The eventLog implements an uptime function which calcualtes uptime
over a given period and a lifespan function which returns the time
when the log began monitoring the channel and, if the channel is
closed, the time when it stopped moitoring it.
2019-10-25 09:51:07 +02:00
fe597b6310 rpc: add forgotten 'address' entity to validEntities
The `address` entity is used by the `WalletKit` sub-server for certain
calls.
2019-10-24 19:44:24 -07:00
b110a3a197 Merge pull request #1160 from guggero/new-macaroon
lncli: add command to create new macaroon
2019-10-24 19:41:51 -07:00
818c302d46 routing: use nodeWithDist instead of vertex to avoid map access
The same nodeWithDist was fetched from the map for every channel it has.
This struct is not mutated so it can be fetched before and reused.
2019-10-24 21:38:07 -03:00
df70095ad0 routing: optimize path finding structures
distance map now holds the edge the current path is coming from,
removing the need for next map.

Both distance map and distanceHeap now hold pointers instead of the full
struct to reduce allocations and copies.

Both these changes reduced path finding time by ~5% and memory usage by
~2mb.
2019-10-24 21:38:07 -03:00
fc36df0e60 routing: avoid unneeded map access
`processEdge` basically had 4 expensive operations: 3 map accesses and
updating the heap. This removes one of those for a small performance
gain.
2019-10-24 21:31:30 -03:00
3e60a23632 routing: pre-allocate the distance map to an estimated node count
Pre-sizing these structures avoids a lot of map resizing, which causes
copies and rehashing of entries. We mostly know that the map won't
exceed that size, and it doesn't affect memory usage in any significant
way.
2019-10-24 21:31:30 -03:00
2141713936 routing: avoid walking all nodes for path finding if we don't need to
Calling `ForEachNode` hits the DB, and allocates and parses every node
in the graph. Walking the channels also loads nodes from the DB, so this
meant that each node was read/parsed/allocated several times per run.

This reduces runtime by ~10ms and memory usage by ~4mb.
2019-10-24 21:31:30 -03:00
5389161162 routing: make log in findPath hot path use logClosure
It generates heap allocations for it's params even if it won't end up
using them.

Reduces memory usage by 2mb
2019-10-24 21:31:30 -03:00
a469d5dc2c Merge pull request #2327 from ottosuess/invoice_macaroon_permissions
lnrpc: allow invoice.macaroon to read onchain transactions
2019-10-24 12:15:16 -04:00
12a1d05127 fundingmanager: make SendToPeer sync
Since the funding flow requires messages to go through, make use of
sync version of SendToPeer. Using the async version we would risk that
the message was dropped and the process would stall (it would properly
continue after a restart though).
2019-10-24 13:08:08 +02:00
6913cd64b6 channeldb: reference migrations in package
This commit removes the migrations from channeldb and references those
in the migrations_01_to_11 package. This creates a one-way dependency on
the migrations. Future changes to channeldb won't be able to break
migrations anymore.
2019-10-24 12:49:40 +02:00
6e463c1634 channeldb: make copy for migrations
This commit is a direct copy of the complete channeldb package. It only
changes the package declaration at the top of every file. We make this
full copy so that review can be focused on the actual changes made.
Otherwise changes may drown in all the file moves.

Linting for the new package is disabled, as it contains lots of
pre-existing issues.
2019-10-24 12:49:38 +02:00
018737a80a Merge pull request #3627 from guggero/witness-timelock-type
input+sweep: log input witness types instead of CSV/CLTV count
2019-10-24 11:08:33 +02:00