In preparation for tests where we will want to spin up SQL DBs many
times, we do some refactoring so that it is easy to re-use postgres
fixtures since those are expensive to spin up.
This commit passes no-op reset callbacks to various places where they
are needed in the KVStore. Upcoming commits will replace the no-ops by
adding reset params to the methods that require it.
This commit expands `MigrateGraphToSQL` to include migration of the
zombie index.
NOTE: we take this opportunity to clean up the zombie index a bit by
first checking for each channel in the zombie index if it has been
marked as closed in the closed-scid index. If it has, then there is no
need to include it in the zombie index. A special case test for this
is added too.
In this commit, the `MigrateGraphToSQL` function is expanded to migrate
the channel and channe policy data. Both of these have the special case
where the kvdb store records may contain invalid TLV. If we encounter a
channel with invalid TLV, we skip it and its policies. If we encounter a
policy with invalid TLV, we skip it.
The `TestMigrateGraphToSQL` and `TestMigrationWithChannelDB` tests are
updated accordingly.
Here, we move TLV validation for the KVStore out of `updateEdgePolicy`
so that we can re-use `updateEdgePolicy` in our tests to write policies
with invalid TLV (since that was possible before the recently added TLV
sanity check) so that we can test that our SQL migration behaves
correctly for these cases.
This commit expands MigrateGraphToSQL so that it migrates all the graph
nodes from kvdb to SQL.
The TestMigrateGraphToSQL test is updated to cover the basic
LightningNode cases. A new test, `TestSQLMigrationEdgeCases`, is also
added and a case is added to tests the edge case where a node exists in
our kvdb store that has invalid TLV bytes.
This commit was inspired by the invoices package
TestMigrationWithChannelDB test helper.
This test is purely for running locally to test the graph SQL migration.
It allows a dev to run the migration against a local `channel.sqlite` or
even `channel.db` file.
In this commit, the basic framework for the graph SQL migration is
added. This sets us up for the commits to follow which will add
migration logic for each table in the graph commit by commit.
In this commit, we move the serialisation details of a channel's
features to the DB layer and change the `models` field to instead use a
more useful `*lnwire.FeatureVector` type.
This makes the features easier to work with and moves the serialisation
to where it is actually used.
In preparation for the kvdb->migration code, this commit updates
`insertChannel` to return the ID of the newly inserted channel along
with the IDs of the nodes that the channel links to.
Refactor channelIDToBytes to return a slice instead of an 8 byte array
so that we dont need to use `[:]` everywhere.
Also make sure we are using this helper everywhere.
Here we start using the newly added message_flags and channel_flags
columns of the channel_policies table. The test added previoulsy to
demonstrate the bug is now updated to show that the bug has been fixed.
This commit adds a new graph store unit test that demonstrates that
there is currently a bug in the SQL version of the graph store where
unknown message or channel flags in a channel_update are not persisted
correctly. This will be fixed in an upcoming commit.
