In preparation for handling retries on the source DB side, we thread
through the `reset` call-backs properly so that we can reset appropriate
variables.
In preparation for making the channel & policy migration logic
idempotent in a step-by-step manner, we add a test here that only tests
the migration of channels _without_ policies so that we can first focus
on just making the channel migration idempotent.
Currently, the graph SQL migration is not retry safe. Meaning that if
the source DB exeutes a retry under the hood, this could result in the
migration failing. In preparation for fixing this, we adust the
migration test accordingly.
This will help us test idempotency later on, but it also ensures that
TestMigrateGraphToSQL is properly testing writes to the
graph_channel_policy_extra_types table.
In this commit, we add a makefile helper that can be used to add the
"-short" flag to the "go test" command when running unit tests via "make
unit ...".
Tests can then be expanded with a `testing.Short()` check to potentially
skip the test in short mode. This is useful for if a dev wants to
quickly run most of the tests in a package but would like to opt out of
running the longer form tests such as stress tests or rapid generation
tests.
This commit adds this check to two graph/db tests.
In preparation for tests where we will want to spin up SQL DBs many
times, we do some refactoring so that it is easy to re-use postgres
fixtures since those are expensive to spin up.
This commit expands `MigrateGraphToSQL` to include migration of the
zombie index.
NOTE: we take this opportunity to clean up the zombie index a bit by
first checking for each channel in the zombie index if it has been
marked as closed in the closed-scid index. If it has, then there is no
need to include it in the zombie index. A special case test for this
is added too.
In this commit, the `MigrateGraphToSQL` function is expanded to migrate
the channel and channe policy data. Both of these have the special case
where the kvdb store records may contain invalid TLV. If we encounter a
channel with invalid TLV, we skip it and its policies. If we encounter a
policy with invalid TLV, we skip it.
The `TestMigrateGraphToSQL` and `TestMigrationWithChannelDB` tests are
updated accordingly.
This commit expands MigrateGraphToSQL so that it migrates all the graph
nodes from kvdb to SQL.
The TestMigrateGraphToSQL test is updated to cover the basic
LightningNode cases. A new test, `TestSQLMigrationEdgeCases`, is also
added and a case is added to tests the edge case where a node exists in
our kvdb store that has invalid TLV bytes.
This commit was inspired by the invoices package
TestMigrationWithChannelDB test helper.
This test is purely for running locally to test the graph SQL migration.
It allows a dev to run the migration against a local `channel.sqlite` or
even `channel.db` file.
In this commit, the basic framework for the graph SQL migration is
added. This sets us up for the commits to follow which will add
migration logic for each table in the graph commit by commit.
