Mission control may have outdated success/failure amounts for node pairs
that have channels with differing capacities. In that case we assume to
still find the liquidity as before and rescale the amounts to the
according range.
If the success and fail amounts indicate that a channel doesn't obey a
bimodal distribution, we fall back to a uniform/linear success
probability model. This also helps to avoid numerical normalization
issues with the bimodal model.
This is achieved by adding a very small summand to the balance
distribution P(x) ~ exp(-x/s) + exp((x-c)/s), 1/c that helps to
regularize the probability distribution. The distribution becomes finite
for intermediate balances where the exponentials would be evaluated to
an exact zero (float) otherwise. This regularization is effective in
edge cases and leads to falling back to a uniform model should the
bimodal model fail.
This affects the normalization to be s * (-2 * exp(-c/s) + 2 + 1/s) and
the primitive function to receive an extra term x/(cs).
The previously added fuzz seed is expected to be resolved with this.
This test demonstrates an error found in a fuzz test by adding a
previously found seed, which will be fixed in an upcoming commit.
The following fuzz test is expected to fail:
go test -v -fuzz=Prob ./routing/
The bimodal model doesn't depend on the unit, which is why updating to
more realistic values doesn't require changes in tests.
We pin the scale in the fuzz test to not invalidate the corpus.
probabilityFormula() is expected to return an error if capacity is 0, so
we should exclude that case from fuzzing.
Previously it was attempted to avoid this case by seeding the corpus
with an input that had capacity 1. That is not an effective solution
since the fuzzer can still generate an input with capacity 0.
The process how we calculate a total probability from the direct and
node probability is modified to give more importance to the direct
probability.
This is important if we know about a recent failure. We should not try
to send over this channel again if we know we can't. Otherwise this can
lead to infinite retrials over the channel, when the probability is
pinned at high probabilities by the other node results.
Implements a new probability estimator based on a probability theory
framework.
The computed probability consists of:
* the direct channel probability, which is estimated based on a
depleted liquidity distribution model, formulas and broader concept derived
after Pickhardt et al. https://arxiv.org/abs/2103.08576
* an extension of the probability model to incorporate knowledge decay
after time for previous successes and failures
* a mixed node probability taking into account successes/failures on other
channels of the node (similar to the apriori approach)
