In this commit, we add a makefile helper that can be used to add the
"-short" flag to the "go test" command when running unit tests via "make
unit ...".
Tests can then be expanded with a `testing.Short()` check to potentially
skip the test in short mode. This is useful for if a dev wants to
quickly run most of the tests in a package but would like to opt out of
running the longer form tests such as stress tests or rapid generation
tests.
This commit adds this check to two graph/db tests.
This commit adds a new flag to shuffle all the test cases before running
them so tests which require lots of blocks to be mined are less likely
to be run in the same tranch.
The other benefit is this approach provides a more efficient way to
figure which tests are broken since all the differnet backends are
running different tranches in their builds, we can identify more failed
tests in one push.
This commit adds a new build tag `integration` and removes the old tag
`rpctest` for clarity. Multiple unnecessary usages of `build !rpctest`
is also removed.
This commit adds a new flag `temptest` so we can run new and old tests
separately. This flag will be removed once the migration from old tests
to new tests is finished.
To make it easier to see what tranche a failed test was running in, we
add the tranche index to the test name. This corresponds to the number
in the .logs-tranche<index> folder name so the logs can be found much
quicker.
Previously we were excluding non-dev test files from our coverage
report, which included interface_test.go as well as the bitcoindnotify
and btcdnotify dev tests.
It seems that our itests don't perform correctly in a high CPU usage
scenario such as running 6 tests in parallel. Some goroutines don't get
enough execution time and causes check to time out.
By reducing the total number of parallel tests, we hope to give all
goroutines some more breathing room.
To remove the need to have an extra make goal for the Windows itests, we
instead add the flag windows=1 that sets the make variable EXEC_SUFFIX
to properly add the ".exe" suffix to all executable names.
To make the Makefile a bit easier to understand, we remove the implicit
ITEST goal/command variable and switch all itest execution over to
explicit goals in the main Makefile.
Previous to this commit, running `make unit-cover pkg=xx`
would ignore the selected package and run unit tests and
coverage for all packages.
After this commit, the package selected with pkg= is the
only one that is tested and coverage output generated for.
If no pkg is selected, the default is as before, all pkgs.
