Per Steffensen skrev:
Spot on! Good arguments.
When you just do not think of randomized tests as a "replacement for
boundary condition tests" etc
Thanks. Will consider randomized for my projects in the future - with
limits :-)
Regards, Per Steffensen
Dawid Weiss skrev:
I see your point about "bringing up bugs nobody thought to cover manually", but it
also has cons - e.g. violating the principal that tests > should be (easily) repeatable (you
will/can end up with tests the sometimes fail and sometimes succeed, and you have to dig out
the > random values of the tests that fail in order to be able to repeat/reconstruct the
fail)
Randomized tests should be identical in their execution given the same
seed, it's the same principle as with regular tests but expands on
different code paths every time you execute with a different seed.
They are not a replacement for boundary condition tests, they're a
complementary thing that should allow picking things you haven't
thought of. Sure, in case of a failure you need to find the seed that
caused the problem but that doesn't seem like a lot of effort given
the potential profit.
If you want identical runs -- fix the initial seed.
If you have a non-deterministic test for a given fixed seed, it'd be
equally non-deterministic if no randomization was used, it's just a
flawed test (or inherently non-deterministic by nature so assertions
should be relaxed).
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
