On Jan 23, 8:59 pm, jswit...@gmail.com (Jason Switzer) wrote: > That sounds useful on the surface but often turns out to be more difficult > to do than you might think. There are many cases where tests are performed > from within loops. Something like S09.237 may or may not be in a loop, may > be difficult to identify in files with many tests.
There are at least two reasons to identify a test (or check): to control it from afar, and to track it's results. If the reason for wanting identity is to control it (e.g. Foo::Bar::Test<S09.237>.disable()), then the fact that it's in a loop isn't necessarily important: if you want to disable it, then you probably want to disable all iterations. If we do want finer grain control, then it is probably possible to do something with resumable exceptions that are thrown each time the test is potentially skipped. If the reason to identifying a check is to track its result, then the obvious solution is to not assume that it's result is pass/fail, but is instead a pair of pass/fail counts (or pass/total -- same thing). A good testing approach is "directed-random", where the same test is run multiple times with different random seeds so as to use different test data. IMO, it is reasonable to think of a one-shot test as an aberration.
