Thanks Chris, That very likely is the reason. I had noticed the seed and realized that it will be controlling the random input generation for the tests to make failures reproducible. However, i didn't consider that it can also cause test skipping.
Thanks! Nawab On Thu, Oct 5, 2017 at 3:13 PM, Chris Hostetter <hossman_luc...@fucit.org> wrote: > > : I am seeing that in different test runs (e.g., by executing 'ant test' on > : the root folder in 'lucene-solr') a different subset of tests are > skipped. > : Where can I find more about it? I am trying to create parity between test > : successes before and after my changes and this is causing confusion. > > The test randomization logic creates an arbitrary "master seed" that is > assigned by ant. This master seed is > then used to generate some randomized default properties for the the > forked JVMs (default timezones, default Locale, default charset, etc...) > > Each test class run in a forked JVM then gets it's own Random seed > (generated fro mthe master seed as well) which the solr test-framework > uses to randomize some more things (that are specific to the solr > test-framework. > > In some cases, tests have @Assume of assumeThat(...) logic in if we know > that certain tests are completely incompatible with certain randomized > aspects of the environemnt -- for example: some tests won't bothe to run > if the randomized Locale uses "tr" because of external third-party > dependencies that break with this Locale (due to upercase/lowercase > behavior). > > This is most likeley the reason you are seeing a diff "set" of tests run > on diff times. But if you want true parity between test runs, use the > same master seed -- which is printed at the begining of every "ant > test" run, as well as any time a test fails, and can be overridden on the > ant command line for future runs. > > run "ant test-help" for the specifics. > > > -Hoss > http://www.lucidworks.com/ >
