I think this information could sit well within the dev-docs in lucene i.e. "randomized testing in lucene". This would make it more discoverable as well and there is already a lack of proper docs as Dawid pointed?. We could add some references to docs like randomized testing core concepts <https://github.com/randomizedtesting/randomizedtesting/wiki/Core-Concepts>. If this idea makes sense I could try to write some doc around it and open a PR. I would like to know your thoughts on this? Thanks!
- Shubham On Thu, Feb 15, 2024 at 10:23 PM Dawid Weiss <dawid.we...@gmail.com> wrote: > > Sorry, the docs are not the best, I know. > > It's documented here - > > https://github.com/randomizedtesting/randomizedtesting/blob/master/randomized-runner/src/main/java/com/carrotsearch/randomizedtesting/SysGlobals.java#L186-L197 > > So: > > 1) if you pass tests.timeoutSuite=1000 this changes the default value for > all classes that don't define any explicit timeout using an annotation; > classes that do have an annotation, > use the annotation's value, > 2) if you pass tests.timeoutSuite=1000! then this overrides everything - > the default value and all annotations. > > I vaguely recall option (2) was added specifically for nightlies which > bumped the iteration multiplier - this affected tests that normally ran > fairly fast > but during nightly runs could run slower than anticipated. > > D. > > > On Thu, Feb 15, 2024 at 3:18 PM David Smiley <dsmi...@apache.org> wrote: > >> Oh; I didn't know that took precedence -- makes sense. Hopefully a >> test subclass (like SolrTestCase) could override it as well. >> >> On Mon, Feb 12, 2024 at 2:09 PM Dawid Weiss <dawid.we...@gmail.com> >> wrote: >> > >> > >> > You can override the defaults using sysprops in your CI builds - >> > >> > -Ptests.timeoutSuite=1000! >> > >> > takes precedence over any annotations (1 second). >> > >> > Dawid >> > >> > On Mon, Feb 12, 2024 at 7:53 PM David Smiley <dsmi...@apache.org> >> wrote: >> >> >> >> Looking at LuceneTestCase, I see the annotation from RandomizedTesting: >> >> @TimeoutSuite(millis = 2 * TimeUnits.HOUR) >> >> This matches my observations of some builds that timed out, perhaps >> >> some flaky test hanging in Solr (that extends LuceneTestCase). >> >> Looking at this annotation, there is further documentation that the >> >> default can be set via sysprop tests.timeoutSuite. Wouldn't doing >> >> that make more sense than hard-coding this figure in LuceneTestCase? >> >> For example, I'd like to have a normal/default test run have a low >> >> timeout (10min?) but on a "nightly" run on CI, use much higher. Not 2 >> >> hours though; individual tests needing so much should have a >> >> TimeoutSuite applied to them. >> >> >> >> ~ David Smiley >> >> Apache Lucene/Solr Search Developer >> >> http://www.linkedin.com/in/davidwsmiley >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >>
