[
https://issues.apache.org/jira/browse/LUCENE-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17181253#comment-17181253
]
Dawid Weiss commented on LUCENE-9472:
-------------------------------------
Please read what's in help/tests.txt. Simple test duplication is already
implemented (as tests.iters). The problem is that in many cases this isn't
sufficient because of static class-level initialization blocks (not just in
tests, but also in third party code), external randomization (locale, encoding,
etc.) and other factors.
In great majority of cases tests.iters is the best and fastest way to multiply
the execution of a single test (include pre/post hooks etc.).
> True concurrent test suite duplication and execution (beasting)
> ---------------------------------------------------------------
>
> Key: LUCENE-9472
> URL: https://issues.apache.org/jira/browse/LUCENE-9472
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Dawid Weiss
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I toyed with this a bit because it's hacker's delight... Forked the default
> Gradle test task and made it emit the same test/ suite name multiple times.
> This results in the same test being executed multiple times:
> {code}
> # ./gradlew -p lucene\analysis\kuromoji testy --tests
> TestToStringUtil.testHepburn -Ptests.dups=10
> > Task :randomizationInfo
> Running tests with randomization seed: tests.seed=88C21B0E8ABB2D46
> > Task :lucene:analysis:kuromoji:testy
> :lucene:analysis:kuromoji:testy (SUCCESS): 10 test(s)
> BUILD SUCCESSFUL in 13s
> 21 actionable tasks: 6 executed, 15 up-to-date
> {code}
> Upsides:
> * Everything configures itself because the custom task is a subtype of
> Gradle's Test task,
> * Tests are run in parallel, on multiple VMs,
> * Opens up the possibility of reordering test suites (for load-balancing).
> Downsides:
> * Requires a custom(ized) clone of Gradle's internal classes. Will require
> maintenance when upgrading gradle. There is no certainty it'll work.
> * Some things visibly don't like duplicated suite names - the progress bar
> shows the suite once, even though it is executed concurrently in multiple VMs,
> * XML reports will drive jenkins and other tooling crazy:
> {code}
> <?xml version="1.0" encoding="UTF-8"?>
> <testsuite name="org.apache.lucene.analysis.ja.util.TestToStringUtil"
> tests="10" skipped="0" failures="0" errors="0"
> timestamp="2020-08-20T12:54:27" hostname="DWEISS-DESKTOP" time="3.177">
> <properties/>
> <testcase name="testHepburn"
> classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.04"/>
> <testcase name="testHepburn"
> classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.045"/>
> <testcase name="testHepburn"
> classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.043"/>
> <testcase name="testHepburn"
> classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.038"/>
> <testcase name="testHepburn"
> classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.008"/>
> <testcase name="testHepburn"
> classname="org.apache.lucene.analysis.ja.util.TestToStringUtil" time="1.005"/>
> ....
> {code}
> * not sure what happens with sysouts from such concurrent runs...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
