[jira] [Commented] (LUCENE-7032) jdk-9-ea+105 breaks MinimizeOperations.minimize()
[ https://issues.apache.org/jira/browse/LUCENE-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15154563#comment-15154563 ] Robert Muir commented on LUCENE-7032: - Here is the openjdk bug: https://bugs.openjdk.java.net/browse/JDK-8150280 > jdk-9-ea+105 breaks MinimizeOperations.minimize() > - > > Key: LUCENE-7032 > URL: https://issues.apache.org/jira/browse/LUCENE-7032 > Project: Lucene - Core > Issue Type: Bug >Affects Versions: master >Reporter: Robert Muir > Labels: Java9 > > As soon as jdk-9-ea+105 was put into test rotation, automaton tests have been > sporatically failing in non-reproducible ways. All of them invoke hopcroft > minimization either directly or indirectly. > The bug manifests in several forms: > * ArrayIndexOutOfBoundsException in minimize() > * IllegalArgumentException for an explicit bounds check > * incorrect resulting automaton > This method is ... let's say not the ideal one to debug something like this, > but I've at least got it where I can make it fail in a few minutes with > beasting, so we can try simple things like turning on/off jvm flags to try to > narrow it more. > It would be really great to make it fail more efficiently, but unfortunately > it takes many thousands of iterations until we understand more about it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7032) jdk-9-ea+105 breaks MinimizeOperations.minimize()
[ https://issues.apache.org/jira/browse/LUCENE-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150845#comment-15150845 ] Uwe Schindler commented on LUCENE-7032: --- Fails on Windows, too (Java HotSpot(TM) 64-Bit Server VM (build 9-ea+105-2016-02-11-003336.javare.4433.nc, mixed mode): {noformat} [junit4] Suite: org.apache.lucene.util.automaton.TestMinimize [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestMinimize -Dtests.method=testBasic -Dtests.seed=5E1BF6106DCA9EC9 -Dtests.locale=fr-GF -Dtests.timezone=Africa/Khartoum -Dtests.asserts=true -Dtests.file.encoding=Cp1252 [junit4] ERROR 0.69s | TestMinimize.testBasic <<< [junit4]> Throwable #1: java.lang.IllegalArgumentException: source=3 is out of bounds (maxState is 2) [junit4]>at __randomizedtesting.SeedInfo.seed([5E1BF6106DCA9EC9:F5E1EB05B21618E7]:0) [junit4]>at org.apache.lucene.util.automaton.Automaton.addTransition(Automaton.java:165) [junit4]>at org.apache.lucene.util.automaton.MinimizationOperations.minimize(MinimizationOperations.java:245) [junit4]>at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:537) [junit4]>at org.apache.lucene.util.automaton.RegExp.findLeaves(RegExp.java:617) [junit4]>at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:519) [junit4]>at org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:495) [junit4]>at org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:426) [junit4]>at org.apache.lucene.util.automaton.AutomatonTestUtil.randomSingleAutomaton(AutomatonTestUtil.java:262) [junit4]>at org.apache.lucene.util.automaton.AutomatonTestUtil.randomAutomaton(AutomatonTestUtil.java:276) [junit4]>at org.apache.lucene.util.automaton.TestMinimize.testBasic(TestMinimize.java:30) [junit4]>at java.lang.Thread.run(Thread.java:804) [junit4] 2> NOTE: test params are: codec=Asserting(Lucene60): {}, docValues:{}, sim=RandomSimilarity(queryNorm=true,coord=no): {}, locale=fr-GF, timezone=Africa/Khartoum [junit4] 2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 9-ea (64-bit)/cpus=8,threads=1,free=94388736,total=130023424 [junit4] 2> NOTE: All tests run in this JVM: [TestMinimize] [junit4] Completed [1/1 (1!)] in 1.60s, 1 test, 1 error <<< FAILURES! [junit4] [junit4] [junit4] Tests with failures [seed: 5E1BF6106DCA9EC9]: [junit4] - org.apache.lucene.util.automaton.TestMinimize.testBasic [junit4] [junit4] [junit4] JVM J0: 0.88 .. 3.61 = 2.73s [junit4] Execution time total: 3.64 sec. [junit4] Tests summary: 1 suite, 1 test, 1 error {noformat} > jdk-9-ea+105 breaks MinimizeOperations.minimize() > - > > Key: LUCENE-7032 > URL: https://issues.apache.org/jira/browse/LUCENE-7032 > Project: Lucene - Core > Issue Type: Bug >Affects Versions: master >Reporter: Robert Muir > Labels: Java9 > > As soon as jdk-9-ea+105 was put into test rotation, automaton tests have been > sporatically failing in non-reproducible ways. All of them invoke hopcroft > minimization either directly or indirectly. > The bug manifests in several forms: > * ArrayIndexOutOfBoundsException in minimize() > * IllegalArgumentException for an explicit bounds check > * incorrect resulting automaton > This method is ... let's say not the ideal one to debug something like this, > but I've at least got it where I can make it fail in a few minutes with > beasting, so we can try simple things like turning on/off jvm flags to try to > narrow it more. > It would be really great to make it fail more efficiently, but unfortunately > it takes many thousands of iterations until we understand more about it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7032) jdk-9-ea+105 breaks MinimizeOperations.minimize()
[ https://issues.apache.org/jira/browse/LUCENE-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150824#comment-15150824 ] Uwe Schindler commented on LUCENE-7032: --- Hi, I think we are now fine to open bug report. I did this several times. Maybe use older bug as template: - How to checkout lucene-core (modify to use GIT) - How to run the test (above command line) - How to get pure "java" command line with "ant -verbose" The OpenJDK developers (e.g., Roland Westrelin) were able to reproduce and debug. I think thats enough. I would not spend too much time in debugging this. > jdk-9-ea+105 breaks MinimizeOperations.minimize() > - > > Key: LUCENE-7032 > URL: https://issues.apache.org/jira/browse/LUCENE-7032 > Project: Lucene - Core > Issue Type: Bug >Affects Versions: master >Reporter: Robert Muir > Labels: Java9 > > As soon as jdk-9-ea+105 was put into test rotation, automaton tests have been > sporatically failing in non-reproducible ways. All of them invoke hopcroft > minimization either directly or indirectly. > The bug manifests in several forms: > * ArrayIndexOutOfBoundsException in minimize() > * IllegalArgumentException for an explicit bounds check > * incorrect resulting automaton > This method is ... let's say not the ideal one to debug something like this, > but I've at least got it where I can make it fail in a few minutes with > beasting, so we can try simple things like turning on/off jvm flags to try to > narrow it more. > It would be really great to make it fail more efficiently, but unfortunately > it takes many thousands of iterations until we understand more about it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7032) jdk-9-ea+105 breaks MinimizeOperations.minimize()
[ https://issues.apache.org/jira/browse/LUCENE-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150748#comment-15150748 ] Robert Muir commented on LUCENE-7032: - Thanks dawid, I optimized test parameters further and experimented with your suggestions. It now fails 100% reproducible for me with a single test method invocation on my linux machine (4 seconds): {noformat} rmuir@beast:~/workspace/lucene-solr/lucene/core$ ant test -Dtestcase=TestMinimize -Dtests.method="testBasic*" -Dtests.seed=5E1BF6106DCA9EC9 -Dargs="-XX:-TieredCompilation -Xbatch" <<< anting >>> -test: [junit4:pickseed] Seed property 'tests.seed' already defined: 5E1BF6106DCA9EC9 [junit4] says g'day! Master seed: 5E1BF6106DCA9EC9 [junit4] Executing 1 suite with 1 JVM. [junit4] [junit4] Started J0 PID(5919@localhost). [junit4] Suite: org.apache.lucene.util.automaton.TestMinimize [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestMinimize -Dtests.method=testBasic -Dtests.seed=5E1BF6106DCA9EC9 -Dtests.locale=fr-GF -Dtests.timezone=Africa/Khartoum -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [junit4] ERROR 0.59s | TestMinimize.testBasic <<< [junit4]> Throwable #1: java.lang.IllegalArgumentException: source=3 is out of bounds (maxState is 2) [junit4]>at __randomizedtesting.SeedInfo.seed([5E1BF6106DCA9EC9:F5E1EB05B21618E7]:0) [junit4]>at org.apache.lucene.util.automaton.Automaton.addTransition(Automaton.java:165) [junit4]>at org.apache.lucene.util.automaton.MinimizationOperations.minimize(MinimizationOperations.java:245) [junit4]>at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:537) [junit4]>at org.apache.lucene.util.automaton.RegExp.findLeaves(RegExp.java:617) [junit4]>at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:519) [junit4]>at org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:495) [junit4]>at org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:426) [junit4]>at org.apache.lucene.util.automaton.AutomatonTestUtil.randomSingleAutomaton(AutomatonTestUtil.java:262) [junit4]>at org.apache.lucene.util.automaton.AutomatonTestUtil.randomAutomaton(AutomatonTestUtil.java:276) [junit4]>at org.apache.lucene.util.automaton.TestMinimize.testBasic(TestMinimize.java:30) [junit4]>at java.lang.Thread.run(Thread.java:804) [junit4] 2> NOTE: test params are: codec=Asserting(Lucene60): {}, docValues:{}, sim=RandomSimilarity(queryNorm=true,coord=no): {}, locale=fr-GF, timezone=Africa/Khartoum [junit4] 2> NOTE: Linux 4.2.0-25-generic amd64/Oracle Corporation 9-ea (64-bit)/cpus=8,threads=1,free=162224128,total=197132288 [junit4] 2> NOTE: All tests run in this JVM: [TestMinimize] [junit4] Completed [1/1 (1!)] in 1.69s, 1 test, 1 error <<< FAILURES! [junit4] [junit4] [junit4] Tests with failures [seed: 5E1BF6106DCA9EC9]: [junit4] - org.apache.lucene.util.automaton.TestMinimize.testBasic [junit4] [junit4] [junit4] JVM J0: 0.58 .. 2.96 = 2.38s [junit4] Execution time total: 2.98 sec. [junit4] Tests summary: 1 suite, 1 test, 1 error BUILD FAILED /home/rmuir/workspace/lucene-solr/lucene/common-build.xml:1457: The following error occurred while executing this line: /home/rmuir/workspace/lucene-solr/lucene/common-build.xml:1014: There were test failures: 1 suite, 1 test, 1 error [seed: 5E1BF6106DCA9EC9] Total time: 4 seconds {noformat} Now, to think about how to boil it down from here. > jdk-9-ea+105 breaks MinimizeOperations.minimize() > - > > Key: LUCENE-7032 > URL: https://issues.apache.org/jira/browse/LUCENE-7032 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > As soon as jdk-9-ea+105 was put into test rotation, automaton tests have been > sporatically failing in non-reproducible ways. All of them invoke hopcroft > minimization either directly or indirectly. > The bug manifests in several forms: > * ArrayIndexOutOfBoundsException in minimize() > * IllegalArgumentException for an explicit bounds check > * incorrect resulting automaton > This method is ... let's say not the ideal one to debug something like this, > but I've at least got it where I can make it fail in a few minutes with > beasting, so we can try simple things like turning on/off jvm flags to try to > narrow it more. > It would be really great to make it fail more efficiently, but unfortunately > it takes many thousands of iterations until we understand more about it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) ---
[jira] [Commented] (LUCENE-7032) jdk-9-ea+105 breaks MinimizeOperations.minimize()
[ https://issues.apache.org/jira/browse/LUCENE-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150074#comment-15150074 ] Dawid Weiss commented on LUCENE-7032: - I'd try to make it more consistent by: 1) trying to make the compiler deterministic (I'm guessing C2), 2) trying to make the compiler run sequentially, 3) maybe we can lower some compilation threshold a bit or deoptimize a throw some noise at the compiler... if it speeds up the repro, otherwise it doesn't make sense as it'd contribute lots of irrelevant compilation logs. Options for 1 and 2 are: {code} -XX:-TieredCompilation -Xbatch -XX:CICompilerCount=1 {code} Potential options for 3 are: {code} -XX:+DeoptimizeALot -XX:+MinInliningThreshold=250 (default) -XX:+CompileThreshold=1 (default) {code} > jdk-9-ea+105 breaks MinimizeOperations.minimize() > - > > Key: LUCENE-7032 > URL: https://issues.apache.org/jira/browse/LUCENE-7032 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > As soon as jdk-9-ea+105 was put into test rotation, automaton tests have been > sporatically failing in non-reproducible ways. All of them invoke hopcroft > minimization either directly or indirectly. > The bug manifests in several forms: > * ArrayIndexOutOfBoundsException in minimize() > * IllegalArgumentException for an explicit bounds check > * incorrect resulting automaton > This method is ... let's say not the ideal one to debug something like this, > but I've at least got it where I can make it fail in a few minutes with > beasting, so we can try simple things like turning on/off jvm flags to try to > narrow it more. > It would be really great to make it fail more efficiently, but unfortunately > it takes many thousands of iterations until we understand more about it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7032) jdk-9-ea+105 breaks MinimizeOperations.minimize()
[ https://issues.apache.org/jira/browse/LUCENE-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150063#comment-15150063 ] Dawid Weiss commented on LUCENE-7032: - There was no script for that, unfortunately -- I did it with mercurial's bisect on the hotspot repo manually (rebuilding the jvm.so each time and re-running Lucene tests on the rebuilt version). > jdk-9-ea+105 breaks MinimizeOperations.minimize() > - > > Key: LUCENE-7032 > URL: https://issues.apache.org/jira/browse/LUCENE-7032 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > As soon as jdk-9-ea+105 was put into test rotation, automaton tests have been > sporatically failing in non-reproducible ways. All of them invoke hopcroft > minimization either directly or indirectly. > The bug manifests in several forms: > * ArrayIndexOutOfBoundsException in minimize() > * IllegalArgumentException for an explicit bounds check > * incorrect resulting automaton > This method is ... let's say not the ideal one to debug something like this, > but I've at least got it where I can make it fail in a few minutes with > beasting, so we can try simple things like turning on/off jvm flags to try to > narrow it more. > It would be really great to make it fail more efficiently, but unfortunately > it takes many thousands of iterations until we understand more about it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7032) jdk-9-ea+105 breaks MinimizeOperations.minimize()
[ https://issues.apache.org/jira/browse/LUCENE-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149918#comment-15149918 ] Robert Muir commented on LUCENE-7032: - The flags passed by our jenkins randomization (-Dargs="-XX:+UseCompressedOops -XX:+UseParallelGC -XX:-UseSuperWord") are all irrelevant. It fails in the same way with any garbage collector and on/off for the other two. So my reproduction line is something like this: {noformat} ant beast -Dtestcase=TestMinimize -Dtests.method="testBasic*" -Dtests.iters=1000 -Dtests.multiplier=3 -Dtests.seed=5E1BF6106DCA9EC9 -Dtests.failfast=yes -Dbeast.iters=5 -Dtests.dups=8 -Dtests.jvms=8 {noformat} I lowered beast.iters to 5, even though so far it always fails in the first iteration... we want to be careful since its only failing once per thousands (inefficient). This is probably good enough to slowly bisect hotspot to find the problematic commit, or we could play a guessing game with jvm flags. [~dawid.weiss], I know you have done this before: do you still have the bisection script? Maybe we should commit it to our dev-tools if you still have it around. > jdk-9-ea+105 breaks MinimizeOperations.minimize() > - > > Key: LUCENE-7032 > URL: https://issues.apache.org/jira/browse/LUCENE-7032 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > As soon as jdk-9-ea+105 was put into test rotation, automaton tests have been > sporatically failing in non-reproducible ways. All of them invoke hopcroft > minimization either directly or indirectly. > The bug manifests in several forms: > * ArrayIndexOutOfBoundsException in minimize() > * IllegalArgumentException for an explicit bounds check > * incorrect resulting automaton > This method is ... let's say not the ideal one to debug something like this, > but I've at least got it where I can make it fail in a few minutes with > beasting, so we can try simple things like turning on/off jvm flags to try to > narrow it more. > It would be really great to make it fail more efficiently, but unfortunately > it takes many thousands of iterations until we understand more about it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7032) jdk-9-ea+105 breaks MinimizeOperations.minimize()
[ https://issues.apache.org/jira/browse/LUCENE-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149858#comment-15149858 ] Robert Muir commented on LUCENE-7032: - Here is the current best I have. Its still very inefficient and slow, but tends to fail in less than five minutes: {noformat} rmuir@beast:~/workspace/lucene-solr/lucene/core$ ant beast -Dtestcase=TestMinimize -Dtests.method="testBasic*" -Dargs="-XX:+UseCompressedOops -XX:+UseParallelGC -XX:-UseSuperWord" -Dtests.iters=1000 -Dtests.multiplier=3 -Dtests.seed=5E1BF6106DCA9EC9 -Dtests.failfast=yes -Dbeast.iters=1000 -Dtests.dups=8 -Dtests.jvms=8 Buildfile: /home/rmuir/workspace/lucene-solr/lucene/core/build.xml -clover.disable: ivy-availability-check: ivy-fail: ivy-configure: [ivy:configure] :: Apache Ivy 2.3.0 - 20130110142753 :: http://ant.apache.org/ivy/ :: [ivy:configure] :: loading settings :: file = /home/rmuir/workspace/lucene-solr/lucene/ivy-settings.xml -clover.load: -clover.classpath: -clover.setup: clover: -check-git-state: -git-cleanroot: -copy-git-state: git-autoclean: resolve: init: compile-core: compile-test-framework: -check-git-state: -git-cleanroot: -copy-git-state: git-autoclean: ivy-availability-check: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/rmuir/workspace/lucene-solr/lucene/ivy-settings.xml resolve: init: compile-lucene-core: -check-git-state: -git-cleanroot: -copy-git-state: git-autoclean: ivy-availability-check: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/rmuir/workspace/lucene-solr/lucene/ivy-settings.xml resolve: init: -clover.disable: -clover.load: -clover.classpath: -clover.setup: clover: compile-core: compile-codecs: -check-git-state: -git-cleanroot: -copy-git-state: git-autoclean: ivy-availability-check: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/rmuir/workspace/lucene-solr/lucene/ivy-settings.xml resolve: common.init: compile-lucene-core: init: -clover.disable: -clover.load: -clover.classpath: -clover.setup: clover: compile-core: -clover.disable: -clover.load: -clover.classpath: -clover.setup: clover: common.compile-core: compile-core: compile-test: install-junit4-taskdef: validate: resolve-groovy: [ivy:cachepath] :: resolving dependencies :: org.codehaus.groovy#groovy-all-caller;working [ivy:cachepath] confs: [default] [ivy:cachepath] found org.codehaus.groovy#groovy-all;2.4.4 in public [ivy:cachepath] :: resolution report :: resolve 12ms :: artifacts dl 0ms - | |modules|| artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| - | default | 1 | 0 | 0 | 0 || 1 | 0 | - -init-totals: -beast: [beaster] Beast round: 1 [junit4] Duplicate suite name used with XML reports: org.apache.lucene.util.automaton.TestMinimize. This may confuse tools that process XML reports. Set 'ignoreDuplicateSuites' to true to skip this message. [beaster] Executing 8 suites with 8 JVMs. [beaster] [beaster] Started J1 PID(8867@localhost). [beaster] Started J6 PID(8857@localhost). [beaster] Started J5 PID(8859@localhost). [beaster] Started J3 PID(8861@localhost). [beaster] Started J4 PID(8862@localhost). [beaster] Started J0 PID(8858@localhost). [beaster] Started J7 PID(8860@localhost). [beaster] Started J2 PID(8863@localhost). [beaster] 2> NOTE: reproduce with: ant test -Dtestcase=TestMinimize -Dtests.method=testBasic -Dtests.seed=5E1BF6106DCA9EC9 -Dtests.multiplier=3 -Dtests.locale=fr-GF -Dtests.timezone=Africa/Khartoum -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [beaster] [23:43:35.303] ERROR 0.62s J0 | TestMinimize.testBasic { seed=[5E1BF6106DCA9EC9:3C97F1414A872260]} <<< [beaster]> Throwable #1: java.lang.IllegalArgumentException: source=3 is out of bounds (maxState is 2) [beaster]>at __randomizedtesting.SeedInfo.seed([5E1BF6106DCA9EC9:3C97F1414A872260]:0) [beaster]>at org.apache.lucene.util.automaton.Automaton.addTransition(Automaton.java:165) [beaster]>at org.apache.lucene.util.automaton.MinimizationOperations.minimize(MinimizationOperations.java:245) [beaster]>at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:537) [beaster]>at org.apache.lucene.util.automaton.RegExp.findLeaves(RegExp.java:617) [beaster]>at org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:521) [beaster]>at org.apache.lucene.util.automat