Re: [JENKINS] Lucene-main-Windows (64bit/jdk-19) - Build # 11672 - Unstable!
Looks scary but I can't reproduce it (on JDK19 as well). D. On Thu, Nov 17, 2022 at 7:55 AM Policeman Jenkins Server < jenk...@thetaphi.de> wrote: > Build: https://jenkins.thetaphi.de/job/Lucene-main-Windows/11672/ > Java: 64bit/jdk-19 -XX:-UseCompressedOops -XX:+UseG1GC > > 1 tests failed. > FAILED: > org.apache.lucene.codecs.simpletext.TestSimpleTextDocValuesFormat.testSortedMergeAwayAllValuesLargeSegment > > Error Message: > java.lang.AssertionError: expected:<2147483647> but was:<0> > > Stack Trace: > java.lang.AssertionError: expected:<2147483647> but was:<0> > at > __randomizedtesting.SeedInfo.seed([3EE72AD2946984D1:1D190FBA7F2B848E]:0) > at org.junit.Assert.fail(Assert.java:89) > at org.junit.Assert.failNotEquals(Assert.java:835) > at org.junit.Assert.assertEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:633) > at > org.apache.lucene.tests.index.BaseDocValuesFormatTestCase.testSortedMergeAwayAllValuesLargeSegment(BaseDocValuesFormatTestCase.java:3398) > at > java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) > at java.base/java.lang.reflect.Method.invoke(Method.java:578) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) > at > org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) > at > org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) > at > org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at > org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) > at > org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) > at > org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at > org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at > org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) >
Re: [lucene] branch main updated: Prevent NPEs while still handling the polar case for horizontal planes right off the pole
Hi Karl,
This commit broke the build because code formatting was off (this was fixed
in a subsequent, unrelated commit).
I spent some time looking for the issue to check what happened and couldn't
find it anywhere. Github's PR infrastructure
makes it quite convenient to ensure everything passes before it's merged
and it leaves a handy
place to add comments in case something doesn't work - I highly recommend
it.
Dawid
On Thu, Nov 17, 2022 at 2:19 AM wrote:
> This is an automated email from the ASF dual-hosted git repository.
>
> kwright pushed a commit to branch main
> in repository https://gitbox.apache.org/repos/asf/lucene.git
>
>
> The following commit(s) were added to refs/heads/main by this push:
> new b6ebfd18610 Prevent NPEs while still handling the polar case for
> horizontal planes right off the pole
> b6ebfd18610 is described below
>
> commit b6ebfd18610c482109c6a38b2327254848508f03
> Author: Karl David Wright
> AuthorDate: Wed Nov 16 11:03:24 2022 -0500
>
> Prevent NPEs while still handling the polar case for horizontal planes
> right off the pole
> ---
> .../java/org/apache/lucene/spatial3d/geom/Plane.java | 20
>
> 1 file changed, 16 insertions(+), 4 deletions(-)
>
> diff --git
> a/lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/Plane.java
> b/lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/Plane.java
> index ef9e9773223..9b46c3553bf 100755
> --- a/lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/Plane.java
> +++ b/lucene/spatial3d/src/java/org/apache/lucene/spatial3d/geom/Plane.java
> @@ -1500,9 +1500,14 @@ public class Plane extends Vector {
>} else {
> // Since a==b==0, any plane including the Z axis suffices.
> // System.err.println(" Perpendicular to z");
> -final GeoPoint[] points =
> +GeoPoint[] points =
> findIntersections(planetModel, normalYPlane, NO_BOUNDS,
> NO_BOUNDS);
> -if (points.length > 0) {
> +if (points.length == 0) {
> + points = findIntersections(planetModel, normalXPlane,
> NO_BOUNDS, NO_BOUNDS);
> +}
> +if (points.length == 0) {
> + boundsInfo.addZValue(new GeoPoint(0.0, 0.0, -this.z));
> +} else {
>boundsInfo.addZValue(points[0]);
> }
>}
> @@ -2042,9 +2047,16 @@ public class Plane extends Vector {
> }
>} else {
> // Horizontal circle. Since a==b, any vertical plane suffices.
> -final GeoPoint[] points =
> +GeoPoint[] points =
> findIntersections(planetModel, normalXPlane, NO_BOUNDS,
> NO_BOUNDS);
> -boundsInfo.addZValue(points[0]);
> +if (points.length == 0) {
> + points = findIntersections(planetModel, normalYPlane,
> NO_BOUNDS, NO_BOUNDS);
> +}
> +if (points.length == 0) {
> + boundsInfo.addZValue(new GeoPoint(0.0, 0.0, -this.z));
> +} else {
> + boundsInfo.addZValue(points[0]);
> +}
>}
>// System.err.println("Done latitude bounds");
> }
>
>
Re: Release Lucene 9.4.2
+1, thanks for the patience. I feel we at least made the effort to root out any more of these and hopefully prevent a 9.4.3 with another overflow bug. On Wed, Nov 16, 2022 at 10:55 AM Adrien Grand wrote: > > It looks like we're good with the changes we wanted to get in for 9.4.2. > > I plan on starting the release process tomorrow if there are no objections. > > On Fri, Nov 11, 2022 at 4:22 PM Robert Muir wrote: >> >> These are the 9.4.2 completed issues: >> >> https://github.com/apache/lucene/pull/11905 <-- bug and associated monster >> test >> https://github.com/apache/lucene/pull/11916 <-- checkindex improvement >> https://github.com/apache/lucene/pull/11919 <-- checkindex improvement >> >> These are the remaining issues: >> >> https://github.com/apache/lucene/pull/11918 <-- better error messages, >> looks close to being merged >> https://github.com/apache/lucene/issues/11910 <-- static analysis: >> after discussion on the issue, let's consider just doing a "one-time" >> pass to look for more problems? >> >> On Fri, Nov 11, 2022 at 9:52 AM Michael Sokolov wrote: >> > >> > +1 makes sense. I do think given this is the second similar-flavored >> > bug we've found that we should be thorough and try to get them all >> > rather than having a 9.4.3 ... >> > >> > On Wed, Nov 9, 2022 at 10:25 AM Julie Tibshirani >> > wrote: >> > > >> > > +1 from me for a bugfix release once we've solidified testing. Thanks to >> > > everyone working on improving tests and static analysis -- this now is >> > > our second time encountering a bad arithmetic bug and it's important to >> > > get ahead of these issues! >> > > >> > > Julie >> > > >> > > On Wed, Nov 9, 2022 at 8:26 AM Robert Muir wrote: >> > >> >> > >> Thank you Adrien! >> > >> >> > >> I created an issue for the static analysis piece, but I'm not >> > >> currently working on it yet. This could be a fun one, if anyone is >> > >> interested, to flush a bunch of these bugs out at once: >> > >> https://github.com/apache/lucene/issues/11910 >> > >> >> > >> On Wed, Nov 9, 2022 at 10:48 AM Adrien Grand wrote: >> > >> > >> > >> > Totally Robert, I was not trying to add any time pressure, next week >> > >> > is totally fine. I mostly wanted to get the discussion started >> > >> > because folks sometimes have one or two bug fixes they'd like to fold >> > >> > into a bugfix release so I wanted to give them time to plan. Friday >> > >> > is also a public holiday here, celebrating the end of World War 1. :) >> > >> > >> > >> > On Wed, Nov 9, 2022 at 4:41 PM Robert Muir wrote: >> > >> >> >> > >> >> Can we please have a few days to improve the test situation? I think >> > >> >> we need to beef up checkindex to exercise seek() on the vectors, also >> > >> >> we need to look at static analysis to try to find other similar bugs. >> > >> >> This would help prevent "whack-a-mole" and improve correctness going >> > >> >> forwards. >> > >> >> >> > >> >> I want to help more but it's difficult timing-wise, lots of stuff >> > >> >> going on this week, and in my country friday is Veteran's Day >> > >> >> holiday. >> > >> >> >> > >> >> On Wed, Nov 9, 2022 at 10:39 AM Adrien Grand >> > >> >> wrote: >> > >> >> > >> > >> >> > Hello all, >> > >> >> > >> > >> >> > A bad integer overflow has been discovered in the KNN vectors >> > >> >> > format, which affects segments that have more than ~16M vectors. >> > >> >> > I'd like to do a bugfix release when the bug is fixed and we have >> > >> >> > a test for such large datasets of KNN vectors. I volunteer to be >> > >> >> > the RM for this release. >> > >> >> > >> > >> >> > -- >> > >> >> > Adrien >> > >> >> >> > >> >> - >> > >> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> > >> >> For additional commands, e-mail: dev-h...@lucene.apache.org >> > >> >> >> > >> > >> > >> > >> > >> > -- >> > >> > Adrien >> > >> >> > >> - >> > >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> > >> For additional commands, e-mail: dev-h...@lucene.apache.org >> > >> >> > >> > - >> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> > For additional commands, e-mail: dev-h...@lucene.apache.org >> > >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> > > > -- > Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Release Lucene 9.4.2
It looks like we're good with the changes we wanted to get in for 9.4.2. I plan on starting the release process tomorrow if there are no objections. On Fri, Nov 11, 2022 at 4:22 PM Robert Muir wrote: > These are the 9.4.2 completed issues: > > https://github.com/apache/lucene/pull/11905 <-- bug and associated > monster test > https://github.com/apache/lucene/pull/11916 <-- checkindex improvement > https://github.com/apache/lucene/pull/11919 <-- checkindex improvement > > These are the remaining issues: > > https://github.com/apache/lucene/pull/11918 <-- better error messages, > looks close to being merged > https://github.com/apache/lucene/issues/11910 <-- static analysis: > after discussion on the issue, let's consider just doing a "one-time" > pass to look for more problems? > > On Fri, Nov 11, 2022 at 9:52 AM Michael Sokolov > wrote: > > > > +1 makes sense. I do think given this is the second similar-flavored > > bug we've found that we should be thorough and try to get them all > > rather than having a 9.4.3 ... > > > > On Wed, Nov 9, 2022 at 10:25 AM Julie Tibshirani > wrote: > > > > > > +1 from me for a bugfix release once we've solidified testing. Thanks > to everyone working on improving tests and static analysis -- this now is > our second time encountering a bad arithmetic bug and it's important to get > ahead of these issues! > > > > > > Julie > > > > > > On Wed, Nov 9, 2022 at 8:26 AM Robert Muir wrote: > > >> > > >> Thank you Adrien! > > >> > > >> I created an issue for the static analysis piece, but I'm not > > >> currently working on it yet. This could be a fun one, if anyone is > > >> interested, to flush a bunch of these bugs out at once: > > >> https://github.com/apache/lucene/issues/11910 > > >> > > >> On Wed, Nov 9, 2022 at 10:48 AM Adrien Grand > wrote: > > >> > > > >> > Totally Robert, I was not trying to add any time pressure, next > week is totally fine. I mostly wanted to get the discussion started because > folks sometimes have one or two bug fixes they'd like to fold into a bugfix > release so I wanted to give them time to plan. Friday is also a public > holiday here, celebrating the end of World War 1. :) > > >> > > > >> > On Wed, Nov 9, 2022 at 4:41 PM Robert Muir > wrote: > > >> >> > > >> >> Can we please have a few days to improve the test situation? I > think > > >> >> we need to beef up checkindex to exercise seek() on the vectors, > also > > >> >> we need to look at static analysis to try to find other similar > bugs. > > >> >> This would help prevent "whack-a-mole" and improve correctness > going forwards. > > >> >> > > >> >> I want to help more but it's difficult timing-wise, lots of stuff > > >> >> going on this week, and in my country friday is Veteran's Day > holiday. > > >> >> > > >> >> On Wed, Nov 9, 2022 at 10:39 AM Adrien Grand > wrote: > > >> >> > > > >> >> > Hello all, > > >> >> > > > >> >> > A bad integer overflow has been discovered in the KNN vectors > format, which affects segments that have more than ~16M vectors. I'd like > to do a bugfix release when the bug is fixed and we have a test for such > large datasets of KNN vectors. I volunteer to be the RM for this release. > > >> >> > > > >> >> > -- > > >> >> > Adrien > > >> >> > > >> >> > - > > >> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > >> >> For additional commands, e-mail: dev-h...@lucene.apache.org > > >> >> > > >> > > > >> > > > >> > -- > > >> > Adrien > > >> > > >> - > > >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > >> For additional commands, e-mail: dev-h...@lucene.apache.org > > >> > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > -- Adrien
Re: [JENKINS] Lucene-9.x-MacOSX (64bit/jdk-18) - Build # 1386 - Failure!
No problem. I resorted to looking at their code... Often it makes me long for the simpler days of ant... Dawid On Wed, Nov 16, 2022 at 1:31 PM Robert Muir wrote: > Thanks again for cleaning this hack up Dawid. I was cursing gradle all > night, could not believe that sometimes it uses java.exe (with a bunch > of internal api violations) and othertimes uses javac.exe. > > On Wed, Nov 16, 2022 at 4:12 AM Dawid Weiss wrote: > > > > > > I've committed a fix on main and checked that it works with error prone, > in process compilation and alt javac. But double checking would be probably > good. :) > > > > Dawid > > > > On Wed, Nov 16, 2022 at 12:18 AM Robert Muir wrote: > >> > >> It is my fault. I will revert my changes and test with "alternate > >> toolchain". I think we have to hold things a bit differently in that > >> case. Sorry for all the noise. > >> > >> On Tue, Nov 15, 2022 at 6:07 PM Policeman Jenkins Server > >> wrote: > >> > > >> > Build: https://jenkins.thetaphi.de/job/Lucene-9.x-MacOSX/1386/ > >> > Java: 64bit/jdk-18 -XX:-UseCompressedOops -XX:+UseSerialGC > >> > > >> > No tests ran. > >> > >> - > >> To unsubscribe, e-mail: builds-unsubscr...@lucene.apache.org > >> For additional commands, e-mail: builds-h...@lucene.apache.org > >> > > - > To unsubscribe, e-mail: builds-unsubscr...@lucene.apache.org > For additional commands, e-mail: builds-h...@lucene.apache.org > >
Re: [JENKINS] Lucene-9.x-MacOSX (64bit/jdk-18) - Build # 1386 - Failure!
Thanks again for cleaning this hack up Dawid. I was cursing gradle all night, could not believe that sometimes it uses java.exe (with a bunch of internal api violations) and othertimes uses javac.exe. On Wed, Nov 16, 2022 at 4:12 AM Dawid Weiss wrote: > > > I've committed a fix on main and checked that it works with error prone, in > process compilation and alt javac. But double checking would be probably > good. :) > > Dawid > > On Wed, Nov 16, 2022 at 12:18 AM Robert Muir wrote: >> >> It is my fault. I will revert my changes and test with "alternate >> toolchain". I think we have to hold things a bit differently in that >> case. Sorry for all the noise. >> >> On Tue, Nov 15, 2022 at 6:07 PM Policeman Jenkins Server >> wrote: >> > >> > Build: https://jenkins.thetaphi.de/job/Lucene-9.x-MacOSX/1386/ >> > Java: 64bit/jdk-18 -XX:-UseCompressedOops -XX:+UseSerialGC >> > >> > No tests ran. >> >> - >> To unsubscribe, e-mail: builds-unsubscr...@lucene.apache.org >> For additional commands, e-mail: builds-h...@lucene.apache.org >> - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-9.x-MacOSX (64bit/jdk-18) - Build # 1386 - Failure!
I've committed a fix on main and checked that it works with error prone, in process compilation and alt javac. But double checking would be probably good. :) Dawid On Wed, Nov 16, 2022 at 12:18 AM Robert Muir wrote: > It is my fault. I will revert my changes and test with "alternate > toolchain". I think we have to hold things a bit differently in that > case. Sorry for all the noise. > > On Tue, Nov 15, 2022 at 6:07 PM Policeman Jenkins Server > wrote: > > > > Build: https://jenkins.thetaphi.de/job/Lucene-9.x-MacOSX/1386/ > > Java: 64bit/jdk-18 -XX:-UseCompressedOops -XX:+UseSerialGC > > > > No tests ran. > > - > To unsubscribe, e-mail: builds-unsubscr...@lucene.apache.org > For additional commands, e-mail: builds-h...@lucene.apache.org > >
