Re: [VOTE] Release Lucene 9.1.0 RC2
I found a way to disable the log4shell patch on my work machine, and got a pass: SUCCESS! [1:00:57.577806] I don't think we need to be concerned about trying to adapt to this thing. It only conflicts with the test framework, and devs can turn it off to run tests. Somehow our automated builds don't seem to be affected. so, +1 from me On Fri, Mar 18, 2022 at 10:38 PM Patrick Zhai wrote: > > Hi Julie, thanks for summarizing the issues, for 4, as far as I understand, > there's nothing that needs to be changed on lucece's end. We're still having > trouble running the smoke test on Amazon's machine, but I've tried it on my > own machine and it passed. > > So here's my +1. SUCCESS! [0:59:35.291572] > > > On Fri, Mar 18, 2022 at 2:26 PM Julie Tibshirani wrote: >> >> Hi everyone, people have run into several test failures, here's a summary to >> make it easier to follow: >> >> Some tests can be very slow with certain random parameters, and this can >> cause the smoke tester to take several hours -- this seems like a test-only >> problem, and we're working to address it in >> https://issues.apache.org/jira/browse/LUCENE-10473. >> TestMatchAllDocsQuery#testEarlyTermination can fail with assertion -- this >> is a test-only issue, and has already been fixed in >> https://issues.apache.org/jira/browse/LUCENE-10472. >> TestIndexWriterMergePolicy#testStressUpdateSameDocumentWithMergeOnGetReader >> can OOM -- this is a known issue and is not expected to occur outside of >> tests (https://issues.apache.org/jira/browse/LUCENE-10088) >> Security manager issues from Amazon's log4shell patch -- not sure this >> requires a change on our end? It sounds like Mike is investigating the >> issues now (thanks!) >> Test suite may also stall on some machines? -- Uwe mentioned "Looks like a >> problem with the test or some wrong assumption or some missing synchronized >> block (the Policeman Jenkins machine has many cores, so issues with not >> synchronized cache lines can happen easily)." Is this a different problem >> from #1 where we just have slow tests? I'm not sure if this is something we >> want to investigate as part of the release or if we think it can wait. >> >> Thanks, >> Julie >> >> On Fri, Mar 18, 2022 at 12:17 PM Michael Sokolov wrote: >>> >>> We had to do a workaround in our internal test suites by setting a >>> system property to trick this thing into not running; Maybe we can >>> apply that also here... >>> >>> On Fri, Mar 18, 2022 at 3:12 PM Dawid Weiss wrote: >>> > >>> > I think this is Amazon trying to cope with log4shell - they've added >>> > external instrumentation. >>> > >>> > On Fri, Mar 18, 2022 at 7:51 PM Robert Muir wrote: >>> > > >>> > > >> 2>at >>> > > >> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>> > > >> 2>at Log4jHotPatch.asmVersion(Log4jHotPatch.java:71) >>> > > >> 2>at Log4jHotPatch.agentmain(Log4jHotPatch.java:93) >>> > > >> 2>... 6 more >>> > > > >>> > > > >>> > > > Does anyone have an idea? >>> > > > >>> > > >>> > > Your JVM is broken, look at this Log4jHotPatch stuff causing trouble. >>> > > >>> > > - >>> > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> > > For additional commands, e-mail: dev-h...@lucene.apache.org >>> > > >>> > >>> > - >>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> > For additional commands, e-mail: dev-h...@lucene.apache.org >>> > >>> >>> - >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [VOTE] Release Lucene 9.1.0 RC2
Hi Julie, thanks for summarizing the issues, for 4, as far as I understand, there's nothing that needs to be changed on lucece's end. We're still having trouble running the smoke test on Amazon's machine, but I've tried it on my own machine and it passed. So here's my +1. SUCCESS! [0:59:35.291572] On Fri, Mar 18, 2022 at 2:26 PM Julie Tibshirani wrote: > Hi everyone, people have run into several test failures, here's a summary > to make it easier to follow: > >1. Some tests can be very slow with certain random parameters, and >this can cause the smoke tester to take several hours -- this seems like a >test-only problem, and we're working to address it in >https://issues.apache.org/jira/browse/LUCENE-10473. >2. TestMatchAllDocsQuery#testEarlyTermination can fail with assertion >-- this is a test-only issue, and has already been fixed in >https://issues.apache.org/jira/browse/LUCENE-10472. >3. > TestIndexWriterMergePolicy#testStressUpdateSameDocumentWithMergeOnGetReader >can OOM -- this is a known issue and is not expected to occur outside of >tests (https://issues.apache.org/jira/browse/LUCENE-10088) >4. *Security manager issues from Amazon's log4shell patch* -- not sure >this requires a change on our end? It sounds like Mike is investigating the >issues now (thanks!) >5. *Test suite may also stall on some machines? *-- Uwe mentioned >"Looks like a problem with the test or some wrong assumption or some >missing synchronized block (the Policeman Jenkins machine has many cores, >so issues with not synchronized cache lines can happen easily)." Is this a >different problem from #1 where we just have slow tests? I'm not sure if >this is something we want to investigate as part of the release or if we >think it can wait. > > Thanks, > Julie > > On Fri, Mar 18, 2022 at 12:17 PM Michael Sokolov > wrote: > >> We had to do a workaround in our internal test suites by setting a >> system property to trick this thing into not running; Maybe we can >> apply that also here... >> >> On Fri, Mar 18, 2022 at 3:12 PM Dawid Weiss >> wrote: >> > >> > I think this is Amazon trying to cope with log4shell - they've added >> > external instrumentation. >> > >> > On Fri, Mar 18, 2022 at 7:51 PM Robert Muir wrote: >> > > >> > > >> 2>at >> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >> > > >> 2>at Log4jHotPatch.asmVersion(Log4jHotPatch.java:71) >> > > >> 2>at Log4jHotPatch.agentmain(Log4jHotPatch.java:93) >> > > >> 2>... 6 more >> > > > >> > > > >> > > > Does anyone have an idea? >> > > > >> > > >> > > Your JVM is broken, look at this Log4jHotPatch stuff causing trouble. >> > > >> > > - >> > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> > > For additional commands, e-mail: dev-h...@lucene.apache.org >> > > >> > >> > - >> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> > For additional commands, e-mail: dev-h...@lucene.apache.org >> > >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >>
Re: [VOTE] Release Lucene 9.1.0 RC2
Hi everyone, people have run into several test failures, here's a summary to make it easier to follow: 1. Some tests can be very slow with certain random parameters, and this can cause the smoke tester to take several hours -- this seems like a test-only problem, and we're working to address it in https://issues.apache.org/jira/browse/LUCENE-10473. 2. TestMatchAllDocsQuery#testEarlyTermination can fail with assertion -- this is a test-only issue, and has already been fixed in https://issues.apache.org/jira/browse/LUCENE-10472. 3. TestIndexWriterMergePolicy#testStressUpdateSameDocumentWithMergeOnGetReader can OOM -- this is a known issue and is not expected to occur outside of tests (https://issues.apache.org/jira/browse/LUCENE-10088) 4. *Security manager issues from Amazon's log4shell patch* -- not sure this requires a change on our end? It sounds like Mike is investigating the issues now (thanks!) 5. *Test suite may also stall on some machines? *-- Uwe mentioned "Looks like a problem with the test or some wrong assumption or some missing synchronized block (the Policeman Jenkins machine has many cores, so issues with not synchronized cache lines can happen easily)." Is this a different problem from #1 where we just have slow tests? I'm not sure if this is something we want to investigate as part of the release or if we think it can wait. Thanks, Julie On Fri, Mar 18, 2022 at 12:17 PM Michael Sokolov wrote: > We had to do a workaround in our internal test suites by setting a > system property to trick this thing into not running; Maybe we can > apply that also here... > > On Fri, Mar 18, 2022 at 3:12 PM Dawid Weiss wrote: > > > > I think this is Amazon trying to cope with log4shell - they've added > > external instrumentation. > > > > On Fri, Mar 18, 2022 at 7:51 PM Robert Muir wrote: > > > > > > >> 2>at > java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > > > >> 2>at Log4jHotPatch.asmVersion(Log4jHotPatch.java:71) > > > >> 2>at Log4jHotPatch.agentmain(Log4jHotPatch.java:93) > > > >> 2>... 6 more > > > > > > > > > > > > Does anyone have an idea? > > > > > > > > > > Your JVM is broken, look at this Log4jHotPatch stuff causing trouble. > > > > > > - > > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
Re: [VOTE] Release Lucene 9.1.0 RC2
We had to do a workaround in our internal test suites by setting a system property to trick this thing into not running; Maybe we can apply that also here... On Fri, Mar 18, 2022 at 3:12 PM Dawid Weiss wrote: > > I think this is Amazon trying to cope with log4shell - they've added > external instrumentation. > > On Fri, Mar 18, 2022 at 7:51 PM Robert Muir wrote: > > > > >> 2>at > > >> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > > >> 2>at Log4jHotPatch.asmVersion(Log4jHotPatch.java:71) > > >> 2>at Log4jHotPatch.agentmain(Log4jHotPatch.java:93) > > >> 2>... 6 more > > > > > > > > > Does anyone have an idea? > > > > > > > Your JVM is broken, look at this Log4jHotPatch stuff causing trouble. > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [VOTE] Release Lucene 9.1.0 RC2
Yeah this is endemic in our world now. I am having the same issue On Fri, Mar 18, 2022 at 2:51 PM Robert Muir wrote: > > >> 2>at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > >> 2>at Log4jHotPatch.asmVersion(Log4jHotPatch.java:71) > >> 2>at Log4jHotPatch.agentmain(Log4jHotPatch.java:93) > >> 2>... 6 more > > > > > > Does anyone have an idea? > > > > Your JVM is broken, look at this Log4jHotPatch stuff causing trouble. > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [VOTE] Release Lucene 9.1.0 RC2
I think this is Amazon trying to cope with log4shell - they've added external instrumentation. On Fri, Mar 18, 2022 at 7:51 PM Robert Muir wrote: > > >> 2>at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > >> 2>at Log4jHotPatch.asmVersion(Log4jHotPatch.java:71) > >> 2>at Log4jHotPatch.agentmain(Log4jHotPatch.java:93) > >> 2>... 6 more > > > > > > Does anyone have an idea? > > > > Your JVM is broken, look at this Log4jHotPatch stuff causing trouble. > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [VOTE] Release Lucene 9.1.0 RC2
>> 2>at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >> 2>at Log4jHotPatch.asmVersion(Log4jHotPatch.java:71) >> 2>at Log4jHotPatch.agentmain(Log4jHotPatch.java:93) >> 2>... 6 more > > > Does anyone have an idea? > Your JVM is broken, look at this Log4jHotPatch stuff causing trouble. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [VOTE] Release Lucene 9.1.0 RC2
Hi
I tried to run the smoke test command on an EC2 instance, and I got several
failures like the one below:
Exception in thread "Attach Listener" Agent failed to start!
>
> org.apache.lucene.index.Test2BPostings > classMethod FAILED
> java.lang.AssertionError: The test or suite printed 48646 bytes to
> stdout and stderr, even though the limit was se$
> to 8192 bytes. Increase the limit with @Limit, ignore it completely with
> @SuppressSysoutChecks or run with -Dtests.ve$
> bose=true
> at __randomizedtesting.SeedInfo.seed([BBF119AE27F6031E]:0)
> at
> org.apache.lucene.tests.util.TestRuleLimitSysouts.afterIfSuccessful(TestRuleLimitSysouts.java:283)
> at
> com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterIfSuccessful(TestRuleAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:37)
> at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:$
> 0)
> at
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826$
> at java.base/java.lang.Thread.run(Thread.java:834)
>
> org.apache.lucene.index.Test2BPostings > test suite's output saved to
> /tmp/smoke_lucene_9.1.0_5b522487ba8e0f1002b50a13$
>
> 817ca037aec9686_2/unpack/lucene-9.1.0/lucene/core/build/test-results/test/outputs/OUTPUT-org.apache.lucene.index.Test2$
> Postings.txt, copied below:
> 2> java.lang.reflect.InvocationTargetException
> 2>at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> 2>at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2>at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2>at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 2>at
> java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
> 2>at
> java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallAgentmain(InstrumentationImpl.java:535)
> 2> Caused by: java.security.AccessControlException: access denied
> ("java.lang.RuntimePermission" "accessClassInPacka$
> e.jdk.internal.org.objectweb.asm")
> 2>at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> 2>at
> java.base/java.security.AccessController.checkPermission(AccessController.java:895)
> 2>at
> java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> 2>at
> java.base/java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1238)
> 2>at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:174)
> 2>at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
> 2>at Log4jHotPatch.asmVersion(Log4jHotPatch.java:71)
> 2>at Log4jHotPatch.agentmain(Log4jHotPatch.java:93)
> 2>... 6 more
>
Does anyone have an idea?
Patrick
On Fri, Mar 18, 2022 at 7:48 AM Ignacio Vera wrote:
> +1
>
> SUCCESS! [1:08:44.248804]
>
> On Thu, Mar 17, 2022 at 11:08 PM Uwe Schindler wrote:
>
>> Hi,
>>
>>
>>
>> the test also failed several times in usual Jenkins jobs (see builds@lao
>> list). Looks like a problem with the test or some wrong assumption or some
>> missing synchronized block (the Policeman Jenkins machine has many cores,
>> so issues with not synchronized cache lines can happen easily).
>>
>>
>>
>> Uwe
>>
>>
>>
>> -
>>
>> Uwe Schindler
>>
>> Achterdiek 19, D-28357 Bremen
>>
>> https://www.thetaphi.de
>>
>> eMail: u...@thetaphi.de
>>
>>
>>
>> *From:* Julie Tibshirani
>> *Sent:* Thursday, March 17, 2022 8:04 PM
>> *To:* dev@lucene.apache.org
>> *Subject:* Re: [VOTE] Release Lucene 9.1.0 RC2
>>
>>
>>
>> I investigated the TestMatchAllDocsQuery failure and filed an issue here:
>> https://issues.apache.org/jira/browse/LUCENE-10472. To me it doesn't
>> seem like a major problem and could potentially be a test issue. We also
>> haven't made changes to this early termination logic in 9.1 (to the best of
>> my knowledge), so it's unlikely to be a regression.
>>
>>
Re: [VOTE] Release Lucene 9.1.0 RC2
+1 SUCCESS! [1:08:44.248804] On Thu, Mar 17, 2022 at 11:08 PM Uwe Schindler wrote: > Hi, > > > > the test also failed several times in usual Jenkins jobs (see builds@lao > list). Looks like a problem with the test or some wrong assumption or some > missing synchronized block (the Policeman Jenkins machine has many cores, > so issues with not synchronized cache lines can happen easily). > > > > Uwe > > > > - > > Uwe Schindler > > Achterdiek 19, D-28357 Bremen > > https://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > *From:* Julie Tibshirani > *Sent:* Thursday, March 17, 2022 8:04 PM > *To:* dev@lucene.apache.org > *Subject:* Re: [VOTE] Release Lucene 9.1.0 RC2 > > > > I investigated the TestMatchAllDocsQuery failure and filed an issue here: > https://issues.apache.org/jira/browse/LUCENE-10472. To me it doesn't seem > like a major problem and could potentially be a test issue. We also haven't > made changes to this early termination logic in 9.1 (to the best of my > knowledge), so it's unlikely to be a regression. > > Julie > > > > On Thu, Mar 17, 2022 at 11:51 AM Anshum Gupta > wrote: > > Hi Julie, > > > > Thanks for volunteering as the RM. > > > > After 4 attempts, I finally got the smoke-tester (jdk11) to pass on the > 5th attempt. > > > > Tested out using some other basic indexing/search sample app code that I > was playing around with. > > > > Here's my +1 to release Lucene 9.1.0 RC2. > > > > SUCCESS! [1:05:31.299375] > > > > On Wed, Mar 16, 2022 at 1:27 PM Julie Tibshirani > wrote: > > Please vote for release candidate 2 for Lucene 9.1.0 > > The artifacts can be downloaded from: > > https://dist.apache.org/repos/dist/dev/lucene/lucene-9.1.0-RC2-rev-5b522487ba8e0f1002b50a136817ca037aec9686 > > You can run the smoke tester directly with this command: > > python3 -u dev-tools/scripts/smokeTestRelease.py \ > > https://dist.apache.org/repos/dist/dev/lucene/lucene-9.1.0-RC2-rev-5b522487ba8e0f1002b50a136817ca037aec9686 > > The vote will be open for at least 72 hours i.e. until 2022-03-19 21:00 > UTC. > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove (and reason why) > > Here is my +1. > > > > Julie > > > > > -- > > Anshum Gupta > >
