Re: ./crave pull .. 'heapdumps/* Fwd: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still Failing!

[Mikhail Khludnev]

Thanks Yuvraaj.
dev@, how to tweak jenkins script?

On Sat, Dec 2, 2023 at 9:25 PM Yuvraaj Kelkar  wrote:

> The new version of crave is in place and will be used automatically on the
> next invocation from Jenkins.
> Can you update the Jenkins script to call crave like this:
>
> ./crave pull --extra-rsync-flags ' --ignore-missing-args'
> '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
>
>
> Release has been marked here:
> https://github.com/accupara/crave/releases/tag/0.2-6879
> 
>
> Thanks,
> -Uv
> On Dec 1 2023, at 11:10 am, Mikhail Khludnev  wrote:
>
> Make sense.
>
> [image: Sent from Mailspring]
> On Fri, Dec 1, 2023 at 7:56 PM Yuvraaj Kelkar  wrote:
>
> I think the second option is what we'll go for.
> I'm going to add a flag to pull that will allow the user to specify extra
> flags to be given to rsync.
> Then we can call crave pull like this:
> ./crave pull --extra-rsync-flags ' --ignore-missing-args'
> '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
>
>
> *** Note the additional space before the hypen in ' --ignore-missing-args'
>  .
>
> This should handle the missing source files/directories.
>
> What do you think?
>
> Thanks,
> -Uv
>
> On Dec 1 2023, at 12:56 am, Mikhail Khludnev  wrote:
>
> Hello Yuvraaj,
> Thanks for taking care of this. Honestly it's not my wheelhouse.
> I seems like there's a consideration that a test getting out of heap will
> create heapdumps folder and put a file into. I don't know wether
> test/gradle can dump heap there ever. At least we don't have tests dumps
> heap there now. So, whether this folder exists or is absent is not certain.
> We have a few options:
>  - drop heapdumps/** from crave pull until someone needs to investigate a
> test falling out of memory.
>  - hack crave pull to ignore path wildcards for absent dir
>  - execute $mkdir heapdumps or  $mkdir -p heapdumps (depending on script's
> error handling more) before $crave pull
>
>
> On Thu, Nov 30, 2023 at 11:24 PM Yuvraaj Kelkar  wrote:
>
> I just started a build with crave:
> crave run ./gradlew --console=plain check integrationTests
>
> And at the end of it, looked for the patterns in the crave pull  command:
>
> admin@171074329f9e:/tmp/src/solr$ find . -name '*.events'
> admin@171074329f9e:/tmp/src/solr$ find . -name 'hs_err_pid*'
> admin@171074329f9e:/tmp/src/solr$
> admin@171074329f9e:/tmp/src/solr$ ls -l heapdumps
> ls: cannot access 'heapdumps': No such file or directory
>
>
> The only thing I could get a lot of output on was
>
> admin@171074329f9e:/tmp/src/solr$ find . | grep 'build.*test.TEST' | head
> ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.JsonRequestApiTest.xml
> ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.UsingSolrJRefGuideExamplesTest.xml
>  
> 
> ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.IndexingNestedDocuments.xml
> ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.ZkConfigFilesTest.xml
> ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.JsonRequestApiHeatmapFacetingTest.xml
> ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.exporter.SolrExporterIntegrationTest.xml
> ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.scraper.SolrStandaloneScraperBasicAuthTest.xml
> ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.exporter.MetricsQueryTemplateTest.xml
> ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.scraper.SolrStandaloneScraperTest.xml
> ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.scraper.SolrCloudScraperTest.xml
>
>
> Is there some other build command required to generate the other file
> patterns?
>
> Thanks,
> -Uv
>
> On Nov 30 2023, at 11:33 am, Yuvraaj Kelkar  wrote:
>
> Investigating.
>
> On Nov 26 2023, at 12:32 am, Mikhail Khludnev  wrote:
>
> Pardon
>
> On Sun, Nov 26, 2023 at 11:28 AM Gautam Worah 
> wrote:
>
> I think you meant to send it to d...@solr.apache.org?
>
> On Sun, Nov 26, 2023 at 12:24 AM Mikhail Khludnev  wrote:
>
> Hello
> It's rather like a logical error in crave pull. How to work around it?
>
> + status=0
> + ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' 
> '**/hs_err_pid*'
> Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such 
> file or 

Re: ./crave pull .. 'heapdumps/* Fwd: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still Failing!

[Yuvraaj Kelkar]

The new version of crave is in place and will be used automatically on the next 
invocation from Jenkins.
Can you update the Jenkins script to call crave like this:

./crave pull --extra-rsync-flags ' --ignore-missing-args' 
'**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Release has been marked here: 
https://github.com/accupara/crave/releases/tag/0.2-6879 
(https://link.getmailspring.com/link/010a76bd-7093-4bfd-980e-990296657...@getmailspring.com/0?redirect=https%3A%2F%2Fgithub.com%2Faccupara%2Fcrave%2Freleases%2Ftag%2F0.2-6879=ZGV2QGx1Y2VuZS5hcGFjaGUub3Jn)
Thanks,
-Uv
On Dec 1 2023, at 11:10 am, Mikhail Khludnev  wrote:
> Make sense.
>
> On Fri, Dec 1, 2023 at 7:56 PM Yuvraaj Kelkar  (mailto:u...@crave.io)> wrote:
> > I think the second option is what we'll go for.
> > I'm going to add a flag to pull that will allow the user to specify extra 
> > flags to be given to rsync.
> > Then we can call crave pull like this:
> > ./crave pull --extra-rsync-flags ' --ignore-missing-args' 
> > '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
> >
> > *** Note the additional space before the hypen in ' --ignore-missing-args' .
> > This should handle the missing source files/directories.
> > What do you think?
> > Thanks,
> > -Uv
> >
> > On Dec 1 2023, at 12:56 am, Mikhail Khludnev  > (mailto:m...@apache.org)> wrote:
> > > Hello Yuvraaj,
> > > Thanks for taking care of this. Honestly it's not my wheelhouse.
> > > I seems like there's a consideration that a test getting out of heap will 
> > > create heapdumps folder and put a file into. I don't know wether 
> > > test/gradle can dump heap there ever. At least we don't have tests dumps 
> > > heap there now. So, whether this folder exists or is absent is not 
> > > certain.
> > > We have a few options:
> > > - drop heapdumps/** from crave pull until someone needs to investigate a 
> > > test falling out of memory.
> > > - hack crave pull to ignore path wildcards for absent dir
> > > - execute $mkdir heapdumps or $mkdir -p heapdumps (depending on script's 
> > > error handling more) before $crave pull
> > >
> > >
> > > On Thu, Nov 30, 2023 at 11:24 PM Yuvraaj Kelkar  > > (mailto:u...@crave.io)> wrote:
> > > > I just started a build with crave:
> > > > crave run ./gradlew --console=plain check integrationTests
> > > >
> > > > And at the end of it, looked for the patterns in the crave pull command:
> > > >
> > > > admin@171074329f9e:/tmp/src/solr$ find . -name '*.events'
> > > > admin@171074329f9e:/tmp/src/solr$ find . -name 'hs_err_pid*'
> > > > admin@171074329f9e:/tmp/src/solr$
> > > > admin@171074329f9e:/tmp/src/solr$ ls -l heapdumps
> > > > ls: cannot access 'heapdumps': No such file or directory
> > > >
> > > > The only thing I could get a lot of output on was
> > > >
> > > > admin@171074329f9e:/tmp/src/solr$ find . | grep 'build.*test.TEST' | 
> > > > head
> > > > ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.JsonRequestApiTest.xml
> > > > ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.UsingSolrJRefGuideExamplesTest.xml
> > > >  
> > > > (https://link.getmailspring.com/link/a1463805-bfd9-43a5-bd19-08e9ae1a4...@getmailspring.com/0?redirect=TEST-org.apache.solr.client.ref_guide_examples.UsingSolrJRefGuideExamplesTest.xml=ZGV2QGx1Y2VuZS5hcGFjaGUub3Jn)
> > > > ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.IndexingNestedDocuments.xml
> > > > ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.ZkConfigFilesTest.xml
> > > > ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.JsonRequestApiHeatmapFacetingTest.xml
> > > > ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.exporter.SolrExporterIntegrationTest.xml
> > > > ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.scraper.SolrStandaloneScraperBasicAuthTest.xml
> > > > ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.exporter.MetricsQueryTemplateTest.xml
> > > > ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.scraper.SolrStandaloneScraperTest.xml
> > > > ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.scraper.SolrCloudScraperTest.xml
> > > >
> > > > Is there some other build command required to generate the other file 
> > > > patterns?
> > > > Thanks,
> > > > -Uv
> > > >
> > > > On Nov 30 2023, at 11:33 am, Yuvraaj Kelkar  > > > (mailto:u...@crave.io)> wrote:
> > > > > Investigating.
> > > > >
> > > > > On Nov 26 2023, at 12:32 am, Mikhail Khludnev  > > > > (mailto:m...@apache.org)> wrote:
> > > > > > Pardon
> > > > > >
> > > > > > On Sun, Nov 26, 2023 at 11:28 AM Gautam Worah 
> > > > > > mailto:worah.gau...@gmail.com)> wrote:
> > > > > > > I think you meant to send it to 

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

[Uwe Schindler]

Ha. Cool! Nice to meet.

I suggested to use this reader to some customers, but they were using Solr or 
Elasticsearch and it's not easy to implement it there. And they didn't want to 
pay the expensive Uwe. 

How do you handle deletes. Because the main issue with those readers is that 
you can't update documents without also updating the main reader (although it's 
a fake update).

If this is used, have you thought of a SynchronizedMergePolicy that just 
applies the same merges in the secondary index?

Uwe

Am 2. Dezember 2023 10:27:20 MEZ schrieb Dawid Weiss :
>> ParallelReader is also seldomly used, maybe we should remove support at
>> some point. I don't know anybody using it, because it is very complicated
>> to maintain consistent indexes. It only works with stable merge policies.
>>
>
>You do know somebody - you know me. We're using it extensively - the
>scenario is for storing data derived from the main document in a separate
>index, merging this data dynamically. The data can then be reindexed/
>modified independently. Yes, we do use stable merge policies.
>
>Dawid

--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

[Dawid Weiss]

> ParallelReader is also seldomly used, maybe we should remove support at
> some point. I don't know anybody using it, because it is very complicated
> to maintain consistent indexes. It only works with stable merge policies.
>

You do know somebody - you know me. We're using it extensively - the
scenario is for storing data derived from the main document in a separate
index, merging this data dynamically. The data can then be reindexed/
modified independently. Yes, we do use stable merge policies.

Dawid

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

[Uwe Schindler]

Found the PR. Somehow the mailinglist didn't get it.

Am 2. Dezember 2023 09:58:45 MEZ schrieb Uwe Schindler :
>Hi Chris,
>
>I can't find the PR.
>
>I am interested, because I wrote the original ParallelReader tests.
>
>IMHO the parallel readers are so sensitive to random changes, the test setup 
>should not use any indexwriter randomization at all.
>
>ParallelReader is also seldomly used, maybe we should remove support at some 
>point. I don't know anybody using it, because it is very complicated to 
>maintain consistent indexes. It only works with stable merge policies.
>
>Uwe
>
>Am 2. Dezember 2023 09:34:46 MEZ schrieb Chris Hegarty 
>:
>>Hi,
>>
>>I noticed this failure locally, and opened a PR for it yesterday. It is a
>>test issues, and indeed related to the recent merge policy test
>>randomization change.
>>
>>-Chris
>>
>>On Saturday, December 2, 2023, Patrick Zhai  wrote:
>>
>>> Seems it's because this MockRandomMergePolicy change
>>> 
>>>  recently
>>> makes ParallelLeafReader unhappy - it's reading two parallel segments from
>>> 2 dir and this MP makes one of the segments' documents order reversed.
>>>
>>> But should be just test util issue and not affecting release.
>>>
>>> Adrien do you want to take a look? I'm not sure what's the best way to fix
>>> it, adding an index sort for that test seems a bit overkill?
>>>
>>> Patrick
>>>
>>> On Fri, Dec 1, 2023 at 2:06 PM Michael McCandless <
>>> luc...@mikemccandless.com> wrote:
>>>
 Hmm this reproduces for me, and looks new/unique.  Could it be related to
 recent 9.9.0 changes / release blocker?

 Mike

 On Fri, Dec 1, 2023 at 3:33 PM Policeman Jenkins Server <
 jenk...@thetaphi.de> wrote:

> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/14204/
> Java: 64bit/hotspot/jdk-11.0.21 -XX:+UseCompressedOops -XX:+UseParallelGC
>
> 1 tests failed.
> FAILED:  org.apache.lucene.index.TestParallelLeafReader.testQueries
>
> Error Message:
> org.junit.ComparisonFailure: expected: but was:
>
> Stack Trace:
> org.junit.ComparisonFailure: expected: but was:
> at __randomizedtesting.SeedInfo.seed([6CA57EA3FB50CA0D:
> 302BB278E1397FA3]:0)
> at org.junit.Assert.assertEquals(Assert.java:117)
> at org.junit.Assert.assertEquals(Assert.java:146)
> at org.apache.lucene.index.TestParallelLeafReader.queryTest(
> TestParallelLeafReader.java:263)
> at org.apache.lucene.index.TestParallelLeafReader.testQueries(
> TestParallelLeafReader.java:48)
> at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.
> invoke(NativeMethodAccessorImpl.java:62)
> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.
> invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(
> RandomizedRunner.java:1758)
> at com.carrotsearch.randomizedtesting.
> RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
> at com.carrotsearch.randomizedtesting.
> RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
> at com.carrotsearch.randomizedtesting.
> RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
> at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$
> 1.evaluate(TestRuleSetupTeardownChained.java:48)
> at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.
> evaluate(AbstractBeforeAfterRule.java:43)
> at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.
> evaluate(TestRuleThreadAndTestName.java:45)
> at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures
> $1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at org.apache.lucene.tests.util.TestRuleMarkFailure$1.
> evaluate(TestRuleMarkFailure.java:44)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at com.carrotsearch.randomizedtesting.rules.
> StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$
> StatementRunner.run(ThreadLeakControl.java:390)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl.
> forkTimeoutingTask(ThreadLeakControl.java:843)
> at com.carrotsearch.randomizedtesting.
> ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.
> runSingleTest(RandomizedRunner.java:955)
> at com.carrotsearch.randomizedtesting.
> 

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

[Chris Hegarty]

Sorry, PR link: https://github.com/apache/lucene/pull/12865

-Chris

On Saturday, December 2, 2023, Uwe Schindler  wrote:

> Hi Chris,
>
> I can't find the PR.
>
>
>
> I am interested, because I wrote the original ParallelReader tests.
>
> IMHO the parallel readers are so sensitive to random changes, the test
> setup should not use any indexwriter randomization at all.
>
> ParallelReader is also seldomly used, maybe we should remove support at
> some point. I don't know anybody using it, because it is very complicated
> to maintain consistent indexes. It only works with stable merge policies.
>
> Uwe
>

>
> Am 2. Dezember 2023 09:34:46 MEZ schrieb Chris Hegarty
> :
>
>> Hi,
>>
>> I noticed this failure locally, and opened a PR for it yesterday. It is a
>> test issues, and indeed related to the recent merge policy test
>> randomization change.
>>
>> -Chris
>>
>> On Saturday, December 2, 2023, Patrick Zhai  wrote:
>>
>>> Seems it's because this MockRandomMergePolicy change
>>> 
>>>  recently
>>> makes ParallelLeafReader unhappy - it's reading two parallel segments from
>>> 2 dir and this MP makes one of the segments' documents order reversed.
>>>
>>> But should be just test util issue and not affecting release.
>>>
>>> Adrien do you want to take a look? I'm not sure what's the best way to
>>> fix it, adding an index sort for that test seems a bit overkill?
>>>
>>> Patrick
>>>
>>> On Fri, Dec 1, 2023 at 2:06 PM Michael McCandless <
>>> luc...@mikemccandless.com> wrote:
>>>
 Hmm this reproduces for me, and looks new/unique.  Could it be related
 to recent 9.9.0 changes / release blocker?

 Mike

 On Fri, Dec 1, 2023 at 3:33 PM Policeman Jenkins Server <
 jenk...@thetaphi.de> wrote:

> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/14204/
> Java: 64bit/hotspot/jdk-11.0.21 -XX:+UseCompressedOops
> -XX:+UseParallelGC
>
> 1 tests failed.
> FAILED:  org.apache.lucene.index.TestParallelLeafReader.testQueries
>
> Error Message:
> org.junit.ComparisonFailure: expected: but was:
>
> Stack Trace:
> org.junit.ComparisonFailure: expected: but was:
> at __randomizedtesting.SeedInfo.s
> eed([6CA57EA3FB50CA0D:302BB278E1397FA3]:0)
> at org.junit.Assert.assertEquals(Assert.java:117)
> at org.junit.Assert.assertEquals(Assert.java:146)
> at org.apache.lucene.index.TestPa
> rallelLeafReader.queryTest(TestParallelLeafReader.java:263)
> at org.apache.lucene.index.TestPa
> rallelLeafReader.testQueries(TestParallelLeafReader.java:48)
> at java.base/jdk.internal.reflect
> .NativeMethodAccessorImpl.invoke0(Native Method)
> at java.base/jdk.internal.reflect
> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at java.base/jdk.internal.reflect
> .DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso
> rImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at com.carrotsearch.randomizedtes
> ting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.
> evaluate(RandomizedRunner.java:946)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.
> evaluate(RandomizedRunner.java:982)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.
> evaluate(RandomizedRunner.java:996)
> at org.apache.lucene.tests.util.T
> estRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardown
> Chained.java:48)
> at org.apache.lucene.tests.util.A
> bstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at org.apache.lucene.tests.util.T
> estRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> at org.apache.lucene.tests.util.T
> estRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfter
> MaxFailures.java:60)
> at org.apache.lucene.tests.util.T
> estRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.
> evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtes
> ting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> at com.carrotsearch.randomizedtes
> ting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.
> evaluate(ThreadLeakControl.java:490)
> at com.carrotsearch.randomizedtes
> ting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
> at 

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

[Uwe Schindler]

Hi Chris,

I can't find the PR.

I am interested, because I wrote the original ParallelReader tests.

IMHO the parallel readers are so sensitive to random changes, the test setup 
should not use any indexwriter randomization at all.

ParallelReader is also seldomly used, maybe we should remove support at some 
point. I don't know anybody using it, because it is very complicated to 
maintain consistent indexes. It only works with stable merge policies.

Uwe

Am 2. Dezember 2023 09:34:46 MEZ schrieb Chris Hegarty 
:
>Hi,
>
>I noticed this failure locally, and opened a PR for it yesterday. It is a
>test issues, and indeed related to the recent merge policy test
>randomization change.
>
>-Chris
>
>On Saturday, December 2, 2023, Patrick Zhai  wrote:
>
>> Seems it's because this MockRandomMergePolicy change
>> 
>>  recently
>> makes ParallelLeafReader unhappy - it's reading two parallel segments from
>> 2 dir and this MP makes one of the segments' documents order reversed.
>>
>> But should be just test util issue and not affecting release.
>>
>> Adrien do you want to take a look? I'm not sure what's the best way to fix
>> it, adding an index sort for that test seems a bit overkill?
>>
>> Patrick
>>
>> On Fri, Dec 1, 2023 at 2:06 PM Michael McCandless <
>> luc...@mikemccandless.com> wrote:
>>
>>> Hmm this reproduces for me, and looks new/unique.  Could it be related to
>>> recent 9.9.0 changes / release blocker?
>>>
>>> Mike
>>>
>>> On Fri, Dec 1, 2023 at 3:33 PM Policeman Jenkins Server <
>>> jenk...@thetaphi.de> wrote:
>>>
 Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/14204/
 Java: 64bit/hotspot/jdk-11.0.21 -XX:+UseCompressedOops -XX:+UseParallelGC

 1 tests failed.
 FAILED:  org.apache.lucene.index.TestParallelLeafReader.testQueries

 Error Message:
 org.junit.ComparisonFailure: expected: but was:

 Stack Trace:
 org.junit.ComparisonFailure: expected: but was:
 at __randomizedtesting.SeedInfo.seed([6CA57EA3FB50CA0D:
 302BB278E1397FA3]:0)
 at org.junit.Assert.assertEquals(Assert.java:117)
 at org.junit.Assert.assertEquals(Assert.java:146)
 at org.apache.lucene.index.TestParallelLeafReader.queryTest(
 TestParallelLeafReader.java:263)
 at org.apache.lucene.index.TestParallelLeafReader.testQueries(
 TestParallelLeafReader.java:48)
 at 
 java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
 Method)
 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.
 invoke(NativeMethodAccessorImpl.java:62)
 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.
 invoke(DelegatingMethodAccessorImpl.java:43)
 at java.base/java.lang.reflect.Method.invoke(Method.java:566)
 at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(
 RandomizedRunner.java:1758)
 at com.carrotsearch.randomizedtesting.
 RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
 at com.carrotsearch.randomizedtesting.
 RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
 at com.carrotsearch.randomizedtesting.
 RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
 at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$
 1.evaluate(TestRuleSetupTeardownChained.java:48)
 at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.
 evaluate(AbstractBeforeAfterRule.java:43)
 at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.
 evaluate(TestRuleThreadAndTestName.java:45)
 at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures
 $1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
 at org.apache.lucene.tests.util.TestRuleMarkFailure$1.
 evaluate(TestRuleMarkFailure.java:44)
 at org.junit.rules.RunRules.evaluate(RunRules.java:20)
 at com.carrotsearch.randomizedtesting.rules.
 StatementAdapter.evaluate(StatementAdapter.java:36)
 at com.carrotsearch.randomizedtesting.ThreadLeakControl$
 StatementRunner.run(ThreadLeakControl.java:390)
 at com.carrotsearch.randomizedtesting.ThreadLeakControl.
 forkTimeoutingTask(ThreadLeakControl.java:843)
 at com.carrotsearch.randomizedtesting.
 ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
 at com.carrotsearch.randomizedtesting.RandomizedRunner.
 runSingleTest(RandomizedRunner.java:955)
 at com.carrotsearch.randomizedtesting.
 RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
 at com.carrotsearch.randomizedtesting.
 RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
 at com.carrotsearch.randomizedtesting.
 

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

[Chris Hegarty]

Hi,

I noticed this failure locally, and opened a PR for it yesterday. It is a
test issues, and indeed related to the recent merge policy test
randomization change.

-Chris

On Saturday, December 2, 2023, Patrick Zhai  wrote:

> Seems it's because this MockRandomMergePolicy change
> 
>  recently
> makes ParallelLeafReader unhappy - it's reading two parallel segments from
> 2 dir and this MP makes one of the segments' documents order reversed.
>
> But should be just test util issue and not affecting release.
>
> Adrien do you want to take a look? I'm not sure what's the best way to fix
> it, adding an index sort for that test seems a bit overkill?
>
> Patrick
>
> On Fri, Dec 1, 2023 at 2:06 PM Michael McCandless <
> luc...@mikemccandless.com> wrote:
>
>> Hmm this reproduces for me, and looks new/unique.  Could it be related to
>> recent 9.9.0 changes / release blocker?
>>
>> Mike
>>
>> On Fri, Dec 1, 2023 at 3:33 PM Policeman Jenkins Server <
>> jenk...@thetaphi.de> wrote:
>>
>>> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/14204/
>>> Java: 64bit/hotspot/jdk-11.0.21 -XX:+UseCompressedOops -XX:+UseParallelGC
>>>
>>> 1 tests failed.
>>> FAILED:  org.apache.lucene.index.TestParallelLeafReader.testQueries
>>>
>>> Error Message:
>>> org.junit.ComparisonFailure: expected: but was:
>>>
>>> Stack Trace:
>>> org.junit.ComparisonFailure: expected: but was:
>>> at __randomizedtesting.SeedInfo.seed([6CA57EA3FB50CA0D:
>>> 302BB278E1397FA3]:0)
>>> at org.junit.Assert.assertEquals(Assert.java:117)
>>> at org.junit.Assert.assertEquals(Assert.java:146)
>>> at org.apache.lucene.index.TestParallelLeafReader.queryTest(
>>> TestParallelLeafReader.java:263)
>>> at org.apache.lucene.index.TestParallelLeafReader.testQueries(
>>> TestParallelLeafReader.java:48)
>>> at 
>>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> Method)
>>> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.
>>> invoke(NativeMethodAccessorImpl.java:62)
>>> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.
>>> invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>>> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(
>>> RandomizedRunner.java:1758)
>>> at com.carrotsearch.randomizedtesting.
>>> RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
>>> at com.carrotsearch.randomizedtesting.
>>> RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
>>> at com.carrotsearch.randomizedtesting.
>>> RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
>>> at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$
>>> 1.evaluate(TestRuleSetupTeardownChained.java:48)
>>> at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.
>>> evaluate(AbstractBeforeAfterRule.java:43)
>>> at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.
>>> evaluate(TestRuleThreadAndTestName.java:45)
>>> at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures
>>> $1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>>> at org.apache.lucene.tests.util.TestRuleMarkFailure$1.
>>> evaluate(TestRuleMarkFailure.java:44)
>>> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>>> at com.carrotsearch.randomizedtesting.rules.
>>> StatementAdapter.evaluate(StatementAdapter.java:36)
>>> at com.carrotsearch.randomizedtesting.ThreadLeakControl$
>>> StatementRunner.run(ThreadLeakControl.java:390)
>>> at com.carrotsearch.randomizedtesting.ThreadLeakControl.
>>> forkTimeoutingTask(ThreadLeakControl.java:843)
>>> at com.carrotsearch.randomizedtesting.
>>> ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
>>> at com.carrotsearch.randomizedtesting.RandomizedRunner.
>>> runSingleTest(RandomizedRunner.java:955)
>>> at com.carrotsearch.randomizedtesting.
>>> RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
>>> at com.carrotsearch.randomizedtesting.
>>> RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
>>> at com.carrotsearch.randomizedtesting.
>>> RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
>>> at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.
>>> evaluate(AbstractBeforeAfterRule.java:43)
>>> at com.carrotsearch.randomizedtesting.rules.
>>> StatementAdapter.evaluate(StatementAdapter.java:36)
>>> at org.apache.lucene.tests.util.TestRuleStoreClassName$1.
>>> evaluate(TestRuleStoreClassName.java:38)
>>> at com.carrotsearch.randomizedtesting.rules.
>>> NoShadowingOrOverridesOnMethodsRule$1.evaluate(
>>> NoShadowingOrOverridesOnMethodsRule.java:40)
>>> at com.carrotsearch.randomizedtesting.rules.
>>>