Re: [JENKINS] Lucene » Lucene-NightlyTests-main - Build # 1209 - Unstable!

Woops, sorry for suggesting this change in the first place! I didn't know we had this validation for points, but not for postings. On Fri, Dec 8, 2023 at 2:16 PM Michael McCandless wrote: > OK I reverted the "optimization" to not pull FieldInfo for a field when > getting Points values from

Re: The need for a Lucene 9.9.1 release

> On 9 Dec 2023, at 09:09, Chris Hegarty wrote: > > Hi, > > We’ve encounter two very serious issues with the recent Lucene 9.9.0 release, > both of which (even if taken by themselves) would warrant a 9.9.1. The issues > are: > > 1. https://github.com/apache/lucene/issues/12895 - Corruption

Re: The need for a Lucene 9.9.1 release

FYI - I added the next bugfix version 9.9.1 to `branch_9_9`, in preparation for the upcoming bug fix release. https://github.com/apache/lucene/commit/1617c0b3a5624adba6e7b380dfeb7fb89b8a2feb -Chris. > On 9 Dec 2023, at 09:09, Chris Hegarty wrote: > > Hi, > > We’ve encounter two very serious

Re: The need for a Lucene 9.9.1 release

I believe your assessment that it is "only" a read problem is correct. I can see how using the "corruption" wording may have caused confusion. It is a severe bug though that affects multi term queries and I thought it's a good idea to patch that, given that folks have reproduced it and found the

Re: The need for a Lucene 9.9.1 release

I don't understand use of the word corruption, isn't it just a bug in intersect() that only affects wildcards etc? e.g. its not gonna merge into new segments or impact written data in any way. And i don't think we should rushout some bugfix release without any test for this? On Sat, Dec 9, 2023

Re: The need for a Lucene 9.9.1 release

Based on the discussions in https://github.com/apache/lucene/issues/12895 , it seems like reverting the change that caused the corruption on read is the quickest fix, so that we can speed up releasing 9.9.1. I opened a PR for that: https://github.com/apache/lucene/pull/12899. Is there additional

Re: The need for a Lucene 9.9.1 release

Oh, and I’m happy to be Release Manager for 9.9.1 (given my recent experience on 9.9.0) -Chris. > On 9 Dec 2023, at 09:09, Chris Hegarty wrote: > > Hi, > > We’ve encounter two very serious issues with the recent Lucene 9.9.0 release, > both of which (even if taken by themselves) would

The need for a Lucene 9.9.1 release

Hi, We’ve encounter two very serious issues with the recent Lucene 9.9.0 release, both of which (even if taken by themselves) would warrant a 9.9.1. The issues are: 1. https://github.com/apache/lucene/issues/12895 - Corruption read on term dictionaries in Lucene 9.9 2.

Re: [JENKINS-EA] Lucene-main-Windows (64bit/hotspot/jdk-22-ea+26) - Build # 13501 - Unstable!

Saw this a couple times so I reproduced locally. Since it repo'd for me as well (against main), I opened an issue: https://github.com/apache/lucene/issues/12896 On Fri, Dec 8, 2023 at 11:00 AM Policeman Jenkins Server < jenk...@thetaphi.de> wrote: > Build:

Re: [JENKINS] Lucene » Lucene-NightlyTests-main - Build # 1209 - Unstable!

OK I reverted the "optimization" to not pull FieldInfo for a field when getting Points values from SlowCompositeCodecReaderWrapper! Clearly it was not safe ;) Mike McCandless http://blog.mikemccandless.com On Fri, Dec 8, 2023 at 8:06 AM Michael McCandless wrote: > Uh oh -- I'll dig. We may

Re: [JENKINS] Lucene » Lucene-NightlyTests-main - Build # 1209 - Unstable!

Uh oh -- I'll dig. We may need to put back the FieldInfo check before pulling points. Tricky! Mike McCandless http://blog.mikemccandless.com On Fri, Dec 8, 2023 at 3:55 AM Apache Jenkins Server < jenk...@builds.apache.org> wrote: > Build: >

Re: [JENKINS] Lucene » Lucene-Check-main - Build # 10815 - Unstable!

It's the same as this issue - https://github.com/apache/lucene/issues/12883 On Thu, Dec 7, 2023 at 1:30 PM Apache Jenkins Server < jenk...@builds.apache.org> wrote: > Build: > https://ci-builds.apache.org/job/Lucene/job/Lucene-Check-main/10815/ > > 4 tests failed. > FAILED: >

Re: [JENKINS] Lucene-MMAPv2-Linux (64bit/openj9/jdk-17.0.8) - Build # 1500 - Unstable!

Oh, nevermind -- we have seen it before, and added a comment on the upstream (Open J9) issue: https://github.com/eclipse-openj9/openj9/issues/18400#issuecomment-1795093834 Mike McCandless http://blog.mikemccandless.com On Thu, Dec 7, 2023 at 8:32 AM Michael McCandless wrote: > Hmm -- this

Re: [JENKINS] Lucene-MMAPv2-Linux (64bit/openj9/jdk-17.0.8) - Build # 1500 - Unstable!

Hmm -- this looks like maybe another Open J9 specific failure? I have not seen this one before I think... Mike McCandless http://blog.mikemccandless.com On Fri, Dec 1, 2023 at 10:20 PM Policeman Jenkins Server < jenk...@thetaphi.de> wrote: > Build:

SIGSEGV with Lucene 9.9.0

Hi, I want to raise awareness of a JVM crash that has been tickled by the Lucene 9.9.0 release. I don’t have all the answers (yet), but I just want to ensure that folk here are aware. Details in this Elasticsearch issue [1] # A fatal error has been detected by the Java Runtime Environment: # #

Re: [VOTE] Release Lucene 9.9.0 RC1

Thanks Adrien, that makes sense. I was wondering how we'd ensure all API breakages in a major release were covered with deprecation messages. Sounds like this is the answer. Cheers, -Greg On Thu, Nov 30, 2023 at 11:14 AM Adrien Grand wrote: > My expectation is that we will do a 9.x minor at

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

Hi Uwe, Apologies for the late reply. I suggested to use this reader to some customers, but they were using Solr > or Elasticsearch and it's not easy to implement it there. And they didn't > want to pay the expensive Uwe. >  > Oh, they should know better - Mr Uwe is worth his price tag for

[ANNOUNCE] Apache Lucene 9.9.0 released

The Lucene PMC is pleased to announce the release of Apache Lucene 9.9.0. Apache Lucene is a high-performance, full-featured search engine library written entirely in Java. It is a technology suitable for nearly any application that requires structured search, full-text search, faceting,

[RESULT] [VOTE] Release Lucene 9.9.0 RC2

Hi, It's been >72h since the vote was initiated and the result is: +1 12 (9 binding) 0 0 -1 0 This vote has PASSED -Chris.

Re: [VOTE] Release Lucene 9.9.0 RC2

+1 Success! [0:33:02.122753] On Thu, Nov 30, 2023 at 11:39 PM Chris Hegarty wrote: > Please vote for release candidate 2 for Lucene 9.9.0 > > > The artifacts can be downloaded from: > > >

Re: ./crave pull .. 'heapdumps/* Fwd: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still Failing!

Investingating On Dec 3 2023, at 2:44 pm, David Smiley wrote: > I updated the script accordingly and I still see the problem: > > https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/6008/console >

Re: ./crave pull .. 'heapdumps/* Fwd: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still Failing!

I updated the script accordingly and I still see the problem: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/6008/console + status=0 + ./crave pull --extra-rsync-flags ' --ignore-missing-args' '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*' Error: rsync:

Re: ./crave pull .. 'heapdumps/* Fwd: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still Failing!

Thanks Yuvraaj. dev@, how to tweak jenkins script? On Sat, Dec 2, 2023 at 9:25 PM Yuvraaj Kelkar wrote: > The new version of crave is in place and will be used automatically on the > next invocation from Jenkins. > Can you update the Jenkins script to call crave like this: > > ./crave pull

Re: ./crave pull .. 'heapdumps/* Fwd: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still Failing!

The new version of crave is in place and will be used automatically on the next invocation from Jenkins. Can you update the Jenkins script to call crave like this: ./crave pull --extra-rsync-flags ' --ignore-missing-args' '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**'

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

Ha. Cool! Nice to meet. I suggested to use this reader to some customers, but they were using Solr or Elasticsearch and it's not easy to implement it there. And they didn't want to pay the expensive Uwe.  How do you handle deletes. Because the main issue with those readers is that you can't

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

> ParallelReader is also seldomly used, maybe we should remove support at > some point. I don't know anybody using it, because it is very complicated > to maintain consistent indexes. It only works with stable merge policies. > You do know somebody - you know me. We're using it extensively - the

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

Found the PR. Somehow the mailinglist didn't get it. Am 2. Dezember 2023 09:58:45 MEZ schrieb Uwe Schindler : >Hi Chris, > >I can't find the PR. > >I am interested, because I wrote the original ParallelReader tests. > >IMHO the parallel readers are so sensitive to random changes, the test setup

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

Sorry, PR link: https://github.com/apache/lucene/pull/12865 -Chris On Saturday, December 2, 2023, Uwe Schindler wrote: > Hi Chris, > > I can't find the PR. > > > > I am interested, because I wrote the original ParallelReader tests. > > IMHO the parallel readers are so sensitive to random

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

Hi Chris, I can't find the PR. I am interested, because I wrote the original ParallelReader tests. IMHO the parallel readers are so sensitive to random changes, the test setup should not use any indexwriter randomization at all. ParallelReader is also seldomly used, maybe we should remove

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

Hi, I noticed this failure locally, and opened a PR for it yesterday. It is a test issues, and indeed related to the recent merge policy test randomization change. -Chris On Saturday, December 2, 2023, Patrick Zhai wrote: > Seems it's because this MockRandomMergePolicy change >

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

Seems it's because this MockRandomMergePolicy change recently makes ParallelLeafReader unhappy - it's reading two parallel segments from 2 dir and this MP makes

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-11.0.21) - Build # 14204 - Unstable!

Hmm this reproduces for me, and looks new/unique. Could it be related to recent 9.9.0 changes / release blocker? Mike On Fri, Dec 1, 2023 at 3:33 PM Policeman Jenkins Server wrote: > Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/14204/ > Java: 64bit/hotspot/jdk-11.0.21

Re: ./crave pull .. 'heapdumps/* Fwd: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still Failing!

Make sense. On Fri, Dec 1, 2023 at 7:56 PM Yuvraaj Kelkar wrote: > I think the second option is what we'll go for. > I'm going to add a flag to pull that will allow the user to specify extra > flags to be given to rsync. > Then we can call crave pull like this: > ./crave pull

Re: ./crave pull .. 'heapdumps/* Fwd: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still Failing!

I think the second option is what we'll go for. I'm going to add a flag to pull that will allow the user to specify extra flags to be given to rsync. Then we can call crave pull like this: ./crave pull --extra-rsync-flags ' --ignore-missing-args' '**/build/**/test/TEST-*.xml' '**/*.events'

Re: [VOTE] Release Lucene 9.9.0 RC2

+1 SUCCESS! [0:20:12.297376] Mike McCandless http://blog.mikemccandless.com On Fri, Dec 1, 2023 at 9:21 AM Uwe Schindler wrote: > Hi, > > I let Policeman Jenkins run the smoke tester with Java 11 and Java 17 > (unfortunately we have no support for 21 yet, so new MMap and Vectors were >

Re: [VOTE] Release Lucene 9.9.0 RC2

Hi, I let Policeman Jenkins run the smoke tester with Java 11 and Java 17 (unfortunately we have no support for 21 yet, so new MMap and Vectors were not tested). But this was tested long enough, so I trust everything. I just did some cross-checking and validated the MR-JAR to contain all

Re: [VOTE] Release Lucene 9.9.0 RC2

SUCCESS! [0:34:53.150902] +1 On Fri, Dec 1, 2023 at 9:06 AM Ignacio Vera wrote: > SUCCESS! [1:20:23.570231] > > > +1 > > On Fri, Dec 1, 2023 at 6:55 AM Shyamsunder Mutcha > wrote: > >> SUCCESS! [0:38:41.054860] >> +1 >> >> On Thu, Nov 30, 2023 at 9:59 PM Nhat Nguyen >> wrote: >> >>>

Re: ./crave pull .. 'heapdumps/* Fwd: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still Failing!

Hello Yuvraaj, Thanks for taking care of this. Honestly it's not my wheelhouse. I seems like there's a consideration that a test getting out of heap will create heapdumps folder and put a file into. I don't know wether test/gradle can dump heap there ever. At least we don't have tests dumps heap

Re: [VOTE] Release Lucene 9.9.0 RC2

SUCCESS! [1:20:23.570231] +1 On Fri, Dec 1, 2023 at 6:55 AM Shyamsunder Mutcha wrote: > SUCCESS! [0:38:41.054860] > +1 > > On Thu, Nov 30, 2023 at 9:59 PM Nhat Nguyen > wrote: > >> SUCCESS! [1:22:43.808415] >> >> +1 >> >> On Thu, Nov 30, 2023 at 6:09 PM Christian Moen wrote: >> >>> SUCCESS!

Re: [VOTE] Release Lucene 9.9.0 RC2

SUCCESS! [0:38:41.054860] +1 On Thu, Nov 30, 2023 at 9:59 PM Nhat Nguyen wrote: > SUCCESS! [1:22:43.808415] > > +1 > > On Thu, Nov 30, 2023 at 6:09 PM Christian Moen wrote: > >> SUCCESS! [1:49:26.873909] >> >> +1 >> >> On Fri, Dec 1, 2023 at 3:09 AM Chris Hegarty >> wrote: >> >>> Please vote

Re: [VOTE] Release Lucene 9.9.0 RC2

SUCCESS! [1:22:43.808415] +1 On Thu, Nov 30, 2023 at 6:09 PM Christian Moen wrote: > SUCCESS! [1:49:26.873909] > > +1 > > On Fri, Dec 1, 2023 at 3:09 AM Chris Hegarty > wrote: > >> Please vote for release candidate 2 for Lucene 9.9.0 >> >> >> The artifacts can be downloaded from: >> >> >>

Re: [VOTE] Release Lucene 9.9.0 RC2

SUCCESS! [1:49:26.873909] +1 On Fri, Dec 1, 2023 at 3:09 AM Chris Hegarty wrote: > Please vote for release candidate 2 for Lucene 9.9.0 > > > The artifacts can be downloaded from: > > > https://dist.apache.org/repos/dist/dev/lucene/lucene-9.9.0-RC2-rev-06070c0dceba07f0d33104192d9ac98ca16fc500

Re: [VOTE] Release Lucene 9.9.0 RC2

SUCCESS! [0:46:20.693134] +1 On Thu, Nov 30, 2023 at 5:50 PM Tomás Fernández Löbbe wrote: > SUCCESS! [0:52:49.337126] > > +1 > > On Thu, Nov 30, 2023 at 12:05 PM Benjamin Trent > wrote: > >> SUCCESS! [0:44:05.132154] >> >> +1 >> >> On Thu, Nov 30, 2023 at 1:09 PM Chris Hegarty >> wrote: >>

Re: [VOTE] Release Lucene 9.9.0 RC2

SUCCESS! [0:52:49.337126] +1 On Thu, Nov 30, 2023 at 12:05 PM Benjamin Trent wrote: > SUCCESS! [0:44:05.132154] > > +1 > > On Thu, Nov 30, 2023 at 1:09 PM Chris Hegarty > wrote: > >> Please vote for release candidate 2 for Lucene 9.9.0 >> >> >> The artifacts can be downloaded from: >> >> >>

Re: GitHub issues vs PRs vs Lucene's CHANGES.txt

Sounds like we could automate assigning the milestone, given that it is a commonly forgotten step, based on the section of CHANGES where the PR gets added? I am pretty sure that I forgot to add entries to CHANGES too. That could be maybe suggested in github. Whenever there's a PR that does not

Re: GitHub issues vs PRs vs Lucene's CHANGES.txt

Hopefully this is relevant. There are useful tools like git-cliff​ for automating changelog generation. https://github.com/orhun/git-cliff Tony X From: Michael McCandless Sent: Thursday, November 30, 2023 4:30 AM To: dev@lucene.apache.org Subject: Re: GitHub

Re: ./crave pull .. 'heapdumps/* Fwd: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still Failing!

I just started a build with crave: crave run ./gradlew --console=plain check integrationTests And at the end of it, looked for the patterns in the crave pull command: admin@171074329f9e:/tmp/src/solr$ find . -name '*.events' admin@171074329f9e:/tmp/src/solr$ find . -name 'hs_err_pid*'

Re: [VOTE] Release Lucene 9.9.0 RC2

SUCCESS! [0:44:05.132154] +1 On Thu, Nov 30, 2023 at 1:09 PM Chris Hegarty wrote: > Please vote for release candidate 2 for Lucene 9.9.0 > > > The artifacts can be downloaded from: > > > https://dist.apache.org/repos/dist/dev/lucene/lucene-9.9.0-RC2-rev-06070c0dceba07f0d33104192d9ac98ca16fc500

Re: ./crave pull .. 'heapdumps/* Fwd: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still Failing!

Investigating. On Nov 26 2023, at 12:32 am, Mikhail Khludnev wrote: > Pardon > > On Sun, Nov 26, 2023 at 11:28 AM Gautam Worah (mailto:worah.gau...@gmail.com)> wrote: > > I think you meant to send it to d...@solr.apache.org > > (mailto:d...@solr.apache.org)? > > > > On Sun, Nov 26, 2023 at

Re: [VOTE] Release Lucene 9.9.0 RC1

My expectation is that we will do a 9.x minor at about the same time as 10.0 anyway, this is what we have done in the past for new majors. This will give an opportunity to make sure we have deprecation warnings for all breaking changes in 10.0. Le jeu. 30 nov. 2023, 10:43, Chris Hegarty a écrit

Re: [VOTE] Release Lucene 9.9.0 RC1

> Thanks for raising the issue. I don’t have a strong opinion on whether or not to do the deprecation in this release, and since you say that it is minor, then I don’t see that it necessitates another respin. Since I had already started an RC2 build, then I just continued with it (and since the

Re: [VOTE] Release Lucene 9.9.0 RC1

For clarity, consider this vote cancelled. A new vote has been started on an RC2 build. > On 30 Nov 2023, at 16:22, Greg Miller wrote: > > If we're spinning a new RC, I'd like to ask this group if it would make sense > to pull this very small method deprecation in: >

[VOTE] Release Lucene 9.9.0 RC2

Please vote for release candidate 2 for Lucene 9.9.0 The artifacts can be downloaded from: https://dist.apache.org/repos/dist/dev/lucene/lucene-9.9.0-RC2-rev-06070c0dceba07f0d33104192d9ac98ca16fc500 You can run the smoke tester directly with this command: python3 -u

Re: [VOTE] Release Lucene 9.9.0 RC1

OK, great. I wanted to post a +1 already. Will wait for 2nd RC. Uwe Am 30.11.2023 um 16:38 schrieb Michael McCandless: On Thu, Nov 30, 2023 at 9:56 AM Chris Hegarty wrote: P.S. I’m less sure about this, but the RC 2 starts a 72hr voting time again? (Just so I know what TTL to put on

Re: [VOTE] Release Lucene 9.9.0 RC1

If we're spinning a new RC, I'd like to ask this group if it would make sense to pull this very small method deprecation in: https://github.com/apache/lucene/pull/12854 If there's a chance we don't release a 9.10 and go directly to 10.0, this would be our last opportunity to mark it deprecated on

Re: [VOTE] Release Lucene 9.9.0 RC1

for the sake of posterity, I did get a successful smoketest: SUCCESS! [1:00:06.512261] but +0 to release I guess since it's moot... On Thu, Nov 30, 2023 at 10:38 AM Michael McCandless < luc...@mikemccandless.com> wrote: > On Thu, Nov 30, 2023 at 9:56 AM Chris Hegarty > wrote: > > P.S. I’m

Re: [VOTE] Release Lucene 9.9.0 RC1

On Thu, Nov 30, 2023 at 9:56 AM Chris Hegarty wrote: P.S. I’m less sure about this, but the RC 2 starts a 72hr voting time > again? (Just so I know what TTL to put on that) > Yeah a new 72 hour clock starts with each new RC :) Mike McCandless http://blog.mikemccandless.com

Re: [VOTE] Release Lucene 9.9.0 RC1

Adrien, > On 30 Nov 2023, at 14:51, Adrien Grand wrote: > > Yet another bug due to ghost fields. :( Thanks for fixing! For reference, I > checked how postings work on SlowCompositeCodecReaderWrapper, since they are > prone to ghost fields as well, and they seem to be ok. Thanks for checking

Re: [VOTE] Release Lucene 9.9.0 RC1

Yet another bug due to ghost fields. :( Thanks for fixing! For reference, I checked how postings work on SlowCompositeCodecReaderWrapper, since they are prone to ghost fields as well, and they seem to be ok. I worry that it could actually occur in practice when enabling recursive graph bisection,

Re: [VOTE] Release Lucene 9.9.0 RC1

SUCCESS! [0:33:10.432870] +1 On Thu, Nov 30, 2023 at 2:59 PM Chris Hegarty wrote: > Hi Mike, > > On 30 Nov 2023, at 11:41, Michael McCandless > wrote: > > +1 to release. > > I hit a corner-case test failure and opened a PR to fix it: > https://github.com/apache/lucene/pull/12859 > > > Good

Re: [VOTE] Release Lucene 9.9.0 RC1

Hi Mike, > On 30 Nov 2023, at 11:41, Michael McCandless > wrote: > > +1 to release. > > I hit a corner-case test failure and opened a PR to fix it: > https://github.com/apache/lucene/pull/12859 Good find! It looks like the fix for this issue is well in hand - great. > I don't think this

Re: [VOTE] Release Lucene 9.9.0 RC1

SUCCESS! [0:47:11.013106] +1 On Thu, Nov 30, 2023 at 7:16 AM Ignacio Vera wrote: > SUCCESS! [0:52:59.891964] > > > +1 > > On Thu, Nov 30, 2023 at 12:42 PM Michael McCandless < > luc...@mikemccandless.com> wrote: > >> +1 to release. >> >> I hit a corner-case test failure and opened a PR to fix

Re: GitHub issues vs PRs vs Lucene's CHANGES.txt

Well, I created a starting tool to at least help us keep the what-should-be-identical-yet-is-nearly-impossible-for-us-to-achieve sections in CHANGES.txt in sync: https://github.com/apache/lucene/pull/12860 Right now it finds a number of mostly minor differences in the 9.9.0 sections in main vs

Re: [VOTE] Release Lucene 9.9.0 RC1

SUCCESS! [0:52:59.891964] +1 On Thu, Nov 30, 2023 at 12:42 PM Michael McCandless < luc...@mikemccandless.com> wrote: > +1 to release. > > I hit a corner-case test failure and opened a PR to fix it: > https://github.com/apache/lucene/pull/12859 > > I don't think this should block the release?

Re: [VOTE] Release Lucene 9.9.0 RC1

+1 to release. I hit a corner-case test failure and opened a PR to fix it: https://github.com/apache/lucene/pull/12859 I don't think this should block the release? -- it looks exotic. Thanks Chris! Mike McCandless http://blog.mikemccandless.com On Thu, Nov 30, 2023 at 1:16 AM Patrick Zhai

Re: [JENKINS] Lucene » Lucene-Check-main - Build # 10750 - Unstable!

I hit this one running the smoke tester on 9.9.0 RC 0, and it repros. I'll open an issue ... I think it's just a missing null check in the SlowCompositeCodecReaderWrapper. Mike McCandless http://blog.mikemccandless.com On Tue, Nov 28, 2023 at 6:37 PM Apache Jenkins Server <

Re: [VOTE] Release Lucene 9.9.0 RC1

SUCCESS! [1:03:54.880200] +1. Thank you Chris! On Wed, Nov 29, 2023 at 8:45 PM Nhat Nguyen wrote: > SUCCESS! [1:11:30.037919] > > +1. Thanks, Chris! > > On Wed, Nov 29, 2023 at 8:53 AM Chris Hegarty > wrote: > >> Hi, >> >> >> Please vote for release candidate 1 for Lucene 9.9.0 >> >> >> The

Re: [VOTE] Release Lucene 9.9.0 RC1

SUCCESS! [1:11:30.037919] +1. Thanks, Chris! On Wed, Nov 29, 2023 at 8:53 AM Chris Hegarty wrote: > Hi, > > > Please vote for release candidate 1 for Lucene 9.9.0 > > > The artifacts can be downloaded from: > > >

[VOTE] Release Lucene 9.9.0 RC1

Hi, Please vote for release candidate 1 for Lucene 9.9.0 The artifacts can be downloaded from: https://dist.apache.org/repos/dist/dev/lucene/lucene-9.9.0-RC1-rev-92a5e5b02e0e083126c4122f2b7a02426c21a037 You can run the smoke tester directly with this command: python3 -u

New branch and feature freeze for Lucene 9.9.0

Hi Lucene Devs, Branch branch_9_9 has been cut and versions updated to 9.10 on stable branch. Please observe the normal rules: * No new features may be committed to the branch. * Documentation patches, build patches and serious bug fixes may be committed to the branch. However, you should

Re: Lucene 9.9.0 Release

Hi Feng, > On 29 Nov 2023, at 11:25, Guo Feng wrote: > > Hi Chris, > > Nightly benchmark shows that #12699 gets back some speed. I've backport it to > 9.9.0. I think it is ready now. Awesome! > Sorry for delaying the release! No apology needed. I appreciate your speedy work here. I’ll

Re: Lucene 9.9.0 Release

Hi Chris, Nightly benchmark shows that #12699 gets back some speed. I've backport it to 9.9.0. I think it is ready now. Sorry for delaying the release! Feng On 2023/11/28 08:32:59 Chris Hegarty wrote: > Hi Guo, > > Thanks for the update. > > Let’s push the 9.9.0 branch cut until tomorrow

Re: GitHub issues vs PRs vs Lucene's CHANGES.txt

Oh, and that the CHANGES.txt entries in e.g. 9.9.0 section match on 9.x and main branches... I think that one we have some automation to catch? Mike McCandless http://blog.mikemccandless.com On Wed, Nov 29, 2023 at 5:58 AM Michael McCandless < luc...@mikemccandless.com> wrote: > Hi Team, > >

GitHub issues vs PRs vs Lucene's CHANGES.txt

Hi Team, I see Chris is tagging issues that were left open after their linked PRs were merged (thanks!). Is there something in our release tooling that cross-checks all the weakly linked metadata today: Milestone marked (or more often: not) on an issue vs commits to the respective branches vs

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-19) - Build # 14180 - Failure!

JVM crashed: # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x7f9fa8545493, pid=2982126, tid=2990096 # # JRE version: OpenJDK Runtime Environment (19.0+36) (build 19+36-2238) # Java VM: OpenJDK 64-Bit Server VM (19+36-2238, mixed mode, sharing,

Re: GDPR compliance

To the valid point Robert makes above about the underlying data still on the disk (old news): https://news.sophos.com/en-us/2022/09/23/morgan-stanley-fined-millions-for-selling-off-devices-full-of-customer-pii/ On Wed, Nov 29, 2023 at 11:01 AM Michael Sokolov wrote: > Another way is to ensure

Re: GDPR compliance

Another way is to ensure that all documents get updated on a regular cadence whether there are changes in the underlying data or not. Or, regenerating the index from scratch all the time. Of course these approaches might be more costly for an index that has intrinsically low update rates, but they

Re: GDPR compliance

It's not that insane, it's about several weeks however the big segment can stay there for quite long if there's not enough update for a merge policy to pick it up On Tue, Nov 28, 2023, 17:14 Dongyu Xu wrote: > What is the expected grace time for the data-deletion request to take > place? > >

Re: [JENKINS] Lucene » Lucene-NightlyTests-main - Build # 1199 - Unstable!

OK I pushed a fix. Mike On Tue, Nov 28, 2023 at 7:32 PM Michael McCandless < luc...@mikemccandless.com> wrote: > I think maybe LuceneTestCase.newSearcher is turning on concurrency > (setting the executor randomly). Since this test explicitly passes a "no > concurrency" collector manager I

Re: [JENKINS] Lucene » Lucene-NightlyTests-main - Build # 1199 - Unstable!

I think maybe LuceneTestCase.newSearcher is turning on concurrency (setting the executor randomly). Since this test explicitly passes a "no concurrency" collector manager I think we should switch to "new IndexSearcher(...)". Mike On Tue, Nov 28, 2023 at 7:29 PM Michael McCandless <

Re: [JENKINS] Lucene » Lucene-NightlyTests-main - Build # 1199 - Unstable!

This reproduces for me. Maybe related to LUCENE-10002 / #240? Mike On Tue, Nov 28, 2023 at 1:58 AM Apache Jenkins Server < jenk...@builds.apache.org> wrote: > Build: > https://ci-builds.apache.org/job/Lucene/job/Lucene-NightlyTests-main/1199/ > > 1 tests failed. > FAILED:

Re: GDPR compliance

Thanks Robert and Dawid, I think what you said is reasonable to me, I can keep the MP private then I guess(and it's not hard to code it out anyway so I guess people can still figure it out easily if they're facing a similar situation). For our case I think we do have some other constraints so we

Re: GDPR compliance

and if you delete those segments, will that data ever be actually removed from the underlying physical storage? equally uncertain. deleting a file from the filesystem is similar to what lucene is doing, it doesn't really delete anything from the disk, just allows it to be overwritten by future

Re: GDPR compliance

What is the expected grace time for the data-deletion request to take place? I'm not expert about the policy but I think something like "I need my data to be gone in next 2 second" is unreasonable. Tony X From: Robert Muir Sent: Tuesday, November 28, 2023

Re: GDPR compliance

Are larger and older segments even certain to ever be merged in practice? I was assuming that if there is not a lot of new indexed content and not a lot of older documents being deleted, large older segment might never have to be merged. On Tue 28 Nov 2023 at 20:53, Robert Muir wrote: > I

Re: GDPR compliance

I don't think there's any problem with GDPR, and I don't think users should be running unnecessary "optimize". GDRP just says data should be erased without "undue" delay. waiting for a merge to nuke the deleted docs isn't "undue", there is a good reason for it. On Tue, Nov 28, 2023 at 2:40 PM

GDPR compliance

Hi Folks, In LinkedIn we need to comply with GDPR for a large part of our data, and an important part of it is that we need to be sure we have completely deleted the data the user requested to delete within a certain period of time. The way we have come up with so far is to: 1. Record the segment

Re: Lucene 9.9.0 Release

Hi Guo, Thanks for the update. Let’s push the 9.9.0 branch cut until tomorrow (rather than today as previously suggested), which should allow time to determine the outstanding issues you mentioned below. That should be more straightforward all round. New 9.9.0 branch cut 12:00 29th Nov 2023

Re: Lucene 9.9.0 Release

+1, thanks for volunteering Chris! #12699 is merged to main. I plan to backport it to 9.9 if it fixes the performance drop, otherwise revert #12699 and #12631 (the PR introduced regression) and push them to the next version. On 2023/11/21 09:51:43 Chris Hegarty wrote: > Hi, > > It's been a

Re: Lucene 9.9.0 Release

Thanks Chris for checking. I had been too optimistic for #12180, I'll push it to 9.10. Fingers crossed that #12699 fixes the performance drop. Le lun. 27 nov. 2023, 07:17, Chris Hegarty a écrit : > Hi Adrien, > > Comments inline. > > On 21 Nov 2023, at 12:31, Adrien Grand wrote: > > +1 9.9

Re: Lucene 9.9.0 Release

Hi Adrien, Comments inline. > On 21 Nov 2023, at 12:31, Adrien Grand wrote: > > +1 9.9 has plenty of great changes indeed! Thanks for volunteering as a RM, > Chris. > > It would be good to try and fix the PKLookup regression that was introduced > since 9.8:

Re: ./crave pull .. 'heapdumps/* Fwd: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still Failing!

Pardon On Sun, Nov 26, 2023 at 11:28 AM Gautam Worah wrote: > I think you meant to send it to d...@solr.apache.org? > > On Sun, Nov 26, 2023 at 12:24 AM Mikhail Khludnev wrote: > >> Hello >> It's rather like a logical error in crave pull. How to work around it? >> >> + status=0 >> + ./crave

Re: ./crave pull .. 'heapdumps/* Fwd: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still Failing!

I think you meant to send it to d...@solr.apache.org? On Sun, Nov 26, 2023 at 12:24 AM Mikhail Khludnev wrote: > Hello > It's rather like a logical error in crave pull. How to work around it? > > + status=0 > + ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' >

./crave pull .. 'heapdumps/* Fwd: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still Failing!

Hello It's rather like a logical error in crave pull. How to work around it? + status=0 + ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*' Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2) rsync error: some

Re: [JENKINS] Lucene-main-Linux (64bit/hotspot/jdk-19) - Build # 45643 - Failure!

Hmm JVM crashed (there's an hs_err file there): > Process 'Gradle Test Executor 33' finished with non-zero exit value 134 Mike McCandless http://blog.mikemccandless.com On Sat, Nov 25, 2023 at 6:34 AM Policeman Jenkins Server < jenk...@thetaphi.de> wrote: > Build:

Re: Test framework can't find SPI implementations from module sandbox

Thanks Mike. (And Happy Thanksgiving!) Yes, I did add it to META-INF folder, as well as to module-info.java. This problem affects to the all tests. E.g. All tests that will be kiced off if I run `./gradlew test -Dtests.postingsformat="Lucene99RandomAccess"` Tony

Re: Lucene 9.9.0 Release

+1 to a 9.9.0 release, thanks for volunteering. And also positive to immediately start planning the 10.0 release. JDK bump -> 17 is a major "feature" in itself, so we can bump main branch to 21 too. And discontinue support for 8.x after the 8.11.3 release. Jan > 22. nov. 2023 kl. 17:11 skrev

Re: Lucene 9.9.0 Release

+1 thanks for volunteering! Hijacking the thread a bit, sorry, I started looking into whether this is a good time to start looking ahead to 10? I know we had some rumblings about releasing that so we can start requiring newer JDKs. But looking at CHANGES it feels like we already back-ported most

Re: Test framework can't find SPI implementations from module sandbox

did you add to the sandbox META-INF file? It looks like maybe sandbox is not included in the scope of the test, but you didn't say which test it was. Is the test also in the sandbox module? On Mon, Nov 20, 2023 at 6:56 PM Dongyu Xu wrote: > Hi devs, > > I tried to plug in my experimental

Re: Lucene 9.9.0 Release

+1, thank you Chris! On Tue, Nov 21, 2023, 06:49 Benjamin Trent wrote: > +1 9.9 will be a stellar release! > > Thank you Chris! > > On Tue, Nov 21, 2023 at 7:31 AM Adrien Grand wrote: > >> +1 9.9 has plenty of great changes indeed! Thanks for volunteering as a >> RM, Chris. >> >> It would be

<    2   3   4   5   6   7   8   9   10   11   >