Re: Format metadata versioning vs. new named Formats

2024-04-12 Thread Adrien Grand
These are indeed two ways to change the on-disk layout of our file formats. In general, I try to follow the following rules: - If the format is not bw-compatible (e.g. formats in lucene/codecs), do the change in-place and bump both VERSION_START and VERSION_CURRENT to make sure users get a

Re: Lucene 10

2024-03-20 Thread Adrien Grand
Thanks Mike and Dawid for the kind words, and thanks Patrick, Luca and Egor for your interest in decoupling index geometry from search concurrency, this would be a great release highlight if we can get it into Lucene 10! I haven't seen pushback on the proposed schedule so I plan on proceeding

Lucene 10

2024-03-13 Thread Adrien Grand
Hello everyone! It's been ~2.5 years since we released Lucene 9.0 (December 2021) and I'd like us to start working towards Lucene 10.0. I'm volunteering for being the release manager and propose the following timeline: - ~September 15th: main gets bumped to 11.x, branch_10x gets created -

Re: [Vote] Bump the Lucene main branch to Java 21

2024-02-23 Thread Adrien Grand
+1 On Fri, Feb 23, 2024 at 12:54 PM Uwe Schindler wrote: > > Here is my +1 > > Uwe > > Am 23.02.2024 um 12:24 schrieb Chris Hegarty: > > Hi, > > > > Since the discussion on bumping the Lucene main branch to Java 21 is > > winding down, let's hold a vote on this important change. > > > > Once

Re: (lucene) branch main updated: Fix bw index generation logic.

2024-02-20 Thread Adrien Grand
x bw index generation logic. > 13d561af1d6 is described below > > commit 13d561af1d624f35f8a27a05490062ac2472e786 > Author: Adrien Grand > AuthorDate: Tue Feb 20 22:10:01 2024 +0100 > > Fix bw index generation logic. > --- > d

[ANNOUNCE] Apache Lucene 9.10.0 released

2024-02-20 Thread Adrien Grand
The Lucene PMC is pleased to announce the release of Apache Lucene 9.10. Apache Lucene is a high-performance, full-featured search engine library written entirely in Java. It is a technology suitable for nearly any application that requires structured search, full-text search, faceting,

Welcome Zhang Chao as Lucene committer

2024-02-20 Thread Adrien Grand
I'm pleased to announce that Zhang Chao has accepted the PMC's invitation to become a committer. Chao, the tradition is that new committers introduce themselves with a brief bio. Congratulations and welcome! -- Adrien

Re: Announcing githubsearch!

2024-02-20 Thread Adrien Grand
Very cool, thank you Mike! On Mon, Feb 19, 2024 at 5:40 PM Michael McCandless < luc...@mikemccandless.com> wrote: > Hi Team, > > ~1.5 years ago (August 2022) we migrated our Lucene issue tracking from > Jira to GitHub. Thank you Tomoko for all the hard work doing such a > complex, multi-phased,

[RESULT][VOTE] Release Lucene 9.10.0 RC1

2024-02-20 Thread Adrien Grand
ster to also allow it to >> pass Java 21. Maybe the best would be to pass multiple Java versions as >> comma spearated list, just the default one must be Java 11 (the baseline). >> This would allo me to spin Policeman Jenkins with Java 11, Java 17, Java >> 19, Java 20, Java

Re: [JENKINS] Lucene » Lucene-NightlyTests-9.x - Build # 825 - Still Unstable!

2024-02-15 Thread Adrien Grand
I removed 8.12 from the versions.txt file since it hasn't been released. On Thu, Feb 15, 2024 at 7:38 AM Apache Jenkins Server < jenk...@builds.apache.org> wrote: > Build: > https://ci-builds.apache.org/job/Lucene/job/Lucene-NightlyTests-9.x/825/ > > 6 tests failed. > FAILED: >

[VOTE] Release Lucene 9.10.0 RC1

2024-02-14 Thread Adrien Grand
Please vote for release candidate 1 for Lucene 9.10.0 The artifacts can be downloaded from: https://dist.apache.org/repos/dist/dev/lucene/lucene-9.10.0-RC1-rev-695c0ac84508438302cd346a812cfa2fdc5a10df You can run the smoke tester directly with this command: python3 -u

Re: Lucene 9.10

2024-02-13 Thread Adrien Grand
t; Am 07.02.2024 um 15:57 schrieb Adrien Grand: > > Hello all, > > It's been 2 months since we released 9.9 and we accumulated a good number > of changes, so I'd like to propose that we release 9.10.0. > > If there are no objections, I volunteer to be the release manager and &

Re: (lucene) branch branch_9_10 created (now 695c0ac8450)

2024-02-12 Thread Adrien Grand
You're so quick Uwe, thank you! On Mon, Feb 12, 2024 at 2:49 PM Uwe Schindler wrote: > Hi Adrien, > > Thanks for creating the branch. I activated Policeman Jenkins tests for it. > > Uwe > > Am 12.02.2024 um 14:30 schrieb jpou...@apache.org: > > This is an automated email from the ASF

New branch and feature freeze for Lucene 9.10.0

2024-02-12 Thread Adrien Grand
NOTICE: Branch branch_9_10 has been cut and versions updated to 9.11 on stable branch. Please observe the normal rules: * No new features may be committed to the branch. * Documentation patches, build patches and serious bug fixes may be committed to the branch. However, you should submit all

Lucene 9.10

2024-02-07 Thread Adrien Grand
Hello all, It's been 2 months since we released 9.9 and we accumulated a good number of changes, so I'd like to propose that we release 9.10.0. If there are no objections, I volunteer to be the release manager and suggest cutting the branch next Monday (February 12th) and starting the release

Re: Computing weight.count() cheaply in the face of deletes?

2024-02-06 Thread Adrien Grand
d be fine if the model in Lucene would > encode deleted docs (it did that in earlier times). As deletes are sparse > (deletes are in most cases <40%), this would help to make the iterator > cheaper. > Uwe > > Am 06.02.2024 um 09:01 schrieb Adrien Grand: > > Hey Michael, >

Re: Computing weight.count() cheaply in the face of deletes?

2024-02-06 Thread Adrien Grand
sufficiently cheap. Hmm... precomputing that int[] for deleted docIDs on > refresh could be an option too. > > Thanks again, > Froh > > On Fri, Feb 2, 2024 at 11:38 PM Adrien Grand wrote: > >> Hi Michael, >> >> Indeed, only MatchAllDocsQuery knows how to produc

Re: Computing weight.count() cheaply in the face of deletes?

2024-02-02 Thread Adrien Grand
Hi Michael, Indeed, only MatchAllDocsQuery knows how to produce a count when there are deletes. Your idea sounds good to me, do you actually need a side car iterator for deletes, or could you use a nextClearBit() operation on the bit set? I don't think we can fold it into Weight#count since

Re: [VOTE] Release Lucene 9.9.2 RC1

2024-01-26 Thread Adrien Grand
+1 SUCCESS! [1:00:39.059480] On Fri, Jan 26, 2024 at 7:54 AM Ignacio Vera wrote: > +1 > > SUCCESS! [0:54:32.772088] > > On Thu, Jan 25, 2024 at 11:23 PM Uwe Schindler wrote: > >> Hi, >> >> +1 to release. >> >> Tested smoketester with Java 11 and 17; results: >>

Re: Welcome Stefan Vodita as Lucene committter

2024-01-18 Thread Adrien Grand
Welcome Stefan! On Thu, Jan 18, 2024 at 6:10 PM Patrick Zhai wrote: > Welcome and Congrats, Stefan. > > Patrick > > On Thu, Jan 18, 2024, 08:45 Chris Hegarty > wrote: > >> Welcome Stefan. >> >> -Chris. >> >> > On 18 Jan 2024, at 15:53, Michael McCandless >> wrote: >> > >> > Hi Team, >> > >> >

Re: Lucene v9.9.1: org.apache.lucene.search.ScoreMode

2024-01-12 Thread Adrien Grand
There have been a few similar reports of the ScoreMode import issue at Elastic with Lucene 9.9.1. It looks like an Intellij-specific issue, which can be addressed by upgrading to the latest version. (I'm not really an Intellij user myself so I don't know more about the problem.) On Sun, Jan 7,

Re: [JENKINS] Lucene-main-Linux (64bit/hotspot/jdk-19) - Build # 45856 - Unstable!

2023-12-20 Thread Adrien Grand
I don't fully understandi it yet. I opened an issue: https://github.com/apache/lucene/issues/12957. On Tue, Dec 19, 2023 at 6:02 PM Adrien Grand wrote: > This looks like a real bug with the default codec when the prefix compares > greater than every indexed term. I'll look into it to

Re: [JENKINS] Lucene-main-Linux (64bit/hotspot/jdk-19) - Build # 45856 - Unstable!

2023-12-19 Thread Adrien Grand
This looks like a real bug with the default codec when the prefix compares greater than every indexed term. I'll look into it tomorrow if nobody beats me to it. On Tue, Dec 19, 2023 at 12:35 PM Policeman Jenkins Server < jenk...@thetaphi.de> wrote: > Build:

Re: UTF-8 well-formedness for SimpleTextCodec

2023-12-19 Thread Adrien Grand
Hey Michael, Writing well-formed UTF-8 with SimpleTextformat sounds desirable indeed, e.g. your PR makes sense. I don't think we would want to be heroic about it, but if we can serialize the same information easily, then it sounds like something we should do. Thanks for improving SimpleTextCodec!

Re: [VOTE] Release Lucene 9.9.1 RC1

2023-12-14 Thread Adrien Grand
+1 SUCCESS! [1:41:08.997307] Thanks Chris for taking care of this release. On Thu, Dec 14, 2023 at 4:40 PM Michael Sokolov wrote: > > +1 > > SUCCESS! [0:50:50.776559] > > Note: we did get some test fails on the mailing list this morning, but I > believe they are not real bugs and will be

Re: [JENKINS] Lucene » Lucene-NightlyTests-main - Build # 1209 - Unstable!

2023-12-11 Thread Adrien Grand
Woops, sorry for suggesting this change in the first place! I didn't know we had this validation for points, but not for postings. On Fri, Dec 8, 2023 at 2:16 PM Michael McCandless wrote: > OK I reverted the "optimization" to not pull FieldInfo for a field when > getting Points values from

Re: [VOTE] Release Lucene 9.9.0 RC1

2023-11-30 Thread Adrien Grand
My expectation is that we will do a 9.x minor at about the same time as 10.0 anyway, this is what we have done in the past for new majors. This will give an opportunity to make sure we have deprecation warnings for all breaking changes in 10.0. Le jeu. 30 nov. 2023, 10:43, Chris Hegarty a écrit

Re: [VOTE] Release Lucene 9.9.0 RC1

2023-11-30 Thread Adrien Grand
Yet another bug due to ghost fields. :( Thanks for fixing! For reference, I checked how postings work on SlowCompositeCodecReaderWrapper, since they are prone to ghost fields as well, and they seem to be ok. I worry that it could actually occur in practice when enabling recursive graph bisection,

Re: Lucene 9.9.0 Release

2023-11-27 Thread Adrien Grand
Thanks Chris for checking. I had been too optimistic for #12180, I'll push it to 9.10. Fingers crossed that #12699 fixes the performance drop. Le lun. 27 nov. 2023, 07:17, Chris Hegarty a écrit : > Hi Adrien, > > Comments inline. > > On 21 Nov 2023, at 12:31, Adrien Grand wr

Re: Lucene 9.9.0 Release

2023-11-21 Thread Adrien Grand
+1 9.9 has plenty of great changes indeed! Thanks for volunteering as a RM, Chris. It would be good to try and fix the PKLookup regression that was introduced since 9.8: http://people.apache.org/~mikemccand/lucenebench/PKLookup.html. Is it just about getting #12699

Heads up: reindex main/9.x indices

2023-11-20 Thread Adrien Grand
Hello all, The 9.9 file format was just updated to encode tail postings using group-vint instead of vint[1], so you need to reindex all indices generated from the main and branch_9x branches. As always, indexes created from a proper Lucene release are still compatible. [1]

Re: [JENKINS] Lucene » Lucene-Check-main - Build # 10678 - Unstable!

2023-11-20 Thread Adrien Grand
A one-in-a-million-runs test failure. I pushed a fix: https://github.com/apache/lucene/commit/194a500323531b66124577167006115c34dfde54 . On Sun, Nov 19, 2023 at 10:00 PM Apache Jenkins Server < jenk...@builds.apache.org> wrote: > Build: >

Re: SPLADE implementation

2023-11-15 Thread Adrien Grand
make use of FeatureField exactly? > > I tried to find some code examples, but couldn't, do you maybe have some > pointers? > > Thanks > > Michael > > > Am 15.11.23 um 10:34 schrieb Adrien Grand: > > Hi Michael, > > What functionality are you missing? Lucene

Re: SPLADE implementation

2023-11-15 Thread Adrien Grand
Hi Michael, What functionality are you missing? Lucene already supports indexing/querying weighted terms using FeatureField. On Wed, Nov 15, 2023 at 10:03 AM Michael Wechner wrote: > Hi > > I have found the following issue re a possible SPLADE implementation > >

Re: [JENKINS] Lucene-9.x-Linux (64bit/openj9/jdk-17.0.8) - Build # 14028 - Unstable!

2023-11-14 Thread Adrien Grand
What a fantastic test, it found another real bug. I opened https://github.com/apache/lucene/pull/12807. On Mon, Nov 13, 2023 at 10:44 PM Policeman Jenkins Server < jenk...@thetaphi.de> wrote: > Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/14028/ > Java: 64bit/openj9/jdk-17.0.8

Re: Welcome Patrick Zhai to the Lucene PMC

2023-11-10 Thread Adrien Grand
Welcome Patrick! Le ven. 10 nov. 2023, 21:18, Greg Miller a écrit : > Congrats and welcome Patrick! > > On Fri, Nov 10, 2023 at 12:05 PM Michael McCandless < > luc...@mikemccandless.com> wrote: > >> I'm happy to announce that Patrick Zhai has accepted an invitation to >> join the Lucene Project

Re: Apach Solr Exercise 1 Index the Techproducts Data step not working

2023-11-04 Thread Adrien Grand
Hi Qizhi, I am moving your question to the Solr users list. Le sam. 4 nov. 2023, 01:58, Qizhi Zheng a écrit : > Hello, > > > > I am trying to run the Solr Tutorial Exercise 1 Index Techproducts Data in > Windows 10. I typed the exact same command following it link: > > >

Re: [JENKINS] Lucene-main-Linux (64bit/hotspot/jdk-20) - Build # 45223 - Unstable!

2023-11-01 Thread Adrien Grand
I pushed a fix: https://github.com/apache/lucene/commit/66324f763fc7fb0d8e7cd6f334e5438f0171c84e . On Thu, Oct 26, 2023 at 4:35 PM Policeman Jenkins Server < jenk...@thetaphi.de> wrote: > Build: https://jenkins.thetaphi.de/job/Lucene-main-Linux/45223/ > Java: 64bit/hotspot/jdk-20

Re: [JENKINS] Lucene-9.x-Linux (64bit/openj9/jdk-17.0.5) - Build # 13732 - Unstable!

2023-10-31 Thread Adrien Grand
I pushed a fix for these failures: https://github.com/apache/lucene/commit/85f5d3bb0bf84fed46ca4c093c1aa084e4a43873 On Fri, Oct 27, 2023 at 9:55 AM Policeman Jenkins Server < jenk...@thetaphi.de> wrote: > Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/13732/ > Java:

Re: [JENKINS] Lucene » Lucene-NightlyTests-9.x - Build # 720 - Unstable!

2023-10-26 Thread Adrien Grand
For reference, Simon pushed a fix for these TestIndexWriter.classMethod failures: https://github.com/apache/lucene/commit/01acb1c37b2826339d95681251dacd7e2a929be9 On Tue, Oct 24, 2023 at 11:12 AM Apache Jenkins Server < jenk...@builds.apache.org> wrote: > Build: >

Re: [JENKINS] Lucene-9.x-Linux (64bit/openj9/jdk-17.0.5) - Build # 13705 - Unstable!

2023-10-25 Thread Adrien Grand
This is mine, I'm looking into it. On Wed, Oct 25, 2023 at 7:54 PM Policeman Jenkins Server < jenk...@thetaphi.de> wrote: > Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/13705/ > Java: 64bit/openj9/jdk-17.0.5 -XX:-UseCompressedOops -Xgcpolicy:metronome > > 1 tests failed. > FAILED: >

Welcome Guo Feng to the Lucene PMC

2023-10-24 Thread Adrien Grand
I'm pleased to announce that Guo Feng has accepted an invitation to join the Lucene PMC! Congratulations Feng, and welcome aboard! -- Adrien

Welcome Luca Cavanna to the Lucene PMC

2023-10-19 Thread Adrien Grand
I'm pleased to announce that Luca Cavanna has accepted an invitation to join the Lucene PMC! Congratulations Luca, and welcome aboard! -- Adrien

Re: PackedInts functionalities

2023-10-17 Thread Adrien Grand
+1 to what Mikhail wrote, this is e.g. how postings work: instead of interleaving doc IDs and frequencies, they always store a block of 128 doc IDs followed by a block of 128 frequencies. For reference, bit packing feels space-inefficient for this kind of data. I would expect docFreqs to have a

Re: Weird HNSW merge performance result

2023-10-10 Thread Adrien Grand
Regarding building time, did you configure a SerialMergeScheduler? Otherwise merges run in separate threads, which would explain the speedup as adding vectors to the graph gets more and more expensive as the size of the graph increases. Le mer. 11 oct. 2023, 05:07, Patrick Zhai a écrit : > Hi

Re: LeafCollector#finish idempotency?

2023-10-09 Thread Adrien Grand
Hi Greg, I agree that LeafCollector implementations should be able to assume that finish() only gets called once. The test framework already makes this assumption:

Re: ConjunctionDISI nextDoc can return immediately when NO_MORE_DOCS

2023-10-01 Thread Adrien Grand
checking. > > This might be a related but more general approach. > > That was almost 30 years ago, so any patents are long-expired. > > wunder > Walter Underwood > wun...@wunderwood.org > http://observer.wunderwood.org/ (my blog) > > On Oct 1, 2023, at 10:12 AM, Adrien Grand

Re: ConjunctionDISI nextDoc can return immediately when NO_MORE_DOCS

2023-10-01 Thread Adrien Grand
not exist ,and then the loop execution may cause > unnecessary overhead.(sorry I have not yet find out whether there is any > filter work before the doNext().. > > Best Regard > > Adrien Grand 于2023年10月1日周日 22:30写道: > >> Hello, >> >> This change wo

Re: ConjunctionDISI nextDoc can return immediately when NO_MORE_DOCS

2023-10-01 Thread Adrien Grand
Hello, This change would be correct, but it would only save work when the conjunction is exhausted, and add overhead otherwise? Le sam. 30 sept. 2023, 16:20, YouPeng Yang a écrit : > Hi > I am reading the code of class ConjunctionDISI .and about the method > nextDoc , Suppose that the

Re: Solr upgrade to Lucene 9.8.0 question

2023-09-28 Thread Adrien Grand
Hi Alex, I believe that your analysis is correct. > is it expected that the 'finish' method is idempotent? I don't expect `finish()` to be idempotent. It should not get called multiple times per segment either, only once and when collection runs successfully. Do you have a Lucene test case that

Re: Can the BooleanQuery execution be optimized with same term queries

2023-09-23 Thread Adrien Grand
Thanks for letting me know, I'm glad you like them! Le ven. 22 sept. 2023, 16:36, YouPeng Yang a écrit : > Hi Adrien >Glad to have your opinion.I am reading your excellent articles on > elastic blog. > > Best regards > > > Adrien Grand 于2023年9月19日周二 21:32写道: >

Re: [VOTE] Release Lucene 9.8.0 RC1

2023-09-22 Thread Adrien Grand
+1 SUCCESS! [0:54:58.932481] On Fri, Sep 22, 2023 at 4:18 PM Uwe Schindler wrote: > > Hi, > > I verified the release with the usual tools and my workflow: > > Policeman Jenkins ran smoketester for me with Java 11 and Java 17: > https://jenkins.thetaphi.de/job/Lucene-Release-Tester/28/console > >

Re: Sitemap to get latest reference manual to rank in Google/Bing?

2023-09-21 Thread Adrien Grand
Hi Walter, You emailed the Lucene dev list (dev@lucene.a.o) but I think you meant to ask this question to the Solr list (dev@solr.a.o). On Wed, Sep 20, 2023 at 8:59 PM Walter Underwood wrote: > > When I get web search results that include the Solr Reference Guide, I often > get older versions

Re: Lucene 9.8 Release

2023-09-21 Thread Adrien Grand
; > On Tue, Sep 19, 2023 at 12:05 AM Adrien Grand wrote: >> >> Thanks Patrick, this PR is now merged. >> >> On Tue, Sep 19, 2023 at 6:22 AM Patrick Zhai wrote: >> > >> > Update: >> > Will wait https://github.com/apache/lucene/pull/12568 to be me

Re: Can the BooleanQuery execution be optimized with same term queries

2023-09-19 Thread Adrien Grand
Hi Yang, It would be legal for Lucene to perform such optimizations indeed. On Tue, Sep 19, 2023 at 3:27 PM YouPeng Yang wrote: > > Hi All > > Sorry to bother you.The happiest thing is studying the Lucene source > codes,thank you for all the great works . > > > About the BooleanQuery.I am

Re: [lucene] branch branch_9x updated: Fix issues with BP tests and the security manager. (#12568)

2023-09-19 Thread Adrien Grand
tory https://gitbox.apache.org/repos/asf/lucene.git > >> > >> > >> The following commit(s) were added to refs/heads/branch_9x by this push: > >> new c241ab006c4 Fix issues with BP tests and the security > >> manager. (#12568) > >> c241ab006c4 is

Re: Lucene 9.8 Release

2023-09-19 Thread Adrien Grand
Thanks Patrick, this PR is now merged. On Tue, Sep 19, 2023 at 6:22 AM Patrick Zhai wrote: > > Update: > Will wait https://github.com/apache/lucene/pull/12568 to be merged to cut the > branch > > > On Mon, Sep 18, 2023 at 11:00 AM Michael Sokolov wrote: >> >> +1 for a release soon, and thanks

Re: [JENKINS] Lucene-MMAPv2-Windows (64bit/hotspot/jdk-21-rc) - Build # 801 - Still Unstable!

2023-09-18 Thread Adrien Grand
cated reader). > > Uwe > > Am 18.09.2023 um 18:18 schrieb Adrien Grand: > > Thanks Uwe for digging. The fork-join pool is optional, I will change > > the test to use a ByteBuffersDirectory. > > > > On Mon, Sep 18, 2023 at 6:15 PM Uwe Schindler wrote: > >

Re: [JENKINS] Lucene-MMAPv2-Windows (64bit/hotspot/jdk-21-rc) - Build # 801 - Still Unstable!

2023-09-18 Thread Adrien Grand
Thanks Uwe for digging. The fork-join pool is optional, I will change the test to use a ByteBuffersDirectory. On Mon, Sep 18, 2023 at 6:15 PM Uwe Schindler wrote: > > Hi, > > this issue is a real one. The problem is: The default ForkJoin thread pool > runs all tasks with zero permissions if a

Re: Lucene 9.8 Release

2023-09-12 Thread Adrien Grand
Thanks Patrick for volunteering as release manager! Le mar. 12 sept. 2023, 08:07, Patrick Zhai a écrit : > Hi all, > It's been a while since the last release and we have quite a few good > changes including new APIs, improvements and bug fixes. Should we release > the 9.8? > > If there's no

Re: Enabling concurrent search only for certain queries

2023-07-19 Thread Adrien Grand
nderstanding is correct > and that's how the future API is going to look like before we do this > refactoring. Thank you. > > -- > Regards, > Alex > > > On Tue, Jul 18, 2023 at 3:26 PM Adrien Grand wrote: >> >> Hi Alexander, >> >> Yo

Re: Enabling concurrent search only for certain queries

2023-07-18 Thread Adrien Grand
Hi Alexander, You mentioned that your current implementation relies on a single IndexSearcher. Could you have two instead? One that configures an executor for long running queries and another one that doesn't? For reference, IndexSearchers are cheap to create, it would be ok to create one per

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-17.0.5) - Build # 11322 - Unstable!

2023-06-27 Thread Adrien Grand
I opened a PR at https://github.com/apache/lucene/pull/12400 with a fix, I tried to explain in the PR description why AssertingScorer has this check. Even though it's not documented in BulkScorer#score, I think it's a good check to keep. On Wed, Jun 28, 2023 at 6:25 AM Adrien Grand wrote

Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-17.0.5) - Build # 11322 - Unstable!

2023-06-27 Thread Adrien Grand
Thanks Patrick, I will look into it this morning. Le mer. 28 juin 2023, 06:20, Patrick Zhai a écrit : > Yeah I think that's the commit, I'm definitely not an expert on scorer as > well so maybe @jpou...@gmail.com could you take a > look? > > Patrick > > On Tue, Jun 27, 2023 at 5:34 AM Michael

[ANNOUNCE] Apache Lucene 9.7.0 released

2023-06-26 Thread Adrien Grand
The Lucene PMC is pleased to announce the release of Apache Lucene 9.7.0. Apache Lucene is a high-performance, full-featured search engine library written entirely in Java. It is a technology suitable for nearly any application that requires structured search, full-text search, faceting,

[RESULTI] [VOTE] Release Lucene 9.7.0 RC1

2023-06-25 Thread Adrien Grand
s >>>>> copypaste! >>>>> >>>>> I verified in addition the following: >>>>> >>>>>- Changes for completeness; I also updated the release notes >>>>>(function query support for vectors was missing) >

[VOTE] Release Lucene 9.7.0 RC1

2023-06-21 Thread Adrien Grand
Please vote for release candidate 1 for Lucene 9.7.0 The artifacts can be downloaded from: https://dist.apache.org/repos/dist/dev/lucene/lucene-9.7.0-RC1-rev-ccf4b198ec328095d45d2746189dc8ca633e8bcf You can run the smoke tester directly with this command: python3 -u

Re: Scorer#getMinScore()

2023-06-21 Thread Adrien Grand
Your guesses sound right to me: - A query that does subtractions could yield negative scores, which are not supported. - We'd need to store the least competitive impacts for each block of postings, which would double the amount of CPU and space we spend on impacts, while min scores would likely

Draft of release notes for 9.7

2023-06-21 Thread Adrien Grand
Hello all, I put up a draft of release notes for 9.7, am I missing important changes? https://cwiki.apache.org/confluence/display/LUCENE/Release+notes+9.7 -- Adrien

Re: Richer Aggregations in Lucene

2023-06-20 Thread Adrien Grand
Hey Shradha, Such a contribution would be welcome. There is no good reason not to support richer aggregations in Lucene. One thing that I have found interesting with faceting/aggregations is that every implementation seems to make different trade-offs, e.g. - Lucene's faceting historically

Welcome Chris Hegarty to the Lucene PMC

2023-06-19 Thread Adrien Grand
I'm pleased to announce that Chris Hegarty has accepted an invitation to join the Lucene PMC! Congratulations Chris, and welcome aboard! -- Adrien

New branch and feature freeze for Lucene 9.7.0

2023-06-16 Thread Adrien Grand
NOTICE: Branch branch_9_7 has been cut and versions updated to 9.8 on stable branch. Please observe the normal rules: * No new features may be committed to the branch. * Documentation patches, build patches and serious bug fixes may be committed to the branch. However, you should submit all

Re: Lucene 9.7 release

2023-06-12 Thread Adrien Grand
ere are bug fixes they won't affect public APIs or the >>> incubator module, right? >>> >>> Your understanding is correct. I do not expect any API changes at this >>> point. >>> >>> The MMapDir changes are already tested all the time, vector AP

Lucene 9.7 release

2023-06-09 Thread Adrien Grand
Hello all, There is some good stuff that is scheduled for 9.7 already, I found the following changes in the changelog that look especially interesting: - Concurrent query rewrites for vector queries. - Speedups to vector indexing/search via integration of the Panama vector API. - Reduced

Re: [Proposal] Remove max number of dimensions for KNN vectors

2023-04-08 Thread Adrien Grand
As Dawid pointed out earlier on this thread, this is the rule for Apache projects: a single -1 vote on a code change is a veto and cannot be overridden. Furthermore, Robert is one of the people on this project who worked the most on debugging subtle bugs, making Lucene more robust and improving

Re: [Proposal] Remove max number of dimensions for KNN vectors

2023-03-31 Thread Adrien Grand
I'm supportive of bumping the limit on the maximum dimension for vectors to something that is above what the majority of users need, but I'd like to keep a limit. We have limits for other things like the max number of docs per index, the max term length, the max number of dimensions of points,

Re: [JENKINS] Lucene-MMAPv2-Linux (64bit/openj9/jdk-17.0.5) - Build # 644 - Unstable!

2023-03-15 Thread Adrien Grand
We have seen this issue a few times over the past months. I would default to assuming a bug in J9, do we have a contact on the J9 team that we should make aware of this? On Sat, Mar 11, 2023 at 12:33 PM Policeman Jenkins Server wrote: > > Build:

Re: Lucene PMC Chair Greg Miller

2023-03-07 Thread Adrien Grand
Thank you Bruno and Greg! Le lun. 6 mars 2023, 18:15, Bruno Roustant a écrit : > Hello Lucene developers, > > Lucene Program Management Committee has elected a new chair, Greg Miller, > and the Board has approved. > > Greg, thank you for stepping up, and congratulations! > > > - Bruno >

Re: [JENKINS] Lucene-9.x-MacOSX (64bit/hotspot/jdk-11.0.15) - Build # 1806 - Failure!

2023-02-07 Thread Adrien Grand
3 at 10:24 AM Adrien Grand wrote: > > I'm looking into it. > > On Tue, Feb 7, 2023 at 8:08 AM Policeman Jenkins Server > wrote: > > > > Build: https://jenkins.thetaphi.de/job/Lucene-9.x-MacOSX/1806/ > > Java: 64bit/hotspot/jdk-11.0.15 -XX:-UseCompressedO

Re: [JENKINS] Lucene-9.x-MacOSX (64bit/hotspot/jdk-11.0.15) - Build # 1806 - Failure!

2023-02-07 Thread Adrien Grand
I'm looking into it. On Tue, Feb 7, 2023 at 8:08 AM Policeman Jenkins Server wrote: > > Build: https://jenkins.thetaphi.de/job/Lucene-9.x-MacOSX/1806/ > Java: 64bit/hotspot/jdk-11.0.15 -XX:-UseCompressedOops -XX:+UseParallelGC > > No tests ran. > >

Welcome Ben Trent as Lucene committer

2023-01-27 Thread Adrien Grand
I'm pleased to announce that Ben Trent has accepted the PMC's invitation to become a committer. Ben, the tradition is that new committers introduce themselves with a brief bio. Congratulations and welcome! -- Adrien - To

Re: Lucene 9.5 release notes draft

2023-01-26 Thread Adrien Grand
Thanks Luca, the release notes look good to me. On Thu, Jan 26, 2023 at 10:11 AM Luca Cavanna wrote: > > Hi all, > I published a draft of the release notes for Lucene 9.5 here: > https://cwiki.apache.org/confluence/display/LUCENE/Release+Notes+9.5 > > Could you please review it? Feel free to

Re: [VOTE] Release Lucene 9.5.0 RC1

2023-01-26 Thread Adrien Grand
+1 I lost my console so I no longer have the time that smoketester took but it passed. Changes look good to me too. On Thu, Jan 26, 2023 at 10:23 AM Ignacio Vera wrote: > > +1 > > SUCCESS! [0:44:15.998020] > > > On Thu, Jan 26, 2023 at 9:19 AM Jan Høydahl wrote: >> >> +1 >> >> SUCCESS!

Re: Lucene 9.5.0 release

2023-01-23 Thread Adrien Grand
We did a major cleanup to the vector API in 9.5 but there are a few things that still annoy me a bit that are worth fixing in my opinion: - VectorValues, the API for float vectors, still exposes a binaryValue() API. We should remove it and only expose floats in the API? - Byte vectors should be

Re: Lucene 9.5.0 release

2023-01-13 Thread Adrien Grand
+1 to doing a 9.5 release, it's overdue Le ven. 13 janv. 2023, 10:54, Luca Cavanna a écrit : > Hi all, > I'd like to propose that we release Lucene 9.5.0. There is a decent amount > of changes that would go into it looking at the github milestone: > https://github.com/apache/lucene/milestone/4

Re: Request for naming help

2023-01-01 Thread Adrien Grand
>>>> >>>> On Tue, Dec 13, 2022 at 5:04 PM Greg Miller wrote: >>>>> >>>>> Thanks for the suggestion Adrien. I like this idea! Marc- what do you >>>>> think? >>>>> >>>>> We might need to rework the

Re: Request for naming help

2022-12-13 Thread Adrien Grand
I wonder if the facets actually require a different name, since they look to me like a generalization of range facets for range fields, while we previously only supported range facets on numeric fields. We could keep calling them range facets? Maybe we could use the same model we used for queries

Re: [lucene] branch main updated: More refactoring work, and fix a distance calculation.

2022-11-24 Thread Adrien Grand
Karl, this commit has been failing precommit because it introduced dead code. I just pushed a fix. On Thu, Nov 24, 2022 at 10:47 AM wrote: > > This is an automated email from the ASF dual-hosted git repository. > > kwright pushed a commit to branch main > in repository

[ANNOUNCE] Apache Lucene 9.4.2 released

2022-11-23 Thread Adrien Grand
The Lucene PMC is pleased to announce the release of Apache Lucene 9.4.2 Apache Lucene is a high-performance, full-featured search engine library written entirely in Java. It is a technology suitable for nearly any application that requires structured search, full-text search, faceting,

Re: Main website not building

2022-11-22 Thread Adrien Grand
master branch). So the production branch was > building but was deployed to Staging. > > I tried to exclude asf.yaml from any merging, but there is no way to put a > "sticky" bit on it. If anybody has an idea how to tell git: never touch > asf.aml while merging, tell me! > >

Main website not building

2022-11-22 Thread Adrien Grand
Hello, I've managed to make changes to the website for 9.4.2 and they are correctly reflected on lucene.staged.apache.org. However pushing to the `production` branch doesn't seem to trigger a build on https://ci2.apache.org/#/builders/3 and I'm not seeing the production website getting updated

[RESULT] [VOTE] Release Lucene 9.4.2 RC1

2022-11-21 Thread Adrien Grand
22 at 8:43 AM Uwe Schindler wrote: >> >> > >> >> > The problem is: it is working like this since years - the 9.4.1 >> release worked fine. No change! >> >> > >> >> > And I can't configure this because GPG uses its own

Re: [VOTE] Release Lucene 9.4.2 RC1

2022-11-18 Thread Adrien Grand
t; u...@odoko.co.uk>" imported [...] gpg: key 051A0FAF76BC6507: public key > "Adrien Grand (CODE SIGNING KEY) " imported [...] > gpg: key 32423B0E264B5CBA: public key "Julie Tibshirani (New code signing > key) " imported gpg: Total number processed: 6

Re: [VOTE] Release Lucene 9.4.2 RC1

2022-11-18 Thread Adrien Grand
ustdb.gpg: > trustdb created gpg: key B83EA82A0AFCEE7C: public key "Yonik Seeley > " imported gpg: can't connect to the > agent: IPC connect call failed gpg: key E48025ED13E57FFC: public key > "Upayavira " imported [...] gpg: key > 051A0FAF76BC6507: public key

[VOTE] Release Lucene 9.4.2 RC1

2022-11-17 Thread Adrien Grand
Please vote for release candidate 1 for Lucene 9.4.2 The artifacts can be downloaded from: https://dist.apache.org/repos/dist/dev/lucene/lucene-9.4.2-RC1-rev-858d9b437047a577fa9457089afff43eefa461db You can run the smoke tester directly with this command: python3 -u

Re: Maven artifacts and releases

2022-11-17 Thread Adrien Grand
d until the voting is closed. This would > make the build side of things much easier and wound't require any extra > infrastructure/ steps. > > Dawid > > On Thu, Nov 17, 2022 at 2:08 PM Adrien Grand wrote: > >> I remember using a 8.x build to do that for the 9.0 release. >&g

Re: Maven artifacts and releases

2022-11-17 Thread Adrien Grand
I remember using a 8.x build to do that for the 9.0 release. On Thu, Nov 17, 2022 at 1:54 PM Dawid Weiss wrote: > > I looked at an old issue related to maven artifact publishing and I > noticed the release wizard still gives instructions to use the ant-based > script (which isn't there

Re: Release Lucene 9.4.2

2022-11-17 Thread Adrien Grand
. I feel we at least made the effort to > root out any more of these and hopefully prevent a 9.4.3 with another > overflow bug. > > On Wed, Nov 16, 2022 at 10:55 AM Adrien Grand wrote: > > > > It looks like we're good with the changes we wanted to get in for 9.4.2.

Re: Release Lucene 9.4.2

2022-11-16 Thread Adrien Grand
ys to improve the test situation? I > think > > >> >> we need to beef up checkindex to exercise seek() on the vectors, > also > > >> >> we need to look at static analysis to try to find other similar > bugs. > > >> >> This would help

Re: Backporting of Nori

2022-11-10 Thread Adrien Grand
Hello Roy and Shad, What you are asking is not straightforward, I worry it would take me a lot of time and I'm not even sure I would succeed, and I would assume that other committers who read your email felt the same way. My preferred path forward would be to delay support of Korean in Lucene.NET

  1   2   3   4   5   6   7   8   9   10   >