Re: Welcome Armin Braun as Lucene comitter

[Vigya Sharma]

Congratulations and welcome, Armin! Volunteering as a firefighter is
amazing, respect!

On Fri, Jul 26, 2024 at 1:46 AM Ignacio Vera  wrote:

> Welcome Armin!
>
> On Fri, Jul 26, 2024 at 10:16 AM Chris Hegarty
>  wrote:
> >
> > Welcome Armin!
> >
> > -Chris.
> >
> > > On 26 Jul 2024, at 05:24, Anshum Gupta  wrote:
> > >
> > > Congratulations and welcome, Armin!
> > >
> > > On Thu, Jul 25, 2024 at 2:10 AM Luca Cavanna 
> wrote:
> > > I'm pleased to announce that Armin Braun has accepted the PMC's
> invitation to become a Lucene committer.
> > >
> > > Armin, the tradition is that new committers introduce themselves with
> a brief bio.
> > >
> > > Thanks for your contributions so far and looking forward to the
> upcoming ones :)
> > >
> > > Congratulations and welcome!
> > >
> > >
> > > --
> > > Anshum Gupta
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-- 
- Vigya

Re: (lucene) branch main updated: Use jdk11 primitives in test to allow backport to branch_9x (#13311)

[Vigya Sharma]

Yes, I'll prefer applying the change only to 9.x in future. I was worried
we'd get recurring conflicts on this file in the backport branch, but it's
not a very high touch test function, so now, I don't think it's a big
concern.

On Tue, Apr 16, 2024 at 11:57 PM Uwe Schindler  wrote:

> Hi,
>
> why did you not first backport and apply this change only to 9.x?
>
> If we have better methods available in Java 21, why not use them? We
> also change large parts of code to "record" classes, also not available
> in Java 11.
>
> Uwe
>
> Am 17.04.2024 um 08:17 schrieb vigyasha...@apache.org:
> > This is an automated email from the ASF dual-hosted git repository.
> >
> > vigyasharma pushed a commit to branch main
> > in repository https://gitbox.apache.org/repos/asf/lucene.git
> >
> >
> > The following commit(s) were added to refs/heads/main by this push:
> >   new bc678ac67e3 Use jdk11 primitives in test to allow backport to
> branch_9x (#13311)
> > bc678ac67e3 is described below
> >
> > commit bc678ac67e32c55a27a4e8950c25144cc89cef66
> > Author: Vigya Sharma 
> > AuthorDate: Tue Apr 16 23:17:43 2024 -0700
> >
> >  Use jdk11 primitives in test to allow backport to branch_9x (#13311)
> > ---
> >   .../src/test/org/apache/lucene/search/BaseKnnVectorQueryTestCase.java
> | 4 ++--
> >   1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git
> a/lucene/core/src/test/org/apache/lucene/search/BaseKnnVectorQueryTestCase.java
> b/lucene/core/src/test/org/apache/lucene/search/BaseKnnVectorQueryTestCase.java
> > index 2ae0ae14a29..2546115ff4f 100644
> > ---
> a/lucene/core/src/test/org/apache/lucene/search/BaseKnnVectorQueryTestCase.java
> > +++
> b/lucene/core/src/test/org/apache/lucene/search/BaseKnnVectorQueryTestCase.java
> > @@ -781,7 +781,7 @@ abstract class BaseKnnVectorQueryTestCase extends
> LuceneTestCase {
> > TimeLimitingKnnCollectorManager noTimeoutManager =
> > new TimeLimitingKnnCollectorManager(delegate, null);
> > KnnCollector noTimeoutCollector =
> > -  noTimeoutManager.newCollector(Integer.MAX_VALUE,
> searcher.leafContexts.getFirst());
> > +  noTimeoutManager.newCollector(Integer.MAX_VALUE,
> searcher.leafContexts.get(0));
> >
> > // Check that a normal collector is created without timeout
> > assertTrue(noTimeoutCollector instanceof TopKnnCollector);
> > @@ -797,7 +797,7 @@ abstract class BaseKnnVectorQueryTestCase extends
> LuceneTestCase {
> > TimeLimitingKnnCollectorManager timeoutManager =
> > new TimeLimitingKnnCollectorManager(delegate, () -> true);
> > KnnCollector timeoutCollector =
> > -  timeoutManager.newCollector(Integer.MAX_VALUE,
> searcher.leafContexts.getFirst());
> > +  timeoutManager.newCollector(Integer.MAX_VALUE,
> searcher.leafContexts.get(0));
> >
> > // Check that a time limiting collector is created, which
> returns partial results
> > assertFalse(timeoutCollector instanceof TopKnnCollector);
> >
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: u...@thetaphi.de
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-- 
- Vigya

Re: Welcome Zhang Chao as Lucene committer

[Vigya Sharma]

Congratulations Zhang!

On Tue, Feb 20, 2024 at 9:51 AM Chris Hegarty
 wrote:

> Congratulations and welcome!!
>
> -Chris.
>
> > On 20 Feb 2024, at 17:28, Adrien Grand  wrote:
> >
> > I'm pleased to announce that Zhang Chao has accepted the PMC's
> > invitation to become a committer.
> >
> > Chao, the tradition is that new committers introduce themselves with a
> > brief bio.
> >
> > Congratulations and welcome!
> >
> > --
> > Adrien
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-- 
- Vigya

Re: Welcome Patrick Zhai to the Lucene PMC

[Vigya Sharma]

Congratulations Patrick!

On Mon, Nov 13, 2023 at 5:42 AM Guo Feng  wrote:

> Congratulations and welcome, Patrick!
>
> Feng
>
> On 2023/11/10 20:04:32 Michael McCandless wrote:
> > I'm happy to announce that Patrick Zhai has accepted an invitation to
> join
> > the Lucene Project Management Committee (PMC)!
> >
> > Congratulations Patrick, thank you for all your hard work improving
> > Lucene's community and source code, and welcome aboard!
> >
> > Mike McCandless
> >
> > http://blog.mikemccandless.com
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-- 
- Vigya

Re: Welcome Guo Feng to the Lucene PMC

[Vigya Sharma]

Congratulations Feng!

On Fri, Oct 27, 2023 at 10:51 AM Tomás Fernández Löbbe <
tomasflo...@gmail.com> wrote:

> Congratulations Feng!
>
> On Fri, Oct 27, 2023 at 4:02 AM Guo Feng  wrote:
>
>> Thanks all. It is a great honor to join the PMC!
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>

-- 
- Vigya

Re: Welcome Ben Trent as Lucene committer

[Vigya Sharma]

Congratulations, Ben! Welcome.

On Fri, Jan 27, 2023 at 11:06 AM Julie Tibshirani 
wrote:

> Congratulations and welcome Ben!!
>
> And hello from another aspiring mathematician turned software engineer :)
>
> On Fri, Jan 27, 2023 at 9:40 AM Benjamin Trent 
> wrote:
>
>> Hey y'all!
>>
>> This is truly an honor!
>>
>> Well, I am Ben Trent and have been writing code for over a decade now.
>> Which I know is not a very long time compared to most folks. I originally
>> wanted to do research and work in pure mathematics (my baccalaureate), but
>> quickly realized I am nowhere near smart enough to make money at that. So,
>> like many folks, I switched to computing and haven't looked back.
>>
>> In my spare time (when not wrangling one of my children), I enjoy movies
>> (especially old kung fu, anything Golden Harvest or Shaw Brothers), good
>> beer, reading, playing guitar, and hiking.
>>
>> Thank you all for the warm welcome! See you online!
>>
>> Ben
>>
>> On Fri, Jan 27, 2023 at 10:26 AM Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com> wrote:
>>
>>> Welcome and congratulations, Ben!
>>>
>>> On Fri, Jan 27, 2023 at 8:48 PM Adrien Grand  wrote:
>>> >
>>> > I'm pleased to announce that Ben Trent has accepted the PMC's
>>> > invitation to become a committer.
>>> >
>>> > Ben, the tradition is that new committers introduce themselves with a
>>> > brief bio.
>>> >
>>> > Congratulations and welcome!
>>> >
>>> > --
>>> > Adrien
>>> >
>>> > -
>>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> > For additional commands, e-mail: dev-h...@lucene.apache.org
>>> >
>>>
>>

-- 
- Vigya

Re: [JENKINS] Lucene » Lucene-Check-9.x - Build # 4239 - Unstable!

[Vigya Sharma]

I see this is failing in main as well. Started some debugging in
https://github.com/apache/lucene/issues/12097.

On Thu, Jan 19, 2023 at 2:02 AM Apache Jenkins Server <
jenk...@builds.apache.org> wrote:

> Build: https://ci-builds.apache.org/job/Lucene/job/Lucene-Check-9.x/4239/
>
> 1 tests failed.
> FAILED:
> org.apache.lucene.search.TestIndexSortSortedNumericDocValuesRangeQuery.testCountBoundary
>
> Error Message:
> java.lang.AssertionError: expected:<2> but was:<0>
>
> Stack Trace:
> java.lang.AssertionError: expected:<2> but was:<0>
> at
> __randomizedtesting.SeedInfo.seed([A11A06AE642497F1:49EE15CFDFF5FE89]:0)
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.failNotEquals(Assert.java:835)
> at org.junit.Assert.assertEquals(Assert.java:647)
> at org.junit.Assert.assertEquals(Assert.java:633)
> at
> org.apache.lucene.search.TestIndexSortSortedNumericDocValuesRangeQuery.testCountBoundary(TestIndexSortSortedNumericDocValuesRangeQuery.java:600)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
> at
> org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
> at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
> at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> 

Re: Is there a way to customize segment names?

[Vigya Sharma]

Hi Patrick,

This is an interesting question, and from what I understood, I see
correctness problems in what you're trying to implement. Let me make sure I
understand correctly...

So indexer-1 created segments 1,2,3,4 and indexer-2 created segments 1',
2', 3', 4' independently (they just have the same segment names, i.e.
name(1).equals(name(1')). Now you have a reader opened on (1, 2, 3) from
indexer-1, and you're trying to call openIfChanged() to only load segment
4' into it - after downloading 1', 2', 3' and 4' into your indexDir.

But indexer-1 and indexer-2 would have flushed at different intervals, and
would likely have different documents (and doc counts) in each segment. So
even if Lucene allowed it, you can miss some data if your reader is opened
on (1,2,3,4'). Worse, you might double count data. In general, it opens all
sorts of corruption bugs, which is indeed what the exception you link says.

If you're sure that you only want the incremental data from segment 4',
have you considered adding it via the addIndexes(Directory...) API, which
simply copies the segment into the dir. openIfChanged() would then work
like you (seem to) want.

It seems like a bad idea to silently replace segments from under open
readers but try to trick them into thinking they are still present.
Thankfully, Lucene provides important protections against it. Apologies if
I misunderstood your use case.


On Fri, Dec 30, 2022 at 10:10 AM Patrick Zhai  wrote:

> Hi Robert, got it, thanks!
>
> Hi Uwe, yes we do have a way to detect whether the segment is created by
> node A or B even if they share the same name, however, lucene does not
> allow such situation (same name but generated by different writer) when
> calling `openIfChanged` to try to incrementally load the new index. So what
> I want is to attach a prefix (or postfix, anything lol) to the segment
> name, say "A_4" and "B_4" so that when DirectoryReader is doing
> `openIfChanged` it will proceed without throwing any exception.
>
> On Sat, Dec 17, 2022 at 4:56 AM Robert Muir  wrote:
>
>> No, you can't control them. And we must not open up anything to try to
>> support this.
>>
>> On Fri, Dec 16, 2022 at 7:28 PM Patrick Zhai  wrote:
>> >
>> > Hi Mike, Robert
>> >
>> > Thanks for replying, the system is almost like what Mike has described:
>> one writer is primary,
>> > and the other is trying to catch up and wait, but in our internal
>> discussion we found there might
>> > be small chances where the secondary mistakenly think itself as primary
>> (due to errors of other component)
>> > while primary is still alive and thus goes into the situation I
>> described.
>> > And because we want to tolerate the error in case we can't prevent it
>> from happening, we're looking for customizing
>> > filenames.
>> >
>> > Thanks again for discussing this with me and I've learnt that playing
>> with filenames can become quite
>> > troublesome, but still, even out of my own curiosity, I want to
>> understand whether we're able to control
>> > the segment names in some way?
>> >
>> > Best
>> > Patrick
>> >
>> >
>> > On Fri, Dec 16, 2022 at 6:36 AM Michael Sokolov 
>> wrote:
>> >>
>> >> +1 trying to coordinate multiple writers running independently will
>> >> not work. My 2c for availability: you can have a single primary active
>> >> writer with a backup one waiting, receiving all the segments from the
>> >> primary. Then if the primary goes down, the secondary one has the most
>> >> recent commit replicated from the primary (identical commit, same
>> >> segments etc) and can pick up from there. You would need a mechanism
>> >> to replay the writes the primary never had a chance to commit.
>> >>
>> >> On Fri, Dec 16, 2022 at 5:41 AM Robert Muir  wrote:
>> >> >
>> >> > You are still talking "Multiple writers". Like i said, going down
>> this
>> >> > path (playing tricks with filenames) isn't going to work out well.
>> >> >
>> >> > On Fri, Dec 16, 2022 at 2:48 AM Patrick Zhai 
>> wrote:
>> >> > >
>> >> > > Hi Robert,
>> >> > >
>> >> > > Maybe I didn't explain it clearly but we're not going to
>> constantly switch
>> >> > > between writers or share effort between writers, it's purely for
>> >> > > availability: the second writer only kicks in when the first
>> writer is not
>> >> > > available for some reason.
>> >> > > And as far as I know the replicator/nrt module has not provided a
>> solution
>> >> > > on when the primary node (main indexer) is down, how would we
>> recover with
>> >> > > a back up indexer?
>> >> > >
>> >> > > Thanks
>> >> > > Patrick
>> >> > >
>> >> > >
>> >> > > On Thu, Dec 15, 2022 at 7:16 PM Robert Muir 
>> wrote:
>> >> > >
>> >> > > > This multiple-writer isn't going to work and customizing names
>> won't
>> >> > > > allow it anyway. Each file also contains a unique identifier
>> tied to
>> >> > > > its commit so that we know everything is intact.
>> >> > > >
>> >> > > > I would look at the segment replication in lucene/replicator and
>> not
>> >> > > > try to 

Re: [JENKINS] Lucene-9.x-Linux (64bit/jdk-11.0.15) - Build # 6455 - Unstable!

[Vigya Sharma]

I tried to repro this on my box, but it seems to pass on repeated runs.
Also, the jenkins build failure page
 returns a 404 not
found for me. Did we clear up all the old failures recently?
Looks similar to https://github.com/apache/lucene/issues/10930.

On Thu, Oct 27, 2022 at 1:14 AM Policeman Jenkins Server <
jenk...@thetaphi.de> wrote:

> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/6455/
> Java: 64bit/jdk-11.0.15 -XX:+UseCompressedOops -XX:+UseG1GC
>
> 1 tests failed.
> FAILED:
> org.apache.lucene.index.TestIndexWriterWithThreads.testIOExceptionDuringWriteSegmentWithThreads
>
> Error Message:
> java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are
> still 30 open files: {_0.pos=1, _1.tvd=1, _1.nvd=1, _0.nvd=1, _2.fdt=1,
> _2.tvx=1, _0.tip=1, _0.tim=1, _2.fdx=1, _2.doc=1, _2.nvd=1, _2.tip=1,
> _2.tvd=1, _0_Lucene90_0.dvd=1, _1.doc=1, _1.fdx=1, _1.fdt=1, _1.tvx=1,
> _1_Lucene90_0.dvd=1, _0.fdx=1, _2.pos=1, _0.doc=1, _0.fdt=1, _0.tvx=1,
> _2.tim=1, _0.tvd=1, _2_Lucene90_0.dvd=1, _1.tim=1, _1.tip=1, _1.pos=1}
>
> Stack Trace:
> java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are
> still 30 open files: {_0.pos=1, _1.tvd=1, _1.nvd=1, _0.nvd=1, _2.fdt=1,
> _2.tvx=1, _0.tip=1, _0.tim=1, _2.fdx=1, _2.doc=1, _2.nvd=1, _2.tip=1,
> _2.tvd=1, _0_Lucene90_0.dvd=1, _1.doc=1, _1.fdx=1, _1.fdt=1, _1.tvx=1,
> _1_Lucene90_0.dvd=1, _0.fdx=1, _2.pos=1, _0.doc=1, _0.fdt=1, _0.tvx=1,
> _2.tim=1, _0.tvd=1, _2_Lucene90_0.dvd=1, _1.tim=1, _1.tip=1, _1.pos=1}
> at
> __randomizedtesting.SeedInfo.seed([99721ECDA7884007:42DAA434CD8BF38C]:0)
> at
> org.apache.lucene.tests.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:876)
> at
> org.apache.lucene.index.TestIndexWriterWithThreads._testMultipleThreadsFailure(TestIndexWriterWithThreads.java:336)
> at
> org.apache.lucene.index.TestIndexWriterWithThreads.testIOExceptionDuringWriteSegmentWithThreads(TestIndexWriterWithThreads.java:484)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
> at
> org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
> at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
> at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)

Re: Welcome Luca Cavanna as Lucene committer

[Vigya Sharma]

Congratulations Luca! And welcome...

Vigya

On Wed, Oct 5, 2022 at 3:36 PM Uwe Schindler  wrote:

> Welcome Luca. This was long overdue. 
>
> Am 5. Oktober 2022 19:03:43 MESZ schrieb Adrien Grand :
>>
>> I'm pleased to announce that Luca Cavanna has accepted the PMC's
>> invitation to become a committer.
>>
>> Luca, the tradition is that new committers introduce themselves with a
>> brief bio.
>>
>> Congratulations and welcome!
>>
>> --
>> Adrien
>>
> --
> Uwe Schindler
> Achterdiek 19, 28357 Bremen
> https://www.thetaphi.de
>


-- 
- Vigya

Re: [VOTE] Release Lucene 9.4.0 RC1

[Vigya Sharma]

The smoke tests passed for me too..


(no vote)

SUCCESS! [1:12:31.588303]

On Thu, Sep 22, 2022 at 2:27 AM Ignacio Vera  wrote:

> +1
>
>
> SUCCESS! [0:46:00.508949]
>
> On Thu, Sep 22, 2022 at 9:53 AM Adrien Grand  wrote:
>
>> +1 SUCCESS! [0:45:35.275017]
>>
>> On Wed, Sep 21, 2022 at 9:19 PM Michael McCandless <
>> luc...@mikemccandless.com> wrote:
>>
>>> +1
>>>
>>>
>>> SUCCESS! [0:27:16.514391]
>>>
>>>
>>> Mike McCandless
>>>
>>> http://blog.mikemccandless.com
>>>
>>>
>>> On Wed, Sep 21, 2022 at 2:32 PM Dawid Weiss 
>>> wrote:
>>>

 +1.
 SUCCESS! [0:53:33.891603]


> Ran the smoketester with both java 11 and 17:
>
> SUCCESS! [2:41:19.024193]
>
> On Tue, Sep 20, 2022 at 10:10 PM Michael Sokolov 
> wrote:
> >
> > Please vote for release candidate 1 for Lucene 9.4.0
> >
> > The artifacts can be downloaded from:
> >
> https://dist.apache.org/repos/dist/dev/lucene/lucene-9.4.0-RC1-rev-f5d0646daa5651f2192282ac85551bca667e34f9
> >
> > You can run the smoke tester directly with this command:
> >
> > python3 -u dev-tools/scripts/smokeTestRelease.py \
> >
> https://dist.apache.org/repos/dist/dev/lucene/lucene-9.4.0-RC1-rev-f5d0646daa5651f2192282ac85551bca667e34f9
> >
> > The vote will be open for at least 72 hours i.e. until 2022-09-24
> 02:00 UTC.
> >
> > [ ] +1  approve
> > [ ] +0  no opinion
> > [ ] -1  disapprove (and reason why)
> >
> > Here is my +1
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>
>>
>> --
>> Adrien
>>
>

-- 
- Vigya

Re: [ANNOUNCE] Issue migration Jira to GitHub starts on Monday, August 22

[Vigya Sharma]

Love this! Thanks for all the hard work, Tomoko.
-
Vigya


On Wed, Aug 24, 2022 at 12:27 PM Michael Sokolov  wrote:

> Thanks! It seems to be working nicely.
>
> Question about the fix-version: tagging. I wonder if going forward we
> want to main that for new issues? I happened to notice there is also
> this "milestone" feature in github -- does that seem like a place to
> put version information?
>
> On Wed, Aug 24, 2022 at 3:20 PM Tomoko Uchida
>  wrote:
> >
> > 
> >
> > Issue migration has been completed (except for minor cleanups).
> > This is the Jira -> GitHub issue number mapping for possible future
> usage.
> https://github.com/apache/lucene-jira-archive/blob/main/migration/mappings-data/issue-map.csv.20220823_final
> >
> > GitHub issue is now fully available for all issues.
> > For issue label management (e.g. "fix-version"), please review this
> manual.
> >
> https://github.com/apache/lucene/blob/main/dev-docs/github-issues-howto.md
> >
> > Tomoko
> >
> >
> > 2022年8月22日(月) 19:46 Michael McCandless :
> >>
> >> Wooot!  Thank you so much Tomoko!!
> >>
> >> Mike
> >>
> >> On Mon, Aug 22, 2022 at 6:44 AM Tomoko Uchida <
> tomoko.uchida.1...@gmail.com> wrote:
> >>>
> >>> 
> >>>
> >>> Issue migration has been started. Jira is now read-only.
> >>>
> >>> GitHub issue is available for new issues.
> >>>
> >>> - You should open new issues on GitHub. E.g.
> https://github.com/apache/lucene/issues/1078
> >>> - Do not touch issues that are in the middle of migration, please.
> E.g. https://github.com/apache/lucene/issues/1072
> >>>   - While you cannot break these issues, migration scripts can
> modify/overwrite your comments on the issues.
> >>> - Pull requests are not affected. You can open/update PRs as usual.
> Please let me know if you have any trouble with PRs.
> >>>
> >>>
> >>> Tomoko
> >>>
> >>>
> >>> 2022年8月18日(木) 18:23 Tomoko Uchida :
> 
>  Hello all,
> 
>  The Lucene project decided to move our issue tracking system from
> Jira to GitHub and migrate all Jira issues to GitHub.
> 
>  We start issue migration on Monday, August 22 at 8:00 UTC.
>  1) We make Jira read-only before migration. You cannot update
> existing issues until the migration is completed.
>  2) You can use GitHub for opening NEW issues or pull requests during
> migration.
> 
>  Note that issues should be raised in Jira at this moment, although
> GitHub issue is already enabled in the Lucene repository.
>  Please do not raise issues in GitHub until we let you know that
> GitHub issue is officially available. We immediately close any issues on
> GitHub until then.
> 
>  Here are the detailed plan/migration steps.
>  https://github.com/apache/lucene-jira-archive/issues/7
> 
>  Tomoko
> >>
> >> --
> >> Mike McCandless
> >>
> >> http://blog.mikemccandless.com
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-- 
- Vigya

Re: Welcome Vigya Sharma as Lucene committer

[Vigya Sharma]

Thanks everyone for the warm welcome. It is an honor to be invited as a
Lucene committer, and I look forward to contributing more to the community.

A little bit about me - I currently work for the Product Search team at
Amazon, and am based out of the San Francisco Bay Area in California, US.
I am interested in a wide variety of computer science areas, and, in the
last few years, have focused more on distributed systems, concurrency,
system software and performance. Outside of tech., I like spending my time
outdoors - running, skiing, and long road trips. I completed my first
marathon (the SFMarathon) last week, and now, getting this invitation has
made this month a highlight of the year.

I had known that Lucene powers some of the most popular search and
analytics use cases across the globe, but as I've gotten more involved, the
depth and breadth of this software has blown my mind. I am deeply impressed
by what this community has built, and how it continues to work together and
grow. It is a great honor to be trusted with committer privileges, and I
look forward to learning and contributing to multiple different parts of
the library.

Thank you,
Vigya


On Thu, Jul 28, 2022 at 12:20 PM Anshum Gupta 
wrote:

> Congratulations and welcome, Vigya!
>
> On Thu, Jul 28, 2022 at 12:34 AM Adrien Grand  wrote:
>
>> I'm pleased to announce that Vigya Sharma has accepted the PMC's
>> invitation to become a committer.
>>
>> Vigya, the tradition is that new committers introduce themselves with a
>> brief bio.
>>
>> Congratulations and welcome!
>>
>> --
>> Adrien
>>
>
>
> --
> Anshum Gupta
>


-- 
- Vigya

Re: [JENKINS] Lucene-main-Linux (64bit/jdk-17.0.3) - Build # 35657 - Unstable!

[Vigya Sharma]

This should also get addressed by https://github.com/apache/lucene/pull/1012.
The PR fixes testWithExceptions failures for subclasses
of BasePointsFormatTestCase.

Vigya

On Sun, Jul 10, 2022 at 8:31 PM Policeman Jenkins Server <
jenk...@thetaphi.de> wrote:

> Build: https://jenkins.thetaphi.de/job/Lucene-main-Linux/35657/
> Java: 64bit/jdk-17.0.3 -XX:+UseCompressedOops -XX:+UseG1GC
>
> 1 tests failed.
> FAILED:
> org.apache.lucene.codecs.simpletext.TestSimpleTextPointsFormat.testWithExceptions
>
> Error Message:
> java.nio.file.NoSuchFileException:
> /home/jenkins/workspace/Lucene-main-Linux/lucene/codecs/build/tmp/tests-tmp/lucene.codecs.simpletext.TestSimpleTextPointsFormat_1F2E321F434B241B-001/tempDir-001/_0.dii
>
> Stack Trace:
> java.nio.file.NoSuchFileException:
> /home/jenkins/workspace/Lucene-main-Linux/lucene/codecs/build/tmp/tests-tmp/lucene.codecs.simpletext.TestSimpleTextPointsFormat_1F2E321F434B241B-001/tempDir-001/_0.dii
> at
> __randomizedtesting.SeedInfo.seed([1F2E321F434B241B:BD7DAE63CCC56BE1]:0)
> at
> java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
> at
> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
> at
> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
> at
> java.base/sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:248)
> at
> java.base/sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:105)
> at
> org.apache.lucene.tests.mockfile.FilterFileSystemProvider.delete(FilterFileSystemProvider.java:134)
> at
> org.apache.lucene.tests.mockfile.FilterFileSystemProvider.delete(FilterFileSystemProvider.java:134)
> at
> org.apache.lucene.tests.mockfile.FilterFileSystemProvider.delete(FilterFileSystemProvider.java:134)
> at
> org.apache.lucene.tests.mockfile.FilterFileSystemProvider.delete(FilterFileSystemProvider.java:134)
> at
> org.apache.lucene.tests.mockfile.FilterFileSystemProvider.delete(FilterFileSystemProvider.java:134)
> at java.base/java.nio.file.Files.delete(Files.java:1152)
> at
> org.apache.lucene.store.FSDirectory.privateDeleteFile(FSDirectory.java:344)
> at
> org.apache.lucene.store.FSDirectory.deleteFile(FSDirectory.java:309)
> at
> org.apache.lucene.tests.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:651)
> at
> org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:37)
> at
> org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:763)
> at
> org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:757)
> at
> org.apache.lucene.index.IndexFileDeleter.deleteNewFiles(IndexFileDeleter.java:729)
> at
> org.apache.lucene.index.IndexWriter.deleteNewFiles(IndexWriter.java:5763)
> at
> org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3177)
> at
> org.apache.lucene.tests.index.RandomIndexWriter.addIndexes(RandomIndexWriter.java:334)
> at
> org.apache.lucene.tests.index.BasePointsFormatTestCase.switchIndex(BasePointsFormatTestCase.java:1195)
> at
> org.apache.lucene.tests.index.BasePointsFormatTestCase.verify(BasePointsFormatTestCase.java:797)
> at
> org.apache.lucene.tests.index.BasePointsFormatTestCase.testWithExceptions(BasePointsFormatTestCase.java:265)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:568)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
> at
> org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44)
> at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at
> 

Re: [JENKINS] Lucene-main-Linux (64bit/jdk-17.0.3) - Build # 35576 - Unstable!

[Vigya Sharma]

Looks like the same failure as Build#4060

.

The test fails on a boolean flag assert that verifies that
ConcurrentMergeScheduler.handleMergeException was called. From the test
stack trace, I do see IndexWriter.handleMergeException() being called,
which should rethrow this IOException to hit the CMS handleMergeException
function.. Will try to beast this test and update if I find more.

Test stack trace:
https://jenkins.thetaphi.de/job/Lucene-main-Linux/35576/testReport/junit/org.apache.lucene/TestMergeSchedulerExternal/testSubclassConcurrentMergeScheduler/


--
Vigya


On Wed, Jul 6, 2022 at 4:34 PM Policeman Jenkins Server 
wrote:

> Build: https://jenkins.thetaphi.de/job/Lucene-main-Linux/35576/
> Java: 64bit/jdk-17.0.3 -XX:-UseCompressedOops -XX:+UseG1GC
>
> 1 tests failed.
> FAILED:
> org.apache.lucene.TestMergeSchedulerExternal.testSubclassConcurrentMergeScheduler
>
> Error Message:
> java.lang.AssertionError
>
> Stack Trace:
> java.lang.AssertionError
> at
> __randomizedtesting.SeedInfo.seed([DDD39440C0DA45D4:5A5229EDC4FA3FD0]:0)
> at org.junit.Assert.fail(Assert.java:87)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at org.junit.Assert.assertTrue(Assert.java:53)
> at
> org.apache.lucene.TestMergeSchedulerExternal.testSubclassConcurrentMergeScheduler(TestMergeSchedulerExternal.java:149)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:568)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
> at
> org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44)
> at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
> at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> 

Re: [JENKINS] Lucene-9.x-Linux (64bit/jdk-18) - Build # 3654 - Unstable!

[Vigya Sharma]

Same as the issue being tracked in
https://issues.apache.org/jira/browse/LUCENE-10617. Did not repro for me
locally with the random seed in build failure.
For reference:

gradlew test --tests
TestIndexWriterWithThreads.testIOExceptionDuringAbortWithThreadsOnlyOnce
-Dtests.seed=7F0F71569300A11D -Dtests.multiplier=3 -Dtests.locale=ceb
-Dtests.timezone=Asia/Aqtobe -Dtests.asserts=true
-Dtests.file.encoding=UTF-8


Jenkins:
https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/3654/testReport/junit/org.apache.lucene.index/TestIndexWriterWithThreads/testIOExceptionDuringAbortWithThreadsOnlyOnce/


On Wed, Jun 15, 2022 at 2:43 PM Policeman Jenkins Server <
jenk...@thetaphi.de> wrote:

> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/3654/
> Java: 64bit/jdk-18 -XX:-UseCompressedOops -XX:+UseSerialGC
>
> 1 tests failed.
> FAILED:
> org.apache.lucene.index.TestIndexWriterWithThreads.testIOExceptionDuringAbortWithThreadsOnlyOnce
>
> Error Message:
> java.lang.AssertionError: hit unexpected Throwable
>
> Stack Trace:
> java.lang.AssertionError: hit unexpected Throwable
> at
> __randomizedtesting.SeedInfo.seed([7F0F71569300A11D:285956B736B1E443]:0)
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at
> org.apache.lucene.index.TestIndexWriterWithThreads._testMultipleThreadsFailure(TestIndexWriterWithThreads.java:301)
> at
> org.apache.lucene.index.TestIndexWriterWithThreads.testIOExceptionDuringAbortWithThreadsOnlyOnce(TestIndexWriterWithThreads.java:447)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
> at java.base/java.lang.reflect.Method.invoke(Method.java:577)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
> at
> org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44)
> at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
> at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at
> 

Re: [DISCUSS] A proposal for migration to GitHub issue (LUCENE-10557)

[Vigya Sharma]

Thanks for starting this thread, Tomoko! Being fairly new to the community,
I can recall some early teething issues and offer a newcomer's perspective.

Jira took a while to get familiar with. I had used GitHub issues before,
for Elasticsearch and Opensearch. Having used Jira now, makes me like
GitHub a whole lot more. I like the standard markdown support, and overall
experience. GitHub labels (and their UX placement), are great for
discovering new issues and narrowing down your search. (I suppose Jira can
do that too, but haven't gotten there yet).

I remember frequently finding myself getting signed out and forgetting my
Jira password in my initial days. Not having to create another account for
Jira, or sign into two different systems is a big plus.


Vigya


On Thu, May 5, 2022 at 2:51 PM Michael Sokolov  wrote:

> > Is the original Jira -> GitHub move just a change of defaults or are we,
> > once moved to GitHub, not letting people use Jira at all anymore ?
>
> Nothing has been decided - it's all open for debate.
>
> I just want to re-state the idea (at least as I heard it) behind this
> proposed move is to make contributing more accessible.
>
> As for github banning people, just google "github banning" and you
> will see that people have been banned for a variety of reasons.
>
> I don't know if the risk of that outweighs the inconvenience of JIRA
> for the new developer.
>
> Please note that the primary avenue for contributions today *is
> already github*. So far we have not had any issues with people being
> banned, and if that were to happen, we still accept patch files in
> JIRA. We have had these two mechanisms in place for quite a while now
> and both are in active use.
>
> But this proposal is about issue tracking anyway: I just point out
> that we already rely on github in spite of its shortcomings.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-- 
- Vigya

AddIndexes(CodecReader...) API Question

[Vigya Sharma]

Hi All,

I am working on a patch that would leverage the MergePolicy and
MergeScheduler to run addIndexes(CodecReader...) triggered merges
concurrently (Lucene-10216
, WIP-PR
). I had some general questions
about the APIs current implementation.

At the start of the API, we trigger a flush(triggerMerge: false,
applyAllDeletes: true)
.
I was wondering why we need this. My understanding is that the readers
brought in by addIndexes() API would be unrelated to any pending updates or
deletes.

I tried removing this call, and testExistingDeletes

(). failed. This leads me to understand that we flush and applyAllDeletes,
so that, if there was a pending delete by term, it does not impact incoming
readers that coincidentally contained docs with the same term.

Is this correct?
Also, since we may still get such a delete before the API completes, and
those deletes would get applied, this is likely a best effort scenario,
right?

On a related note, the regular merge for existing segments writes all
pending DV updates before merging, but we skip this in the addIndexes API.
Should we be doing this in both places?

Thanks,
Vigya

Re: Welcome Guo Feng as Lucene committer

[Vigya Sharma]

Congratulations Feng!

On Tue, Jan 25, 2022 at 1:12 PM Julie Tibshirani 
wrote:

> Welcome!!
>
> On Tue, Jan 25, 2022 at 11:00 AM Marcus Eagan 
> wrote:
>
>> Congratulations Feng!
>>
>> On Tue, Jan 25, 2022 at 10:51 AM Anshum Gupta 
>> wrote:
>>
>>> Congratulations and welcome, Feng!
>>>
>>> On Tue, Jan 25, 2022 at 1:09 AM Adrien Grand  wrote:
>>>
 I'm pleased to announce that Guo Feng has accepted the PMC's
 invitation to become a committer.

 Feng, the tradition is that new committers introduce themselves with a
 brief bio.

 Congratulations and welcome!

 --
 Adrien

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


>>>
>>> --
>>> Anshum Gupta
>>>
>> --
>> Marcus Eagan
>>
>>

-- 
warm regards,
Vigya

Re: VectorField renamed to KnnVectorField?

[Vigya Sharma]

Hi Michael,

Glad you got it working. There is also a KNN vector search demo that was
added not long ago. You might want to check it out. It has references for
example, to compute embeddings and build knn vector queries
,
among other things.

   - Search Files:
   
https://github.com/apache/lucene/blob/main/lucene/demo/src/java/org/apache/lucene/demo/SearchFiles.java

   - Index Files:
   
https://github.com/apache/lucene/blob/main/lucene/demo/src/java/org/apache/lucene/demo/IndexFiles.java

   - knn demo files:
   
https://github.com/apache/lucene/tree/main/lucene/demo/src/java/org/apache/lucene/demo/knn


- Vigya


On Mon, Nov 1, 2021 at 2:44 PM Michael Wechner 
wrote:

> I was able to update my code
>
> -FieldType vectorFieldType =
> VectorField.createHnswType(vector.length,
> VectorValues.SimilarityFunction.DOT_PRODUCT, 16, 500);
> -VectorField vectorField = new VectorField(VECTOR_FIELD, vector,
> vectorFieldType);
> +FieldType vectorFieldType =
> KnnVectorField.createFieldType(vector.length,
> VectorSimilarityFunction.DOT_PRODUCT);
> +KnnVectorField vectorField = new KnnVectorField(VECTOR_FIELD,
> vector, vectorFieldType);
>
> and
>
> -return new TopDocScorer(this,
> context.reader().searchNearestVectors(field, vector, topK, fanout));
> +return new TopDocScorer(this,
> context.reader().searchNearestVectors(field, vector, topK, null));
>
> the indexing and searching works again :-)
>
> Thanks
>
> Michael
>
> Am 01.11.21 um 18:53 schrieb Michael Wechner:
>
> Hi
>
> In May 2021 I have done a Vector Search implementation based on Lucene 
> 9.0.0-SNAPSHOT with the following code
>
> FieldType vectorFieldType = VectorField.createHnswType(vector.length, 
> VectorValues.SimilarityFunction.DOT_PRODUCT, 16, 500);
> VectorField vectorField = new VectorField(VECTOR_FIELD, vector, 
> vectorFieldType);
> doc.add(vectorField)
>
> and
> class KnnWeight extends Weight {
>
> KnnWeight() {
> super(KnnQuery.this);
> }
>
> @Overridepublic Scorer scorer(LeafReaderContext context) throws 
> IOException {
> log.debug("Get scorer.");
> return new TopDocScorer(this, 
> context.reader().searchNearestVectors(field, vector, topK, fanout));
> }
>
> whereas fanout is of type "int"
>
> I have now updated Lucene source and rebuilt 9.0.0-SNAPSHOT and get various 
> compile errors.
>
> I assume VectorField got renamed to KnnVectorField, right?
>
> Does somebody maybe have some sample code how Vector search is being 
> implemented with the most recent Lucene code?
>
> Thanks
>
> Michael
>
>
>

-- 
- Vigya

Question on HowToContribute Documentation

[Vigya Sharma]

Hi All,

I am new to Lucene and recently made a small contribution. While figuring
out the contributing process, I felt that the documentation may not be up
to date.

The repository home page 
links to this HowToContribute cwiki
, which
says that we should upload a patch file with our changes to the relevant
JIRA. However, I observed that contributions could also be made via Github
pull requests that one could raise from a Lucene fork.

Is patch-upload-to-jira, the preferred method of accepting contributions?
Or is there some other documentation that I missed?

The way I understand it, it is easier to review and comment on a pull
request. Perhaps if a reviewer wants to run/benchmark the code change,
downloading and applying a patch file may be easier.

I can help update the documentation if we need to.
We can also include some minor details - like prefixing the PR name with
all caps "LUCENE-#JIRA:", and making an entry in the changes file (these
were things that I had missed). Also, if we make these changes, would we
want to continue using the cwiki page, or just add a contributing.md to the
main repo?

-- 
- Vigya