Re: Adding a new PointDocValuesField

2022-05-25 Thread Robert Muir
On Wed, May 25, 2022 at 8:04 AM Michael Sokolov wrote: > > Also, there should be examples from other fields. Suppose you are > indexing map data and want to support a UI that shows "hot spots" on > the map where there is a lot of let's say ... activity of some sort. > You'd like to facet on 2-d

Re: Adding a new PointDocValuesField

2022-05-24 Thread Robert Muir
This seems really exotic feature to add a dedicated docvalues field for. We should let BINARY be the catchall for stuff like this. On Mon, May 23, 2022 at 10:17 PM Marc D'Mello wrote: > > Hi, > > Some background: I've been working on this PR to add hyper rectangle faceting > capabilities to

Re: [VOTE] Release Lucene 9.2.0 RC1

2022-05-18 Thread Robert Muir
I opened issue about this. It shouldn't block the release, but it is pretty crazy and something to improve. https://issues.apache.org/jira/browse/LUCENE-10579 On Wed, May 18, 2022 at 3:10 PM Robert Muir wrote: > > It seems strange the way that > confirmAllReleasesAreTestedForBackCompat

Re: [VOTE] Release Lucene 9.2.0 RC1

2022-05-18 Thread Robert Muir
ed because something randomly got printed to stdout at that time (its passing tests.verbose, could be anything)? Would it be better to just inspect the backward_index folder and look at the .zip filenames? On Wed, May 18, 2022 at 2:54 PM Robert Muir wrote: > > The smoketester failed for me. S

Re: [VOTE] Release Lucene 9.2.0 RC1

2022-05-18 Thread Robert Muir
The smoketester failed for me. Strange that it didn't fail for anyone else. My command line: $ export JAVA_HOME=/usr/lib/jvm/java-11-openjdk/ $ python3 -u dev-tools/scripts/smokeTestRelease.py --test-java17 /home/rmuir/Downloads/jdk

Re: How to see test case logs in Intellij

2022-05-18 Thread Robert Muir
Tests generally don't print or log anything. If you want them to print tons of stuff, pass -Ptests.verbose=true to gradle. You can do it in idea IDE by going to "Run" menu, "Edit Configurations", and adding -Ptests.verbose=true to the command line of the particular test (e.g. TestDemo) On Wed,

Re: Lucene 9.2 release

2022-05-17 Thread Robert Muir
or similar. I'm also ok with failing hard on the build if someone tries to use such old stuff. On Tue, May 17, 2022 at 11:39 PM Robert Muir wrote: > > But that's not the case at all. I suspect it is simply that Alan used > an older version of java 11, one that has a javadocs bug. > > On Tu

Re: Lucene 9.2 release

2022-05-17 Thread Robert Muir
th the javadocs bug that emits broken links in this situation. > > Yes - I think it'd be great if we can detect bugs in our code and the > language itself we depend on (Java 11) on nightly CI builds, not on the > release process (if possible!) :) > > Tomoko > > > 2022

Re: Lucene 9.2 release

2022-05-17 Thread Robert Muir
nes? > If there are gaps between the nightly build and the wizard, we could try to > fill the gap. > > Tomoko > > > 2022年5月17日(火) 21:54 Robert Muir : > >> On Tue, May 17, 2022 at 5:59 AM Alan Woodward >> wrote: >> > >> > It was pa

Re: Lucene 9.2 release

2022-05-17 Thread Robert Muir
On Tue, May 17, 2022 at 5:59 AM Alan Woodward wrote: > > It was part of the release process, which runs with Java 11. It should be > fixed now. > > > Newer java versions won't make a broken link, you just won't have a link at > > all. > > This seems a bit of a shame, as some of the problems

Re: Lucene 9.2 release

2022-05-16 Thread Robert Muir
On Mon, May 16, 2022 at 9:28 AM Dawid Weiss wrote: > - different jvms producing different javadoc? Could this be the cause? > This is it, it is always a bug when javadoc produces broken links like this. It happens "at release time" because otherwise nobody is messing with java 11. Newer java

Re: [DISCUSS] A proposal for migration to GitHub issue (LUCENE-10557)

2022-05-09 Thread Robert Muir
On Mon, May 9, 2022 at 7:59 PM Tomoko Uchida wrote: > > Also, I have a great deal to be concerned about "what active > committers/contributors on Lucene actually think of it". > I have a couple thoughts. 1. What percentage of contributions are done via pull requests versus via patch files

Re: [DISCUSS] A proposal for migration to GitHub issue (LUCENE-10557)

2022-05-05 Thread Robert Muir
On Thu, May 5, 2022 at 8:14 AM Tomoko Uchida wrote: > > Just for a supplement - I did not know that either, but I think it's not only > abouit Jira. > I've seen several times developers in China sometimes struggle also with > connecting to GitHub - may be due to the so-called Great Firewall. >

Re: [DISCUSS] A proposal for migration to GitHub issue (LUCENE-10557)

2022-05-05 Thread Robert Muir
On Thu, May 5, 2022 at 7:49 AM Michael Sokolov wrote: > 5. I find the Github commenting UI confusing -- what is the difference > between conversation and review comments? How can I reply to a > comment? What if I want to batch up a bunch of replies and send one > email with all of them? Maybe

Re: [DISCUSS] A proposal for migration to GitHub issue (LUCENE-10557)

2022-05-05 Thread Robert Muir
On Wed, May 4, 2022 at 10:24 PM Tomoko Uchida wrote: > > According to the INFRA issue for the RocketMQ project (Michael McCandless > gave the pointer in a comment on the issue; thanks!), a PMC agreement or Vote > result is needed for the decision. >

Re: Changing type of the tokens generated by pattern tokenizer

2022-05-03 Thread Robert Muir
As an alternative to writing a custom tokenizer, you can use built-in PatternTypingFilter which does exactly this (sets type based on whether it matches some regex). https://lucene.apache.org/core/9_1_0/analysis/common/org/apache/lucene/analysis/pattern/PatternTypingFilter.html On Tue, May 3,

Re: [JENKINS] Lucene » Lucene-Check-9.1 - Build # 454 - Unstable!

2022-04-28 Thread Robert Muir
at 4:09 AM Robert Muir wrote: > > to be clear, it makes sense to backport the fix anyway, as it prevents > another failure we don't want. > > but by itself, it won't fix this one. we'd need to do more (e.g. force > FSDirectory always) > > On Thu, Apr 28, 2022 at 4:

Re: [JENKINS] Lucene » Lucene-Check-9.1 - Build # 454 - Unstable!

2022-04-28 Thread Robert Muir
to be clear, it makes sense to backport the fix anyway, as it prevents another failure we don't want. but by itself, it won't fix this one. we'd need to do more (e.g. force FSDirectory always) On Thu, Apr 28, 2022 at 4:06 AM Robert Muir wrote: > > That change looks like it only d

Re: [JENKINS] Lucene » Lucene-Check-9.1 - Build # 454 - Unstable!

2022-04-28 Thread Robert Muir
, 2022 at 4:03 AM Dawid Weiss wrote: > > It's addressed on main but wasn't backported - > > ae29a3b326478a > LUCENE-10088: adec73dd284 > > I'll backport. > > D. > > On Thu, Apr 28, 2022 at 9:56 AM Robert Muir wrote: > > > > Sounds good to me. There may b

Re: [JENKINS] Lucene » Lucene-Check-9.1 - Build # 454 - Unstable!

2022-04-28 Thread Robert Muir
at the simpletext code here to confirm: is there really a heap issue, or is the heap filled by Directory contents and this is just red herring On Thu, Apr 28, 2022 at 3:56 AM Robert Muir wrote: > > Sounds good to me. There may be some followup to do in simpletext > codec too, so that it is less pr

Re: [JENKINS] Lucene » Lucene-Check-9.1 - Build # 454 - Unstable!

2022-04-28 Thread Robert Muir
Sounds good to me. There may be some followup to do in simpletext codec too, so that it is less problematic, but let's stabilize this test. On Thu, Apr 28, 2022 at 3:50 AM Dawid Weiss wrote: > > Failed with simple text codec - perhaps we should disallow it on this test? > > On Thu, Apr 28, 2022

Re: [JENKINS] Lucene » Lucene-NightlyTests-9.1 - Build # 42 - Unstable!

2022-04-27 Thread Robert Muir
On Wed, Apr 27, 2022 at 11:58 AM Michael McCandless wrote: > > Maybe we should make a dedicated exception class (instead of the generic > IllegalArgumentException) for this situation and catch it in this test? Or > change this test to index synthetic (randomly generated) text instead? But >

Re: [JENKINS] Lucene » Lucene-NightlyTests-9.1 - Build # 42 - Unstable!

2022-04-26 Thread Robert Muir
On Tue, Apr 26, 2022 at 8:40 AM Michael McCandless wrote: > > But then I don't think our testing of too-long terms, which happens to real > users, is great. We have a dedicated unit test case > (TestIndexWriter.testWickedLongTerm) which specifically confirms that the > inverted index will be

Re: [JENKINS] Lucene » Lucene-NightlyTests-9.1 - Build # 42 - Unstable!

2022-04-26 Thread Robert Muir
On Tue, Apr 26, 2022 at 8:40 AM Michael McCandless wrote: > > But then I don't think our testing of too-long terms, which happens to real > users, is great. We have a dedicated unit test case > (TestIndexWriter.testWickedLongTerm) which specifically confirms that the > inverted index will be

Re: [JENKINS] Lucene-main-MacOSX (64bit/jdk-18) - Build # 7969 - Still Failing!

2022-04-25 Thread Robert Muir
On Mon, Apr 25, 2022 at 7:48 PM Gautam Worah wrote: > > Not sure of what is causing these builds to fail. They have been continuously > failing since the last 3-4 tries? > The `./gradlew :lucene:documentation:checkBrokenLinks` command on the latest > `main` works fine on my Mac. > I don't see

Re: [JENKINS] Lucene » Lucene-NightlyTests-9.1 - Build # 42 - Unstable!

2022-04-23 Thread Robert Muir
we can write unit tests for what happens on too-big terms. unit tests that are simple and don't require downloading gigabyte files off the internet. i don't think we should intentionally allow our tests to be flaky, i strongly disagree. On Sat, Apr 23, 2022 at 6:12 AM Michael McCandless wrote:

Re: is gradle supposed to work with java 18?

2022-04-18 Thread Robert Muir
issues/19283 > Seems it has been fixed in the master branch, but not yet released as proper > build. > > > On Mon, Apr 18, 2022 at 4:49 PM Robert Muir wrote: >> >> I passed various flags such as --debug with no help. "Failed to create >> Jar file" is

Re: is gradle supposed to work with java 18?

2022-04-18 Thread Robert Muir
er.java:66) at org.gradle.internal.classpath.ClasspathBuilder.jar(ClasspathBuilder.java:53) ... 10 more So it looks like gradle has java 18 compatibility issues? Can we document a workaround or document "don't use this version, download 17 for now" ? On Mon, Apr 18, 202

is gradle supposed to work with java 18?

2022-04-18 Thread Robert Muir
On my operating system, java package has been upgraded to jdk 18, since it is the latest release. Do i need to download 17 and stash it off somewhere else for gradle to work? Is there some other workaround? Thanks for any pointers. think:lucene[main]$ git status On branch main nothing to commit,

Re: spotless targets

2022-04-07 Thread Robert Muir
On Thu, Apr 7, 2022 at 1:32 PM Dawid Weiss wrote: > > > This can be made in a number of ways, I guess. I just didn't have time/ care > for it much. Typically everything would run just fine - the JVM options that > open up modules for spotless is the first thing that caused serious issues... >

Re: [VOTE] Release Lucene 9.1.0 RC2

2022-03-18 Thread Robert Muir
>> 2>at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >> 2>at Log4jHotPatch.asmVersion(Log4jHotPatch.java:71) >> 2>at Log4jHotPatch.agentmain(Log4jHotPatch.java:93) >> 2>... 6 more > > > Does anyone have an idea? > Your JVM is broken, look at this

Re: [VOTE] Release Lucene 9.1.0 RC2

2022-03-17 Thread Robert Muir
On Thu, Mar 17, 2022 at 4:08 PM Dawid Weiss wrote: > > The smoke tester passed for me on the first try. I'm not sure what > takes so long there (nightly tests?)... It'd be nice to have > timestamps in smoke tester output so that we can determine which phase > takes so darn long. A full gradle

Re: [VOTE] Release Lucene 9.1.0 RC2

2022-03-17 Thread Robert Muir
On Thu, Mar 17, 2022 at 10:28 AM Uwe Schindler wrote: > > The first one passed all tests, but for some strange reason it failed with > the following error: > > gpg: can't connect to the agent: IPC connect call failed > > Not sure where this came from. Looks like a problem with this GPG agent

Re: [VOTE] Release Lucene 9.1.0 RC1

2022-03-15 Thread Robert Muir
On Tue, Mar 15, 2022 at 8:13 AM Uwe Schindler wrote: > > The “tests.multiplier=3” looks like the problem. I have no idea how to stop > this, because the gradle properties are injected through the config file. Is > there a way to pass custom parameters. Maybe we should add >

Re: Experience re OpenAI embeddings in combination with Lucene vector search

2022-02-15 Thread Robert Muir
On Tue, Feb 15, 2022 at 2:33 PM Michael Wechner wrote: > > There seems to be no light at the end of the tunnel for the JDK vector > api, I think OpenJDK will incubate this API until the sun supernovas and > java is dead :) > It is frustrating, as that could give current implementation a needed >

Re: Experience re OpenAI embeddings in combination with Lucene vector search

2022-02-15 Thread Robert Muir
Sure, but lucene should be able to have limits. We have this discussion with every single limit we attempt to implement :) There will always be extreme use cases using too many dimensions or whatever. It is open source! I think if what you are doing is strange enough, you can modify the sources.

Re: The Retired Open Relevance Project

2022-01-20 Thread Robert Muir
On Thu, Jan 20, 2022 at 9:57 PM Marcus Eagan wrote: > > Hi everyone, > > Is there anyone here that has any information on what happened with the Open > Relevance Project? I saw that there was a vote and it was discontinued here. > I could not track down the vote email thread, perhaps because it

Re: [Heads up] Test framework package rename

2021-12-21 Thread Robert Muir
On Tue, Dec 21, 2021 at 8:20 AM Mark Jens wrote: > > You can use > https://patch-diff.githubusercontent.com/raw/apache/lucene/pull/551.diff to > render plain text and keep the browser responsive. > Another option is > https://patch-diff.githubusercontent.com/raw/apache/lucene/pull/551.patch to

Re: [Heads up] Test framework package rename

2021-12-20 Thread Robert Muir
Thanks for doing this!! Give me a few minutes, I'll verify everything passes on linux, and review it locally. And I agree, sooner than later is better as it will conflict with anything and everything. Due to the expected noise of the import statements, the change is so large that it makes the

Re: Welcome Haoyu (Patrick) Zhai as Lucene Committer

2021-12-19 Thread Robert Muir
Congratulations! On Sun, Dec 19, 2021 at 4:12 AM Dawid Weiss wrote: > > Hello everyone! > > Please welcome Haoyu Zhai as the latest Lucene committer. You may also > know Haoyu as Patrick - this is perhaps his kind gesture to those of > us whose tongues are less flexible in pronouncing difficult

Re: Any potential benefits to a SSDV#bulkLookupOrd(long ord) impl?

2021-12-17 Thread Robert Muir
On Fri, Dec 17, 2021 at 10:19 AM Michael Sokolov wrote: > > > And you get two single-valued fields instead of one big multi-valued > > field... so I'm not sure I am convinced that "dim mixing" is typically a > > good thing. > > Mixing enables the user to model multiple (of their) fields within

Re: Any potential benefits to a SSDV#bulkLookupOrd(long ord) impl?

2021-12-17 Thread Robert Muir
On Fri, Dec 17, 2021 at 10:02 AM Greg Miller wrote: > > I suppose the last thing I'd say is that there are valid use-cases for > wanting the "top" dims along with their "top" children, and getAllDims > provides a reasonable way to do this. For example, in Amazon's product > search, we have a

Re: Any potential benefits to a SSDV#bulkLookupOrd(long ord) impl?

2021-12-16 Thread Robert Muir
On Thu, Dec 16, 2021 at 5:57 PM Greg Miller wrote: > > This is separate from adding hierarchical support. I'm probably not > communicating the current state well, but here's where SSDV faceting > does ordinal lookups: >

Re: Any potential benefits to a SSDV#bulkLookupOrd(long ord) impl?

2021-12-16 Thread Robert Muir
On Thu, Dec 16, 2021 at 5:05 PM Greg Miller wrote: > > On Thu, Dec 16, 2021 at 1:31 PM Robert Muir wrote: > > > > On Thu, Dec 16, 2021 at 3:53 PM Greg Miller wrote: > > > > > > > > TaxonomyReader was recently updated > > > to support bulk

Re: Any potential benefits to a SSDV#bulkLookupOrd(long ord) impl?

2021-12-16 Thread Robert Muir
On Thu, Dec 16, 2021 at 3:53 PM Greg Miller wrote: > > TaxonomyReader was recently updated > to support bulk ordinal resolution (LUCENE-9476), but SSDV faceting is > stuck looking up paths one-at-a-time via SSDV#lookupOrd(ord). This > results in a separate TermsEnum#seekExact() call down in >

Re: [VOTE] Release Lucene/Solr 8.11.1 RC1

2021-12-14 Thread Robert Muir
+1 Jan, thank you for being the release manager! Everyone, I know ppl are busy, maybe haven't slept or ate, but please/begging let's just test out this RC, report back with VOTEs, and get these fixes to the users as quick as possible. Let's take care of addressing issues like this for the

Re: Strange error in Solr with Lucene 9.0.0

2021-12-08 Thread Robert Muir
a serious bug. In > Lucene tests we havent seen this! > > Uwe > > - > Uwe Schindler > Achterdiek 19, D-28357 Bremen > https://www.thetaphi.de > eMail: u...@thetaphi.de > > > -Original Message- > > From: Robert Muir > > Sent: Wednesday, Dec

Re: Strange error in Solr with Lucene 9.0.0

2021-12-08 Thread Robert Muir
I also generally tried to "beast" this SimpleFacetsTest.testRangeFacetFilterVsDocValuesRandom without any seed at all (JDK 17/linux), but I didn't manage to provoke any failures. On Wed, Dec 8, 2021 at 1:08 PM Robert Muir wrote: > > I checked out the commit hash and re-ran the te

Re: Strange error in Solr with Lucene 9.0.0

2021-12-08 Thread Robert Muir
reproduces failures for given random seeds? On Wed, Dec 8, 2021 at 12:44 PM Robert Muir wrote: > > Uwe, it looks a little crazy: > > we've got asserts here that "index" is in bounds and certainly not -1 > right before the method call! > https://github.com/apache/lucene/blo

Re: Strange error in Solr with Lucene 9.0.0

2021-12-08 Thread Robert Muir
Uwe, it looks a little crazy: we've got asserts here that "index" is in bounds and certainly not -1 right before the method call! https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90DocValuesProducer.java#L1123-L1125 On Wed, Dec 8, 2021 at

Re: Closing GitHub PRs

2021-12-08 Thread Robert Muir
I'm also now even -1 against bulk-comment. You guys are trying to be too sneaky/passive-aggressive/bypass consensus. I'm stopping this shit right now in its tracks On Wed, Dec 8, 2021 at 8:50 AM Robert Muir wrote: > > I'm -1 against auto-closing issues, as I already stated on this

Re: Closing GitHub PRs

2021-12-08 Thread Robert Muir
> This is a proposal for a one-time action, introducing a stale-bot for the > project, which I can see is more controversial and annoying for sure. > > Jan > > > 8. des. 2021 kl. 13:04 skrev Robert Muir : > > > > i mean you dont even have anything close to fucking cons

Re: Closing GitHub PRs

2021-12-08 Thread Robert Muir
i mean you dont even have anything close to fucking consensus about "bulk close" on this thread. most are against it. why be so fucking sneaky about it? I don't get it! On Wed, Dec 8, 2021 at 7:03 AM Robert Muir wrote: > > On Wed, Dec 8, 2021 at 7:01 AM Robert Muir wrote: &g

Re: Closing GitHub PRs

2021-12-08 Thread Robert Muir
On Wed, Dec 8, 2021 at 7:01 AM Robert Muir wrote: > > I added my vote against bulk close functionality. > it is pretty clear from this thread that several of us are opposed to > bulk close. > > somehow the discussion jumped from bulk commenting to bulk close. fuck that! >

Re: Closing GitHub PRs

2021-12-08 Thread Robert Muir
r way. In JIRA can we keep these old issues > >> while hiding them from default searches. Can we "archive" old issues > >> in some way? Maybe there is a "Status" like Archived that is different > >> from Closed. Anything but Open! > >> >

Re: [JENKINS] Lucene-9.x-Linux (64bit/jdk-17) - Build # 221 - Still Failing!

2021-12-07 Thread Robert Muir
To me, it seems like jenkins fails because it can't find an archiving file. Perhaps something is up, or maybe it just needs the allowEmptyArchive: true, in case files are missing? Still, this doesn't seem to be a typical failure... InterruptedException -> NullPointerException, looks buggy as shit.

Re: [VOTE] Release Lucene 9.0.0 RC4

2021-12-04 Thread Robert Muir
; > > On Sat, Dec 4, 2021 at 12:26 AM Dawid Weiss wrote: >> >> Yeah. Could you please upgrade the JDK first? I suspect javadoc >> generation has changed in between and something in there is causing >> invalid/ different HTML to be emitted for you. >> >> D.

Re: [VOTE] Release Lucene 9.0.0 RC4

2021-12-04 Thread Robert Muir
such ancient versions of openjdk probably have bugs generating the javadocs. On Sat, Dec 4, 2021 at 3:04 AM Gautam Worah wrote: > > openjdk 11 2018-09-25 > OpenJDK Runtime Environment 18.9 (build 11+28) > > On Fri, Dec 3, 2021 at 11:27 PM Dawid Weiss wrote: >> >> Can you check which exact Java

Re: Welcome Julie Tibshirani to the Lucene PMC

2021-11-30 Thread Robert Muir
Congratulations! On Tue, Nov 30, 2021 at 4:49 PM Adrien Grand wrote: > > I'm pleased to announce that Julie Tibshirani has accepted an invitation to > join the Lucene PMC! > > Congratulations Julie, and welcome aboard! > > -- > Adrien

Re: Closing GitHub PRs

2021-11-29 Thread Robert Muir
On Mon, Nov 29, 2021 at 2:49 PM Michael McCandless wrote: > > Could we maybe instead bulk-add a comment explaining the split and how to > take the PR forwards if someone (in the future) has itch/time? > > I know we humans love to clean things up, but I think leaving such "unclean" > things open

Re: [VOTE] Release Lucene 9.0.0 RC3

2021-11-28 Thread Robert Muir
On Sun, Nov 28, 2021 at 6:24 PM Uwe Schindler wrote: > FYI, > > looks like this: > > > > I also tried to compile, test and assemble. All works. Only the src.tgz > can’t be built from itsself, because the extracted folder is no git > repository : > > > > > Task :checkWorkingCopyClean > >

Re: [VOTE] Release Lucene 9.0.0 RC3

2021-11-28 Thread Robert Muir
On Sun, Nov 28, 2021 at 11:12 AM Uwe Schindler wrote: > > Thanks Robert! > > This is of course not a problem, but 7zip is the defacto unzipper used on > windows nowadays, so we should be prepared because of questions. > > It's not an issue at all, the hidden file contains useful information

Re: simplifying precommit checks / GH actions

2021-11-28 Thread Robert Muir
On Sun, Nov 28, 2021 at 12:37 PM Dawid Weiss wrote: > > > OK, I see. I was thinking that a simple 'gradle check' as the github > > action would be easier to understand / reproduce locally for > > contributors, and that by combining the two tasks into one, we'd use > > less server-side resources

Re: simplifying precommit checks / GH actions

2021-11-28 Thread Robert Muir
On Sun, Nov 28, 2021 at 11:34 AM Dawid Weiss wrote: > > Hi Robert, > > > * gradle check -x test: 8 minutes, 15 seconds > > * gradle test: 9 minutes, 8 seconds > > These are done in parallel on separate VMs - if we merge them the > combined time will be longer (that's why I split them into two >

simplifying precommit checks / GH actions

2021-11-28 Thread Robert Muir
Hello, Currently we run 3 actions on every PR/update. I'm including sample runtimes from the most recent PR: https://github.com/apache/lucene/pull/478 * gradle check -x test: 8 minutes, 15 seconds * gradle test: 9 minutes, 8 seconds * lift: 84 minutes First question: should we combine the first

Re: [VOTE] Release Lucene 9.0.0 RC3

2021-11-28 Thread Robert Muir
at 9:09 AM Robert Muir wrote: > > On Sun, Nov 28, 2021 at 5:15 AM Dawid Weiss wrote: > >> > >> > >> Wasn't the src.tgz not created by Gradle? Was it done by using the git > >> command line inside the release wizard? > > > > > > No -

Re: [VOTE] Release Lucene 9.0.0 RC3

2021-11-28 Thread Robert Muir
On Sun, Nov 28, 2021 at 5:15 AM Dawid Weiss wrote: >> >> >> Wasn't the src.tgz not created by Gradle? Was it done by using the git >> command line inside the release wizard? > > > No - we use external git because it simplifies excludes/ includes (git will > only compress versioned files). >

Re: [VOTE] Release Lucene 9.0.0 RC3

2021-11-27 Thread Robert Muir
+1 (with --test-java17) SUCCESS! [0:26:01.193203] On Sat, Nov 27, 2021 at 8:40 AM Michael McCandless wrote: > > +1 > > SUCCESS! [0:06:46.020662] > > What a crazy speedup to smoke tester!! > > > > Mike McCandless > > http://blog.mikemccandless.com > > > On Sat, Nov 27, 2021 at 3:42 AM Ignacio

Re: [VOTE] Release Lucene 9.0.0 RC2

2021-11-23 Thread Robert Muir
Thanks Adrien for addressing one half of the back compat problem. Unfortunately, the "branch_8x" branch is still live and running hot, same as before. So I have to again give my -1 VOTE, due to the fact that backwards compatibility can't be guaranteed. Given that it is being expanded to TWO major

Re: What should we do of branch_8x?

2021-11-21 Thread Robert Muir
21 Nov, 2021, 6:31 pm Robert Muir, wrote: >> >> Sorry, I just don't understand the implications of what you are suggesting. >> >> The code in question is lucene+solr combined, and the build system and >> packaging and everything only knows how to do that. So are you fo

Re: What should we do of branch_8x?

2021-11-21 Thread Robert Muir
t; branch. > > > On Sun, Nov 21, 2021 at 6:14 PM Robert Muir wrote: >> >> I don't think the solr PMC should issue Lucene 8.12 either. >> >> On Sun, Nov 21, 2021 at 7:42 AM Ishan Chattopadhyaya >> wrote: >> > >> > Sounds good, Rob. Shoul

Re: What should we do of branch_8x?

2021-11-21 Thread Robert Muir
> On Sun, 21 Nov, 2021, 6:10 pm Robert Muir, wrote: >> >> Nope, it isn't crazy. I am trying to ensure the backwards >> compatibility that we have is on solid, sustainable footing before we >> release a new version promising double the back compat. >> >> O

Re: What should we do of branch_8x?

2021-11-21 Thread Robert Muir
olr and just add >> Bugfixes. This also applies to Solr. Later this is decoupled, so Solr 9.1234 >> may use Lucene 10.4711. >> >> As said before: let's close branch 8.x and add protection to it in GitHub. >> Anybox may merge Bugfixes directly from Solr or Lucene mai

Re: [VOTE] Release Lucene 9.0.0 RC1

2021-11-21 Thread Robert Muir
and yes, I think it is reasonable to be a blocker. If we release 9.0, promising 2 major versions of back compat, some of these options get removed from the table. On Sun, Nov 21, 2021 at 7:23 AM Robert Muir wrote: > > Thanks Ignacio, > > I see several choices, but the status quo of

Re: [VOTE] Release Lucene 9.0.0 RC1

2021-11-21 Thread Robert Muir
onments. > > Regarding your concerns in the subject, there is a healthy discussion in the > issue and there are sound proposals to ease the pain and they can be > implemented any time, do you think it is still a blocker? > > > > On Sun, Nov 21, 2021 at 12:59 PM Robert Muir wr

Re: [VOTE] Release Lucene 9.0.0 RC1

2021-11-21 Thread Robert Muir
compat, people are quick to demand more of it every time. But when it comes to addressing the necessary issues to make it work...crickets. On Sun, Nov 21, 2021 at 5:11 AM Robert Muir wrote: > > -1 to release lucene 9.0, as long as branch_8x remains. > > I know you made a sep

Re: What should we do of branch_8x?

2021-11-21 Thread Robert Muir
has no weight." > > On Sun, 21 Nov, 2021, 3:30 pm Robert Muir, wrote: >> >> I think we should remove this branch. >> >> personally, i'll probably -1 any commit to it. I'll see if i can >> automate such an email response with a gmail rule. >> >

Re: [VOTE] Release Lucene 9.0.0 RC1

2021-11-21 Thread Robert Muir
-1 to release lucene 9.0, as long as branch_8x remains. I know you made a separate thread for this, but it is a real problem. The problem is that we can't support backwards compatibility like this: releasing 9.0 then 8.12's and stuff. It isn't how the back compat testing works, it is completely

Re: What should we do of branch_8x?

2021-11-21 Thread Robert Muir
I think we should remove this branch. personally, i'll probably -1 any commit to it. I'll see if i can automate such an email response with a gmail rule. we already released lucene 9.0, we can't change backwards compatibility for some 8.12, same old story, lets move on people. On Sat, Nov 20,

Re: [VOTE] Release Lucene 9.0.0 RC1

2021-11-20 Thread Robert Muir
I also ran the smoketester with --test-java17 /usr/lib/jvm/java-17-openjdk, which ran all the checks on both java 11 and java 17. SUCCESS! [0:34:34.251839] On Sat, Nov 20, 2021 at 7:25 AM Robert Muir wrote: > > I ran the smoketester and it passed: > > SUCCESS! [0:18:08.361431] >

Re: [VOTE] Release Lucene 9.0.0 RC1

2021-11-20 Thread Robert Muir
I ran the smoketester and it passed: SUCCESS! [0:18:08.361431] Thanks to all the hard work on smoke testing, release scripts, and build! Like Uwe, I plan to do some manual inspection and look for trouble before voting. But things look to be in a good place. On Sat, Nov 20, 2021 at 3:25 AM

Re: Lucene 9.0 release candidate

2021-11-19 Thread Robert Muir
i, Nov 19, 2021 at 4:44 PM Greg Miller wrote: > > > > Heads up that both LUCENE-10122 and LUCENE-10062 have been merged onto > > branch_9_0 now. @Adrien Grand I know you're aware already, but > > following up here just for completeness. Thanks! > > > > -Greg &

Re: Lucene 9.0 release candidate

2021-11-15 Thread Robert Muir
On Mon, Nov 15, 2021 at 2:02 PM Michael McCandless wrote: > > > Yeah I love that idea, but that's not what Patrick's PR explored (yet?). > > His explored switching away from custom token positions to NumericDocValues > to store the same data (ordinal -> parent mapping), but it still loaded all

Re: Lucene 9.0 release candidate

2021-11-15 Thread Robert Muir
On Mon, Nov 15, 2021 at 12:57 PM Michael McCandless wrote: > > I think for PR 420 (https://github.com/apache/lucene/pull/420) we are > (confusingly!) not really seeing performance benefits -- taxonomy index got a > bit bigger, and loading the parent arrays no faster? So Patrick closed that >

Re: Bump minimum Java version to 17 on main (10.0)

2021-11-04 Thread Robert Muir
On Thu, Nov 4, 2021 at 10:42 AM Dawid Weiss wrote: > > > we don't have to "support" users on our main trunk branch. That's why > > we make releases. > > Nah. I don't agree with this one - you have to be aware about who is > using your project (especially a library) and in what context. Maybe > we

Re: Bump minimum Java version to 17 on main (10.0)

2021-11-04 Thread Robert Muir
t will > probably be for a good reason, and we should prepare users for that. > > On Thu, Nov 4, 2021 at 7:54 AM Robert Muir wrote: > > > > With the current release frequency, JDK 11 will be EOL by the time > > Lucene 10 is released. > > > > Users can use luc

Re: [lucene] branch main updated: LUCENE-10088: allow per-class override in HandleLimitFS. Bump the limit a bit for nightlies in TestIndexWriterMergePolicy. (#424)

2021-11-04 Thread Robert Muir
On Thu, Nov 4, 2021 at 9:17 AM Dawid Weiss wrote: > > I think there should be a better solution based on throttling. This test runs > with 4 threads - I can imagine processes where an order of magnitude more > threads will index in parallel (+ merges). So it should just work. I don't > know

Re: Bump minimum Java version to 17 on main (10.0)

2021-11-04 Thread Robert Muir
Lucene. >> >> >> >> Uwe >> >> >> >> - >> >> Uwe Schindler >> >> Achterdiek 19, D-28357 Bremen >> >> https://www.thetaphi.de >> >> eMail: u...@thetaphi.de >> >> >> >> From: Da

Re: [lucene] branch main updated: LUCENE-10088: allow per-class override in HandleLimitFS. Bump the limit a bit for nightlies in TestIndexWriterMergePolicy. (#424)

2021-11-04 Thread Robert Muir
Hopefully we can find a better long-term fix for this test? If fixing IndexWriter throttling is too tricky, maybe we could simply improve the test with a short-term fix to use less handles (e.g. force NRTCachingDir or RAMDir) 2048 is already way too high, the default OS limit is 1024 on every

Re: Bump minimum Java version to 17 on main (10.0)

2021-11-03 Thread Robert Muir
On Wed, Nov 3, 2021 at 1:36 PM Dawid Weiss wrote: > > I principally agree with you - we should leverage new Java features and I'm > all for it. I just don't see much difference between > Java 11 and 17 in the context of Lucene... Upgrading for the sake of > upgrading doesn't justify the move to

Re: Bump minimum Java version to 17 on main (10.0)

2021-11-03 Thread Robert Muir
On Wed, Nov 3, 2021 at 12:51 PM Robert Muir wrote: > > > > It will be released, eventually, right? In six months, a year maybe? Then > > it's people like me who would be affected: we use Lucene internally and > > this one dependency would push the entire stack to

Re: Bump minimum Java version to 17 on main (10.0)

2021-11-03 Thread Robert Muir
On Wed, Nov 3, 2021 at 12:32 PM Dawid Weiss wrote: > > >> Who would we be forcing to upgrade? It is the 'main' branch: unreleased. > > > It will be released, eventually, right? In six months, a year maybe? Then > it's people like me who would be affected: we use Lucene internally and this > one

Re: Bump minimum Java version to 17 on main (10.0)

2021-11-03 Thread Robert Muir
Who would we be forcing to upgrade? It is the 'main' branch: unreleased. +1 to bump to 17 On Wed, Nov 3, 2021 at 11:19 AM Dawid Weiss wrote: > > > Do we gain much from such a requirement? Are there APIs that make things run > faster or perform better? > > The downside for me is that if Lucene

Re: [JENKINS] Lucene » Lucene-Maven-Snapshots-main - Build # 791 - Failure!

2021-10-31 Thread Robert Muir
Dawid do you know what code is doing this? Can we configure retries for the network calls? On Sun, Oct 31, 2021 at 4:03 AM Dawid Weiss wrote: > > > Some temporary glitch, it seems: > > Could not GET >

Re: Multi-valued xxValue / xxValueSource implementations?

2021-10-26 Thread Robert Muir
On Tue, Oct 26, 2021 at 8:01 PM Robert Muir wrote: > > Hi Greg, I think the general issue is one of the API, the ValueSource > seems really geared at returning values from single-valued fields. I think really, this is the core issue. This ValueSource thing was created before

Re: Multi-valued xxValue / xxValueSource implementations?

2021-10-26 Thread Robert Muir
and common case, and you only need MAX in the rare case someone clicks an arrow to reverse the sort, as it requires consuming all the ordinals for each doc :) On Tue, Oct 26, 2021 at 8:01 PM Robert Muir wrote: > > Hi Greg, I think the general issue is one of the API, the ValueSource >

Re: Multi-valued xxValue / xxValueSource implementations?

2021-10-26 Thread Robert Muir
Hi Greg, I think the general issue is one of the API, the ValueSource seems really geared at returning values from single-valued fields. IMO, for the way the API is used (e.g. sorting), it makes sense to define a selector that works in O(1) time per-document, and use these existing valuesources:

Re: Slow DV equivalent of TermInSetQuery

2021-10-26 Thread Robert Muir
. > > > Joel Bernstein > http://joelsolr.blogspot.com/ > > > On Tue, Oct 26, 2021 at 5:31 PM Robert Muir wrote: >> >> Sorry, I don't think there is a need to use any top-level ordinals. >> none of these docvalues-based query implementations need it. >> &

Re: Slow DV equivalent of TermInSetQuery

2021-10-26 Thread Robert Muir
redicates. >> >> >> Joel Bernstein >> http://joelsolr.blogspot.com/ >> >> >> On Tue, Oct 26, 2021 at 3:52 PM Adrien Grand wrote: >>> >>> I opened https://issues.apache.org/jira/browse/LUCENE-10207 about these >>> ideas. >>> &g

Re: Slow DV equivalent of TermInSetQuery

2021-10-26 Thread Robert Muir
On Tue, Oct 26, 2021 at 1:37 PM Adrien Grand wrote: > > > And then we could make an IndexOrDocValuesQuery with both the > > TermInSetQuery and this SDV.newSlowInSetQuery? > > Unfortunately IndexOrDocValuesQuery relies on the fact that the "index" query > can evaluate its cost

<    1   2   3   4   5   6   7   8   9   10   >