Re: SIP-10: Solr 9 examples: Can we use Ref Guide as a dogfood example?

[David Smiley]

It's tempting to accomplish two goals at once (tutorial & searchable ref
guide) but I think the realities of making a *good* searchable ref guide
may distract someone from learning as it tries to do both well.  A
searchable ref-guide could very well be its own project that we point
people learning at who move beyond some of the very early basics.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Tue, Sep 1, 2020 at 1:23 PM Alexandre Rafalovitch 
wrote:

> That Jeopardy set reads very dubious. Content that was collected by
> scraping and available on various sharing sites (including Mega!). I
> would not feel comfortable working with that in our context.
>
> There are other dataset sources. I like the ones that Data is Plural
> newsletter collects: https://tinyletter.com/data-is-plural (full list
> at:
> https://docs.google.com/spreadsheets/d/1wZhPLMCHKJvwOkP4juclhjFgqIY8fQFMemwKL2c64vk/edit#gid=0
> ). Again, copyright is important and I think having a local copy is
> important too, for at least tutorial purposes.
>
> But I wish we could figure out a way to include the RefGuide. It is
> just so much more triple-bottom line solution than just random other
> dataset. We could do a graph of cross-references in the guide, figure
> out how to extract java path references, etc.
>
> Anyway, it is not something that is super-urgent. I don't even know
> whether our new build processes can be augmented to do this. I guess
> it is a bit similar to how we run tests.
>
> I just wanted to get a strong yay/nay on the idea. So far it feels
> like I got one strong yay, one caution and one soft nay.
>
> Regards,
>Alex.
>
>
>
> On Tue, 1 Sep 2020 at 12:28, Jan Høydahl  wrote:
> >
> > What about 200.000 Jeopardy questions in JSON format?
> >
> https://www.reddit.com/r/datasets/comments/1uyd0t/20_jeopardy_questions_in_a_json_file/
> > I downloaded the file in a few seconds, and it also has some structured
> content, e.g.
> >
> >   {
> > "category": "NOVELS",
> > "air_date": "2005-01-27",
> > "question": "'Even the epilogue is lengthy in this 1869 Tolstoy
> epic; it comes out in 2 parts &, in our copy, is 105 pages long'",
> > "value": "$400",
> > "answer": "War and Peace",
> > "round": "Jeopardy!",
> > "show_number": "4699"
> >   },
> >   {
> > "category": "BRIGHT IDEAS",
> > "air_date": "2005-01-27",
> > "question": "'In 1948 scientists at Bristol-Meyers \"buffered\" this
> medicine for the first time'",
> > "value": "$400",
> > "answer": "aspirin",
> > "round": "Jeopardy!",
> > "show_number": "4699"
> >   },
> >
> > Lots of docs. Enough free-text to learn some analysis, enough metadata
> for some meaningful facets / filters…
> >
> > As long as we only provide a URL and not re-distribute the content,
> licensing is less of a concern.
> >
> > Jan
> >
> > 1. sep. 2020 kl. 15:59 skrev Alexandre Rafalovitch :
> >
> > I've thought of providing instructions. But for good indexing, we
> > should use adoc format as source, rather than html (as Cassandra's
> > presentation showed), so that means dependencies to build by user to
> > get asciidoctor library. And the way to get content, so either git
> > clone or download the whole source and unpack and figure out the
> > directory locations. It feels messy. Then, it may as well be an
> > external package or even an external independent project. And
> > therefore, it would lose value as a shipped tutorial material.
> >
> > We could also discuss actually shipping the Solr Reference Guide with
> > Solr now that the release cycles align, but that would actually not
> > help my sub-project too much, again because of adoc vs. html formats.
> >
> > In terms of other datasets:
> > *) I could just stay with limited full-text in the one I am thinking
> > of. The bulk download mode allows for fields such as Occupation,
> > Company and Vehicle model which are 2-7 words long. That's about the
> > same length as current examples we ship. It does not allow for a
> > meaningful discussion about longer-text issues such as
> > length-normalization, but we don't have those now anyway.
> > *) I could use a public domain book and break it into parts. From
> > somewhere like https://standardebooks.org/ . But there is a question
> > about licensing and also whether we will be able to show interesting
> > effects with that.
> > *) I was also told that there is Wikipedia, but again, would we just
> > include a couple of articles at random? What's the license?
> > *) It is possible to index Stack Overflow questions, either from the
> > feed (DIH was doing that) or as a download. I think the license was
> > compatible.
> > *) I could augment the dataset with some mix of the above, like a
> > "favourite quote" field with random book sentences. This feels like
> > fun, but possibly a whole separate project of its own.
> >
> > Anyway, I am open to further thoughts. It is quite likely I missed
> something.
> 

Re: Checking dependencies in gradle after code is removed

[David Smiley]

Thanks Dawid for that fantastic explanation!
What's the path of the ".lock" file you refer to?

gradlew -p solr/solrj dependencies
I observe it has many named dependency trees; some quite large.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Tue, Sep 1, 2020 at 3:19 AM Dawid Weiss  wrote:

> bq. you get a lot of entries there
>
> That's why this is to large degree automated. You do have a lot of
> entries because it's a union of all dependencies across all
> sub-projects. While this may seem like a negative, it's not - it
> ensures everyone stays on the same set of dependencies. Makes life
> easier in the long run.
>
> Also, you can check what depends on what by looking at the .lock file;
> for example:
>
> com.github.jnr:jffi:1.2.18 (1 constraints: b20902ab)
>
> if you ask for this hash, you'll see where the dependency comes from
> (and so on, recursively). Note that some of these dependencies are
> built-time, not runtime.
>
> O:\repos\lucene-master>gradlew why --hash=b20902ab
> Starting a Gradle Daemon (subsequent builds will be faster)
>
> > Task :why
> com.github.jnr:jffi:1.2.18
> org.jruby:jruby-core -> 1.2.18
>
> O:\repos\lucene-master>gradlew why --hash=0f08b57d
>
> > Task :why
> org.jruby:jruby-core:9.2.6.0
> org.jruby:jruby -> 9.2.6.0
>
> org.jruby:jruby-stdlib:9.2.6.0
> org.jruby:jruby -> 9.2.6.0
>
> O:\repos\lucene-master>gradlew why --hash=490d7d28
>
> > Task :why
> org.jruby:jruby:9.2.6.0
> org.asciidoctor:asciidoctorj -> 9.2.6.0
>
> etc.
>
> Dawid
>
> On Tue, Sep 1, 2020 at 1:20 AM Erick Erickson 
> wrote:
> >
> > Excellent, I’ve put it in my notes… you get a lot of entries there…
> Thanks!
> >
> > > On Aug 31, 2020, at 2:59 PM, Dawid Weiss 
> wrote:
> > >
> > >> Is there an easy way to insure that versions.props only has necessary
> dependencies listed? Ideally it would be just a top-level command.
> > >
> > > If a dependency isn't used, check will fail. It's a feature of
> > > palantir's version control plugin. The concrete task that does it is
> > > named checkUnusedConstraints but you should not need to know it (or
> > > care). Use check at the top level and it'll be run automatically.
> > >
> > >> gradlew dependencies lists pretty much everything, is it the case
> that if a dependency does _not_ show up in a project that it’s unused?
> > >
> > > That's because solr depends on pretty much everything...
> > >
> > > D.
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > > For additional commands, e-mail: dev-h...@lucene.apache.org
> > >
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: SIP-10 Improve Getting Started experience

[David Smiley]

Thank you, thank you, thank you!

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Wed, Aug 26, 2020 at 3:42 PM Alexandre Rafalovitch 
wrote:

> Dear all,
>
> Based on the discussion in SOLR-14726, Slack, and in many other JIRA
> issues, I am proposing to spearhead the Big Cleanup focused on
> improving getting started experience, targeting Solr 9 only. This is
> mostly about examples, but it also touches quite heavily on default
> configuration files and - less heavily - on the directory layout
> issues.
>
> As this is bigger than any of my previous contributions to the project
> and is quite cross-cutting, I've made a Solr Improvement Proposal.
>
>
> https://cwiki.apache.org/confluence/display/SOLR/SIP-10+Improve+Getting+Started+experience
>
> Regards,
>Alex.
> Ps. As a side note, in many of the discussions, people say "I would
> prefer if we did X", but with the reality of "X" not being complete,
> actually fully tested or even applicable to the
> standalone/cloud/whatever aspect of the discussion. Those preferences
> are still welcome, but I am begging for explicit qualification of
> their current availability for the critical path. This proposal is an
> attempt to minimize dependence on actual new code features.
> Pps. I am also very aware that the proposal implies lots of
> semi-related changes and I will end up potentially stepping on
> people's feet. And I don't yet know the proper flow of work to do this
> (individual Jiras, combined Jiras, separate branch(s), etc). I would
> really need advice or a mentor or even a group of mentors to make this
> happen in a community-minded fashion.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Request to release packages(Apache Solr Solrj,Apache Solr Core) with latest netty

[David Smiley]

It was just recently updated:
https://issues.apache.org/jira/browse/SOLR-14606
8.7 will use 4.1.50.Final.  I just checked by using a local distribution
(cd solr; ant dist) on branch_8x and looking for the netty jar files
therein.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Fri, Aug 21, 2020 at 8:51 AM ODI DEV  wrote:

> Hi Team,
>
>
> Regarding using the latest netty version, May I know why are we using
> 'Netty' version 4.1.47.final in released branch whereas we are using netty
> 4.1.50.final version in build files of master branch which was updated
> before the previous 2 releases?
>
> I tried successful local builds of this package "Apache Solr Solrj" of the
> master branch, on arm and x86 both architectures using 'ant' and 'Gradle'
> building methods.
>
> Master branch is tested locally, it is downloading netty's 4.1.50.final
> version, while building with both Gradle and ant.
> however netty version used in its recent & latest [maven repo release](
> https://mvnrepository.com/artifact/org.apache.solr/solr-solrj/8.6.1) is
> 4.1.47.final.
>
> Netty version from below files of the master branch and release branch are
> not same:
> - [ivy-versions.properties](
> https://github.com/apache/lucene-solr/blob/master/lucene/ivy-versions.properties#L76
> )
> - [versions.props](
> https://github.com/apache/lucene-solr/blob/master/versions.props#L36)
>
> Kindly let us know when can we see netty latest version as the update
> netty ver 4.1.50 includes both security fixes and AArch64 performance
> improvements.
> Refer release notes for detail:
>   -https://netty.io/news/2020/05/13/4-1-50-Final.html
>   -https://netty.io/news/2020/07/09/4-1-51-Final.html
>
> Thanks,
> Odidev
>
>

Re: Name for the directory above solr.home?

[David Smiley]

The parent directory of Solr home is not special.  Where Solr Home is (as
you know) configurable.  It's default location is different too, since it's
in /var/solr for both the Docker & service install script.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Tue, Aug 25, 2020 at 8:31 PM Alexandre Rafalovitch 
wrote:

> Hello,
>
> What do we call the directory above the solr.home?
> E.g.
> - "schemaless" in example/schemaless/solr/gettingstarted
> - "node1" in example/cloud/node1/solr/gettingstarted_shard1_replica_n2
> - "solr" in server/solr/book/conf
>
> In solr.in.cmd we "may be" calling it "solr start dir"
> In bin/solr, we call it SOLR_SERVER_DIR
> In install_solr-service.sh, we sort-of call it SOLR_VAR_DIR "Directory
> for live/writable Solr files", which is not quite the same because
> apparently pid files go there, while for distribution they seem to go
> into bin
>
> I guess it mostly matters when you have multiple Solr instances
> running on the same machine and you need to separate log locations,
> etc.
>
> Regards,
>   Alex.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Atomic updates, copyField and stored=true

[David Smiley]

Sounds reasonable to me.  I suggest that you try to find which people
worked specifically on the existing restriction at the time RTG came about
and ask them.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Tue, Aug 18, 2020 at 7:56 PM Erick Erickson 
wrote:

> It _finally_ occurred to me to ask why we have the restriction that the
> destination of a copyField must have stored=false. I understand what
> currently happens when that’s the case, you get repeats.
>
> What I wondered is why we can’t detect that a field is the destination of
> a copyField and _not_ pull the stored values out of it during atomic
> updates?
>
> Or do we run afoul of things in tlog retrieval or RTG?
>
> Is this a silly idea or should I raise a JIRA?
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: [VOTE] Lucene logo contest, third time's a charm

[David Smiley]

(binding)
vote: D, A1


(thanks Ryan for your thorough vote instructions & preparation)

Re: Code Analysis during CI?

[David Smiley]

Sounds great to me!  I'm really glad to hear it works with the PR workflow,
and only on the files touched in the PR.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Thu, Sep 3, 2020 at 8:03 PM Tom DuBuisson  wrote:

> Tomás,
> Oof, thanks for the note on TOS.  I fixed the link.  The tool can be
> configured and I'm happy to make things work better for your use case.
> Muse is free for public repos and will remain free for open source
> indefinitely.  You can try it and remove it any time - github is in charge
> of access control and provides you as the repository owner with control via
> the website.
>
> On Thu, Sep 3, 2020 at 4:37 PM Tomás Fernández Löbbe <
> tomasflo...@gmail.com> wrote:
>
>> Thanks Tom. I think this could be very useful as long as it can be
>> configurable. (The "terms of use here[1] link to "google.com", so I
>> couldn't check that, but they claim it's free for public repos, so...). We
>> could always try it and remove it if we don't like it? What do others think?
>>
>>
>> [1] https://github.com/apps/muse-dev
>>
>> On Thu, Sep 3, 2020 at 3:06 PM Tom DuBuisson  wrote:
>>
>>> Hello Lucene/Solr folks,
>>>
>>> During Lucene development CI is used for build and unit tests to gate
>>> merges.  The CI doesn't yet include any analysis tools though, but their
>>> use has been discussed [1].  I fixed some issues flagged by Facebook's
>>> Infer and was prompted to bring up the topic here [2].
>>>
>>> The recent PR fixed some low-hanging fruit that was reported when I ran
>>> Muse [3] - a github app that is a platform for static analysis tools.
>>>  Muse's platform bundles the most useful analysis tools, all open source
>>> with many of them developed by FANG, and triggers analysis on PRs
>>> then delivers results as comments.
>>>
>>> Because of the PR-centric workflow you only see issues related to the
>>> changes in the pull request.  This means that even a project where tools
>>> give a daunting list of issues can still have quiet day-to-day operation.
>>> Muse also has options to configure individual tools and turn tools or
>>> warnings off entirely.  If there are concerns in addition to noise and
>>> added mental tax on development then I'd really like to hear those thoughts.
>>>
>>> Would you be up for running Muse on the lucene-solr repo?  Let me know,
>>> and I hope to hear your thoughts on analysis tools either way.
>>>
>>> -Tom
>>>
>>> [1] https://issues.apache.org/jira/projects/LUCENE/issues/LUCENE-8847
>>> [2] https://issues.apache.org/jira/projects/SOLR/issues/SOLR-14819
>>> [3] Muse result on Lucene:
>>> https://console.muse.dev/result/TomMD/lucene-solr/01EH5WXS6C1RH1NFYHP6ATXTZ9?tab=results
>>> Muse app link: https://github.com/apps/muse-dev
>>> [4] https://github.com/TomMD/lucene-solr/pulls
>>> [5] Example of muse commenting on an issue
>>> https://github.com/TomMD/shiro/pull/2
>>>
>>>

Re: Code Analysis during CI?

[Tom DuBuisson]

Tomás,
Oof, thanks for the note on TOS.  I fixed the link.  The tool can be
configured and I'm happy to make things work better for your use case.
Muse is free for public repos and will remain free for open source
indefinitely.  You can try it and remove it any time - github is in charge
of access control and provides you as the repository owner with control via
the website.

On Thu, Sep 3, 2020 at 4:37 PM Tomás Fernández Löbbe 
wrote:

> Thanks Tom. I think this could be very useful as long as it can be
> configurable. (The "terms of use here[1] link to "google.com", so I
> couldn't check that, but they claim it's free for public repos, so...). We
> could always try it and remove it if we don't like it? What do others think?
>
>
> [1] https://github.com/apps/muse-dev
>
> On Thu, Sep 3, 2020 at 3:06 PM Tom DuBuisson  wrote:
>
>> Hello Lucene/Solr folks,
>>
>> During Lucene development CI is used for build and unit tests to gate
>> merges.  The CI doesn't yet include any analysis tools though, but their
>> use has been discussed [1].  I fixed some issues flagged by Facebook's
>> Infer and was prompted to bring up the topic here [2].
>>
>> The recent PR fixed some low-hanging fruit that was reported when I ran
>> Muse [3] - a github app that is a platform for static analysis tools.
>>  Muse's platform bundles the most useful analysis tools, all open source
>> with many of them developed by FANG, and triggers analysis on PRs
>> then delivers results as comments.
>>
>> Because of the PR-centric workflow you only see issues related to the
>> changes in the pull request.  This means that even a project where tools
>> give a daunting list of issues can still have quiet day-to-day operation.
>> Muse also has options to configure individual tools and turn tools or
>> warnings off entirely.  If there are concerns in addition to noise and
>> added mental tax on development then I'd really like to hear those thoughts.
>>
>> Would you be up for running Muse on the lucene-solr repo?  Let me know,
>> and I hope to hear your thoughts on analysis tools either way.
>>
>> -Tom
>>
>> [1] https://issues.apache.org/jira/projects/LUCENE/issues/LUCENE-8847
>> [2] https://issues.apache.org/jira/projects/SOLR/issues/SOLR-14819
>> [3] Muse result on Lucene:
>> https://console.muse.dev/result/TomMD/lucene-solr/01EH5WXS6C1RH1NFYHP6ATXTZ9?tab=results
>> Muse app link: https://github.com/apps/muse-dev
>> [4] https://github.com/TomMD/lucene-solr/pulls
>> [5] Example of muse commenting on an issue
>> https://github.com/TomMD/shiro/pull/2
>>
>>

Re: Code Analysis during CI?

[Tomás Fernández Löbbe]

Thanks Tom. I think this could be very useful as long as it can be
configurable. (The "terms of use here[1] link to "google.com", so I
couldn't check that, but they claim it's free for public repos, so...). We
could always try it and remove it if we don't like it? What do others think?


[1] https://github.com/apps/muse-dev

On Thu, Sep 3, 2020 at 3:06 PM Tom DuBuisson  wrote:

> Hello Lucene/Solr folks,
>
> During Lucene development CI is used for build and unit tests to gate
> merges.  The CI doesn't yet include any analysis tools though, but their
> use has been discussed [1].  I fixed some issues flagged by Facebook's
> Infer and was prompted to bring up the topic here [2].
>
> The recent PR fixed some low-hanging fruit that was reported when I ran
> Muse [3] - a github app that is a platform for static analysis tools.
>  Muse's platform bundles the most useful analysis tools, all open source
> with many of them developed by FANG, and triggers analysis on PRs
> then delivers results as comments.
>
> Because of the PR-centric workflow you only see issues related to the
> changes in the pull request.  This means that even a project where tools
> give a daunting list of issues can still have quiet day-to-day operation.
> Muse also has options to configure individual tools and turn tools or
> warnings off entirely.  If there are concerns in addition to noise and
> added mental tax on development then I'd really like to hear those thoughts.
>
> Would you be up for running Muse on the lucene-solr repo?  Let me know,
> and I hope to hear your thoughts on analysis tools either way.
>
> -Tom
>
> [1] https://issues.apache.org/jira/projects/LUCENE/issues/LUCENE-8847
> [2] https://issues.apache.org/jira/projects/SOLR/issues/SOLR-14819
> [3] Muse result on Lucene:
> https://console.muse.dev/result/TomMD/lucene-solr/01EH5WXS6C1RH1NFYHP6ATXTZ9?tab=results
> Muse app link: https://github.com/apps/muse-dev
> [4] https://github.com/TomMD/lucene-solr/pulls
> [5] Example of muse commenting on an issue
> https://github.com/TomMD/shiro/pull/2
>
>

Code Analysis during CI?

[Tom DuBuisson]

Hello Lucene/Solr folks,

During Lucene development CI is used for build and unit tests to gate
merges.  The CI doesn't yet include any analysis tools though, but their
use has been discussed [1].  I fixed some issues flagged by Facebook's
Infer and was prompted to bring up the topic here [2].

The recent PR fixed some low-hanging fruit that was reported when I ran
Muse [3] - a github app that is a platform for static analysis tools.
 Muse's platform bundles the most useful analysis tools, all open source
with many of them developed by FANG, and triggers analysis on PRs
then delivers results as comments.

Because of the PR-centric workflow you only see issues related to the
changes in the pull request.  This means that even a project where tools
give a daunting list of issues can still have quiet day-to-day operation.
Muse also has options to configure individual tools and turn tools or
warnings off entirely.  If there are concerns in addition to noise and
added mental tax on development then I'd really like to hear those thoughts.

Would you be up for running Muse on the lucene-solr repo?  Let me know, and
I hope to hear your thoughts on analysis tools either way.

-Tom

[1] https://issues.apache.org/jira/projects/LUCENE/issues/LUCENE-8847
[2] https://issues.apache.org/jira/projects/SOLR/issues/SOLR-14819
[3] Muse result on Lucene:
https://console.muse.dev/result/TomMD/lucene-solr/01EH5WXS6C1RH1NFYHP6ATXTZ9?tab=results
Muse app link: https://github.com/apps/muse-dev
[4] https://github.com/TomMD/lucene-solr/pulls
[5] Example of muse commenting on an issue
https://github.com/TomMD/shiro/pull/2

Re: [VOTE] Lucene logo contest, third time's a charm

[Kevin Risden]

A1, A2, D (binding)

Kevin Risden



On Thu, Sep 3, 2020 at 4:44 PM jim ferenczi  wrote:

> A1 (binding)
>
> Le jeu. 3 sept. 2020 à 07:09, Noble Paul  a écrit :
>
>> A1, A2, D binding
>>
>> On Thu, Sep 3, 2020 at 7:22 AM Jason Gerlowski 
>> wrote:
>> >
>> > A1, A2, D (binding)
>> >
>> > On Wed, Sep 2, 2020 at 10:47 AM Michael McCandless
>> >  wrote:
>> > >
>> > > A2, A1, C5, D (binding)
>> > >
>> > > Thank you to everyone for working so hard to make such cool looking
>> possible future Lucene logos!  And to Ryan for the challenging job of
>> calling this VOTE :)
>> > >
>> > > Mike McCandless
>> > >
>> > > http://blog.mikemccandless.com
>> > >
>> > >
>> > > On Tue, Sep 1, 2020 at 4:21 PM Ryan Ernst  wrote:
>> > >>
>> > >> Dear Lucene and Solr developers!
>> > >>
>> > >> Sorry for the multiple threads. This should be the last one.
>> > >>
>> > >> In February a contest was started to design a new logo for Lucene
>> [jira-issue]. The initial attempt [first-vote] to call a vote resulted in
>> some confusion on the rules, as well the request for one additional
>> submission. The second attempt [second-vote] yesterday had incorrect links
>> for one of the submissions. I would like to call a new vote, now with more
>> explicit instructions on how to vote, and corrected links.
>> > >>
>> > >> Please read the following rules carefully before submitting your
>> vote.
>> > >>
>> > >> Who can vote?
>> > >>
>> > >> Anyone is welcome to cast a vote in support of their favorite
>> submission(s). Note that only PMC member's votes are binding. If you are a
>> PMC member, please indicate with your vote that the vote is binding, to
>> ease collection of votes. In tallying the votes, I will attempt to verify
>> only those marked as binding.
>> > >>
>> > >> How do I vote?
>> > >>
>> > >> Votes can be cast simply by replying to this email. It is a
>> ranked-choice vote [rank-choice-voting]. Multiple selections may be made,
>> where the order of preference must be specified. If an entry gets more than
>> half the votes, it is the winner. Otherwise, the entry with the lowest
>> number of votes is removed, and the votes are retallied, taking into
>> account the next preferred entry for those whose first entry was removed.
>> This process repeats until there is a winner.
>> > >>
>> > >> The entries are broken up by variants, since some entries have
>> multiple color or style variations. The entry identifiers are first a
>> capital letter, followed by a variation id (described with each entry
>> below), if applicable. As an example, if you prefer variant 1 of entry A,
>> followed by variant 2 of entry A, variant 3 of entry C, entry D, and lastly
>> variant 4e of entry B, the following should be in your reply:
>> > >>
>> > >> (binding)
>> > >> vote: A1, A2, C3, D, B4e
>> > >>
>> > >> Entries
>> > >>
>> > >> The entries are as follows:
>> > >>
>> > >> A. Submitted by Dustin Haver. This entry has two variants, A1 and A2.
>> > >>
>> > >> [A1]
>> https://issues.apache.org/jira/secure/attachment/12999548/Screen%20Shot%202020-04-10%20at%208.29.32%20AM.png
>> > >> [A2]
>> https://issues.apache.org/jira/secure/attachment/12997172/LuceneLogo.png
>> > >>
>> > >> B. Submitted by Stamatis Zampetakis. This has several variants.
>> Within the linked entry there are 7 patterns and 7 color palettes. Any vote
>> for B should contain the pattern number followed by the lowercase letter of
>> the color palette. For example, B3e or B1a.
>> > >>
>> > >> [B]
>> https://issues.apache.org/jira/secure/attachment/12997768/zabetak-1-7.pdf
>> > >>
>> > >> C. Submitted by Baris Kazar. This entry has 8 variants.
>> > >>
>> > >> [C1]
>> https://issues.apache.org/jira/secure/attachment/13006392/lucene_logo1_full.pdf
>> > >> [C2]
>> https://issues.apache.org/jira/secure/attachment/13006393/lucene_logo2_full.pdf
>> > >> [C3]
>> https://issues.apache.org/jira/secure/attachment/13006394/lucene_logo3_full.pdf
>> > >> [C4]
>> https://issues.apache.org/jira/secure/attachment/13006395/lucene_logo4_full.pdf
>> > >> [C5]
>> https://issues.apache.org/jira/secure/attachment/13006396/lucene_logo5_full.pdf
>> > >> [C6]
>> https://issues.apache.org/jira/secure/attachment/13006397/lucene_logo6_full.pdf
>> > >> [C7]
>> https://issues.apache.org/jira/secure/attachment/13006398/lucene_logo7_full.pdf
>> > >> [C8]
>> https://issues.apache.org/jira/secure/attachment/13006399/lucene_logo8_full.pdf
>> > >>
>> > >> D. The current Lucene logo.
>> > >>
>> > >> [D]
>> https://lucene.apache.org/theme/images/lucene/lucene_logo_green_300.png
>> > >>
>> > >> Please vote for one of the above choices. This vote will close about
>> one week from today, Mon, Sept 7, 2020 at 11:59PM.
>> > >>
>> > >> Thanks!
>> > >>
>> > >> [jira-issue] https://issues.apache.org/jira/browse/LUCENE-9221
>> > >> [first-vote]
>> http://mail-archives.apache.org/mod_mbox/lucene-dev/202006.mbox/%3cCA+DiXd74Mz4H6o9SmUNLUuHQc6Q1-9mzUR7xfxR03ntGwo=d...@mail.gmail.com%3e
>> > >> [second-vote]
>> 

Re: [VOTE] Lucene logo contest, third time's a charm

[jim ferenczi]

A1 (binding)

Le jeu. 3 sept. 2020 à 07:09, Noble Paul  a écrit :

> A1, A2, D binding
>
> On Thu, Sep 3, 2020 at 7:22 AM Jason Gerlowski 
> wrote:
> >
> > A1, A2, D (binding)
> >
> > On Wed, Sep 2, 2020 at 10:47 AM Michael McCandless
> >  wrote:
> > >
> > > A2, A1, C5, D (binding)
> > >
> > > Thank you to everyone for working so hard to make such cool looking
> possible future Lucene logos!  And to Ryan for the challenging job of
> calling this VOTE :)
> > >
> > > Mike McCandless
> > >
> > > http://blog.mikemccandless.com
> > >
> > >
> > > On Tue, Sep 1, 2020 at 4:21 PM Ryan Ernst  wrote:
> > >>
> > >> Dear Lucene and Solr developers!
> > >>
> > >> Sorry for the multiple threads. This should be the last one.
> > >>
> > >> In February a contest was started to design a new logo for Lucene
> [jira-issue]. The initial attempt [first-vote] to call a vote resulted in
> some confusion on the rules, as well the request for one additional
> submission. The second attempt [second-vote] yesterday had incorrect links
> for one of the submissions. I would like to call a new vote, now with more
> explicit instructions on how to vote, and corrected links.
> > >>
> > >> Please read the following rules carefully before submitting your vote.
> > >>
> > >> Who can vote?
> > >>
> > >> Anyone is welcome to cast a vote in support of their favorite
> submission(s). Note that only PMC member's votes are binding. If you are a
> PMC member, please indicate with your vote that the vote is binding, to
> ease collection of votes. In tallying the votes, I will attempt to verify
> only those marked as binding.
> > >>
> > >> How do I vote?
> > >>
> > >> Votes can be cast simply by replying to this email. It is a
> ranked-choice vote [rank-choice-voting]. Multiple selections may be made,
> where the order of preference must be specified. If an entry gets more than
> half the votes, it is the winner. Otherwise, the entry with the lowest
> number of votes is removed, and the votes are retallied, taking into
> account the next preferred entry for those whose first entry was removed.
> This process repeats until there is a winner.
> > >>
> > >> The entries are broken up by variants, since some entries have
> multiple color or style variations. The entry identifiers are first a
> capital letter, followed by a variation id (described with each entry
> below), if applicable. As an example, if you prefer variant 1 of entry A,
> followed by variant 2 of entry A, variant 3 of entry C, entry D, and lastly
> variant 4e of entry B, the following should be in your reply:
> > >>
> > >> (binding)
> > >> vote: A1, A2, C3, D, B4e
> > >>
> > >> Entries
> > >>
> > >> The entries are as follows:
> > >>
> > >> A. Submitted by Dustin Haver. This entry has two variants, A1 and A2.
> > >>
> > >> [A1]
> https://issues.apache.org/jira/secure/attachment/12999548/Screen%20Shot%202020-04-10%20at%208.29.32%20AM.png
> > >> [A2]
> https://issues.apache.org/jira/secure/attachment/12997172/LuceneLogo.png
> > >>
> > >> B. Submitted by Stamatis Zampetakis. This has several variants.
> Within the linked entry there are 7 patterns and 7 color palettes. Any vote
> for B should contain the pattern number followed by the lowercase letter of
> the color palette. For example, B3e or B1a.
> > >>
> > >> [B]
> https://issues.apache.org/jira/secure/attachment/12997768/zabetak-1-7.pdf
> > >>
> > >> C. Submitted by Baris Kazar. This entry has 8 variants.
> > >>
> > >> [C1]
> https://issues.apache.org/jira/secure/attachment/13006392/lucene_logo1_full.pdf
> > >> [C2]
> https://issues.apache.org/jira/secure/attachment/13006393/lucene_logo2_full.pdf
> > >> [C3]
> https://issues.apache.org/jira/secure/attachment/13006394/lucene_logo3_full.pdf
> > >> [C4]
> https://issues.apache.org/jira/secure/attachment/13006395/lucene_logo4_full.pdf
> > >> [C5]
> https://issues.apache.org/jira/secure/attachment/13006396/lucene_logo5_full.pdf
> > >> [C6]
> https://issues.apache.org/jira/secure/attachment/13006397/lucene_logo6_full.pdf
> > >> [C7]
> https://issues.apache.org/jira/secure/attachment/13006398/lucene_logo7_full.pdf
> > >> [C8]
> https://issues.apache.org/jira/secure/attachment/13006399/lucene_logo8_full.pdf
> > >>
> > >> D. The current Lucene logo.
> > >>
> > >> [D]
> https://lucene.apache.org/theme/images/lucene/lucene_logo_green_300.png
> > >>
> > >> Please vote for one of the above choices. This vote will close about
> one week from today, Mon, Sept 7, 2020 at 11:59PM.
> > >>
> > >> Thanks!
> > >>
> > >> [jira-issue] https://issues.apache.org/jira/browse/LUCENE-9221
> > >> [first-vote]
> http://mail-archives.apache.org/mod_mbox/lucene-dev/202006.mbox/%3cCA+DiXd74Mz4H6o9SmUNLUuHQc6Q1-9mzUR7xfxR03ntGwo=d...@mail.gmail.com%3e
> > >> [second-vote]
> http://mail-archives.apache.org/mod_mbox/lucene-dev/202009.mbox/%3cCA+DiXd7eBrQu5+aJQ3jKaUtUTJUqaG2U6o+kUZfNe-m=smn...@mail.gmail.com%3e
> > >> [rank-choice-voting]
> https://en.wikipedia.org/wiki/Instant-runoff_voting
> >
> > 

Re: Solr configuration options

[Tomás Fernández Löbbe]

Thanks Ishan,
I still don't think it covers the cases very well. The possibilities of how
that handler could be screwing up things are infinite (it could be
corrupting local cores, causing OOMs, it could be spawning infinite loops,
you name it). If the new handler requires initialization that reaches out
to external system, having a large enough cluster means this can hit
throttling or even take down something if you start them all atomically.
I'm fine with Solr supporting atomic deployments with packages and such,
but I'm not fine with that being the only way to deploy Solr, it may not be
suitable for all use cases.

Also, your workaround requires a ton of knowledge of Solr APIs and
internals, vs a simpler and more standard approach where there are two
versions (Docker images, AMIs, tars, whatever you use): old and new.  Add
"new" and remove "old" in your preferred way. This is exactly the same
you'll do when you need to upgrade Solr BTW, so it needs to be handled
anyways.

On Thu, Sep 3, 2020 at 11:35 AM Erick Erickson 
wrote:

> Hmmm, interesting point about deliberately changing one solr.xml for
> testing purposes. To emulate that on a per-node basis you’d have to have
> something like a “node props” associated with each node, which my instant
> reaction to is “y”.
>
> As far as API only, I’d assumed changes to clusterprops could be either
> way. If we allow Solr to start with no clusterprops, then the API route
> would create one. Pros can go ahead and hand-edit one and push it up if
> they want.
>
> In your nightmare scenario, where are the ZK’s located? Are they still
> running somewhere? Could you hand-edit clusterprops and push it to ZK?
>
> I wish everyone would just use Solr the way I think about it ;)
>
> > On Sep 3, 2020, at 2:11 PM, Tomás Fernández Löbbe 
> wrote:
> >
> > I can see that some of these configurations should be moved to
> clusterporps.json, I don’t believe this is the case for all of them. Some
> are configurations that are targeting the local node (i.e sharedLib path),
> some are needed before connecting to ZooKeeper (zk config). Configuration
> of global handlers and components, while in general you do want to see the
> same conf across all nodes, you may not want the changes to reflect
> atomically and instead rely on a phased upgrade (rolling, blue/green, etc),
> where the conf goes together with the binaries that are being deployed. I
> also fear that making the configuration of some of these components dynamic
> means we have to make the code handle them dynamically (i.e. recreate the
> CollectionsHandler based on callback from ZooKeeper). This would be very
> hardly used in reality, but all our code needs to be restructured to handle
> this, I fear this will complicate the code needlessly, and may introduce
> leaks and races of all kinds. If those components can have configuration
> that should be dynamic (some toggle, threshold, etc), I’d love to see those
> as clusterporps, key-value mostly.
> >
> > If we were to put this configuration in clusterprops, would that mean
> that I’m only able to do config changes via API? On a new cluster, do I
> need to start Solr, make a collections API call to change the collections
> handler? Or am I supposed to manually change the clusterporps file before
> starting Solr and push it to Zookeeper (having a file intended for manual
> edits and API edits is bad IMO)? Maybe via the cli, but still, I’d need to
> do this for every cluster I create (vs have the solr.xml in my source
> repository and Docker image, for example). Also I lose the ability to have
> this configuration in my git repo?
> >
> > I'm +1 to keep a node configuration local to the node in the filesystem.
> Currently, it's solr.xml. I've seen comments about xml difficult to
> read/write, I think that's personal preference so, while I don't see it
> that way, I understand lots of people do and things have been moving away
> to other formats, I'm open to discuss that as a change.
> >
> > > However, 1, 2, and 3, are not trivial for a large number of Solr nodes
> and if they aren’t right diagnosing them can be “challenging”…
> > In my mind, solr.xml goes with your code. Having it up to date means
> having all your nodes running the same version of your code. As I said,
> this is the "desired state" of the cluster, but may not be the case all the
> time (i.e. during deployments), and that's fine. Depending on how you
> manage the cluster, you may want to live with different versions for some
> time (you may have canaries or be doing a blue/green deployment, etc).
> Realistically speaking, if you have a 500+ node cluster, you must have a
> system in place to manage configuration and versions, let's not try to bend
> backwards for a situation that isn't that realistic.
> >
> > Let me put an example of things I fear with making these changes atomic.
> Let's say I want to start using a new, custom HealthCheckHandler
> implementation, that I have put in a jar (and let's 

Re: Solr configuration options

[Ishan Chattopadhyaya]

On Fri, 4 Sep, 2020, 12:05 am Erick Erickson, 
wrote:

>
>
> I wish everyone would just use Solr the way I think about it ;)
>

https://twitter.com/ichattopadhyaya/status/1210868171814473728


> > On Sep 3, 2020, at 2:11 PM, Tomás Fernández Löbbe 
> wrote:
> >
> > I can see that some of these configurations should be moved to
> clusterporps.json, I don’t believe this is the case for all of them. Some
> are configurations that are targeting the local node (i.e sharedLib path),
> some are needed before connecting to ZooKeeper (zk config). Configuration
> of global handlers and components, while in general you do want to see the
> same conf across all nodes, you may not want the changes to reflect
> atomically and instead rely on a phased upgrade (rolling, blue/green, etc),
> where the conf goes together with the binaries that are being deployed. I
> also fear that making the configuration of some of these components dynamic
> means we have to make the code handle them dynamically (i.e. recreate the
> CollectionsHandler based on callback from ZooKeeper). This would be very
> hardly used in reality, but all our code needs to be restructured to handle
> this, I fear this will complicate the code needlessly, and may introduce
> leaks and races of all kinds. If those components can have configuration
> that should be dynamic (some toggle, threshold, etc), I’d love to see those
> as clusterporps, key-value mostly.
> >
> > If we were to put this configuration in clusterprops, would that mean
> that I’m only able to do config changes via API? On a new cluster, do I
> need to start Solr, make a collections API call to change the collections
> handler? Or am I supposed to manually change the clusterporps file before
> starting Solr and push it to Zookeeper (having a file intended for manual
> edits and API edits is bad IMO)? Maybe via the cli, but still, I’d need to
> do this for every cluster I create (vs have the solr.xml in my source
> repository and Docker image, for example). Also I lose the ability to have
> this configuration in my git repo?
> >
> > I'm +1 to keep a node configuration local to the node in the filesystem.
> Currently, it's solr.xml. I've seen comments about xml difficult to
> read/write, I think that's personal preference so, while I don't see it
> that way, I understand lots of people do and things have been moving away
> to other formats, I'm open to discuss that as a change.
> >
> > > However, 1, 2, and 3, are not trivial for a large number of Solr nodes
> and if they aren’t right diagnosing them can be “challenging”…
> > In my mind, solr.xml goes with your code. Having it up to date means
> having all your nodes running the same version of your code. As I said,
> this is the "desired state" of the cluster, but may not be the case all the
> time (i.e. during deployments), and that's fine. Depending on how you
> manage the cluster, you may want to live with different versions for some
> time (you may have canaries or be doing a blue/green deployment, etc).
> Realistically speaking, if you have a 500+ node cluster, you must have a
> system in place to manage configuration and versions, let's not try to bend
> backwards for a situation that isn't that realistic.
> >
> > Let me put an example of things I fear with making these changes atomic.
> Let's say I want to start using a new, custom HealthCheckHandler
> implementation, that I have put in a jar (and let's assume the jar is
> already in all nodes). If I use solr.xml (where one can currently
> configures this implementation), I can do a phased deployment (yes, this is
> a restart of all nodes), if the healthcheck handler is buggy and fails
> request, the nodes with the new code will never show as healthy, so the
> deployment will likely stop (i.e. if you are using Kubernetes and using
> probes, those instances will keep restarting, if you use ASG in AWS you can
> do the same thing). If you make it an atomic change, bye-bye cluster, all
> nodes will start reporting unhealthy (Kubernetes and ASG will kill all
> those nodes). Good luck doing API changes to revert now, there is no node
> to respond to those requests. Hopefully you were using some sort of stable
> storage because all ephemeral is gone. Bringing back that cluster is going
> to be a PITA. I have seen similar things happen.
> >
> >
> > On Thu, Sep 3, 2020 at 9:40 AM Erick Erickson 
> wrote:
> > bq.  Isn’t solr.xml is a way to hardcode config in a more flexible way
> that a Java class?
> >
> > Yes, and the problem word here is “flexible”. For a single-node system
> that flexibility is desirable. Flexibility comes at the cost of complexity,
> especially in the SolrCloud case. In this case, not so much Solr code
> complexity as operations complexity.
> >
> > For me this isn’t so much a question of functionality as
> administration/troubleshooting/barrier to entry.
> >
> > If:
> > 1. you can guarantee that every solr.xml file on every node in your
> entire 500 node cluster is up to date

Re: Solr configuration options

[Erick Erickson]

Hmmm, interesting point about deliberately changing one solr.xml for testing 
purposes. To emulate that on a per-node basis you’d have to have something like 
a “node props” associated with each node, which my instant reaction to is 
“y”.

As far as API only, I’d assumed changes to clusterprops could be either way. If 
we allow Solr to start with no clusterprops, then the API route would create 
one. Pros can go ahead and hand-edit one and push it up if they want.

In your nightmare scenario, where are the ZK’s located? Are they still running 
somewhere? Could you hand-edit clusterprops and push it to ZK?

I wish everyone would just use Solr the way I think about it ;)

> On Sep 3, 2020, at 2:11 PM, Tomás Fernández Löbbe  
> wrote:
> 
> I can see that some of these configurations should be moved to 
> clusterporps.json, I don’t believe this is the case for all of them. Some are 
> configurations that are targeting the local node (i.e sharedLib path), some 
> are needed before connecting to ZooKeeper (zk config). Configuration of 
> global handlers and components, while in general you do want to see the same 
> conf across all nodes, you may not want the changes to reflect atomically and 
> instead rely on a phased upgrade (rolling, blue/green, etc), where the conf 
> goes together with the binaries that are being deployed. I also fear that 
> making the configuration of some of these components dynamic means we have to 
> make the code handle them dynamically (i.e. recreate the CollectionsHandler 
> based on callback from ZooKeeper). This would be very hardly used in reality, 
> but all our code needs to be restructured to handle this, I fear this will 
> complicate the code needlessly, and may introduce leaks and races of all 
> kinds. If those components can have configuration that should be dynamic 
> (some toggle, threshold, etc), I’d love to see those as clusterporps, 
> key-value mostly. 
> 
> If we were to put this configuration in clusterprops, would that mean that 
> I’m only able to do config changes via API? On a new cluster, do I need to 
> start Solr, make a collections API call to change the collections handler? Or 
> am I supposed to manually change the clusterporps file before starting Solr 
> and push it to Zookeeper (having a file intended for manual edits and API 
> edits is bad IMO)? Maybe via the cli, but still, I’d need to do this for 
> every cluster I create (vs have the solr.xml in my source repository and 
> Docker image, for example). Also I lose the ability to have this 
> configuration in my git repo?
> 
> I'm +1 to keep a node configuration local to the node in the filesystem. 
> Currently, it's solr.xml. I've seen comments about xml difficult to 
> read/write, I think that's personal preference so, while I don't see it that 
> way, I understand lots of people do and things have been moving away to other 
> formats, I'm open to discuss that as a change.
> 
> > However, 1, 2, and 3, are not trivial for a large number of Solr nodes and 
> > if they aren’t right diagnosing them can be “challenging”…
> In my mind, solr.xml goes with your code. Having it up to date means having 
> all your nodes running the same version of your code. As I said, this is the 
> "desired state" of the cluster, but may not be the case all the time (i.e. 
> during deployments), and that's fine. Depending on how you manage the 
> cluster, you may want to live with different versions for some time (you may 
> have canaries or be doing a blue/green deployment, etc). Realistically 
> speaking, if you have a 500+ node cluster, you must have a system in place to 
> manage configuration and versions, let's not try to bend backwards for a 
> situation that isn't that realistic.
> 
> Let me put an example of things I fear with making these changes atomic. 
> Let's say I want to start using a new, custom HealthCheckHandler 
> implementation, that I have put in a jar (and let's assume the jar is already 
> in all nodes). If I use solr.xml (where one can currently configures this 
> implementation), I can do a phased deployment (yes, this is a restart of all 
> nodes), if the healthcheck handler is buggy and fails request, the nodes with 
> the new code will never show as healthy, so the deployment will likely stop 
> (i.e. if you are using Kubernetes and using probes, those instances will keep 
> restarting, if you use ASG in AWS you can do the same thing). If you make it 
> an atomic change, bye-bye cluster, all nodes will start reporting unhealthy 
> (Kubernetes and ASG will kill all those nodes). Good luck doing API changes 
> to revert now, there is no node to respond to those requests. Hopefully you 
> were using some sort of stable storage because all ephemeral is gone. 
> Bringing back that cluster is going to be a PITA. I have seen similar things 
> happen.
> 
> 
> On Thu, Sep 3, 2020 at 9:40 AM Erick Erickson  wrote:
> bq.  Isn’t solr.xml is a way to hardcode config in a more flexible way 

Re: Solr configuration options

[Ishan Chattopadhyaya]

Hi Tomas,
This type of a problem can be solved using alternate strategies. Here's how
you can do so: register the updated version of the plugin in /healthcheck2
(while simultaneously the older version continues to work at /healthcheck).
Make sure it works. Once it does, update the /healthcheck endpoint to use
the latest version and unregister the /healthcheck2. Same can be done with
other types of plugins. This is totally supported using the package manager
today. I apologize for not documenting all such wonderful things about the
package manager in details, but I shall attempt to do so with examples
shortly so that you can consider adopting such approaches without resorting
to solr.xml (which should go away for SolrCloud due to all the issues Erick
mentioned).
Regards,
Ishan

On Thu, Sep 3, 2020 at 11:41 PM Tomás Fernández Löbbe 
wrote:

> I can see that some of these configurations should be moved to
> clusterporps.json, I don’t believe this is the case for all of them. Some
> are configurations that are targeting the local node (i.e sharedLib path),
> some are needed before connecting to ZooKeeper (zk config). Configuration
> of global handlers and components, while in general you do want to see the
> same conf across all nodes, you may not want the changes to reflect
> atomically and instead rely on a phased upgrade (rolling, blue/green, etc),
> where the conf goes together with the binaries that are being deployed. I
> also fear that making the configuration of some of these components dynamic
> means we have to make the code handle them dynamically (i.e. recreate the
> CollectionsHandler based on callback from ZooKeeper). This would be very
> hardly used in reality, but all our code needs to be restructured to handle
> this, I fear this will complicate the code needlessly, and may introduce
> leaks and races of all kinds. If those components can have configuration
> that should be dynamic (some toggle, threshold, etc), I’d love to see those
> as clusterporps, key-value mostly.
>
> If we were to put this configuration in clusterprops, would that mean that
> I’m only able to do config changes via API? On a new cluster, do I need to
> start Solr, make a collections API call to change the collections handler?
> Or am I supposed to manually change the clusterporps file before starting
> Solr and push it to Zookeeper (having a file intended for manual edits and
> API edits is bad IMO)? Maybe via the cli, but still, I’d need to do this
> for every cluster I create (vs have the solr.xml in my source repository
> and Docker image, for example). Also I lose the ability to have this
> configuration in my git repo?
>
> I'm +1 to keep a node configuration local to the node in the filesystem.
> Currently, it's solr.xml. I've seen comments about xml difficult to
> read/write, I think that's personal preference so, while I don't see it
> that way, I understand lots of people do and things have been moving away
> to other formats, I'm open to discuss that as a change.
>
> > However, 1, 2, and 3, are not trivial for a large number of Solr nodes
> and if they aren’t right diagnosing them can be “challenging”…
> In my mind, solr.xml goes with your code. Having it up to date means
> having all your nodes running the same version of your code. As I said,
> this is the "desired state" of the cluster, but may not be the case all the
> time (i.e. during deployments), and that's fine. Depending on how you
> manage the cluster, you may want to live with different versions for some
> time (you may have canaries or be doing a blue/green deployment, etc).
> Realistically speaking, if you have a 500+ node cluster, you must have a
> system in place to manage configuration and versions, let's not try to bend
> backwards for a situation that isn't that realistic.
>
> Let me put an example of things I fear with making these changes atomic.
> Let's say I want to start using a new, custom HealthCheckHandler
> implementation, that I have put in a jar (and let's assume the jar is
> already in all nodes). If I use solr.xml (where one can currently
> configures this implementation), I can do a phased deployment (yes, this is
> a restart of all nodes), if the healthcheck handler is buggy and fails
> request, the nodes with the new code will never show as healthy, so the
> deployment will likely stop (i.e. if you are using Kubernetes and using
> probes, those instances will keep restarting, if you use ASG in AWS you can
> do the same thing). If you make it an atomic change, bye-bye cluster, all
> nodes will start reporting unhealthy (Kubernetes and ASG will kill all
> those nodes). Good luck doing API changes to revert now, there is no node
> to respond to those requests. Hopefully you were using some sort of stable
> storage because all ephemeral is gone. Bringing back that cluster is going
> to be a PITA. I have seen similar things happen.
>
>
> On Thu, Sep 3, 2020 at 9:40 AM Erick Erickson 
> wrote:
>
>> bq.  Isn’t solr.xml is a way to 

Re: Solr configuration options

[Tomás Fernández Löbbe]

I can see that some of these configurations should be moved to
clusterporps.json, I don’t believe this is the case for all of them. Some
are configurations that are targeting the local node (i.e sharedLib path),
some are needed before connecting to ZooKeeper (zk config). Configuration
of global handlers and components, while in general you do want to see the
same conf across all nodes, you may not want the changes to reflect
atomically and instead rely on a phased upgrade (rolling, blue/green, etc),
where the conf goes together with the binaries that are being deployed. I
also fear that making the configuration of some of these components dynamic
means we have to make the code handle them dynamically (i.e. recreate the
CollectionsHandler based on callback from ZooKeeper). This would be very
hardly used in reality, but all our code needs to be restructured to handle
this, I fear this will complicate the code needlessly, and may introduce
leaks and races of all kinds. If those components can have configuration
that should be dynamic (some toggle, threshold, etc), I’d love to see those
as clusterporps, key-value mostly.

If we were to put this configuration in clusterprops, would that mean that
I’m only able to do config changes via API? On a new cluster, do I need to
start Solr, make a collections API call to change the collections handler?
Or am I supposed to manually change the clusterporps file before starting
Solr and push it to Zookeeper (having a file intended for manual edits and
API edits is bad IMO)? Maybe via the cli, but still, I’d need to do this
for every cluster I create (vs have the solr.xml in my source repository
and Docker image, for example). Also I lose the ability to have this
configuration in my git repo?

I'm +1 to keep a node configuration local to the node in the filesystem.
Currently, it's solr.xml. I've seen comments about xml difficult to
read/write, I think that's personal preference so, while I don't see it
that way, I understand lots of people do and things have been moving away
to other formats, I'm open to discuss that as a change.

> However, 1, 2, and 3, are not trivial for a large number of Solr nodes
and if they aren’t right diagnosing them can be “challenging”…
In my mind, solr.xml goes with your code. Having it up to date means having
all your nodes running the same version of your code. As I said, this is
the "desired state" of the cluster, but may not be the case all the time
(i.e. during deployments), and that's fine. Depending on how you manage the
cluster, you may want to live with different versions for some time (you
may have canaries or be doing a blue/green deployment, etc). Realistically
speaking, if you have a 500+ node cluster, you must have a system in place
to manage configuration and versions, let's not try to bend backwards for a
situation that isn't that realistic.

Let me put an example of things I fear with making these changes atomic.
Let's say I want to start using a new, custom HealthCheckHandler
implementation, that I have put in a jar (and let's assume the jar is
already in all nodes). If I use solr.xml (where one can currently
configures this implementation), I can do a phased deployment (yes, this is
a restart of all nodes), if the healthcheck handler is buggy and fails
request, the nodes with the new code will never show as healthy, so the
deployment will likely stop (i.e. if you are using Kubernetes and using
probes, those instances will keep restarting, if you use ASG in AWS you can
do the same thing). If you make it an atomic change, bye-bye cluster, all
nodes will start reporting unhealthy (Kubernetes and ASG will kill all
those nodes). Good luck doing API changes to revert now, there is no node
to respond to those requests. Hopefully you were using some sort of stable
storage because all ephemeral is gone. Bringing back that cluster is going
to be a PITA. I have seen similar things happen.


On Thu, Sep 3, 2020 at 9:40 AM Erick Erickson 
wrote:

> bq.  Isn’t solr.xml is a way to hardcode config in a more flexible way
> that a Java class?
>
> Yes, and the problem word here is “flexible”. For a single-node system
> that flexibility is desirable. Flexibility comes at the cost of complexity,
> especially in the SolrCloud case. In this case, not so much Solr code
> complexity as operations complexity.
>
> For me this isn’t so much a question of functionality as
> administration/troubleshooting/barrier to entry.
>
> If:
> 1. you can guarantee that every solr.xml file on every node in your entire
> 500 node cluster is up to date
> 2. or you can guarantee that the solr.xml stored on Zookeeper
> 3. and you can guarantee that clusterprops.json in cloud mode is
> interacting properly with whichever solr.xml is read
> 4. Then I’d have no problem with solr.xml.
>
> However, 1, 2, and 3, are not trivial for a large number of Solr nodes and
> if they aren’t right diagnosing them can be “challenging”…
>
> Imagine all the ways that “somehow” the solr.xml 

Re: Solr configuration options

[Erick Erickson]

bq.  Isn’t solr.xml is a way to hardcode config in a more flexible way that a 
Java class?

Yes, and the problem word here is “flexible”. For a single-node system that 
flexibility is desirable. Flexibility comes at the cost of complexity, 
especially in the SolrCloud case. In this case, not so much Solr code 
complexity as operations complexity.

For me this isn’t so much a question of functionality as 
administration/troubleshooting/barrier to entry.

If:
1. you can guarantee that every solr.xml file on every node in your entire 500 
node cluster is up to date
2. or you can guarantee that the solr.xml stored on Zookeeper
3. and you can guarantee that clusterprops.json in cloud mode is interacting 
properly with whichever solr.xml is read
4. Then I’d have no problem with solr.xml.

However, 1, 2, and 3, are not trivial for a large number of Solr nodes and if 
they aren’t right diagnosing them can be “challenging”…

Imagine all the ways that “somehow” the solr.xml file on one node or more nodes 
of a 500 node cluster didn’t get updated and you’re trying to track down why 
query X isn’t working as you expect. Some of the time. When you happen to hit 
conditions X, Y and Z on a subrequest that goes to the node in question (which 
won’t be all of the time, or even possibly a significant fraction of the time). 
Do Containers matter here? Some glitch in Puppet or similar? Somebody didn’t 
follow every step in the process in the playbook? It doesn’t matter how you got 
into this situation, tracking it down would be a nightmare.

Or, for that matter, you’ve solved all the distribution concerns and _can_ 
guarantee 1 and 3. Then somebody pushes a solr.xml to ZK either intentionally 
or by mistake (OH, I thought I was on the QA system, oops). Now I get to spend 
a week tracking down why the guarantee of 1 is still true, it’s just not 
relevant any more.

To me, it’s the same problem that is solved by the blob store for jar files, or 
having configsets in ZK. When I want something available to all my Solr 
instances, I do not want to have to run around to every node and determine that 
the object I copied there is the right one, especially if I’m trying to track 
down a problem.

Sure, all my concerns can be solved, but why make it harder than it needs to 
be? Distributed systems are hard enough already…

FWIW,
Erick




> On Sep 3, 2020, at 11:00 AM, Ilan Ginzburg  wrote:
> 
>  Isn’t solr.xml is a way to hardcode config in a more flexible way that a 
> Java class?


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Migrate Solr helm chart into apache/lucene-solr repo

[Christine Poerschke (BLOOMBERG/ LONDON)]

I'd suggest Apache Community Development as a potential place for such a 
cross-project discussion:

http://community.apache.org

https://lists.apache.org/list.html?d...@community.apache.org

https://issues.apache.org/jira/projects/COMDEV

From: dev@lucene.apache.org At: 09/03/20 15:46:52To:  dev@lucene.apache.org
Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo

The CouchDB example is a good one -- it is not a part of the core repository.

It might be a wide audience, but I would start a discussion at
committ...@apache.org for this.

Atri

On Thu, Sep 3, 2020 at 8:05 PM LEE Ween Jiann
 wrote:
>
> No, where and who would be the right place and person to get this discussion 
going?
>
>
>
> Some charts have already begun moving like 
https://github.com/apache/couchdb-helm, whereas others like Airflow, Hadoop, 
Spark, etc. (+ others that I might have missed) have yet to move.
>
>
>
> Other than that, I think zookeeper is being maintained by Bitnami.
>
>
>
>
>
> From: Atri Sharma 
> Reply to: "dev@lucene.apache.org" 
> Date: Thursday, 3 September 2020 at 10:17 PM
> To: "dev@lucene.apache.org" 
> Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo
>
>
>
> Is this a consensus between all the Apache projects? Is there a discussion 
going on?
>
>
>
> On Thu, 3 Sep 2020 at 19:43, LEE Ween Jiann  
wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> The ideal case would be to have Apache/charts with all the helm charts that 
are orphaned.
>
>
>
> If not I guess, we will just wait and see if anyone picks up those charts.
>
>
>
>
>
>
>
>
>
>
>
>
>
> From: Atri Sharma 
>
>
> Reply to: "dev@lucene.apache.org" 
>
>
> Date: Thursday, 3 September 2020 at 10:08 PM
>
>
> To: "dev@lucene.apache.org" 
>
>
> Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> I still don’t see this as a reason to add it to the main repo. Maybe create a 
new repo just for this helm chart?
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Anyways, as David said, the third party plug-in repository will be perfect 
for this.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, 3 Sep 2020 at 19:36, LEE Ween Jiann  
wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hey there,
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> The thing is there will **not** be repository after Nov 13.
>
>
>
>
>
>
>
> Please refer to this link which was previously posted,
>
>
>
>
>
>
>
>
>
>
>
> https://github.com/helm/charts/blob/master/README.md#deprecation-timeline
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Cheers,
>
>
>
>
>
>
>
> WJ
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> From:
>
> Atri Sharma 
>
>
>
>
>
>
>
>
> Reply to: "dev@lucene.apache.org" 
>
>
>
>
>
>
>
>
> Date: Thursday, 3 September 2020 at 10:02 PM
>
>
>
>
>
>
>
>
> To: "dev@lucene.apache.org" 
>
>
>
>
>
>
>
>
> Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Jumping in late, but I don’t see why the help chart needs to be a part of the 
core repository.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Like the HDFS repository, why not just add a link to the existing repository 
somewhere on the main website?
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, 3 Sep 2020 at 19:29, LEE Ween Jiann  
wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hi both,
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Is there any updates on this? The helm stable chart repo is now deprecated, 
could we move the helm chart to the apache GitHub repo?
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Cheers,
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> WJ
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> From:
>
>
>
>
>
> David Smiley 
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Reply to: "dev@lucene.apache.org" 
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Date: Monday, 3 February 2020 at 1:07 AM
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> To: Solr/Lucene Dev 
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> I was looking for a "Why" Helm is ending its registry and didn't find it, 
though I did find the 2x vs 3x thing so I thought it was about version 
migration more than anything.  I'm wondering
>
>
>
>
>
> "Why" because this very project, Solr, 

Re: Migrate Solr helm chart into apache/lucene-solr repo

[Ishan Chattopadhyaya]

Hi Lee, Atri,

I'd like to volunteer to maintaining Solr Helm charts in a public GitHub
repository outside of Apache, unless we want to do so as part of
Lucene-Solr or elsewhere in Apache.

Regards,
Ishan

On Thu, 3 Sep, 2020, 8:16 pm Atri Sharma,  wrote:

> The CouchDB example is a good one -- it is not a part of the core
> repository.
>
> It might be a wide audience, but I would start a discussion at
> committ...@apache.org for this.
>
> Atri
>
> On Thu, Sep 3, 2020 at 8:05 PM LEE Ween Jiann
>  wrote:
> >
> > No, where and who would be the right place and person to get this
> discussion going?
> >
> >
> >
> > Some charts have already begun moving like
> https://github.com/apache/couchdb-helm, whereas others like Airflow,
> Hadoop, Spark, etc. (+ others that I might have missed) have yet to move.
> >
> >
> >
> > Other than that, I think zookeeper is being maintained by Bitnami.
> >
> >
> >
> >
> >
> > From: Atri Sharma 
> > Reply to: "dev@lucene.apache.org" 
> > Date: Thursday, 3 September 2020 at 10:17 PM
> > To: "dev@lucene.apache.org" 
> > Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo
> >
> >
> >
> > Is this a consensus between all the Apache projects? Is there a
> discussion going on?
> >
> >
> >
> > On Thu, 3 Sep 2020 at 19:43, LEE Ween Jiann 
> wrote:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > The ideal case would be to have Apache/charts with all the helm charts
> that are orphaned.
> >
> >
> >
> > If not I guess, we will just wait and see if anyone picks up those
> charts.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > From: Atri Sharma 
> >
> >
> > Reply to: "dev@lucene.apache.org" 
> >
> >
> > Date: Thursday, 3 September 2020 at 10:08 PM
> >
> >
> > To: "dev@lucene.apache.org" 
> >
> >
> > Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > I still don’t see this as a reason to add it to the main repo. Maybe
> create a new repo just for this helm chart?
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Anyways, as David said, the third party plug-in repository will be
> perfect for this.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Thu, 3 Sep 2020 at 19:36, LEE Ween Jiann 
> wrote:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Hey there,
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > The thing is there will **not** be repository after Nov 13.
> >
> >
> >
> >
> >
> >
> >
> > Please refer to this link which was previously posted,
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> https://github.com/helm/charts/blob/master/README.md#deprecation-timeline
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Cheers,
> >
> >
> >
> >
> >
> >
> >
> > WJ
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > From:
> >
> > Atri Sharma 
> >
> >
> >
> >
> >
> >
> >
> >
> > Reply to: "dev@lucene.apache.org" 
> >
> >
> >
> >
> >
> >
> >
> >
> > Date: Thursday, 3 September 2020 at 10:02 PM
> >
> >
> >
> >
> >
> >
> >
> >
> > To: "dev@lucene.apache.org" 
> >
> >
> >
> >
> >
> >
> >
> >
> > Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Jumping in late, but I don’t see why the help chart needs to be a part
> of the core repository.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Like the HDFS repository, why not just add a link to the existing
> repository somewhere on the main website?
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Thu, 3 Sep 2020 at 19:29, LEE Ween Jiann 
> wrote:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Hi both,
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Is there any updates on this? The helm stable chart repo is now
> deprecated, could we move 

Re: Solr configuration options

[Ilan Ginzburg]

Everything can of course be hard coded. Isn’t solr.xml is a way to hardcode
config in a more flexible way that a Java class?

I hoped to have the config passed to the plugin be overridable via system
properties like solr.xml allows (free disk size thresholds, log vs. fail
when some constraints can’t be honored etc). This will not happen,
overrides will instead be persisted in clusterprops.json.

Is there structured storage in clusterprops.json to define configuration
values to be passed to the plugin?

Note that given the interfaces defined for the plugins, the choice of where
config is stored and read from is isolated from how the plugin consumes it!
The changes being discussed are limited to the implementation classes not
seen by the plugin writer. This wouldn’t be the case if the plugin just
reached into existing configuration abstractions in Solr code.

Ilan

On Thu 3 Sep 2020 at 15:10, Erick Erickson  wrote:

> I’d also like solr.xml to go away if at all possible, at least in cloud
> mode.
>
>
>
> Some important points (to me at least).
>
>
>
> - having to hand-edit _anything_ is A Bad Thing in a large, distributed
> system since you need:
>
>
>
> — an infrastructure to automagically distribute whatever it is you’re
> editing to an arbitrary-but-possibly-very-large number of nodes
>
>
>
> — a process to verify your edits do what you expect (and whether you’re
> hand-editing JSON or XML is immaterial, you can still screw up)
>
>
>
> — a process to refresh those values on running Solr instances
> (cluster-wide restart? Yuck). Although even with everything in
> clusterprops.json I can still imagine some kind of restart would be
> required for some changes...
>
>
>
> These are non-trivial to implement and highly non trivial to debug when
> the process doesn’t work, again in a system with 100s of nodes.
>
>
>
> Also, there’s no chance to sanity check the changes and reject invalid
> ones, say a typo in a property name or specifying an unloadable class or...
>
>
>
> In standalone mode, IDK. Replacing solr.xml with clusterprops.json is
> confusing as well, although it’s explainable at least.
>
> That said, I think it might be easier to explain that:
>
>
>
> “in standalone mode, certain defaults are kept in solr.xml which is read
> locally from disk. In cloud mode, solr.xml is replaced by clusterprops.json
> on ZK”.
>
>
>
> To me, that’s certainly less confusing than
>
>
>
> “in standalone mode, certain defaults are kept in solr.xml. In cloud mode,
> those defaults may kept in solr.xml on ZK. If that znode is absent,
> solr.xml is loaded from disk. In either case, clusterprops.json will
> override anything in solr.xml. Oh, and if you’re running in cloud mode and
> haven’t put solr.xml on ZK, good luck keeping them all coordinated.”.
>
>
>
> The upgrade path does suck though. If I’m going from Solr X to Solr X+1
> where X supports solr.xml and X+1 does not what happens in cloud mode? We’d
> have to do something like detect that situation and rewrite solr.xml to
> clusterprops.xml. Or provide a command/utility that does that under user
> control that’d have to be run first. Or…. Coordinating automatic rewrite
> from solr.xml->clusterprops.json that when simultaneously starting 100 X+1
> instances would be …challenging… And we’d need to respect anything
> _already_ in clusterprops.json.
>
>
>
> Ilan:
>
>
>
> Why do we need to prime Zookeeper? If we hard-code the defaults can’t we
> carry on if cluster.props isn’t there with the hard-coded defaults? Of
> course overwriting those defaults if clusterprops.json _is_ present. The
> fact that changing defaults requires recompilation isn’t a problem IMO.
> Currently if we change them, _we_ change solr.xml and distribute that with
> the new release. This pre-supposes that we can get everything we need early
> enough in the process that we can look at ZK in the first place. If we need
> to read clusterprops before talking to ZK, well, that’s a problem that
> maybe we’d have to solve with a system var or something.
>
>
>
> Of course that’s my 10,000 ft. view. The details are always hairy.
>
>
>
> FWIW,
>
> Erick
>
>
>
>
>
>
>
>
>
> > On Sep 3, 2020, at 5:58 AM, Noble Paul  wrote:
>
> >
>
> >> to stick with solr.xml for current and ongoing feature development and
> not hold up development based on things that may or may not happen in the
> future.
>
> >
>
> > Where did I say that? There are so many features built recently using
>
> > clusterprops.json. The least we can do is stop piling more into
>
> > solr.xml. If you cannot make something better the least you can do is
>
> > stop making life more difficult for everyone.
>
> >
>
> >> In order for clusterpropos.json to replace what's currently done with
> solr.xml, we'd need to introduce a mechanism to make configuration
> available out of the box
>
> >
>
> > How hard is it to insert a JSON into your ZK ?
>
> >
>
> > The problem with Solr developers is they only have one workflow in
>
> > their mind, the way they use it 

Re: Migrate Solr helm chart into apache/lucene-solr repo

[Atri Sharma]

The CouchDB example is a good one -- it is not a part of the core repository.

It might be a wide audience, but I would start a discussion at
committ...@apache.org for this.

Atri

On Thu, Sep 3, 2020 at 8:05 PM LEE Ween Jiann
 wrote:
>
> No, where and who would be the right place and person to get this discussion 
> going?
>
>
>
> Some charts have already begun moving like 
> https://github.com/apache/couchdb-helm, whereas others like Airflow, Hadoop, 
> Spark, etc. (+ others that I might have missed) have yet to move.
>
>
>
> Other than that, I think zookeeper is being maintained by Bitnami.
>
>
>
>
>
> From: Atri Sharma 
> Reply to: "dev@lucene.apache.org" 
> Date: Thursday, 3 September 2020 at 10:17 PM
> To: "dev@lucene.apache.org" 
> Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo
>
>
>
> Is this a consensus between all the Apache projects? Is there a discussion 
> going on?
>
>
>
> On Thu, 3 Sep 2020 at 19:43, LEE Ween Jiann  
> wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> The ideal case would be to have Apache/charts with all the helm charts that 
> are orphaned.
>
>
>
> If not I guess, we will just wait and see if anyone picks up those charts.
>
>
>
>
>
>
>
>
>
>
>
>
>
> From: Atri Sharma 
>
>
> Reply to: "dev@lucene.apache.org" 
>
>
> Date: Thursday, 3 September 2020 at 10:08 PM
>
>
> To: "dev@lucene.apache.org" 
>
>
> Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> I still don’t see this as a reason to add it to the main repo. Maybe create a 
> new repo just for this helm chart?
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Anyways, as David said, the third party plug-in repository will be perfect 
> for this.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, 3 Sep 2020 at 19:36, LEE Ween Jiann  
> wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hey there,
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> The thing is there will **not** be repository after Nov 13.
>
>
>
>
>
>
>
> Please refer to this link which was previously posted,
>
>
>
>
>
>
>
>
>
>
>
> https://github.com/helm/charts/blob/master/README.md#deprecation-timeline
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Cheers,
>
>
>
>
>
>
>
> WJ
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> From:
>
> Atri Sharma 
>
>
>
>
>
>
>
>
> Reply to: "dev@lucene.apache.org" 
>
>
>
>
>
>
>
>
> Date: Thursday, 3 September 2020 at 10:02 PM
>
>
>
>
>
>
>
>
> To: "dev@lucene.apache.org" 
>
>
>
>
>
>
>
>
> Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Jumping in late, but I don’t see why the help chart needs to be a part of the 
> core repository.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Like the HDFS repository, why not just add a link to the existing repository 
> somewhere on the main website?
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, 3 Sep 2020 at 19:29, LEE Ween Jiann  
> wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hi both,
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Is there any updates on this? The helm stable chart repo is now deprecated, 
> could we move the helm chart to the apache GitHub repo?
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Cheers,
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> WJ
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> From:
>
>
>
>
>
> David Smiley 
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Reply to: "dev@lucene.apache.org" 
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Date: Monday, 3 February 2020 at 1:07 AM
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> To: Solr/Lucene Dev 
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> I was looking for a "Why" Helm is ending its registry and didn't find it, 
> though I did find the 2x vs 3x thing so I thought it was about version 
> migration more than anything.  I'm wondering
>
>
>
>
>
> "Why" because this very project, Solr, will soon
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> be grappling with how to create a repository of 3rd party plugins.  
> Successful projects like Docker do this so I wonder why Helm seems to be 
> backing out.  Perhaps we will have one main repository of 3rd party plugins 
> we've vetted in some way (have high confidence
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

Re: Migrate Solr helm chart into apache/lucene-solr repo

[Atri Sharma]

Is this a consensus between all the Apache projects? Is there a discussion
going on?

On Thu, 3 Sep 2020 at 19:43, LEE Ween Jiann 
wrote:

>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> The ideal case would be to have Apache/charts with all the helm charts
> that are orphaned.
>
>
> If not I guess, we will just wait and see if anyone picks up those charts.
>
>
>
>
>
>
>
>
>
>
> *From: *Atri Sharma 
>
>
> *Reply to: *"dev@lucene.apache.org" 
>
>
> *Date: *Thursday, 3 September 2020 at 10:08 PM
>
>
> *To: *"dev@lucene.apache.org" 
>
>
> *Subject: *Re: Migrate Solr helm chart into apache/lucene-solr repo
>
>
>
>
>
>
>
>
>
>
>
>
>
> I still don’t see this as a reason to add it to the main repo. Maybe
> create a new repo just for this helm chart?
>
>
>
>
>
>
>
>
>
>
>
>
>
> Anyways, as David said, the third party plug-in repository will be perfect
> for this.
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, 3 Sep 2020 at 19:36, LEE Ween Jiann 
> wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hey there,
>
>
>
>
>
>
>
>
>
>
>
> The thing is there will **not** be repository after Nov 13.
>
>
>
>
>
> Please refer to this link which was previously posted,
>
>
>
>
>
>
>
>
>
>
>
> https://github.com/helm/charts/blob/master/README.md#deprecation-timeline
>
>
>
>
>
>
>
>
>
>
>
> Cheers,
>
>
>
>
>
> WJ
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *From:*Atri Sharma 
>
>
>
>
>
>
>
>
> *Reply to: *"dev@lucene.apache.org" 
>
>
>
>
>
>
>
>
> *Date: *Thursday, 3 September 2020 at 10:02 PM
>
>
>
>
>
>
>
>
> *To: *"dev@lucene.apache.org" 
>
>
>
>
>
>
>
>
> *Subject: *Re: Migrate Solr helm chart into apache/lucene-solr repo
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Jumping in late, but I don’t see why the help chart needs to be a part of
> the core repository.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Like the HDFS repository, why not just add a link to the existing
> repository somewhere on the main website?
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, 3 Sep 2020 at 19:29, LEE Ween Jiann 
> wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hi both,
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Is there any updates on this? The helm stable chart repo is now
> deprecated, could we move the helm chart to the apache GitHub repo?
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Cheers,
>
>
>
>
>
>
>
>
>
>
>
> WJ
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *From:*David Smiley 
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *Reply to: *"dev@lucene.apache.org" 
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *Date: *Monday, 3 February 2020 at 1:07 AM
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *To: *Solr/Lucene Dev 
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *Subject: *Re: Migrate Solr helm chart into apache/lucene-solr repo
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> I was looking for a "Why" Helm is ending its registry and didn't find it,
> though I did find the 2x vs 3x thing so I thought it was about version
> migration more than anything.  I'm wondering
>
>
>
>
>
> "Why" because this very project, Solr, will soon
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> be grappling with how to create a repository of 3rd party plugins.
> Successful projects like Docker do this so I wonder why Helm seems to be
> backing out.  Perhaps we will have one main repository of 3rd party plugins
> we've vetted in some way (have high confidence
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> in) and then for others we simply list URLs to "known" repos of packages
> of plugins that we have no opinion of.  This is uncharted territory for
> us.  Unlike Helm & Docker, I don't expect a vast number of entries.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ~ David Smiley
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Apache Lucene/Solr Search Developer
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> http://www.linkedin.com/in/davidwsmiley
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Sat, Feb 1, 2020 at 5:52 PM Marcus Eagan  wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

Re: Migrate Solr helm chart into apache/lucene-solr repo

[LEE Ween Jiann]

The ideal case would be to have Apache/charts with all the helm charts that are 
orphaned.
If not I guess, we will just wait and see if anyone picks up those charts.


From: Atri Sharma 
Reply to: "dev@lucene.apache.org" 
Date: Thursday, 3 September 2020 at 10:08 PM
To: "dev@lucene.apache.org" 
Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo

I still don’t see this as a reason to add it to the main repo. Maybe create a 
new repo just for this helm chart?

Anyways, as David said, the third party plug-in repository will be perfect for 
this.

On Thu, 3 Sep 2020 at 19:36, LEE Ween Jiann 
mailto:wjlee.2...@phdcs.smu.edu.sg>> wrote:













Hey there,



The thing is there will **not** be repository after Nov 13.

Please refer to this link which was previously posted,



https://github.com/helm/charts/blob/master/README.md#deprecation-timeline



Cheers,

WJ




From: Atri Sharma mailto:a...@apache.org>>


Reply to: "dev@lucene.apache.org" 
mailto:dev@lucene.apache.org>>


Date: Thursday, 3 September 2020 at 10:02 PM


To: "dev@lucene.apache.org" 
mailto:dev@lucene.apache.org>>


Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo







Jumping in late, but I don’t see why the help chart needs to be a part of the 
core repository.







Like the HDFS repository, why not just add a link to the existing repository 
somewhere on the main website?







On Thu, 3 Sep 2020 at 19:29, LEE Ween Jiann 
mailto:wjlee.2...@phdcs.smu.edu.sg>> wrote:










































Hi both,







Is there any updates on this? The helm stable chart repo is now deprecated, 
could we move the helm chart to the apache GitHub repo?







Cheers,



WJ














From:

David Smiley mailto:david.w.smi...@gmail.com>>








Reply to: "dev@lucene.apache.org" 
mailto:dev@lucene.apache.org>>








Date: Monday, 3 February 2020 at 1:07 AM








To: Solr/Lucene Dev mailto:dev@lucene.apache.org>>








Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo



















I was looking for a "Why" Helm is ending its registry and didn't find it, 
though I did find the 2x vs 3x thing so I thought it was about version 
migration more than anything.  I'm wondering

"Why" because this very project, Solr, will soon





be grappling with how to create a repository of 3rd party plugins.  Successful 
projects like Docker do this so I wonder why Helm seems to be backing out.  
Perhaps we will have one main repository of 3rd party plugins we've vetted in 
some way (have high confidence





in) and then for others we simply list URLs to "known" repos of packages of 
plugins that we have no opinion of.  This is uncharted territory for us.  
Unlike Helm & Docker, I don't expect a vast number of entries.

























~ David Smiley






Apache Lucene/Solr Search Developer









http://www.linkedin.com/in/davidwsmiley









































On Sat, Feb 1, 2020 at 5:52 PM Marcus Eagan 
mailto:marcusea...@gmail.com>> wrote:















David, I think he is referring to the note here:



































https://github.com/helm/charts/blob/master/README.md#deprecation-timeline


























All the best,



















Marcus



















On Sat, Feb 1, 2020 at 14:37 David Smiley 
mailto:david.w.smi...@gmail.com>> wrote:















Hi Lee,
















We're currently in-process of adopting Solr's Docker image.  I confess I've 
never used Helm so I have no clue how it's maintained, tested, etc and so I'll 
leave this sort of decision

to others here.



















BTW I went to that link about Helm deprecation and AFAICT it seems about 2.x 
version of Helm and not about 3.x.  See











https://helm.sh/blog/2019-10-22-helm-2150-released/#helm-2-support-plan too.  
Am I missing something?








































~ David Smiley






Apache Lucene/Solr Search Developer









http://www.linkedin.com/in/davidwsmiley












































On Fri, Jan 31, 2020 at 1:09 AM LEE Ween Jiann 
mailto:wjlee.2...@phdcs.smu.edu.sg>> wrote:















Hi,







Since helm stable repo is deprecated, it’s taking a lot of time to get the pull 
requests approved.



Also the helm chart for Solr has to go somewhere soon, refer to





https://github.com/helm/charts#deprecation-timeline.







Would you guys be adopting it?
























--









Marcus Eagan































--





Regards,


Atri


Apache Concerted







--
Regards,
Atri
Apache Concerted

Re: Migrate Solr helm chart into apache/lucene-solr repo

[Atri Sharma]

I still don’t see this as a reason to add it to the main repo. Maybe create
a new repo just for this helm chart?

Anyways, as David said, the third party plug-in repository will be perfect
for this.

On Thu, 3 Sep 2020 at 19:36, LEE Ween Jiann 
wrote:

>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hey there,
>
>
>
>
>
> The thing is there will **not** be repository after Nov 13.
>
>
> Please refer to this link which was previously posted,
>
>
>
> https://github.com/helm/charts/blob/master/README.md#deprecation-timeline
>
>
>
>
>
> Cheers,
>
>
> WJ
>
>
>
>
>
>
>
> *From: *Atri Sharma 
>
>
> *Reply to: *"dev@lucene.apache.org" 
>
>
> *Date: *Thursday, 3 September 2020 at 10:02 PM
>
>
> *To: *"dev@lucene.apache.org" 
>
>
> *Subject: *Re: Migrate Solr helm chart into apache/lucene-solr repo
>
>
>
>
>
>
>
>
>
>
>
>
>
> Jumping in late, but I don’t see why the help chart needs to be a part of
> the core repository.
>
>
>
>
>
>
>
>
>
>
>
>
>
> Like the HDFS repository, why not just add a link to the existing
> repository somewhere on the main website?
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, 3 Sep 2020 at 19:29, LEE Ween Jiann 
> wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hi both,
>
>
>
>
>
>
>
>
>
>
>
> Is there any updates on this? The helm stable chart repo is now
> deprecated, could we move the helm chart to the apache GitHub repo?
>
>
>
>
>
>
>
>
>
>
>
> Cheers,
>
>
>
>
>
> WJ
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *From:*David Smiley 
>
>
>
>
>
>
>
>
> *Reply to: *"dev@lucene.apache.org" 
>
>
>
>
>
>
>
>
> *Date: *Monday, 3 February 2020 at 1:07 AM
>
>
>
>
>
>
>
>
> *To: *Solr/Lucene Dev 
>
>
>
>
>
>
>
>
> *Subject: *Re: Migrate Solr helm chart into apache/lucene-solr repo
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> I was looking for a "Why" Helm is ending its registry and didn't find it,
> though I did find the 2x vs 3x thing so I thought it was about version
> migration more than anything.  I'm wondering
>
> "Why" because this very project, Solr, will soon
>
>
>
>
>
> be grappling with how to create a repository of 3rd party plugins.
> Successful projects like Docker do this so I wonder why Helm seems to be
> backing out.  Perhaps we will have one main repository of 3rd party plugins
> we've vetted in some way (have high confidence
>
>
>
>
>
> in) and then for others we simply list URLs to "known" repos of packages
> of plugins that we have no opinion of.  This is uncharted territory for
> us.  Unlike Helm & Docker, I don't expect a vast number of entries.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ~ David Smiley
>
>
>
>
>
>
>
>
>
>
>
> Apache Lucene/Solr Search Developer
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> http://www.linkedin.com/in/davidwsmiley
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Sat, Feb 1, 2020 at 5:52 PM Marcus Eagan  wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> David, I think he is referring to the note here:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> https://github.com/helm/charts/blob/master/README.md#deprecation-timeline
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> All the best,
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Marcus
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Sat, Feb 1, 2020 at 14:37 David Smiley 
> wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hi Lee,
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> We're currently in-process of adopting Solr's Docker image.  I confess
> I've never used Helm so I have no clue how it's maintained, tested, etc and
> so I'll leave this sort of decision
>
> to others here.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> BTW I went to that link about Helm deprecation and AFAICT it seems about
> 2.x version of Helm and not about 3.x.  See
>
>
>
>
>
>
>
>
>
>
>
> https://helm.sh/blog/2019-10-22-helm-2150-released/#helm-2-support-plan
> too.  Am I missing something?
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ~ David Smiley
>
>
>
>
>
>
>
>
>
>
>
> Apache Lucene/Solr Search Developer
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> http://www.linkedin.com/in/davidwsmiley
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Fri, Jan 31, 2020 at 1:09 AM LEE Ween Jiann <
> wjlee.2...@phdcs.smu.edu.sg> wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hi,
>
>
>
>
>
>
>
>
>
>
>
> Since helm stable repo is deprecated, it’s taking a lot of time to get the
> pull requests approved.
>
>
>
>
>
> Also the helm chart for Solr has to go somewhere soon, refer to

Re: Migrate Solr helm chart into apache/lucene-solr repo

[LEE Ween Jiann]

Hey there,

The thing is there will **not** be repository after Nov 13.
Please refer to this link which was previously posted, 
https://github.com/helm/charts/blob/master/README.md#deprecation-timeline

Cheers,
WJ

From: Atri Sharma 
Reply to: "dev@lucene.apache.org" 
Date: Thursday, 3 September 2020 at 10:02 PM
To: "dev@lucene.apache.org" 
Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo

Jumping in late, but I don’t see why the help chart needs to be a part of the 
core repository.

Like the HDFS repository, why not just add a link to the existing repository 
somewhere on the main website?

On Thu, 3 Sep 2020 at 19:29, LEE Ween Jiann 
mailto:wjlee.2...@phdcs.smu.edu.sg>> wrote:













Hi both,



Is there any updates on this? The helm stable chart repo is now deprecated, 
could we move the helm chart to the apache GitHub repo?



Cheers,

WJ




From: David Smiley mailto:david.w.smi...@gmail.com>>


Reply to: "dev@lucene.apache.org" 
mailto:dev@lucene.apache.org>>


Date: Monday, 3 February 2020 at 1:07 AM


To: Solr/Lucene Dev mailto:dev@lucene.apache.org>>


Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo







I was looking for a "Why" Helm is ending its registry and didn't find it, 
though I did find the 2x vs 3x thing so I thought it was about version 
migration more than anything.  I'm wondering "Why" because this very project, 
Solr, will soon

be grappling with how to create a repository of 3rd party plugins.  Successful 
projects like Docker do this so I wonder why Helm seems to be backing out.  
Perhaps we will have one main repository of 3rd party plugins we've vetted in 
some way (have high confidence

in) and then for others we simply list URLs to "known" repos of packages of 
plugins that we have no opinion of.  This is uncharted territory for us.  
Unlike Helm & Docker, I don't expect a vast number of entries.









~ David Smiley


Apache Lucene/Solr Search Developer



http://www.linkedin.com/in/davidwsmiley















On Sat, Feb 1, 2020 at 5:52 PM Marcus Eagan 
mailto:marcusea...@gmail.com>> wrote:





David, I think he is referring to the note here:













https://github.com/helm/charts/blob/master/README.md#deprecation-timeline










All the best,







Marcus







On Sat, Feb 1, 2020 at 14:37 David Smiley 
mailto:david.w.smi...@gmail.com>> wrote:





Hi Lee,






We're currently in-process of adopting Solr's Docker image.  I confess I've 
never used Helm so I have no clue how it's maintained, tested, etc and so I'll 
leave this sort of decision to others here.







BTW I went to that link about Helm deprecation and AFAICT it seems about 2.x 
version of Helm and not about 3.x.  See



https://helm.sh/blog/2019-10-22-helm-2150-released/#helm-2-support-plan too.  
Am I missing something?














~ David Smiley


Apache Lucene/Solr Search Developer



http://www.linkedin.com/in/davidwsmiley
















On Fri, Jan 31, 2020 at 1:09 AM LEE Ween Jiann 
mailto:wjlee.2...@phdcs.smu.edu.sg>> wrote:





Hi,



Since helm stable repo is deprecated, it’s taking a lot of time to get the pull 
requests approved.

Also the helm chart for Solr has to go somewhere soon, refer to

https://github.com/helm/charts#deprecation-timeline.



Would you guys be adopting it?








--



Marcus Eagan









--
Regards,
Atri
Apache Concerted

Re: Migrate Solr helm chart into apache/lucene-solr repo

[Atri Sharma]

Jumping in late, but I don’t see why the help chart needs to be a part of
the core repository.

Like the HDFS repository, why not just add a link to the existing
repository somewhere on the main website?

On Thu, 3 Sep 2020 at 19:29, LEE Ween Jiann 
wrote:

>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hi both,
>
>
>
>
>
> Is there any updates on this? The helm stable chart repo is now
> deprecated, could we move the helm chart to the apache GitHub repo?
>
>
>
>
>
> Cheers,
>
>
> WJ
>
>
>
>
>
>
>
> *From: *David Smiley 
>
>
> *Reply to: *"dev@lucene.apache.org" 
>
>
> *Date: *Monday, 3 February 2020 at 1:07 AM
>
>
> *To: *Solr/Lucene Dev 
>
>
> *Subject: *Re: Migrate Solr helm chart into apache/lucene-solr repo
>
>
>
>
>
>
>
>
>
>
>
>
>
> I was looking for a "Why" Helm is ending its registry and didn't find it,
> though I did find the 2x vs 3x thing so I thought it was about version
> migration more than anything.  I'm wondering "Why" because this very
> project, Solr, will soon
>
> be grappling with how to create a repository of 3rd party plugins.
> Successful projects like Docker do this so I wonder why Helm seems to be
> backing out.  Perhaps we will have one main repository of 3rd party plugins
> we've vetted in some way (have high confidence
>
> in) and then for others we simply list URLs to "known" repos of packages
> of plugins that we have no opinion of.  This is uncharted territory for
> us.  Unlike Helm & Docker, I don't expect a vast number of entries.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ~ David Smiley
>
>
>
>
> Apache Lucene/Solr Search Developer
>
>
>
>
>
>
> http://www.linkedin.com/in/davidwsmiley
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Sat, Feb 1, 2020 at 5:52 PM Marcus Eagan  wrote:
>
>
>
>
>
>
>
>
>
>
> David, I think he is referring to the note here:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> https://github.com/helm/charts/blob/master/README.md#deprecation-timeline
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> All the best,
>
>
>
>
>
>
>
>
>
>
>
>
>
> Marcus
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Sat, Feb 1, 2020 at 14:37 David Smiley 
> wrote:
>
>
>
>
>
>
>
>
>
>
> Hi Lee,
>
>
>
>
>
>
>
>
>
>
>
> We're currently in-process of adopting Solr's Docker image.  I confess
> I've never used Helm so I have no clue how it's maintained, tested, etc and
> so I'll leave this sort of decision to others here.
>
>
>
>
>
>
>
>
>
>
>
>
>
> BTW I went to that link about Helm deprecation and AFAICT it seems about
> 2.x version of Helm and not about 3.x.  See
>
>
>
> https://helm.sh/blog/2019-10-22-helm-2150-released/#helm-2-support-plan
> too.  Am I missing something?
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ~ David Smiley
>
>
>
>
> Apache Lucene/Solr Search Developer
>
>
>
>
>
>
> http://www.linkedin.com/in/davidwsmiley
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Fri, Jan 31, 2020 at 1:09 AM LEE Ween Jiann <
> wjlee.2...@phdcs.smu.edu.sg> wrote:
>
>
>
>
>
>
>
>
>
>
> Hi,
>
>
>
>
>
> Since helm stable repo is deprecated, it’s taking a lot of time to get the
> pull requests approved.
>
>
> Also the helm chart for Solr has to go somewhere soon, refer to
>
> https://github.com/helm/charts#deprecation-timeline.
>
>
>
>
>
> Would you guys be adopting it?
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
>
>
>
>
>
>
> Marcus Eagan
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
Regards,

Atri
Apache Concerted

Re: Migrate Solr helm chart into apache/lucene-solr repo

[LEE Ween Jiann]

Hi both,

Is there any updates on this? The helm stable chart repo is now deprecated, 
could we move the helm chart to the apache GitHub repo?

Cheers,
WJ

From: David Smiley 
Reply to: "dev@lucene.apache.org" 
Date: Monday, 3 February 2020 at 1:07 AM
To: Solr/Lucene Dev 
Subject: Re: Migrate Solr helm chart into apache/lucene-solr repo

I was looking for a "Why" Helm is ending its registry and didn't find it, 
though I did find the 2x vs 3x thing so I thought it was about version 
migration more than anything.  I'm wondering "Why" because this very project, 
Solr, will soon be grappling with how to create a repository of 3rd party 
plugins.  Successful projects like Docker do this so I wonder why Helm seems to 
be backing out.  Perhaps we will have one main repository of 3rd party plugins 
we've vetted in some way (have high confidence in) and then for others we 
simply list URLs to "known" repos of packages of plugins that we have no 
opinion of.  This is uncharted territory for us.  Unlike Helm & Docker, I don't 
expect a vast number of entries.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Sat, Feb 1, 2020 at 5:52 PM Marcus Eagan 
mailto:marcusea...@gmail.com>> wrote:
David, I think he is referring to the note here:


https://github.com/helm/charts/blob/master/README.md#deprecation-timeline


All the best,

Marcus

On Sat, Feb 1, 2020 at 14:37 David Smiley 
mailto:david.w.smi...@gmail.com>> wrote:
Hi Lee,

We're currently in-process of adopting Solr's Docker image.  I confess I've 
never used Helm so I have no clue how it's maintained, tested, etc and so I'll 
leave this sort of decision to others here.

BTW I went to that link about Helm deprecation and AFAICT it seems about 2.x 
version of Helm and not about 3.x.  See 
https://helm.sh/blog/2019-10-22-helm-2150-released/#helm-2-support-plan too.  
Am I missing something?

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Fri, Jan 31, 2020 at 1:09 AM LEE Ween Jiann 
mailto:wjlee.2...@phdcs.smu.edu.sg>> wrote:
Hi,

Since helm stable repo is deprecated, it’s taking a lot of time to get the pull 
requests approved.
Also the helm chart for Solr has to go somewhere soon, refer to 
https://github.com/helm/charts#deprecation-timeline.

Would you guys be adopting it?
--
Marcus Eagan

Re: Approach towards solving split package issues?

[Tomoko Uchida]

I also opened SOLR-14826 as the placeholder. I'm not fully sure of its
priority but at least Alexandre expressed an interest in fixing it for
Solr, thanks.
If there is someone who wants to take the ownership, please feel free to
join. I will leave it there until LUCENE-9499 is done anyway.



2020年9月3日(木) 0:12 Tomoko Uchida :

> I opened LUCENE-9499 as the umbrella.
> I set "Fix version" to 9.0 - means once we make a commit on it, this will
> be a blocker for release 9.0.0. (I don't think the changes should be
> delivered across two major releases; all changes have to be out at once in
> a major release.) If there are any objections or concerns, please leave
> comments. For now I have no idea about the total volume of changes or
> technical obstacles that have to be handled.
>
> About Solr - do we really need to fix split packages? Solr is a server
> app, the benefits seem to be smaller than Lucene (a library) for me. I'd
> like to hear opinions/thoughts from others.
>
> Tomoko
>
>
> 2020年9月2日(水) 9:05 Gus Heck :
>
>> +1 to fixing and +1 to doing it in a major release.
>>
>> On Tue, Sep 1, 2020 at 4:32 PM Adrien Grand  wrote:
>>
>>> +1 Changing packages of many classes should be done in a major.
>>>
>>> On Tue, Sep 1, 2020 at 5:50 PM Tomoko Uchida <
>>> tomoko.uchida.1...@gmail.com> wrote:
>>>
 Just to make sure, could I confirm "when the changes will be out"...
 Resolving split package issues should break backward compatibility
 (changing package names and moving classes from one module to another
 modules). So we have just two options, we can have these changes only on
 major releases - 9.0.0 or 10.0.0; we cannot release such changes at minor
 versions. Is that correct?

 Tomoko


 2020年9月1日(火) 22:08 Tomoko Uchida :

> > As I recall one issue was around where to put analysis packages?
>
> It's LUCENE-9317. I had worked on it before, you can see what changes
> will be needed for analyzers-common (and core).
>
>
> 2020年9月1日(火) 22:00 Michael Sokolov :
>
>> I'm in favor - there may be some difficult choices though. As I recall
>> one issue was around where to put analysis packages? I forget the
>> details, but there was some pretty strong feeling that you should have
>> a functioning system with core only. However some basic analysis tools
>> are required for that, while most of our analyzers and so on are in a
>> separate analysis module. Perhaps we will need to move some basic
>> analyzers out of com.amazon.lucene.analysis? Or the reverse - move all
>> the analysis code into the analysis module and acknowledge that it is
>> a fundamental dependency (more essential than core, really).
>>
>> On Tue, Sep 1, 2020 at 8:26 AM Tomoko Uchida
>>  wrote:
>> >
>> > yes, Jigsaw was on my mind too...
>> >
>> > > why not go ahead and try to clean it up right away?
>> >
>> > > So a strong +1 to clean this up!
>> >
>> > OK, maybe I should open two issues, one for Lucene and one for
>> Solr, and link existing wip issues to them.
>> > Once we start it, these will be blockers for 9.0.0 release I
>> believe (for now I have no idea about the volume of the changes or
>> technical obstacles). Are there any objections or comments?
>> >
>> >
>> > 2020年9月1日(火) 19:34 Uwe Schindler :
>> >>
>> >> Hi,
>> >>
>> >> The biggest issue is that split packages make migrating to the
>> Java 9 module system impossible. It's not allowed to have same package 
>> name
>> (with classes) in different JAR files.
>> >>
>> >> Some of those require to open up visibility of classes. Some split
>> packages issues were done because of package private access, which is 
>> very
>> bad between JAR files. This also affects the test framework, although 
>> this
>> is not such a big deal (I would exclude that for now), because you would
>> never run UNIT tests inside a module system, only integration tests.
>> >>
>> >> So a strong +1 to clean this up!
>> >> Uwe
>> >>
>> >> -
>> >> Uwe Schindler
>> >> Achterdiek 19, D-28357 Bremen
>> >> https://www.thetaphi.de
>> >> eMail: u...@thetaphi.de
>> >>
>> >> > -Original Message-
>> >> > From: Dawid Weiss 
>> >> > Sent: Tuesday, September 1, 2020 9:22 AM
>> >> > To: Lucene Dev 
>> >> > Subject: Re: Approach towards solving split package issues?
>> >> >
>> >> > This is a big headache for many things. I wouldn't mind doing
>> this
>> >> > even for 9x. This is a major release, why not go ahead and try to
>> >> > clean it up right away?
>> >> >
>> >> > Dawid
>> >> >
>> >> > On Mon, Aug 31, 2020 at 11:50 PM Tomoko Uchida
>> >> >  wrote:
>> >> > >
>> >> > > Hello devs,
>> >> > >
>> >> > > we have lots of package name conflicts (shared package names)
>> 

Re: Solr configuration options

[Erick Erickson]

I’d also like solr.xml to go away if at all possible, at least in cloud mode.

Some important points (to me at least).

- having to hand-edit _anything_ is A Bad Thing in a large, distributed system 
since you need:

— an infrastructure to automagically distribute whatever it is you’re editing 
to an arbitrary-but-possibly-very-large number of nodes

— a process to verify your edits do what you expect (and whether you’re 
hand-editing JSON or XML is immaterial, you can still screw up)

— a process to refresh those values on running Solr instances (cluster-wide 
restart? Yuck). Although even with everything in clusterprops.json I can still 
imagine some kind of restart would be required for some changes...

These are non-trivial to implement and highly non trivial to debug when the 
process doesn’t work, again in a system with 100s of nodes.

Also, there’s no chance to sanity check the changes and reject invalid ones, 
say a typo in a property name or specifying an unloadable class or...

In standalone mode, IDK. Replacing solr.xml with clusterprops.json is confusing 
as well, although it’s explainable at least. 
That said, I think it might be easier to explain that:

“in standalone mode, certain defaults are kept in solr.xml which is read 
locally from disk. In cloud mode, solr.xml is replaced by clusterprops.json on 
ZK”.

To me, that’s certainly less confusing than

“in standalone mode, certain defaults are kept in solr.xml. In cloud mode, 
those defaults may kept in solr.xml on ZK. If that znode is absent, solr.xml is 
loaded from disk. In either case, clusterprops.json will override anything in 
solr.xml. Oh, and if you’re running in cloud mode and haven’t put solr.xml on 
ZK, good luck keeping them all coordinated.”.

The upgrade path does suck though. If I’m going from Solr X to Solr X+1 where X 
supports solr.xml and X+1 does not what happens in cloud mode? We’d have to do 
something like detect that situation and rewrite solr.xml to clusterprops.xml. 
Or provide a command/utility that does that under user control that’d have to 
be run first. Or…. Coordinating automatic rewrite from 
solr.xml->clusterprops.json that when simultaneously starting 100 X+1 instances 
would be …challenging… And we’d need to respect anything _already_ in 
clusterprops.json.

Ilan:

Why do we need to prime Zookeeper? If we hard-code the defaults can’t we carry 
on if cluster.props isn’t there with the hard-coded defaults? Of course 
overwriting those defaults if clusterprops.json _is_ present. The fact that 
changing defaults requires recompilation isn’t a problem IMO. Currently if we 
change them, _we_ change solr.xml and distribute that with the new release. 
This pre-supposes that we can get everything we need early enough in the 
process that we can look at ZK in the first place. If we need to read 
clusterprops before talking to ZK, well, that’s a problem that maybe we’d have 
to solve with a system var or something.

Of course that’s my 10,000 ft. view. The details are always hairy.

FWIW,
Erick




> On Sep 3, 2020, at 5:58 AM, Noble Paul  wrote:
> 
>> to stick with solr.xml for current and ongoing feature development and not 
>> hold up development based on things that may or may not happen in the future.
> 
> Where did I say that? There are so many features built recently using
> clusterprops.json. The least we can do is stop piling more into
> solr.xml. If you cannot make something better the least you can do is
> stop making life more difficult for everyone.
> 
>> In order for clusterpropos.json to replace what's currently done with 
>> solr.xml, we'd need to introduce a mechanism to make configuration available 
>> out of the box
> 
> How hard is it to insert a JSON into your ZK ?
> 
> The problem with Solr developers is they only have one workflow in
> their mind, the way they use it in their org. How many of us
> understand the pain of restarting the entire cluster because we wish
> to change a single config param for replica placement.
> 
> We really do not care if a newcomer will be able to edit an XML file
> in every node to use this feature. You know why? We really do not want
> anyone to use this feature, Most of us are paid my our respective
> organizations and as long as it meets their needs they're totally
> happy. Why don't you all go back, and work on your own internal fork
> of Solr if that is all that you guys want. Why even pretend that
> something is pluggable & offer some value to our users.
> 
> The existing autoscaling config was stored in ZK , did users complain
> that they have difficulty in storing that file in ZK?
> 
> On Thu, Sep 3, 2020 at 7:20 PM Jan Høydahl  wrote:
>> 
>> Let’s not do another magic overlay json file on top of xml with all the 
>> confusion it creates.
>> +1 to stick with solr.xml for current and ongoing feature development and 
>> not hold up development based on things that may or may not happen in the 
>> future. Discussing removal of solr.xml is a typical SIP 

Re: [JENKINS] Lucene-Solr-8.x-Linux (64bit/jdk-14.0.1) - Build # 4265 - Still Failing!

[Adrien Grand]

I pushed a fix for these failures about solr.cmd having tabs on branch_8x.

On Thu, Sep 3, 2020 at 10:42 AM Policeman Jenkins Server <
jenk...@thetaphi.de> wrote:

> Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Linux/4265/
> Java: 64bit/jdk-14.0.1 -XX:+UseCompressedOops -XX:+UseSerialGC
>
> All tests passed
>
> Build Log:
> [...truncated 26229 lines...]
> BUILD FAILED
> /home/jenkins/workspace/Lucene-Solr-8.x-Linux/build.xml:634: The following
> error occurred while executing this line:
> /home/jenkins/workspace/Lucene-Solr-8.x-Linux/build.xml:128: Found 1
> violations in source files (tabs instead spaces).
>
> Total time: 34 minutes 6 seconds
> Build step 'Invoke Ant' marked build as failure
> Setting
> ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
> Archiving artifacts
> [Java] Skipping execution of recorder since overall result is 'FAILURE'
> Setting
> ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
> Recording test results
> Setting
> ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
> Setting
> ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
> Setting
> ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
> Setting
> ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
> Setting
> ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
> Setting
> ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
> Setting
> ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
>
> -
> To unsubscribe, e-mail: builds-unsubscr...@lucene.apache.org
> For additional commands, e-mail: builds-h...@lucene.apache.org



-- 
Adrien

Re: Solr configuration options

[Noble Paul]

> to stick with solr.xml for current and ongoing feature development and not 
> hold up development based on things that may or may not happen in the future.

Where did I say that? There are so many features built recently using
clusterprops.json. The least we can do is stop piling more into
solr.xml. If you cannot make something better the least you can do is
stop making life more difficult for everyone.

>In order for clusterpropos.json to replace what's currently done with 
>solr.xml, we'd need to introduce a mechanism to make configuration available 
>out of the box

How hard is it to insert a JSON into your ZK ?

The problem with Solr developers is they only have one workflow in
their mind, the way they use it in their org. How many of us
understand the pain of restarting the entire cluster because we wish
to change a single config param for replica placement.

We really do not care if a newcomer will be able to edit an XML file
in every node to use this feature. You know why? We really do not want
anyone to use this feature, Most of us are paid my our respective
organizations and as long as it meets their needs they're totally
happy. Why don't you all go back, and work on your own internal fork
of Solr if that is all that you guys want. Why even pretend that
something is pluggable & offer some value to our users.

The existing autoscaling config was stored in ZK , did users complain
that they have difficulty in storing that file in ZK?

On Thu, Sep 3, 2020 at 7:20 PM Jan Høydahl  wrote:
>
> Let’s not do another magic overlay json file on top of xml with all the 
> confusion it creates.
> +1 to stick with solr.xml for current and ongoing feature development and not 
> hold up development based on things that may or may not happen in the future. 
> Discussing removal of solr.xml is a typical SIP candidate?
>
> Jan
>
> 3. sep. 2020 kl. 10:06 skrev Ilan Ginzburg :
>
> Noble,
>
> In order for clusterpropos.json to replace what's currently done with 
> solr.xml, we'd need to introduce a mechanism to make configuration available 
> out of the box (when Zookeeper is still empty). And if clusterpropos.json is 
> to be used in standalone mode, it also must live somewhere on disk as well 
> (no Zookeeper in standalone mode).
>
> I believe shardHandlerFactoryConfig is in solr.xml for all nodes to know 
> which shard handler to use, not for configuring a different one on each node.
>
> Priming an empty Zookeeper with an initial version of clusterpropos.json on 
> startup is easy (first node up pushes its local copy). But after this 
> happened once, if Solr is upgraded with a new default clusterprops.json, it 
> is hard (to very hard) to update the Zookeeper version, high risk of erasing 
> configuration that the user added or does not want to change.
>
> How about a variant? Keep a local solr.xml file with default configs and 
> support overriding of these configs from Zookeeper's clusterprops.json. This 
> approach does not have the out of the box issue mentioned above, and 
> practically also solves the "updating defaults" issue: if the user cherry 
> picked some solr.xml configuration values and overrode them 
> clusterprops.json, it is then his responsibility to maintain them there. 
> Newly introduced configuration values in solr.xml (due to a new Solr version) 
> are not impacted since they were not overridden.
>
> I believe this approach is not too far from a suggestion you seem to make to 
> hard code default configs to get rid of solr.xml. The difference is that the 
> hard coding is done in solr.xml rather than in some defaultSolrConfig.java 
> class. This makes changing default configuration easy and not requiring 
> recompilation but is otherwise not conceptually different.
>
> Ilan
>
>
> On Thu, Sep 3, 2020 at 7:05 AM Noble Paul  wrote:
>>
>> Let's take a step back and take a look at the history of Solr.
>>
>> Long ago there was only standalone Solr with a single core
>> there were 3 files
>>
>> * solr.xml : everything required for CoreContainer went here
>> * solr.config.xml : per core configurations go here
>> * schema.xml: this is not relevant for this discussion
>>
>> Now we are in the cloud world where everything lives in ZK. This also
>> means there are potentially 1000's of nodes reading configuration from
>> ZK. This is quite a convenient setup. The same configset is being
>> shared by a very large no:of nodes and everyone is happy.
>>
>> But, solr.xml still stands out like a sore thumb. We have no idea what
>> it is for? is it a node specific configuration? or is it something
>> that every single node in the cluster should have in common?
>>
>> e:g: shardHandlerFactoryConfig.
>>
>> Does it even make sense for to use a separate
>> "shardHandlerFactoryConfig" for each node? Or should we have every
>> node have the same shardHandlerFactoryConfig? It totally makes no
>> sense to have a different config in each node. Here is an exhaustive
>> list of parameters that we can configure in solr.xml

Re: Solr configuration options

[Jan Høydahl]

Let’s not do another magic overlay json file on top of xml with all the 
confusion it creates.
+1 to stick with solr.xml for current and ongoing feature development and not 
hold up development based on things that may or may not happen in the future. 
Discussing removal of solr.xml is a typical SIP candidate?

Jan

> 3. sep. 2020 kl. 10:06 skrev Ilan Ginzburg :
> 
> Noble,
> 
> In order for clusterpropos.json to replace what's currently done with 
> solr.xml, we'd need to introduce a mechanism to make configuration available 
> out of the box (when Zookeeper is still empty). And if clusterpropos.json is 
> to be used in standalone mode, it also must live somewhere on disk as well 
> (no Zookeeper in standalone mode).
> 
> I believe shardHandlerFactoryConfig is in solr.xml for all nodes to know 
> which shard handler to use, not for configuring a different one on each node.
> 
> Priming an empty Zookeeper with an initial version of clusterpropos.json on 
> startup is easy (first node up pushes its local copy). But after this 
> happened once, if Solr is upgraded with a new default clusterprops.json, it 
> is hard (to very hard) to update the Zookeeper version, high risk of erasing 
> configuration that the user added or does not want to change.
> 
> How about a variant? Keep a local solr.xml file with default configs and 
> support overriding of these configs from Zookeeper's clusterprops.json. This 
> approach does not have the out of the box issue mentioned above, and 
> practically also solves the "updating defaults" issue: if the user cherry 
> picked some solr.xml configuration values and overrode them 
> clusterprops.json, it is then his responsibility to maintain them there. 
> Newly introduced configuration values in solr.xml (due to a new Solr version) 
> are not impacted since they were not overridden.
> 
> I believe this approach is not too far from a suggestion you seem to make 
>  to 
> hard code default configs to get rid of solr.xml. The difference is that the 
> hard coding is done in solr.xml rather than in some defaultSolrConfig.java 
> class. This makes changing default configuration easy and not requiring 
> recompilation but is otherwise not conceptually different.
> 
> Ilan
> 
> 
> On Thu, Sep 3, 2020 at 7:05 AM Noble Paul  > wrote:
> Let's take a step back and take a look at the history of Solr.
> 
> Long ago there was only standalone Solr with a single core
> there were 3 files
> 
> * solr.xml : everything required for CoreContainer went here
> * solr.config.xml : per core configurations go here
> * schema.xml: this is not relevant for this discussion
> 
> Now we are in the cloud world where everything lives in ZK. This also
> means there are potentially 1000's of nodes reading configuration from
> ZK. This is quite a convenient setup. The same configset is being
> shared by a very large no:of nodes and everyone is happy.
> 
> But, solr.xml still stands out like a sore thumb. We have no idea what
> it is for? is it a node specific configuration? or is it something
> that every single node in the cluster should have in common?
> 
> e:g: shardHandlerFactoryConfig.
> 
> Does it even make sense for to use a separate
> "shardHandlerFactoryConfig" for each node? Or should we have every
> node have the same shardHandlerFactoryConfig? It totally makes no
> sense to have a different config in each node. Here is an exhaustive
> list of parameters that we can configure in solr.xml
> 
> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/core/NodeConfig.java
>  
> 
> 
> 99% of these parameters information should not be configured on a per
> node basis. It should be shared across the cluster. If that is the
> case, we should always have this xml file stored in ZK and not in
> every node.So, if this file is in ZK, does it make sense to be able to
> update this file in ZK and not reload all nodes? Yes totally. Anyone
> who has 1000's of nodes in a cluster will definitely hesitate to
> restart their clusters. Editing XML file is extremely hard. Users hate
> XML. Everyone is familiar with JSON and they love JSON. The entire
> security framework runs off of security.json. It's extremely easy to
> manipulate and read.
> 
> So we introduced clusterprops.json for some attributes we may change
> in a live cluster. The advantages were
> 
> * It's easy to manipulate this data. Simple APIs can be provided.
> These APIs may validate your input. there is a near 1:1 mapping
> between API input and the actual config
> * users can directly edit this JSON using simple tools
> * We can change data live and it's quite easy to reload a specific component
> 
> My preference is
> * Support everything in solr.xml in clusterprops.json. Get rid of XML 
> everywhere
> * This file 

Re: Solr configuration options

[Ilan Ginzburg]

Noble,

In order for *clusterpropos.json* to replace what's currently done with
*solr.xml*, we'd need to introduce a mechanism to make configuration
available out of the box (when Zookeeper is still empty). And if
*clusterpropos.json* is to be used in standalone mode, it also must live
somewhere on disk as well (no Zookeeper in standalone mode).

I believe shardHandlerFactoryConfig is in *solr.xml* for all nodes to know
which shard handler to use, not for configuring a different one on each
node.

Priming an empty Zookeeper with an initial version of *clusterpropos.json*
on startup is easy (first node up pushes its local copy). But after this
happened once, if Solr is upgraded with a new default *clusterprops.json*,
it is hard (to very hard) to update the Zookeeper version, high risk of
erasing configuration that the user added or does not want to change.

How about a variant? Keep a local *solr.xml* file with default configs and
support overriding of these configs from Zookeeper's *clusterprops.json*.
This approach does not have the out of the box issue mentioned above, and
practically also solves the "updating defaults" issue: if the user cherry
picked some *solr.xml* configuration values and overrode them
*clusterprops.json*, it is then his responsibility to maintain them there.
Newly introduced configuration values in *solr.xml* (due to a new Solr
version) are not impacted since they were not overridden.

I believe this approach is not too far from a suggestion you seem to make
 to
hard code default configs to get rid of *solr.xml*. The difference is that
the hard coding is done in *solr.xml* rather than in some
*defaultSolrConfig.java* class. This makes changing default configuration
easy and not requiring recompilation but is otherwise not conceptually
different.

Ilan


On Thu, Sep 3, 2020 at 7:05 AM Noble Paul  wrote:

> Let's take a step back and take a look at the history of Solr.
>
> Long ago there was only standalone Solr with a single core
> there were 3 files
>
> * solr.xml : everything required for CoreContainer went here
> * solr.config.xml : per core configurations go here
> * schema.xml: this is not relevant for this discussion
>
> Now we are in the cloud world where everything lives in ZK. This also
> means there are potentially 1000's of nodes reading configuration from
> ZK. This is quite a convenient setup. The same configset is being
> shared by a very large no:of nodes and everyone is happy.
>
> But, solr.xml still stands out like a sore thumb. We have no idea what
> it is for? is it a node specific configuration? or is it something
> that every single node in the cluster should have in common?
>
> e:g: shardHandlerFactoryConfig.
>
> Does it even make sense for to use a separate
> "shardHandlerFactoryConfig" for each node? Or should we have every
> node have the same shardHandlerFactoryConfig? It totally makes no
> sense to have a different config in each node. Here is an exhaustive
> list of parameters that we can configure in solr.xml
>
>
> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/core/NodeConfig.java
>
> 99% of these parameters information should not be configured on a per
> node basis. It should be shared across the cluster. If that is the
> case, we should always have this xml file stored in ZK and not in
> every node.So, if this file is in ZK, does it make sense to be able to
> update this file in ZK and not reload all nodes? Yes totally. Anyone
> who has 1000's of nodes in a cluster will definitely hesitate to
> restart their clusters. Editing XML file is extremely hard. Users hate
> XML. Everyone is familiar with JSON and they love JSON. The entire
> security framework runs off of security.json. It's extremely easy to
> manipulate and read.
>
> So we introduced clusterprops.json for some attributes we may change
> in a live cluster. The advantages were
>
> * It's easy to manipulate this data. Simple APIs can be provided.
> These APIs may validate your input. there is a near 1:1 mapping
> between API input and the actual config
> * users can directly edit this JSON using simple tools
> * We can change data live and it's quite easy to reload a specific
> component
>
> My preference is
> * Support everything in solr.xml in clusterprops.json. Get rid of XML
> everywhere
> * This file shall be supported in standalone mode as well. There is no harm
> * May be there are a few attributes we do not want to configure on a
> cluster-wide setup. Use a simple node.properties file for that. Nobody
> likes XML configuration. It's error-prone to edit xml files
>
> --Noble
>
>
>
>
> On Sat, Aug 29, 2020 at 5:32 AM Alexandre Rafalovitch
>  wrote:
> >
> > This is way above my head, but I wonder if we could dogfood any of
> > this with a future Solr cloud example? At the moment, it sets up 2-4
> > nodes, 1 collection, any number of shards/replicas. And it does it by
> > directory 

Re: [VOTE] Lucene logo contest, third time's a charm

[Simon Willnauer]

A1, A2, D (binding)

On Thu, Sep 3, 2020 at 7:09 AM Noble Paul  wrote:

> A1, A2, D binding
>
> On Thu, Sep 3, 2020 at 7:22 AM Jason Gerlowski 
> wrote:
> >
> > A1, A2, D (binding)
> >
> > On Wed, Sep 2, 2020 at 10:47 AM Michael McCandless
> >  wrote:
> > >
> > > A2, A1, C5, D (binding)
> > >
> > > Thank you to everyone for working so hard to make such cool looking
> possible future Lucene logos!  And to Ryan for the challenging job of
> calling this VOTE :)
> > >
> > > Mike McCandless
> > >
> > > http://blog.mikemccandless.com
> > >
> > >
> > > On Tue, Sep 1, 2020 at 4:21 PM Ryan Ernst  wrote:
> > >>
> > >> Dear Lucene and Solr developers!
> > >>
> > >> Sorry for the multiple threads. This should be the last one.
> > >>
> > >> In February a contest was started to design a new logo for Lucene
> [jira-issue]. The initial attempt [first-vote] to call a vote resulted in
> some confusion on the rules, as well the request for one additional
> submission. The second attempt [second-vote] yesterday had incorrect links
> for one of the submissions. I would like to call a new vote, now with more
> explicit instructions on how to vote, and corrected links.
> > >>
> > >> Please read the following rules carefully before submitting your vote.
> > >>
> > >> Who can vote?
> > >>
> > >> Anyone is welcome to cast a vote in support of their favorite
> submission(s). Note that only PMC member's votes are binding. If you are a
> PMC member, please indicate with your vote that the vote is binding, to
> ease collection of votes. In tallying the votes, I will attempt to verify
> only those marked as binding.
> > >>
> > >> How do I vote?
> > >>
> > >> Votes can be cast simply by replying to this email. It is a
> ranked-choice vote [rank-choice-voting]. Multiple selections may be made,
> where the order of preference must be specified. If an entry gets more than
> half the votes, it is the winner. Otherwise, the entry with the lowest
> number of votes is removed, and the votes are retallied, taking into
> account the next preferred entry for those whose first entry was removed.
> This process repeats until there is a winner.
> > >>
> > >> The entries are broken up by variants, since some entries have
> multiple color or style variations. The entry identifiers are first a
> capital letter, followed by a variation id (described with each entry
> below), if applicable. As an example, if you prefer variant 1 of entry A,
> followed by variant 2 of entry A, variant 3 of entry C, entry D, and lastly
> variant 4e of entry B, the following should be in your reply:
> > >>
> > >> (binding)
> > >> vote: A1, A2, C3, D, B4e
> > >>
> > >> Entries
> > >>
> > >> The entries are as follows:
> > >>
> > >> A. Submitted by Dustin Haver. This entry has two variants, A1 and A2.
> > >>
> > >> [A1]
> https://issues.apache.org/jira/secure/attachment/12999548/Screen%20Shot%202020-04-10%20at%208.29.32%20AM.png
> > >> [A2]
> https://issues.apache.org/jira/secure/attachment/12997172/LuceneLogo.png
> > >>
> > >> B. Submitted by Stamatis Zampetakis. This has several variants.
> Within the linked entry there are 7 patterns and 7 color palettes. Any vote
> for B should contain the pattern number followed by the lowercase letter of
> the color palette. For example, B3e or B1a.
> > >>
> > >> [B]
> https://issues.apache.org/jira/secure/attachment/12997768/zabetak-1-7.pdf
> > >>
> > >> C. Submitted by Baris Kazar. This entry has 8 variants.
> > >>
> > >> [C1]
> https://issues.apache.org/jira/secure/attachment/13006392/lucene_logo1_full.pdf
> > >> [C2]
> https://issues.apache.org/jira/secure/attachment/13006393/lucene_logo2_full.pdf
> > >> [C3]
> https://issues.apache.org/jira/secure/attachment/13006394/lucene_logo3_full.pdf
> > >> [C4]
> https://issues.apache.org/jira/secure/attachment/13006395/lucene_logo4_full.pdf
> > >> [C5]
> https://issues.apache.org/jira/secure/attachment/13006396/lucene_logo5_full.pdf
> > >> [C6]
> https://issues.apache.org/jira/secure/attachment/13006397/lucene_logo6_full.pdf
> > >> [C7]
> https://issues.apache.org/jira/secure/attachment/13006398/lucene_logo7_full.pdf
> > >> [C8]
> https://issues.apache.org/jira/secure/attachment/13006399/lucene_logo8_full.pdf
> > >>
> > >> D. The current Lucene logo.
> > >>
> > >> [D]
> https://lucene.apache.org/theme/images/lucene/lucene_logo_green_300.png
> > >>
> > >> Please vote for one of the above choices. This vote will close about
> one week from today, Mon, Sept 7, 2020 at 11:59PM.
> > >>
> > >> Thanks!
> > >>
> > >> [jira-issue] https://issues.apache.org/jira/browse/LUCENE-9221
> > >> [first-vote]
> http://mail-archives.apache.org/mod_mbox/lucene-dev/202006.mbox/%3cCA+DiXd74Mz4H6o9SmUNLUuHQc6Q1-9mzUR7xfxR03ntGwo=d...@mail.gmail.com%3e
> > >> [second-vote]
> http://mail-archives.apache.org/mod_mbox/lucene-dev/202009.mbox/%3cCA+DiXd7eBrQu5+aJQ3jKaUtUTJUqaG2U6o+kUZfNe-m=smn...@mail.gmail.com%3e
> > >> [rank-choice-voting]
> https://en.wikipedia.org/wiki/Instant-runoff_voting
> >
> >