Re: [VOTE] Release Apache Solr 9.7.0 RC2

[Jason Gerlowski]

+1 (binding)

I ran the smoketester, and did some manual testing of our backup,
restore, and "install shard" functionality.

Jason

On Fri, Sep 6, 2024 at 10:50 AM Tomás Fernández Löbbe
 wrote:
>
> +1
>
> SUCCESS! [1:00:35.810469]
>
> On Thu, Sep 5, 2024 at 6:59 AM Jan Høydahl  wrote:
>
> > +1 (binding)
> >
> > SUCCESS! [0:42:01.595127]
> >
> > Once again on third attempt.
> >
> > Also built docker images and spun it up.
> >
> > Jan
> >
> > > 4. sep. 2024 kl. 03:00 skrev Anshum Gupta :
> > >
> > > Please vote for release candidate 2 for Solr 9.7.0
> > >
> > > The artifacts can be downloaded from:
> > >
> > https://dist.apache.org/repos/dist/dev/solr/solr-9.7.0-RC2-rev-675a41516e3f3bacfc975590773e7abdca444ff4
> > >
> > > You can run the smoke tester directly with this command:
> > >
> > > python3 -u dev-tools/scripts/smokeTestRelease.py \
> > >
> > https://dist.apache.org/repos/dist/dev/solr/solr-9.7.0-RC2-rev-675a41516e3f3bacfc975590773e7abdca444ff4
> > >
> > > You can build a release-candidate of the official docker images (full &
> > > slim) using the following command:
> > >
> > > SOLR_DOWNLOAD_SERVER=
> > >
> > https://dist.apache.org/repos/dist/dev/solr/solr-9.7.0-RC2-rev-675a41516e3f3bacfc975590773e7abdca444ff4/solr
> > > && \
> > >  docker build
> > $SOLR_DOWNLOAD_SERVER/9.7.0/docker/Dockerfile.official-full \
> > >--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
> > >-t solr-rc:9.7.0-2 && \
> > >  docker build
> > $SOLR_DOWNLOAD_SERVER/9.7.0/docker/Dockerfile.official-slim \
> > >--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
> > >-t solr-rc:9.7.0-2-slim
> > >
> > > The vote will be open for at least 72 hours i.e. until 2024-09-07 01:00
> > UTC.
> > >
> > > [ ] +1  approve
> > > [ ] +0  no opinion
> > > [ ] -1  disapprove (and reason why)
> > >
> > > Here is my +1
> > >
> > > -Anshum Gupta
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
> >

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [Draft] Release notes for Apache Solr 9.7.0

[Jason Gerlowski]

I update the Release Notes, bringing in the update "project
description" text we've settled on in SOLR-16295.

With that done, the notes look great to me.  Thanks for seeing this
through Anshum - there's a ton of stuff in 9.7 that'll be awesome to
get out there!

On Thu, Sep 5, 2024 at 10:04 PM David Smiley  wrote:
>
> I finished editing / cutting it down.  And I added a sentence at the end
> reminding the reader that this is an editorialized summary that doesn't
> list everything.
>
> Eric, please add that summary about the CLI.  It's somewhat long for a
> release highlight but it's useful.
>
> List of contributors is still needed.  If you like Anshum, I will run the
> script.
>
> On Wed, Aug 28, 2024 at 5:42 AM Eric Pugh  wrote:
>
> > Would grouping all the changes to Solr CLI be worth a mention?  I'd love
> > folks to know that the CLI is changing, so if folks *do* run into bugs,
> > it's not a total surprise.
> >
> > "Solr's CLI (i.e bin/solr) embraces traditional Unix style formatting of
> > long options.  For example, -zkHost is now --zk-host.  Additionally it now
> > supports advanced Zookeeper operations for setting cluster properties,
> > linking configsets, updating ACLs, replacing zkcli.sh.  Lastly, you can now
> > choose to use either a ZK host or Solr URL with command line tools to
> > establish a connection to Solr."
> >
> > On 2024/08/28 02:02:44 Anshum Gupta wrote:
> > > Thanks David!
> > >
> > > The current list is far from something that includes all the updates but
> > > this release has reasonably more stuff than most other minor releases. We
> > > should be able to remove a few of those but a lot of the currently listed
> > > changes would fall into the highlight category.
> > >
> > >
> > > Anshum Gupta
> > >
> > >
> > > On Tue, Aug 27, 2024 at 18:56 David Smiley  wrote:
> > >
> > > > I restructured the announcement so that the CHANGES.txt reference goes
> > > > after the highlights, since people will want to only consider clicking
> > > > that *after* they've read the highlights IMO.  I also added a link to
> > > > our upgrade notes, which I think is critical info to link to.
> > > >
> > > > I noticed the current state of the "Highlights" seems to list
> > > > everything, which is not what this should be.  I prefer to start with
> > > > nothing and add what we think passes a subjective bar of a release
> > > > highlight.  I'll cut much of it tomorrow.
> > > >
> > > > On Tue, Aug 27, 2024 at 9:26 PM David Smiley 
> > wrote:
> > > > >
> > > > > The following script can generate the contributors:
> > > > > https://github.com/apache/solr/pull/2424
> > > > >
> > > > > Need to get that PR merged and integrated into the release
> > > > process/wizard.
> > > > >
> > > > > On Tue, Aug 27, 2024 at 3:14 PM Anshum Gupta 
> > wrote:
> > > > > >
> > > > > > Hi everyone,
> > > > > >
> > > > > > Here are the draft release notes for Apache Solr 9.7.0:
> > > > > > https://cwiki.apache.org/confluence/display/SOLR/ReleaseNote9_7_0
> > > > > >
> > > > > > Please review and update.
> > > > > >
> > > > > > -Anshum
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > > > For additional commands, e-mail: dev-h...@solr.apache.org
> > > >
> > > >
> > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
> >

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Highlighting the Vector Search aspect of Apache Solr

[Jason Gerlowski]

I think it'd be great to update the website, etc. to better highlight
our support for vector search!

The first (and hopefully easiest?) step in this direction would be to
update the "blurb" describing Solr that appears on the website, in
release announcements, Dockerhub etc.  There's actually already a
ticket for this (SOLR-16295) that had some great discussion and
consensus - it just didn't get quite over the line for whatever
reason.  It shouldn't be too hard to add "vector search" to the
largely agreed-upon text from that ticket?  If no one objects I'll
post a revised blurb on that ticket, and we can start pushing it out
in PRs if it gets the general thumbs up on that ticket...

Longer term we could do something more involved, such as adding a
vector search "tile" similar to the "Docker" and "Kubernetes" widgets
on solr.apache.org.  But I imagine it'd be tougher to do that to
coincide with 9.7?

Best,

Jason


On Wed, Aug 28, 2024 at 8:06 PM Anshum Gupta  wrote:
>
> Hi everyone,
>
> For a long time we've referred to Solr as an enterprise full-text search
> engine. With all the support of vector search in Lucene and Solr, I think
> we should add that to how we define our project.
>
> I'm thinking about rightfully highlighting Solr as a vector search engine
> as part of the upcoming Solr 9.7 release. It would also require a change to
> our website and calling out the vector search support on the home page,
> among other places before the release announcement mentions 'vector search'
> or 'multi-modal search'.
>
> This change would allow new users to discover Solr as a vector search
> solution, as well as let the existing users know about the capabilities
> that they might have not realized exist.
>
> What do you all think?

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Guidance on new functionality for query string generation

[Jason Gerlowski]

No concrete feedback for the refactor (i.e. move to SolrJ) you've got underway.

But I just wanted to add a note of encouragement and a +1 to the
general idea.  If it's done right a query-builder would be a huge
boost for Solr's users IMO!

Good luck w/ the refactor Geoffrey and thanks for bringing this up!

Jason

On Sun, Aug 18, 2024 at 4:36 PM Geoffrey Slinker
 wrote:
>
> I will refactor my idea to be found under SolrJ. Also, I will refactor trying 
> to better fit the domain terminology.
>
> After I get the refactoring finished I will post a new message.
>
> If there are those that have time to look at the branch that is in my fork of 
> Apache Solr, especially at the classes and then how to use them in the unit 
> tests I hope they might share more guidance on the subject matter.
>
> For example, a QueryTerm has a value, may have a field, and may have options 
> such as proximity, fuzziness, boost, or a constant score.
>
> title: "pink panther" - is an example of a QueryTerm in my current 
> implementation. Other examples:
> title: "pink panther"~2^0.3 (proximity and boost)
> "pink panther"^=1.4 (default field and constant score)
> firstname:maria~2 (fuzziness)
>
> A QueryTermGroup contains query terms and other query term groups. I think 
> these are probably known as sub-queries.
> An example of a group:
> ( title:"treasure island"^0.5 title:"war and peace"^0.5 )^2.0   A group that 
> has two terms that each have a boost and the entire group has a boost.
>
> A query term group can have its own boost or constant score.
>
> Geoffrey
>
>
> > On Aug 17, 2024, at 10:36 PM, David Smiley  wrote:
> >
> > Perhaps Solr could adopt Elastic's Java QueryBuilder API, more or
> > less?: 
> > https://artifacts.elastic.co/javadoc/co/elastic/clients/elasticsearch-java/8.15.0/co/elastic/clients/elasticsearch/_types/query_dsl/package-summary.html
> > Is this what you referred to?  It needs an example (like what you
> > shared for your builder), which I didn't find.  First iteration could
> > be a minimal useful subset that grows as people have the inclination
> > to do so.  Of course Elastic's outputs Elastic JSON; we'd probably do
> > Solr's "lucene" syntax with local-params when needed.  Solr Query DSL
> > is an option but (A) I think most users would prefer something more
> > compact & simple for the basic use-cases, and (B) the JSON query DSL
> > can only be used in a couple places; there is no QParser for it
> > (although there could/should be one easily).
> >
> >> What is the proper domain name for:
> >> title:"pink panther"
> >
> > In Lucene-speak, this is a PhraseQuery.  Although a query parser will
> > usually run the query analysis chain on it, which can produce
> > different or additional terms that will show up in the actual
> > PhraseQuery.  Lucene's QueryBuilder will do this.  I imagine a SolrJ
> > QueryBuilder would/could have a .fieldQuery(fieldName,foobar) and the
> > phrase vs single term matter is a detail in the output based on
> > whether the foobar value has spaces or not (here it didn't).
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Solr 9.7 Release update

[Jason Gerlowski]

Not to say it's what you're running into here, but I've seen this in
the past when the 'signJarsPublication' task is waiting for your gpg
passphrase on stdin, and times out.  You can use gpg-agent or other
setups to avoid entering your passphrase mid-Gradle execution, but
I've found that to be a bit finicky in my personal experience at
least, and the build sometimes waits for a passphrase that I would've
expected it to get from gpg-agent.

The prompt, when Gradle is waiting on the passphrase, can be hard to
spot.  I guess a giveaway is if Gradle execution pauses and doesn't
output anything for several minutes before 'signJarsPublication'
fails?

Best,

Jason

On Sat, Aug 17, 2024 at 2:37 PM Anshum Gupta  wrote:
>
> The step completed fine, but seems like there's something else going on.
> The RC build is consistently failing at the following step
>
> > Task :solr:api:signJarsPublication
> > gpg: signing failed: Timeout
> > gpg: signing failed: Timeout
> > > Task :solr:api:signJarsPublication FAILED
> > FAILURE: Build failed with an exception.
> > * What went wrong:
> > Execution failed for task ':solr:api:signJarsPublication'.
> > > Process 'command 'gpg'' finished with non-zero exit value 2
>
>
> Creating a test file and signing it works just fine on the terminal though.
>
> > > echo test > test-file
> > > ls -ltr test-file
> > -rw-r--r--  1 anshumg  staff  5 Aug 17 02:29 test-file
> > > gpg --use-agent --sign test-file
> > > ls -ltr test-file.gpg
> > -rw-r--r--  1 anshumg  staff  610 Aug 17 02:29 test-file.gpg
>
>
> I've tried a bunch of things including restarting the gpg client,
> downgrading my gpg version, etc. but nothing seems to solve the issue.
>
> Has anyone seen this before?
>
>
> On Fri, Aug 16, 2024 at 2:02 PM David Smiley  wrote:
>
> > The key stuff is the biggest pain.  I'm glad you're through it.
> >
> > On Fri, Aug 16, 2024 at 12:30 PM Anshum Gupta  wrote:
> > >
> > > Hi everyone,
> > >
> > > I started building the RC last night (Pacific Time) so please don't merge
> > > anything into 9.7 at this point without checking with me.
> > >
> > > I've had a few issues with GPG but they are mostly fixed now.
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
> >
>
> --
> Anshum Gupta

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Making sure updates to v2 APIs happen as v1 changes happen

[Jason Gerlowski]

Happy to give another overview in our Community Meetup on the 21st.
It is complex unfortunately, and we could use help getting to a more
unified place.

On Wed, Aug 14, 2024 at 12:19 PM David Smiley  wrote:
>
> I just want to say, the "flavors" of our APIs makes this difficult.
> It's not even simply V1 vs V2, both have different sub-flavors to be
> decomposed, including core level vs node level variations, original V2
> vs JAX-RS V2, annotation based vs something else... and we have some
> REST framework in here for resources affecting a handful of APIs.
> Some V2 calling V1 instead of the other way around.  I'm looking
> forward to another overview of what's going on :-)
>
> On Wed, Aug 14, 2024 at 8:16 AM David Eric Pugh  
> wrote:
> >
> > Hi all, it's been great to see some of the new improvements to the APIs 
> > recently.   One thing that jumped to mind is that we need to make sure that 
> > any changes/improvements to a V1 API are also made to the corresponding V2 
> > API.
> > We're in that awkward world where we have two apis that are maintained, and 
> > that many of our users are still on the V1 apis, so hence they want those 
> > fixed.  However I feel that we need to make sure that any improvements ALSO 
> > happen in V2, or we will never achieve our goal of getting to the V2 API.
> > I want to draw your attention to this spreadsheet that has the mappings of 
> > the V1 to V2 APIs:
> > https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit#gid=1549066254
> > As we look at improvements to V1 apis, please make sure that the 
> > corresponding V2 apis are updated.
> > I'd be interested in other ideas on how to make sure that our V2 apis 
> > become the focus of improvement, maybe it's time to start talking about how 
> > we deprecate and remove some of the V1 apis?
> > Eric
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [DISCUSS] Community Virtual Meetup, August 2024

[Jason Gerlowski]

Works for me, thanks Sanjay!

On Thu, Aug 15, 2024 at 5:44 AM Marcus Eagan  wrote:
>
> Arm
>
> Marcus Eagan
>
>
>
> On Wed, Aug 14, 2024 at 21:17 sanjay dutt 
> wrote:
>
> > Hey everyone,
> >
> > How does *9:00 AM PST on August 21st* sound for our next community virtual
> > meetup? Let me know if that works for all, or if we need to adjust the
> > time.
> >
> > I'll be sharing the meeting notes and link shortly.
> >
> > Thanks!
> >
> > Sanjay
> >
> > On Wed., Aug. 14, 2024, 9:57 p.m. sanjay dutt, <
> > sanjaydutt.unoffic...@gmail.com> wrote:
> >
> > > I’ll take care of it this month. I reviewed the link Jason shared, but if
> > > I have any questions, I'll reach out to him. Wednesday the 21st at 9 AM
> > > Pacific works for me!
> > >
> > > On Wed, Aug 14, 2024 at 9:34 PM David Smiley  wrote:
> > >
> > >> I very much like the idea of a "standing meeting" as it allows
> > >> attendees to plan and put it on their calendars.  A number of times,
> > >> people wanted to attend yet didn't simply because they didn't know
> > >> when it would be.  FWIW 9am Pacific on Wednesdays is very good for me.
> > >>
> > >> I won't lead next week but I can probably do the one after.
> > >>
> > >> On Tue, Aug 13, 2024 at 9:39 PM Jason Gerlowski 
> > >> wrote:
> > >> >
> > >> > Hey all,
> > >> >
> > >> > It's time once again to start thinking ahead to this month's virtual
> > >> meetup!
> > >> >
> > >> > As always, two questions:
> > >> >
> > >> > 1. Does anyone have an interest in organizing?  Duties are light but
> > >> > it's an important job.  I'm happy to organize by default if there
> > >> > aren't any volunteers in the next day or two.  (Addtl details:
> > >> > https://cwiki.apache.org/confluence/display/SOLR/Meeting+notes)
> > >> >
> > >> > 2. Does anyone have preferences on the date or time-of-day?  Maybe we
> > >> > could shoot for a time-slot sometime in the middle of next week?
> > >> >
> > >> > One suggestion that came up was that we might consider setting a
> > >> > standing date/time for these calls.  That's what we've unintentionally
> > >> ended
> > >> > up doing, since most of our meetings have been on the 3rd or 4th
> > >> > Wednesday at 9am PT
> > >> >
> > >> > Best,
> > >> >
> > >> > Jason
> > >> >
> > >> > -
> > >> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > >> > For additional commands, e-mail: dev-h...@solr.apache.org
> > >> >
> > >>
> > >> -
> > >> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > >> For additional commands, e-mail: dev-h...@solr.apache.org
> > >>
> > >>
> >

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

[DISCUSS] Community Virtual Meetup, August 2024

[Jason Gerlowski]

Hey all,

It's time once again to start thinking ahead to this month's virtual meetup!

As always, two questions:

1. Does anyone have an interest in organizing?  Duties are light but
it's an important job.  I'm happy to organize by default if there
aren't any volunteers in the next day or two.  (Addtl details:
https://cwiki.apache.org/confluence/display/SOLR/Meeting+notes)

2. Does anyone have preferences on the date or time-of-day?  Maybe we
could shoot for a time-slot sometime in the middle of next week?

One suggestion that came up was that we might consider setting a
standing date/time for these calls.  That's what we've unintentionally ended
up doing, since most of our meetings have been on the 3rd or 4th
Wednesday at 9am PT

Best,

Jason

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Changes in JAX-RS APIs mean may need to "gradlew clean" ?

[Jason Gerlowski]

Hi David,

Maybe this deserves its own thread?

I also see the behavior you described - the openApiGenerate task
appears to regenerate on each build.  Perhaps this is "expected" when
using the task's "cleanOutput=true" setting?  Or maybe the task just
doesn't track "inputs" the way Gradle tasks typically do, to know when
it needs to run?  Or I guess it's also possible that the Gradle task
in general would behave as desired, but that we've set it up
incorrectly somehow?

In any case, definitely worth a JIRA ticket or a dedicated thread to discuss.

Best,

Jason

On Tue, Aug 13, 2024 at 12:38 PM David Smiley  wrote:
>
> If I do "gw compile" twice without doing anything, we should expect a
> fully up-to-date gradle build.   Instead, openApiGenerate outputs a
> bunch of stuff (perhaps it's our most noisy task?) including that it
> cleaned the output and did generation.  My expectation is that it
> should do nothing and ideally print nothing either.
>
> On Thu, Jun 6, 2024 at 9:16 AM David Smiley  wrote:
> >
> > Thanks Jason!
> >
> > On Thu, Jun 6, 2024 at 9:02 AM Jason Gerlowski  
> > wrote:
> > >
> > > It worked!  There's a PR here with the fix:
> > > https://github.com/apache/solr/pull/2502. Take a look when you get a
> > > chance and let me know what you think!
> > >
> > > (I haven't created a JIRA ticket, since it's a minor change to our
> > > build.  If anyone would prefer a JIRA ticket to document this beyond
> > > what this dev-thread and Github PR provide, let me know.)
> > >
> > > Best,
> > >
> > > Jason
> > >
> > > On Thu, Jun 6, 2024 at 7:05 AM Jason Gerlowski  
> > > wrote:
> > > >
> > > > Yeah, this is definitely a pain from time to time.
> > > >
> > > > For anyone who hasn't hit this, steps to reproduce are:
> > > >
> > > > 1. Start on a branch that has a new JAX-RS-annotated interface in
> > > > Solr's 'api' module.
> > > > 2. Any gradle task that compiles SolrJ will build Solr's OpenAPI spec,
> > > > generate SolrRequest implementations from that spec, and add them to
> > > > solrj's build directory as a part of its "source set".  These
> > > > generated classes often reference 'model' classes (e.g. request or
> > > > response POJOs) that exist on the current branch, but may not be on
> > > > other branches.
> > > > 3. Switch to a new branch, which lacks the new JAX-RS API.
> > > > 4. If solrj is compiled without running "clean" first, the previously
> > > > generated SolrRequest implementations will fail to compile (because
> > > > the request/response POJOs they rely on are missing).
> > > >
> > > > In terms of *when* this happens, I often see it when changing from a
> > > > PR-branch to main.  Though you can also see it going from 'main' to
> > > > 'branch_9x', if 'main' has a JAX-RS API that hasn't been backported
> > > > yet.
> > > >
> > > > I've been a little despairing in the past about fixing this- I know it
> > > > should be done, but my gradle knowledge is pretty lacking.  Though I
> > > > notice in writing this email that the 'openApiGenerate' task itself
> > > > has a few options that might fix this without any broader gradle
> > > > changes, particularly the "cleanupOutput" and "skipOverwrite" options.
> > > > I'll try playing with those a bit and report back if there's any
> > > > promise.
> > > >
> > > > Best,
> > > >
> > > > Jason
> > > >
> > > >
> > > > On Tue, Jun 4, 2024 at 6:52 PM David Smiley  wrote:
> > > > >
> > > > > I noticed some generated source files, and in particular
> > > > > solr/solrj/build/generated/src/main/java/org/apache/solr/client/solrj/request/FileStoreApi.java
> > > > > that suddenly had a compilation issue.  This is almost certainly due
> > > > > to the API evolving, and SOLR-17302 in particular.  Just do "gradlew
> > > > > clean" to start fresh.  I've hit this a couple times for different
> > > > > specific API issues over some months.
> > > > >
> > > > > Still... should the build be smart enough to avoid this?  For example
> > > > > if the generator is

Re: Javadoc links needed from V1 APIs to V2 APIs

[Jason Gerlowski]

Good idea!

One thing that'd make this a bit awkward is the way individual v1
RequestHandlers often map to many many v2 APIs.  (e.g.
CollectionsHandler covers > 40 "operation" types, for instance!).
Maybe using Javadoc's '@see' directive would scale a little better for
these cases?

In any case, +1 to the idea.  Did you have a JIRA?  (Or will you
create one following this thread?)

Best,

Jason

On Fri, Aug 9, 2024 at 5:07 PM David Smiley  wrote:
>
> Recently, a colleague and I enhanced a V1 API and it wasn't obvious
> how to even find the V2 equivalent.  It was a face-palm moment for me
> after the work was done & merged:
>
> https://issues.apache.org/jira/browse/SOLR-17381?focusedCommentId=17872457&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17872457
>
> I'd like to propose that our V1 APIs link to the corresponding V2 APIs
> in Javadocs in a consistent fashion.  I'm thinking of something like:
>  Superceded by V2: {@link ClusterAPI} and {@link ListAliasesApi}
> and {@link CollectionStatusAPI}.
>
> Not sure if "original V2" vs "JAX-RS V2" should make a difference.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: JSON Parsing, Jackson / Noggit

[Jason Gerlowski]

My hunch is that Jackson would be more performant than Noggit, but I
don't have any hard numbers to back that up so it's just an educated
guess.  I swear there was some other issue that gave Noggit vs.
Jackson numbers but I can't find it now.  SOLR-16691 (where Noble
switched at least some things over to using Jackson) mentions perf
improvements in the issue description but doesn't quantify those.
Maybe someone else with context can chime in with data?

Personally, I'd rather see us use Jackson across the board.  I'm sure
we can write and maintain great serialization code if we want to spend
that effort, but do we?  Ultimately we're here for Search - it's hard
to imagine us wanting to spend anywhere near the amount of time on
serde code that a project like Jackson does as their raison d'etre.

The Noggit lenient parsing *is* really nice for making requests by
hand, but that's a minority use case.  If there's evidence that
Jackson is faster, is it worth slowing down 99% of JSON requests just
so that we can leniently parse the 0.1% of malformed reqs that need
it?  Is it worth the cost of maintaining our own JSON parsing code in
perpetuity?

Best,

Jason

On Mon, Aug 5, 2024 at 11:54 AM David Smiley  wrote:
>
> We have a couple JSON Parsing libraries -- "Noggit" (internal to Solr)
> and "Jackson".  Noggit is more lenient in parsing.  I suppose Solr
> should use Noggit for parsing JSON coming into it, but AFAIK Solr only
> returns/emits valid JSON; yes?  For parsing JSON that we assume is
> compliant (e.g. from Solr), should we prefer Jackson or Noggit?  Are
> there performance advantages?
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Cleaning generated jmh classes

[Jason Gerlowski]

+1 - that makes sense afaict.

On Thu, Aug 1, 2024 at 8:31 AM Gus Heck  wrote:
>
> I keep having to waste time on errors like this:
>
> > Task :solr:benchmark:compileJava
> error: Annotation generator had thrown the exception.
> javax.annotation.processing.FilerException: Attempt to recreate a file for
> type
> org.apache.solr.bench.search.jmh_generated.QueryResponseWriters_query_jmhTest
> at
> jdk.compiler/com.sun.tools.javac.processing.JavacFiler.checkNameAndExistence(JavacFiler.java:732)
> at
> jdk.compiler/com.sun.tools.javac.processing.JavacFiler.createSourceOrClassFile(JavacFiler.java:498)
> at
> jdk.compiler/com.sun.tools.javac.processing.JavacFiler.createSourceFile(JavacFiler.java:435)
> at
> org.gradle.api.internal.tasks.compile.processing.IncrementalFiler.createSourceFile(IncrementalFiler.java:45)
> at
> org.openjdk.jmh.generators.annotations.APGeneratorDestinaton.newClass(APGeneratorDestinaton.java:62)
> at
> org.openjdk.jmh.generators.core.BenchmarkGenerator.generateClass(BenchmarkGenerator.java:448)
> at
> org.openjdk.jmh.generators.core.BenchmarkGenerator.generate(BenchmarkGenerator.java:86)
> ... etc
>
> Does anyone object to adding this to :solr:benchmark so that clean actually
> cleans everything?
>
> clean {
>   delete "${layout.projectDirectory}/src/java/generated"
> }
>
>
> --
> http://www.needhamsoftware.com (work)
> https://a.co/d/b2sZLD9 (my fantasy fiction book)

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: SIP-7 New Admin UI

[Jason Gerlowski]

Perfect!  Yeah, I think getting SIP-7 updated is a big step as it'll
let folks catch up on where things stand and on the "big picture"
decisions.

Thanks for the PR link - will take a look shortly!

On Tue, Jul 30, 2024 at 11:59 AM Christos Malliaridis
 wrote:
>
> Nothing easier than that, and probably a better idea than referencing a
> branch that may be deleted at some point.
>
> Here are the updated links:
> - Github pull-request
> <https://github.com/apache/solr/pull/2605#issue-2438171887>
> - Dev Docs for getting started
> <https://github.com/apache/solr/blob/15402bd66b4108340f05549c923ced9e66a581d6/dev-docs/ui/introduction.adoc>
>
> I have linked the PR to the initial Jira ticket, but I would like to
> introduce an epic for breaking down the scope of the UI into smaller
> pieces, once the migration officially begins. We would also have to update
> the SIP-7 with newer information, which I am working on. :)
>
> On Tue, Jul 30, 2024 at 6:07 PM Jason Gerlowski 
> wrote:
>
> > > *For the implementation* I'd like to know from the committers how
> > > comfortable they feel with the code base for the new UI. Input about code
> > > complexity, code readability and suitability of the framework in the
> > > overall Solr project would be helpful to decide whether we could proceed
> > > with this framework or not.
> >
> > Happy to chime in there!  Speaking for myself at least, I didn't
> > realize there was "POC" code ready to look at.  It might be a little
> > easier for folks to review and ask questions about the code if you use
> > it to create a "Draft" PR from your fork/branch - is that something
> > you'd be willing to do?
> >
> > Jason
> >
> > On Sun, Jul 28, 2024 at 1:24 PM Christos Malliaridis
> >  wrote:
> > >
> > > With the input and feedback from the virtual meetup I'd like to wrap up
> > the
> > > current progress with a couple of references:
> > >
> > > *Figma Design*
> > > I have created a Figma design (WIP)
> > > <
> > https://www.figma.com/design/VdbEfcWQ8mirFNquBzbPk2/Apache-Solr-Admin-UI-v2-Concept?node-id=2-100&t=wW7ta6KH9VHTZhgq-1
> > >
> > > that represents the new UI, inspired by the current documentation style
> > and
> > > webapp. I have already received some community feedback via Slack
> > > <https://apachesolr.slack.com/archives/C01GVPZSSK0/p1718289047297999>
> > and
> > > will share it the next few days in the users mailing list, as proposed in
> > > the meetup, to get further feedback. The designs are far from final, but
> > > can be used as a starting point for discussions.
> > >
> > > The figma file includes references to Jira issues that are addressed with
> > > the new design ideas. Elements from the current UI are restructured,
> > > replaced or even completely removed. I've done this by following some
> > > thoughts of improvement I had, so some changes may not be practical and
> > may
> > > be updated. Note that the current focus of the new UI is not to add new
> > > features, but rather include features that the current UI has and are
> > > useful, so that it can at some point replace the EOL UI implementation.
> > > Once we have a good foundation and have replaced the old UI, we can plan
> > > new features (IMO).
> > >
> > > *Compose Implementation*
> > > I've already started implementing the UI as a proof-of-concept
> > > <https://github.com/malliaridis/solr/tree/composeui> with Compose
> > > Multiplatform. I started with the environment section
> > > <
> > https://www.figma.com/design/VdbEfcWQ8mirFNquBzbPk2/Apache-Solr-Admin-UI-v2-Concept?node-id=30-6414
> > >
> > > that displays version information, java properties and command line
> > > arguments in one place. The implementation includes the base for the UI
> > > module and one screen.
> > >
> > > The goal of the current proof-of-concept is to keep the changes simple
> > and
> > > focused to a single use case, so that it is possible to decide whether we
> > > want to proceed with this framework or not. Therefore, many elements are
> > > not implemented yet or not working correctly (even the styling is not
> > > finished). If you think further elements are necessary to make a
> > decision,
> > > let me know so that I can have a look into it.
> > >
> > > The new UI works with localhost:8983 (without auth) and is automatically
> 

Re: SIP-7 New Admin UI

[Jason Gerlowski]

> *For the implementation* I'd like to know from the committers how
> comfortable they feel with the code base for the new UI. Input about code
> complexity, code readability and suitability of the framework in the
> overall Solr project would be helpful to decide whether we could proceed
> with this framework or not.

Happy to chime in there!  Speaking for myself at least, I didn't
realize there was "POC" code ready to look at.  It might be a little
easier for folks to review and ask questions about the code if you use
it to create a "Draft" PR from your fork/branch - is that something
you'd be willing to do?

Jason

On Sun, Jul 28, 2024 at 1:24 PM Christos Malliaridis
 wrote:
>
> With the input and feedback from the virtual meetup I'd like to wrap up the
> current progress with a couple of references:
>
> *Figma Design*
> I have created a Figma design (WIP)
> 
> that represents the new UI, inspired by the current documentation style and
> webapp. I have already received some community feedback via Slack
>  and
> will share it the next few days in the users mailing list, as proposed in
> the meetup, to get further feedback. The designs are far from final, but
> can be used as a starting point for discussions.
>
> The figma file includes references to Jira issues that are addressed with
> the new design ideas. Elements from the current UI are restructured,
> replaced or even completely removed. I've done this by following some
> thoughts of improvement I had, so some changes may not be practical and may
> be updated. Note that the current focus of the new UI is not to add new
> features, but rather include features that the current UI has and are
> useful, so that it can at some point replace the EOL UI implementation.
> Once we have a good foundation and have replaced the old UI, we can plan
> new features (IMO).
>
> *Compose Implementation*
> I've already started implementing the UI as a proof-of-concept
>  with Compose
> Multiplatform. I started with the environment section
> 
> that displays version information, java properties and command line
> arguments in one place. The implementation includes the base for the UI
> module and one screen.
>
> The goal of the current proof-of-concept is to keep the changes simple and
> focused to a single use case, so that it is possible to decide whether we
> want to proceed with this framework or not. Therefore, many elements are
> not implemented yet or not working correctly (even the styling is not
> finished). If you think further elements are necessary to make a decision,
> let me know so that I can have a look into it.
>
> The new UI works with localhost:8983 (without auth) and is automatically
> built and distributed together with the current webapp via `gradlew dev` and
> after launching a solr instance (build times may be optimized and may take
> longer during the first build of the new UI). A button has been added in
> the java properties tab of the current UI (see top right corner) to
> demonstrate how the linking would work during the migration phase. The new
> UI can also be launched as a desktop app via `gradlew :solr:compose-ui:run`.
>
> I've also added a couple of resources for getting started in dev-docs/ui
> , but I am
> not a good writer so it is kept simple and some parts are written with
> ChatGPT's support.
>
>
> *Next Actions**For the designs* I'd like to gather more input from
> experienced Solr users and developers, because I haven't used Solr in
> production yet and have no clue if the designs would still be good and work
> out for larger clusters and real-world scenarios. You can include
> references to specific sections for your feedback by providing the page +
> screen title or by selecting the element in the left menu and copying a
> link via "Copy as -> Copy link to selection". Sadly, feedback via Figma
> comments is not available in the free plan and also not available if the
> designs are published to the Figma Community.
>
> If you have a better way for gathering feedback for the designs, please let
> me know.
>
> *For the implementation* I'd like to know from the committers how
> comfortable they feel with the code base for the new UI. Input about code
> complexity, code readability and suitability of the framework in the
> overall Solr project would be helpful to decide whether we could proceed
> with this framework or not.
>
> *P.S. Excuse me for the overwhelmingly long message again.*

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional comman

Re: Use MockDirectoryFactory not RAMDirectoryFactory in test configs

[Jason Gerlowski]

+1

I took a crack at fixing this (on 'main' at least) here:
https://github.com/apache/solr/pull/2598

Let me know what you guys think when you get a chance to review.

On Sun, Jul 7, 2024 at 7:57 PM Jan Høydahl  wrote:
>
> +1
>
> Experienced the same locally, tried to override lock factory but did not dig 
> deeper and ended up running some tests in gradle instead of IntelliJ. 
> Annoying!
>
> Jan Høydahl
>
> > 1. juli 2024 kl. 22:57 skrev David Smiley :
> >
> > Some of our tests don't run correctly/consistently via IntelliJ's
> > internal JUnit runner (compared to Gradle).  IntelliJ's JUnit runner
> > is faster and better integrated, saving developer productivity.
> >
> > I debugged the issue.  The test TestInPlaceUpdatesDistrib failed
> > because "RAMDirectory can only be used with the 'single' lock factory
> > type.".  This test's solrconfig.xml is very typical:
> >class="${solr.directoryFactory:solr.RAMDirectoryFactory}"
> > The code for this does *not* set the property.  It turns out the
> > randomization.gradle configures all tests to run with this property
> > set to "org.apache.solr.core.MockDirectoryFactory".  At least this is
> > the only one; the others set there have null values.
> > https://github.com/apache/solr/blob/bde8c14bddecc2a417d2fd36abe965675e8e670e/gradle/testing/randomization.gradle#L129
> >
> > Proposal: remove this from the gradle config.  Instead, replace all
> > "solr.directoryFactory:solr.RAMDirectoryFactory" with
> > "solr.directoryFactory:solr.MockDirectoryFactory" in all test
> > solrconfig files.
> > This is ultimately a minor refactoring of sorts; an improvement to the
> > build.  No JIRA.
> >
> > Any insights / concerns?
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Exception handling in background tasks

[Jason Gerlowski]

Hi Andrey,

It's hard to say how much this affects users in practice, but I agree
that it sounds concerning.  Definitely worth auditing our existing
uses of 'submit' IMO.

Some cases might be tough to handle.  If an API call is intentionally
queueing up an operation to run asynchronously and then returns back
to a user, it's hard to surface that information to them after the
fact.  But we should make sure that the exception gets logged at an
absolute minimum - it sounds like that's not the case by default for
these usages of "submit" that don't hold onto the 'Future'?

Best,

Jason

On Tue, Jul 23, 2024 at 12:55 PM Andrey Bozhko  wrote:
>
> Hi all,
>
> I'd like to bring up for discussion how Solr handles failures of various
> background tasks.
>
>
> Typically with an ExecutorService, the task can be offloaded to a
> background thread via `execute(...)` or `submit(...)` methods:
> - if using `execute(Runnable)` method, any exception thrown by the task
> (assuming that the task doesn't have a try-catch) is intercepted and
> handled by the thread's UncaughtExceptionHandler - e.g., printed to console;
> - if using `submit(Callable)` method, the caller *must* hold on to the
> future instance returned from the method, as the task result (or the task
> failure) can only be
> retrieved by invoking `get()` on the future instance.
>
>
> That said, there are quite a few places in Solr codebase where the
> background task is created by invoking `submit(...)` method but which do
> not retan any reference to the returned future.
> So if the background task fails for any reason, the failure will go
> completely unnoticed.
>
> Some of these places are:
> - CoreContainer#runAsync (
> https://github.com/apache/solr/blob/06950c656f21577db624102b913fb659ef1f0306/solr/core/src/java/org/apache/solr/core/CoreContainer.java#L2588
> )
> - SolrZkClient.ProcessWatchWithExecutor#process (
> https://github.com/apache/solr/blob/06950c656f21577db624102b913fb659ef1f0306/solr/solrj-zookeeper/src/java/org/apache/solr/common/cloud/SolrZkClient.java#L1077-L1085
> )
>
> For example, CoreContainer#runAsync may be used to asynchronously reload
> the collection schema in certain cases - so if the reloading fails, I
> imagine the users would want to be aware of the failure and not let it go
> unnoticed.
>
>
> Does any of the above describe a real issue? Well, so far I tried searching
> the codebase for usage of ExecutorService `submit(...)` methods and
> replacing them with `execute(...)` where it makes sense - and then running
> the tests. Doing so broke 200+ tests due to uncaught exceptions in
> background threads. But I did not go through those uncaught exceptions to
> see which ones indicate a real issue and which ones are harmless.
>
> Thoughts?
>
>
> Best,
> Andrey Bozhko

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: JIRA "pull-request-available" label

[Jason Gerlowski]

Neat, thanks David!

On Wed, Jul 17, 2024 at 2:05 PM David Smiley  wrote:
>
> Our JIRA issues, according to .asf.yaml configuration, *should have*
> been getting a label "pull-request-available".  This wasn't working
> because an ASF bot needed committer permissions in JIRA, in accordance
> with ASF docs on this.  I addressed that matter last night.  A nice
> side-effect of the bot doing this is that it notifies anyone watching
> the JIRA issue so that we're aware of the PR.  The PR auto-link
> doesn't do that.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [DISCUSS] Community Virtual Meetup, July 2024

[Jason Gerlowski]

Alright, sorry for the delay in populating these, but here are the
links for this month's meetup:

Confluence Meeting Notes Page:
https://cwiki.apache.org/confluence/display/SOLR/2024-07-25+Meeting+notes
Google Meet: https://meet.google.com/bmc-uvwr-omd
TimeAndDate.com:
https://www.timeanddate.com/worldclock/fixedtime.html?msg=Solr+Community+Meetup%2C+July+2024&iso=20240725T09&p1=224&ah=1

As always, if you have a topic you'd like to discuss please add it to
the Meeting Notes page.  Looking forward to seeing you all there!

On Fri, Jul 19, 2024 at 4:16 AM Eric Pugh
 wrote:
>
> Looking forward to it!
>
> > On Jul 18, 2024, at 6:54 PM, David Smiley  wrote:
> >
> > Thanks; yes that works for me.
> >
> > On Thu, Jul 18, 2024 at 9:05 AM Jason Gerlowski  
> > wrote:
> >>
> >> Alright,
> >>
> >> I'm happy to organize this month.  I'll create the meeting notes pages
> >> and links shortly.
> >>
> >> As for a time/date, how would 9am PT (noon ET) on Thursday the 25th
> >> work for everyone?
> >>
> >> Best,
> >>
> >> Jason
> >>
> >> On Mon, Jul 15, 2024 at 8:31 AM Jason Gerlowski  
> >> wrote:
> >>>
> >>> Hey all,
> >>>
> >>> It's time once again to start thinking ahead to this month's virtual 
> >>> meetup!
> >>>
> >>> As always, two questions:
> >>>
> >>> 1. Does anyone have an interest in organizing?  Duties are light but
> >>> it's an important job.  I'm happy to organize by default if there
> >>> aren't any volunteers in the next day or two.  (Addtl details:
> >>> https://cwiki.apache.org/confluence/display/SOLR/Meeting+notes)
> >>>
> >>> 2. Does anyone have preferences on the date or time-of-day?  Maybe we
> >>> could shoot for time-slot sometime in the middle of next week?
> >>>
> >>> Best,
> >>>
> >>> Jason
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> >> For additional commands, e-mail: dev-h...@solr.apache.org
> >>
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
>
> ___
> Eric Pugh | Founder | OpenSource Connections, LLC | 434.466.1467 | 
> http://www.opensourceconnections.com <http://www.opensourceconnections.com/> 
> | My Free/Busy <http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 
> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
> This e-mail and all contents, including attachments, is considered to be 
> Company Confidential unless explicitly stated otherwise, regardless of 
> whether attachments are marked as such.
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [DISCUSS] Community Virtual Meetup, July 2024

[Jason Gerlowski]

Alright,

I'm happy to organize this month.  I'll create the meeting notes pages
and links shortly.

As for a time/date, how would 9am PT (noon ET) on Thursday the 25th
work for everyone?

Best,

Jason

On Mon, Jul 15, 2024 at 8:31 AM Jason Gerlowski  wrote:
>
> Hey all,
>
> It's time once again to start thinking ahead to this month's virtual meetup!
>
> As always, two questions:
>
> 1. Does anyone have an interest in organizing?  Duties are light but
> it's an important job.  I'm happy to organize by default if there
> aren't any volunteers in the next day or two.  (Addtl details:
> https://cwiki.apache.org/confluence/display/SOLR/Meeting+notes)
>
> 2. Does anyone have preferences on the date or time-of-day?  Maybe we
> could shoot for time-slot sometime in the middle of next week?
>
> Best,
>
> Jason

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

[DISCUSS] Community Virtual Meetup, July 2024

[Jason Gerlowski]

Hey all,

It's time once again to start thinking ahead to this month's virtual meetup!

As always, two questions:

1. Does anyone have an interest in organizing?  Duties are light but
it's an important job.  I'm happy to organize by default if there
aren't any volunteers in the next day or two.  (Addtl details:
https://cwiki.apache.org/confluence/display/SOLR/Meeting+notes)

2. Does anyone have preferences on the date or time-of-day?  Maybe we
could shoot for time-slot sometime in the middle of next week?

Best,

Jason

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: IntelliJ not working with latest main

[Jason Gerlowski]

> Cannot resolve symbol CollectionsApi, ListCollectionsResponse.

+1 to what Christos said.

In short - 'CollectionsApi' and 'ListCollectionsResponse' are both
Java classes that we generate from our OAS.  That generation is setup
in gradle to always happen prior to compiling "solr-solrj" (which of
course happens before compiling "solr-core").  If you're not seeing
that happen and have a short script to reproduce, I'm happy to take a
look!

> SolrCore 'collection1' is not available due to init failure: RAMDirectory can 
> only be used with the 'single' lock factory type.

Not sure if there's a fix yet, but David discussed this a bit recently
in the thread "Use MockDirectoryFactory not RAMDirectoryFactory in
test configs".

As discussed there at least, it sounded like this only happens when
using IntelliJ's "native" test-runner to run tests, instead of having
IntelliJ utilize our gradle build.  So a workaround might be to have
IntelliJ run the tests using gradle for the time being.

(And, now that I think about it, if IntelliJ isn't using the gradle
build for some reason, it makes sense that the code-generation
wouldn't be happening either.  So maybe switching IntelliJ to using
gradle for the test-runner might solve both of your issues?)

Jason

On Sun, Jul 14, 2024 at 7:40 AM Christos Malliaridis
 wrote:
>
> Hey Ishan, I'm sorry to hear that. The test uses a generated class to
> verify the bug fix, since the issue occurs in a template fro class
> generation.
>
> The error you sent points to one of generated classes that I used
> (CollectionsApi) and that is also in SolrJ module. Since it is a generated
> class, is maybe something broken with your build cache? If I am not
> mistaken, the classes should always be generated in advance for SolrJ.
>
> I also tried to reproduce the error and it said for "./gradlew main" that
> main does not exist. Is it maybe "./gradlew dev" what you are trying to
> execute?
>
>
> On Sun, 14 Jul 2024, 11:54 Ishan Chattopadhyaya, 
> wrote:
>
> > I removed the ApiMustacheTemplateTests.java temporarily, and all errors
> > went away.
> > However, tried running tests, and ran into this:
> >
> > BasicFunctionalityTest.java:
> > java.lang.RuntimeException:
> > org.apache.solr.core.SolrCoreInitializationException: SolrCore
> > 'collection1' is not available due to init failure: RAMDirectory can only
> > be used with the 'single' lock factory type.
> >
> > Any ideas, please?
> >
> >
> > On Sun, 14 Jul 2024 at 14:17, Ishan Chattopadhyaya <
> > ichattopadhy...@gmail.com> wrote:
> >
> > > Hi all,
> > > I pulled latest commits, but ./gradlew main is resulting in a project
> > that
> > > doesn't load without errors:
> > > ApiMustacheTemplateTests.java: Cannot resolve symbol CollectionsApi,
> > > ListCollectionsResponse.
> > >
> > > Any ideas, please?
> > >
> > > If I rollback to the commit before the following one, it works fine:
> > >
> > > commit 461955f00118c69d06f50e72addeff12c8dd8169
> > > Author: Christos Malliaridis 
> > > Date:   Tue Jun 11 18:15:01 2024 +0200
> > >
> > > SOLR-17326: Fix references in generated SolrRequest impls (#2510)
> > >
> > > A handful of the v2 SolrRequest implementations generated
> > > by our OAS spec relied on response model classes whose names
> > > conflicted with other (unrelated) classes in solrj.  This caused
> > > errors at request time as JacksonParsingResponse would try to
> > > deserialize the JSON, XML, etc. response body into these
> > > unintended classes.
> > >
> > > This commit fixes this by modifying the 'api.mustache' template
> > > so that generated SolrRequest classes now reference their
> > > response model using the fully-qualified classname (i.e. including
> > > the package).  This resolves the ambiguity.
> > >
> > > -
> > >
> > > Co-authored-by: Jason Gerlowski 
> > >
> > >
> > >
> >

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: ASF Jenkins Build Timeouts

[Jason Gerlowski]

Thanks for the tip Uwe!

Not hearing any objections, so I'll give the dynamic-timeout a try on our
jobs.  Can always revert on the off chance that objections trickle in late
here...

Best,

Jason

On Tue, Jul 9, 2024 at 7:55 AM Uwe Schindler  wrote:

> Hi,
>
> on Policeman Jenkins I use the dynamic timeout. This behaves quite well
> since a while. It averages the build time over the history and if a new
> build is taking significantly longer is kills it, you can choose between
> "likely stuck" (that's what I use) and "elastic" (give a percentage):
>
> This makes maintenance easier to handle, especially when build times
> differ per node.
>
> Uwe
> Am 09.07.2024 um 13:07 schrieb Jason Gerlowski:
>
> What is the current timeout?  You propose setting it to 3 hours?
>
> For all the jobs I listed above, the timeout on the Jenkins job itself
> is currently 24 hours (!), and I'm proposing lowering that to 3 hours
> on each of those listed jobs.
>
> Maybe we should have timeouts on specific test cases or test suites at
> the JUnit level, but I'm not proposing any change to those here.  I
> think that would need more thought.
>
> Jason
>
> On Mon, Jul 8, 2024 at 10:53 PM David Smiley  
>  wrote:
>
> What is the current timeout?  You propose setting it to 3 hours?  Are
> you referring to a Jenkins job specific thing or a Java based test
> suite timeout?  I've looked into the latter, which is set to 2 hours
> in LuceneTestCase (see @TimeoutSuite) -- pretty crazy and IMO should
> not be set there at all.  I've overridden this for the Crave build
> with an unsatisfying hack.
>
> On Mon, Jul 8, 2024 at 1:58 PM Jason Gerlowski  
>  wrote:
>
> Hey all,
>
> I spent some time last week looking into a few ASF Jenkins jobs that
> hung indefinitely until the timeout killed them nearly a day later.
> The specific cause has since been fixed, but it raised the question:
> why is the timeout for our builds so long?
>
> Allowing "stuck" builds to linger blocks other jobs from running, and
> seemingly even our longest-running jobs should complete in a few
> hours.  Would anyone object to my lowering some of these long build
> timeouts to, say, 3 hours?
>
> The following jobs would be affected:
>
> - Solr-BadApples-Tests-main
> - Solr-check-9.6
> - Solr-Check-9.x
> - Solr-Check-main
> - Solr-Check-main-s390x
> - Solr-Docker-Nightly-9.6
> - Solr-Docker-Nightly-9.x
> - Solr-Docker-Nightly-main
> - Solr-Docker-Official-Test-main
> - Solr-Docker-Test-main
> - Solr-NightlyTests-main
> - Solr-Smoketest-9.6
> - Solr-Smoketest-9.x
>
> Best,
>
> Jason
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremenhttps://www.thetaphi.de
> eMail: u...@thetaphi.de
>
>

Re: ASF Jenkins Build Timeouts

[Jason Gerlowski]

> What is the current timeout?  You propose setting it to 3 hours?

For all the jobs I listed above, the timeout on the Jenkins job itself
is currently 24 hours (!), and I'm proposing lowering that to 3 hours
on each of those listed jobs.

Maybe we should have timeouts on specific test cases or test suites at
the JUnit level, but I'm not proposing any change to those here.  I
think that would need more thought.

Jason

On Mon, Jul 8, 2024 at 10:53 PM David Smiley  wrote:
>
> What is the current timeout?  You propose setting it to 3 hours?  Are
> you referring to a Jenkins job specific thing or a Java based test
> suite timeout?  I've looked into the latter, which is set to 2 hours
> in LuceneTestCase (see @TimeoutSuite) -- pretty crazy and IMO should
> not be set there at all.  I've overridden this for the Crave build
> with an unsatisfying hack.
>
> On Mon, Jul 8, 2024 at 1:58 PM Jason Gerlowski  wrote:
> >
> > Hey all,
> >
> > I spent some time last week looking into a few ASF Jenkins jobs that
> > hung indefinitely until the timeout killed them nearly a day later.
> > The specific cause has since been fixed, but it raised the question:
> > why is the timeout for our builds so long?
> >
> > Allowing "stuck" builds to linger blocks other jobs from running, and
> > seemingly even our longest-running jobs should complete in a few
> > hours.  Would anyone object to my lowering some of these long build
> > timeouts to, say, 3 hours?
> >
> > The following jobs would be affected:
> >
> > - Solr-BadApples-Tests-main
> > - Solr-check-9.6
> > - Solr-Check-9.x
> > - Solr-Check-main
> > - Solr-Check-main-s390x
> > - Solr-Docker-Nightly-9.6
> > - Solr-Docker-Nightly-9.x
> > - Solr-Docker-Nightly-main
> > - Solr-Docker-Official-Test-main
> > - Solr-Docker-Test-main
> > - Solr-NightlyTests-main
> > - Solr-Smoketest-9.6
> > - Solr-Smoketest-9.x
> >
> > Best,
> >
> > Jason
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: SIP-7 New Admin UI

[Jason Gerlowski]

Hey Christos,

Sorry for the delay responding here - lots of context to read up on!

Firstly, thanks for the huge effort you've put into writing this all
up!  Quite the thorough job, and it's really helpful to enable us
non-UI folks to follow along haha.

If I understand things correctly, there's a few distinct aspects to
your proposal:

1. New UI would live alongside the existing one (for a time)
2. The code for the new UI would live in the main repository.
3. Development would be piece-meal (i.e. not one big code-dump)

Overall, this sounds like a reasonable approach to me.

I think a big concern with putting code in the main repo is that it's
pretty far from the (current) PMC's/community's wheelhouse to
maintain.  I definitely share that concern.  But IMO we're already
sortof at a "worst case" in that regard with our existing Admin UI
code.  Doing the "refresh" in the main repo gives us a forcing
function (i.e. the review process itself) to ensure that at least a
few community members will understand the code to at least some
extent.  That'll be a huge improvement over where we are today.

Anyway, I'm a cautious '+1' based on these details at least.  To quote
a message from Jan in Slack: "I'd rather see some imperfect movement
than a perfect plan never realized."

(Here's hoping my reply will bump this to the top of folks' Inboxes,
and get you some more feedback.)

Best,

Jason

On Mon, Jul 1, 2024 at 12:25 PM Christos Malliaridis
 wrote:
>
> Hello everyone,
>
> In regards to SIP-7
> 
> and SIP-10
> 
> I would like to add my perspective and address the current concerns about
> implementing a new UI, so that we can take some actions and improve the
> overall quality and experience of Solr Admin UI.
>
> There are many discussions and opinions about the UI and how to resolve the
> current issues, but they all led to the topic becoming stale. In my
> opinion, developing and introducing a new UI into the main repository piece
> by piece without replacing the current UI until feature-complete could
>
> - address all the issues currently reported (and not),
> - add new features,
> - replace the EOL framework and
> - improve the overall user experience.
>
> And the maintenance, which is one of the most important parts, could be
> addressed with the right choice of framework.
>
> I created a detailed writeup
> 
> for those who are interested, where I also write about the alternative
> approaches proposed in the past and listing the pros and cons of each one
> individually.
>
> I also started to improve this part by simply designing a new UI
> 
> and addressing multiple issues at once. I have already received some community
> feedback
> , but
> it is far from production-ready and needs more input. I think this could be
> further refined and moved to development if there is consensus on that as
> the initial approach.
>
> What's your opinion about this approach and do you have any concerns that
> have not been addressed?
>
> Best regards,
> Christos

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

ASF Jenkins Build Timeouts

[Jason Gerlowski]

Hey all,

I spent some time last week looking into a few ASF Jenkins jobs that
hung indefinitely until the timeout killed them nearly a day later.
The specific cause has since been fixed, but it raised the question:
why is the timeout for our builds so long?

Allowing "stuck" builds to linger blocks other jobs from running, and
seemingly even our longest-running jobs should complete in a few
hours.  Would anyone object to my lowering some of these long build
timeouts to, say, 3 hours?

The following jobs would be affected:

- Solr-BadApples-Tests-main
- Solr-check-9.6
- Solr-Check-9.x
- Solr-Check-main
- Solr-Check-main-s390x
- Solr-Docker-Nightly-9.6
- Solr-Docker-Nightly-9.x
- Solr-Docker-Nightly-main
- Solr-Docker-Official-Test-main
- Solr-Docker-Test-main
- Solr-NightlyTests-main
- Solr-Smoketest-9.6
- Solr-Smoketest-9.x

Best,

Jason

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Solr "benchmark", wikipedia

[Jason Gerlowski]

I've found the "benchmark" module easier to extend and write custom
benchmarks with.  And from what I recall it was easier to attach
profilers, get flamegraphs, etc - which seems useful if there are
questions around parallel-segment search.

But it depends on exactly what you want.  I've found solr-bench to be
useful in other scenarios as well.

On Fri, Jun 28, 2024 at 4:03 PM Ishan Chattopadhyaya
 wrote:
>
> Might need to install the prerequisites on Linux as follows:
>
> apt install wget unzip zip ant ivy lsof git netcat make openjdk-11-jdk
> maven jq
>
> Something similar for MacOS can be installed using homebrew etc. The jq and
> maven are important.
>
> On Sat, 29 Jun, 2024, 1:04 am Ishan Chattopadhyaya, <
> ichattopadhy...@gmail.com> wrote:
>
> > Off the top of my head:
> >
> > git clone https://GitHub.com/fullstorydev/solr-bench
> > cd solr-bench
> > git checkout ishan/local-mode-fixes
> >
> > mvn clean compile assembly:single
> > ./cleanup.sh && ./stress.sh -c  suites/stress-facets-local.json
> >
> > (Replace commit with the commit id hash of the latest tip of branch_9x)
> >
> > On Sat, 29 Jun, 2024, 12:32 am Ishan Chattopadhyaya, <
> > ichattopadhy...@gmail.com> wrote:
> >
> >> Hi David,
> >> I'll publish benchmarks and steps to reproduce tests we are doing using
> >> solr-bench. Let's collaborate a bit more to make sure we can extract
> >> maximum performance out of the feature. Right now, I'm not seeing good
> >> numbers and it is a bit worrisome.
> >> Thanks and regards,
> >> Ishan
> >>
> >> On Sat, 29 Jun, 2024, 12:14 am Gus Heck,  wrote:
> >>
> >>> To comment further, I think it would be good if we had query benchmarks
> >>> that tried to map directly to the queries benchmarked for lucene, against
> >>> the same data. This would give us a notion of which half of the equation
> >>> any slow down comes from (or speed up!)
> >>>
> >>> On Fri, Jun 28, 2024 at 2:42 PM Gus Heck  wrote:
> >>>
> >>> > Yes. In fact I have an example in JesterJ of indexing the luceneutil
> >>> > data... (but it's still in early stages, I think it's still against
> >>> > _default schema, perhaps... (need to look again, haven't had time to
> >>> work
> >>> > on it recently)
> >>> >
> >>> > https://github.com/nsoft/jesterj/tree/master/code/examples/wikidocs
> >>> >
> >>> > On Fri, Jun 28, 2024 at 11:33 AM David Smiley 
> >>> wrote:
> >>> >
> >>> >> I was thinking of using Solr's "benchmark" module/thing to benchmark
> >>> >> parallel segment search (coming to Solr 9.7 but needs more love).  I
> >>> >> don't notice any substantial data to query for in this module,
> >>> >> however.  Has anyone considered adding wikipedia, like how Lucene's
> >>> >> "luceneutil" does?  Or something else?  Is this a bad idea for this
> >>> >> benchmark module or should I be looking elsewhere like Searchscale's
> >>> >> solr-bench[1]?
> >>> >>
> >>> >> https://github.com/searchscale/solr-bench
> >>> >>
> >>> >> ~ David Smiley
> >>> >> Apache Lucene/Solr Search Developer
> >>> >> http://www.linkedin.com/in/davidwsmiley
> >>> >>
> >>> >> -
> >>> >> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> >>> >> For additional commands, e-mail: dev-h...@solr.apache.org
> >>> >>
> >>> >>
> >>> >
> >>> > --
> >>> > http://www.needhamsoftware.com (work)
> >>> > https://a.co/d/b2sZLD9 (my fantasy fiction book)
> >>> >
> >>>
> >>>
> >>> --
> >>> http://www.needhamsoftware.com (work)
> >>> https://a.co/d/b2sZLD9 (my fantasy fiction book)
> >>>
> >>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Looking for final review of SOLR-16842

[Jason Gerlowski]

> I wanted to confirm that our Jenkins builds are still happy, and while our 
> normal Jenkins boxes are busted

The boxes are fixed!  Or, lucene-solr-1 should be at least.  I'm still
looking into lucene-solr-2.  So any failures at this point are
legitimate issues.

Case in point, it looks like our 9x BATs tests are broken.  I'm able
to reproduce the error in this build on two different Macs. (Looking
into a fix, now, so hopefully I can handle this one) [1].

Slight tangent - do we have any time-constraints on these builds?
Following a bats failure the job hangs, seemingly indefinitely.
(Current one has been running for ~17 hrs)  It's a real holdup
considering we only have 1 functional build machine at this point.

> I’m running out of time before vacation for three weeks (hello Spain!) and so 
> thinking [list of 3 or 4 different options]

Option (3) i.e. the "skip backporting until after your trip" option,
sounds best to me.  I agree with Jan that users will benefit from 9.x
deprecation warnings, and that it's worth getting something on
branch_9x at some point.  But IMO there's no rush on that, as long as
you're still up for it following your time away.  It's less stress,
afaik there's no proposed 9.x release that you'd miss by waiting, and
since the build-machines have only recently gotten fixed the change
would probably benefit from a bit more time to "bake" on 'main'
anyway.

Just my two-cents.

When you're back around later this summer I'm willing to pair on some
of those backporting difficulties, fwiw.

Jason

[1] https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/6066/

On Fri, Jun 28, 2024 at 9:39 AM Jan Høydahl  wrote:
>
> From an upgrade 9->10 perspective, it would be good to start getting 
> deprecation warnings in 9.x for old options and change scripts etc so you 
> don't have a big-bang moment in 10.0.
>
> A potential option 5) is to introduce the new options and deprecate old in 
> 10.0, meaning they won't be removed until 11.0
>
> Jan
>
> > 28. juni 2024 kl. 13:18 skrev Eric Pugh :
> >
> > Hi all….  Quick update that I merged SOLR-16842 into main.
> >
> > I wanted to confirm that our Jenkins builds are still happy, and while our 
> > normal Jenkins boxes are busted, I see that we have 
> > “Solr-Check-main-s390x”?   Looking at it, it appears that recent code 
> > merges are fine, that the one test that failed on run 
> > https://ci-builds.apache.org/job/Solr/job/Solr-Check-main-s390x/646/ is 
> > probably just flaky.   So feeling good about that!
> >
> > https://ci-builds.apache.org/job/Solr/job/Solr-Check-main-s390x/
> >
> > I have started a back port PR to branch_9x, however it’s not going well…. 
> > :-(.   https://github.com/apache/solr/pull/2540.  There are more 
> > differences between main and branch_9x for the CLI than I quite realized.   
> > Bats tests on main that aren’t on branch_9x, all the changes in how we 
> > craft Solr urls, the fact that we never back ported basic auth for SolrCLI 
> > tools….  We replaced “CreateCoreTool" and “CreateCollectionTool" tools with 
> > “CreateTool” on main...  I haven’t committed all my local changes as the 
> > tests are failing, but may try that….
> >
> > I’m running out of time before vacation for three weeks (hello Spain!) and 
> > so thinking:
> >
> > 1) Ask for help.  If someone else wanted to take a crack at the back port 
> > who has more Git-fu than I do, more than welcome.
> > 2) Push up the code to the branch even though tests etc fail.
> > 3) Not worry about back port to branch_9x till I get back last week of July.
> > 4) Pivot on the back port plan and declare SOLR-16842 to be a Solr 10 only 
> > feature.  Figure out a new plan for removing deprecated options from code.  
> > (That is starting to feel the path of least resistance to me).
> >
> > I would very much like to NOT rollback the change so didn’t list that as an 
> > option….
> >
> >
> > Thanks
> >
> > Eric
> >
> >
> >> On Jun 22, 2024, at 10:05 AM, Eric Pugh  >> <mailto:ep...@opensourceconnections.com>> wrote:
> >>
> >> Thanks Jason, I’m happy to stall till end of day on Monday to click 
> >> “merge”!   Thanks.
> >>
> >>
> >>> On Jun 21, 2024, at 12:33 PM, Jason Gerlowski  
> >>> wrote:
> >>>
> >>> Hey Eric,
> >>>
> >>> Sorry for the delay - I am hoping to review that PR.  I'll try to have
> >>> a review done by Monday (if not today).  That should leave a week or
> >>> so for any followups prior to yo

lucene-solr-1 Jenkins Agent Management

[Jason Gerlowski]

Hey all,

Sending this email to discuss the ASF Jenkins' 'lucene-solr-1' worker
node, which both Lucene and Solr use to run builds.

While investigating a recent issue with some Solr builds, I asked
INFRA to restart 'lucene-solr-1'. They were happy to oblige (and I'm
now unblocked), but in the process they mentioned that 'lucene-solr-1'
was a "project run VM" and that someone in the project should have the
requisite access and permissions. [1] [2]

This was all a surprise to me, so I wanted to follow up here with a
few questions.

1. Is it true that our projects run or manage this VM in some way?
2. If so, does anyone remember the context around how or why this came
about? (As opposed to using one of the "standard" Jenkins build
machines that INFRA provides...)
3. Who all can access the box?  How are credentials managed?  (My
suspicion is that no one currently active on the Solr lists is able to
access the box, which seems worth remedying...)

Best,

Jason

[1] https://lists.apache.org/thread/g0y0ohczdctv0v5fn8sqv4t0j4y6hp68
[2] https://the-asf.slack.com/archives/CBX4TSBQ8/p1719318685682309

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Integration test failures on Jenkins

[Jason Gerlowski]

The failures are ongoing, so I suspect no one has (yet) reached out to
Infra about this.

I pinged them this morning in #asfinfra, so hopefully we can get this
resolved shortly.

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [DISCUSS] Community Virtual Meetup, June 2024

[Jason Gerlowski]

Alright, Wednesday June 26th it is!

If anyone has any topics for discussion, please add them to the
Meeting Notes page.  It might be a "light" month given summer holidays
and the conferences earlier this month.  (But alternatively, maybe a
CoC Europe or Berlin Buzzwords attendee can be tempted into describing
their experiences!)

https://cwiki.apache.org/confluence/display/SOLR/2024-06-26+Meeting+notes

The Google Meet link to use can be found on the Meeting Notes page (or
below).  See you all on Wednesday!

https://meet.google.com/uny-zfca-sny

Best,

Jason

On Thu, Jun 20, 2024 at 8:29 AM Jason Gerlowski  wrote:
>
> Hey all,
>
> I can organize this month.  I haven't heard any suggestions on
> scheduling, so maybe we can aim for noon ET on Wednesday of next week
> (June 26th)?  Would that work for everyone who'd like to attend?
>
> If I don't hear any objections, I'll create the Meeting Notes page and
> meeting link shortly.
>
> Best,
>
> Jason
>
>
> On Wed, Jun 12, 2024 at 11:58 AM Jason Gerlowski  
> wrote:
> >
> > Hey all,
> >
> > It's time once again to start thinking ahead to this month's virtual meetup!
> >
> > As always, two questions:
> >
> > 1. Does anyone have an interest in organizing?  Duties are light but
> > it's an important job.  I'm happy to organize by default if there
> > aren't any volunteers by mid-next-week.  (Addtl details:
> > https://cwiki.apache.org/confluence/display/SOLR/Meeting+notes)
> >
> > 2. Does anyone have preferences on the date or time-of-day?
> >
> > Best,
> >
> > Jason

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Looking for final review of SOLR-16842

[Jason Gerlowski]

Hey Eric,

Sorry for the delay - I am hoping to review that PR.  I'll try to have
a review done by Monday (if not today).  That should leave a week or
so for any followups prior to your big trip!  (Safe travels!)

If Monday isn't early enough and you need to merge on Saturday, IMO
that's fine.  The PR's been open for nearly a year and you've been
more than patient at this point.  In that case I'll just review
post-merge and we can handle any follow ups as needed.

Best,

Jason

On Fri, Jun 21, 2024 at 12:07 PM Eric Pugh
 wrote:
>
> Hi all, I’m planning on merging https://github.com/apache/solr/pull/1768, 
> SOLR-16824: Adopt Linux Command line tool pattern of -- for long option 
> commands tomorrow (Saturday).
>
> I’m going to be traveling (Spain!) for three weeks starting June 29th, and 
> I’d like to make sure this fairly big change has plenty time for any panicky 
> follow up fixes that turn out to be needed when it goes into main and 
> branch_9x.
>
> Eric
>
>
> ___
> Eric Pugh | Founder | OpenSource Connections, LLC | 434.466.1467 | 
> http://www.opensourceconnections.com  
> | My Free/Busy 
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 
> 
> This e-mail and all contents, including attachments, is considered to be 
> Company Confidential unless explicitly stated otherwise, regardless of 
> whether attachments are marked as such.
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [DISCUSS] Community Virtual Meetup, June 2024

[Jason Gerlowski]

Hey all,

I can organize this month.  I haven't heard any suggestions on
scheduling, so maybe we can aim for noon ET on Wednesday of next week
(June 26th)?  Would that work for everyone who'd like to attend?

If I don't hear any objections, I'll create the Meeting Notes page and
meeting link shortly.

Best,

Jason


On Wed, Jun 12, 2024 at 11:58 AM Jason Gerlowski  wrote:
>
> Hey all,
>
> It's time once again to start thinking ahead to this month's virtual meetup!
>
> As always, two questions:
>
> 1. Does anyone have an interest in organizing?  Duties are light but
> it's an important job.  I'm happy to organize by default if there
> aren't any volunteers by mid-next-week.  (Addtl details:
> https://cwiki.apache.org/confluence/display/SOLR/Meeting+notes)
>
> 2. Does anyone have preferences on the date or time-of-day?
>
> Best,
>
> Jason

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

[DISCUSS] Community Virtual Meetup, June 2024

[Jason Gerlowski]

Hey all,

It's time once again to start thinking ahead to this month's virtual meetup!

As always, two questions:

1. Does anyone have an interest in organizing?  Duties are light but
it's an important job.  I'm happy to organize by default if there
aren't any volunteers by mid-next-week.  (Addtl details:
https://cwiki.apache.org/confluence/display/SOLR/Meeting+notes)

2. Does anyone have preferences on the date or time-of-day?

Best,

Jason

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: CI build failures

[Jason Gerlowski]

> I recall getting a direct email to me if I contributed a
> change to a CI failure.  I would like this.  I would also like a dev
> list email periodically (weekly?) listing the CI job status

+1 to any effort that aims to summarize "builds@".

Even for folks that check "builds@" regularly it can be hard to catch
the "signal" through all the noise.  "builds@" sent out so much (~65
failure/unstable emails this week, with 11 yesterday alone!) that it's
all too easy to miss a failure that's new, or potentially related to
something I changed recently.  That's why Fucit is so valuable for us
IMO - aggregation.  It's giving us information at a glance that you
could otherwise only get by slogging through 50-70 emails.

Any other creative ways we can find to aggregate or digest that
"build@" content is going to be helpful IMO.

> You mention “PR validation doesn’t run BATS tests”….   Maybe it should?

+1 to this as well.  I don't have opinions on the specific form of
that additional PR validation (e.g. BATS vs docker).  But the general
idea of trading a bit of speed on our PR checks for additional
validation/coverage is a good one IMO.

Jason

On Thu, Jun 6, 2024 at 9:32 AM David Smiley  wrote:
>
> We could add (yet) another Github workflow validation.  Or maybe take
> the Docker one, which looks at certain paths (including bin/solr), and
> generalizes to be not just testing docker but also running BATS
> integration tests.  Then think of this as the "integration test"
> build.  The "paths" it looks at could be expanded to include some of
> the Java source CLI patterns so this runs in more cases that are
> likely to be impacted.
>
> I'm slightly reluctant to take the Crave build and slow it down to run
> those integration tests that run serially.  It's amazing, particularly
> when working locally, to get fast feedback on all Java based tests.
> Admittedly it's not an either-or; it's a matter of indicating the
> desired targets in the build, so I shouldn't be reluctant here.
> Anyway, I'm more keen to expand the scope of the Docker build.
>
> On Thu, Jun 6, 2024 at 9:17 AM Eric Pugh
>  wrote:
> >
> > I think a weekly heads up would be great to have.
> >
> > You mention “PR validation doesn’t run BATS tests”….   Maybe it should?  
> > We’ve had a lot of churn in the CLI, and we’ll probably continue to have 
> > that till 10x comes out, so that would be a nice check.   Plus, if we add 
> > more integration style BATS tests, why not have them be run on the PR’s?
> >
> > > On May 31, 2024, at 11:40 AM, David Smiley  wrote:
> > >
> > > I'm concerned that too few people look at the bui...@solr.apache.org
> > > From time to time I go look but we have no notifications other than
> > > emailing that list.
> > >
> > > If hypothetically nobody looked, we might as well not have CI at all;
> > > we'd only have PR based validation.  We'd lose out on historic test
> > > failure tracking and detection of introducing a problem that got
> > > merged anyway.  PR validation doesn't run BATS tests or "Nightly"
> > > tests.
> > >
> > > Long ago, I recall getting a direct email to me if I contributed a
> > > change to a CI failure.  I would like this.  I would also like a dev
> > > list email periodically (weekly?) listing the CI job status.
> > >
> > > Any opinions on what to do here?
> > >
> > > ~ David Smiley
> > > Apache Lucene/Solr Search Developer
> > > http://www.linkedin.com/in/davidwsmiley
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > > For additional commands, e-mail: dev-h...@solr.apache.org
> > >
> >
> > ___
> > Eric Pugh | Founder | OpenSource Connections, LLC | 434.466.1467 | 
> > http://www.opensourceconnections.com 
> >  | My Free/Busy 
> > 
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 
> > 
> > This e-mail and all contents, including attachments, is considered to be 
> > Company Confidential unless explicitly stated otherwise, regardless of 
> > whether attachments are marked as such.
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Changes in JAX-RS APIs mean may need to "gradlew clean" ?

[Jason Gerlowski]

It worked!  There's a PR here with the fix:
https://github.com/apache/solr/pull/2502. Take a look when you get a
chance and let me know what you think!

(I haven't created a JIRA ticket, since it's a minor change to our
build.  If anyone would prefer a JIRA ticket to document this beyond
what this dev-thread and Github PR provide, let me know.)

Best,

Jason

On Thu, Jun 6, 2024 at 7:05 AM Jason Gerlowski  wrote:
>
> Yeah, this is definitely a pain from time to time.
>
> For anyone who hasn't hit this, steps to reproduce are:
>
> 1. Start on a branch that has a new JAX-RS-annotated interface in
> Solr's 'api' module.
> 2. Any gradle task that compiles SolrJ will build Solr's OpenAPI spec,
> generate SolrRequest implementations from that spec, and add them to
> solrj's build directory as a part of its "source set".  These
> generated classes often reference 'model' classes (e.g. request or
> response POJOs) that exist on the current branch, but may not be on
> other branches.
> 3. Switch to a new branch, which lacks the new JAX-RS API.
> 4. If solrj is compiled without running "clean" first, the previously
> generated SolrRequest implementations will fail to compile (because
> the request/response POJOs they rely on are missing).
>
> In terms of *when* this happens, I often see it when changing from a
> PR-branch to main.  Though you can also see it going from 'main' to
> 'branch_9x', if 'main' has a JAX-RS API that hasn't been backported
> yet.
>
> I've been a little despairing in the past about fixing this- I know it
> should be done, but my gradle knowledge is pretty lacking.  Though I
> notice in writing this email that the 'openApiGenerate' task itself
> has a few options that might fix this without any broader gradle
> changes, particularly the "cleanupOutput" and "skipOverwrite" options.
> I'll try playing with those a bit and report back if there's any
> promise.
>
> Best,
>
> Jason
>
>
> On Tue, Jun 4, 2024 at 6:52 PM David Smiley  wrote:
> >
> > I noticed some generated source files, and in particular
> > solr/solrj/build/generated/src/main/java/org/apache/solr/client/solrj/request/FileStoreApi.java
> > that suddenly had a compilation issue.  This is almost certainly due
> > to the API evolving, and SOLR-17302 in particular.  Just do "gradlew
> > clean" to start fresh.  I've hit this a couple times for different
> > specific API issues over some months.
> >
> > Still... should the build be smart enough to avoid this?  For example
> > if the generator is blind to the output directory's contents, we may
> > as well clean the generated directory fully first.  On the other hand,
> > maybe it smartly recognizes existing generated stuff and can tell that
> > it doesn't need to re-generate (like javac).
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Changes in JAX-RS APIs mean may need to "gradlew clean" ?

[Jason Gerlowski]

Yeah, this is definitely a pain from time to time.

For anyone who hasn't hit this, steps to reproduce are:

1. Start on a branch that has a new JAX-RS-annotated interface in
Solr's 'api' module.
2. Any gradle task that compiles SolrJ will build Solr's OpenAPI spec,
generate SolrRequest implementations from that spec, and add them to
solrj's build directory as a part of its "source set".  These
generated classes often reference 'model' classes (e.g. request or
response POJOs) that exist on the current branch, but may not be on
other branches.
3. Switch to a new branch, which lacks the new JAX-RS API.
4. If solrj is compiled without running "clean" first, the previously
generated SolrRequest implementations will fail to compile (because
the request/response POJOs they rely on are missing).

In terms of *when* this happens, I often see it when changing from a
PR-branch to main.  Though you can also see it going from 'main' to
'branch_9x', if 'main' has a JAX-RS API that hasn't been backported
yet.

I've been a little despairing in the past about fixing this- I know it
should be done, but my gradle knowledge is pretty lacking.  Though I
notice in writing this email that the 'openApiGenerate' task itself
has a few options that might fix this without any broader gradle
changes, particularly the "cleanupOutput" and "skipOverwrite" options.
I'll try playing with those a bit and report back if there's any
promise.

Best,

Jason


On Tue, Jun 4, 2024 at 6:52 PM David Smiley  wrote:
>
> I noticed some generated source files, and in particular
> solr/solrj/build/generated/src/main/java/org/apache/solr/client/solrj/request/FileStoreApi.java
> that suddenly had a compilation issue.  This is almost certainly due
> to the API evolving, and SOLR-17302 in particular.  Just do "gradlew
> clean" to start fresh.  I've hit this a couple times for different
> specific API issues over some months.
>
> Still... should the build be smart enough to avoid this?  For example
> if the generator is blind to the output directory's contents, we may
> as well clean the generated directory fully first.  On the other hand,
> maybe it smartly recognizes existing generated stuff and can tell that
> it doesn't need to re-generate (like javac).
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Purpose of IndexUpgraderTool

[Jason Gerlowski]

 clients has taken
> >> my advice and simply re-indexed (though not always without grumbling!), so
> >> I have to admit I've not actually seen it used, but in theory it makes some
> >> sense.
> >>
> >> -Gus
> >>
> >> On Mon, Jun 3, 2024 at 2:20 PM David Smiley  wrote:
> >>
> >>> FWIW, in my experience I've never run this tool (nor colleagues) at
> >>> any stage in my career that I can remember.  For one reason, all the
> >>> systems could re-index if they needed to.
> >>> It may be best to remove this information, as it could introduce more
> >>> confusion than it helps.
> >>>
> >>> On Mon, Jun 3, 2024 at 1:34 PM Jason Gerlowski
> >>> wrote:
> >>>> Hey all,
> >>>>
> >>>> I was poking around the ref-guide a bit recently and noticed our page
> >>>> on the "IndexUpgraderTool" that Lucene produces. [1]
> >>>>
> >>>> AFAICT, the page doesn't hint at when/why a user might want to use the
> >>>> IndexUpgraderTool.  Maybe at one point the tool might've been
> >>>> preferred to loading the index in an upgraded Solr version, but I
> >>>> haven't heard of anyone doing that in the last few years.
> >>>>
> >>>> Is this something we expect users to still do?  If so, for what
> >>>> usecase?  And if not - should we drop it from the ref-guide - it seems
> >>>> like it might confuse folks since it's not actually needed to upgrade
> >>>> Solr versions...
> >>>>
> >>>> Best,
> >>>>
> >>>> Jason
> >>>>
> >>>> [1]
> >>> https://solr.apache.org/guide/solr/latest/deployment-guide/indexupgrader-tool.html
> >>>> -
> >>>> To unsubscribe, e-mail:dev-unsubscr...@solr.apache.org
> >>>> For additional commands, e-mail:dev-h...@solr.apache.org
> >>>>
> >>> -
> >>> To unsubscribe, e-mail:dev-unsubscr...@solr.apache.org
> >>> For additional commands, e-mail:dev-h...@solr.apache.org
> >>>
> >>>
> >> --
> >> http://www.needhamsoftware.com  (work)
> >> https://a.co/d/b2sZLD9  (my fantasy fiction book)
> >
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail:u...@thetaphi.de

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Purpose of IndexUpgraderTool

[Jason Gerlowski]

Hey all,

I was poking around the ref-guide a bit recently and noticed our page
on the "IndexUpgraderTool" that Lucene produces. [1]

AFAICT, the page doesn't hint at when/why a user might want to use the
IndexUpgraderTool.  Maybe at one point the tool might've been
preferred to loading the index in an upgraded Solr version, but I
haven't heard of anyone doing that in the last few years.

Is this something we expect users to still do?  If so, for what
usecase?  And if not - should we drop it from the ref-guide - it seems
like it might confuse folks since it's not actually needed to upgrade
Solr versions...

Best,

Jason

[1] 
https://solr.apache.org/guide/solr/latest/deployment-guide/indexupgrader-tool.html

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [VOTE] Release Solr 9.6.1 RC1

[Jason Gerlowski]

+1 (binding)

Ran the smoketester and did a few manual backup/restore and installshard tests.

Best,

Jason

On Tue, May 28, 2024 at 4:18 AM Arrieta, Alejandro
 wrote:
>
> +1 not binding
>
> SUCCESS! [0:38:34.087315]
>
> Dockers
> [+] Building 45.1s (10/10) FINISHED
> [+] Building 28.5s (10/10) FINISHED
>
> Kind regards,
> Alejandro Arrieta
>
> On Mon, May 27, 2024, 2:58 PM Jan Høydahl  wrote:
>
> > +1
> >
> > SUCCESS! [0:45:03.659058]
> >
> > -Jan
> >
> > > 23. mai 2024 kl. 21:39 skrev Houston Putman :
> > >
> > > Please vote for release candidate 1 for Solr 9.6.1
> > >
> > > The artifacts can be downloaded from:
> > >
> > https://dist.apache.org/repos/dist/dev/solr/solr-9.6.1-RC1-rev-d7f7166567f52f1b31e3315b0188e11f2c4c9b60
> > >
> > > You can run the smoke tester directly with this command:
> > >
> > > python3 -u dev-tools/scripts/smokeTestRelease.py \
> > >
> > https://dist.apache.org/repos/dist/dev/solr/solr-9.6.1-RC1-rev-d7f7166567f52f1b31e3315b0188e11f2c4c9b60
> > >
> > > You can build a release-candidate of the official docker images (full &
> > > slim) using the following command:
> > >
> > > SOLR_DOWNLOAD_SERVER=
> > >
> > https://dist.apache.org/repos/dist/dev/solr/solr-9.6.1-RC1-rev-d7f7166567f52f1b31e3315b0188e11f2c4c9b60/solr
> > > && \
> > >  docker build
> > $SOLR_DOWNLOAD_SERVER/9.6.1/docker/Dockerfile.official-full \
> > >--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
> > >-t solr-rc:9.6.1-1 && \
> > >  docker build
> > $SOLR_DOWNLOAD_SERVER/9.6.1/docker/Dockerfile.official-slim \
> > >--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
> > >-t solr-rc:9.6.1-1-slim
> > >
> > > The vote will be open for at least 72 hours i.e. until 2024-05-29 20:00
> > UTC.
> > > (Extended due to weekend and US holiday)
> > >
> > > [ ] +1  approve
> > > [ ] +0  no opinion
> > > [ ] -1  disapprove (and reason why)
> > >
> > > Here is my +1
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
> >

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Welcome Sanjay Dutt as Solr committer!

[Jason Gerlowski]

Welcome and congratulations!

On Mon, May 20, 2024 at 2:04 PM Ishan Chattopadhyaya
 wrote:
>
> Welcome Sanjay. Many congratulations!
>
> On Mon, 20 May, 2024, 9:53 pm David Smiley,  wrote:
>
> > The Project Management Committee (PMC) for Apache Solr has invited
> > Sanjay Dutt to become a committer and we are pleased to announce that
> > they have accepted.
> >
> > Sanjay, the tradition is that new committers introduce themselves with
> > a brief bio.
> >
> > Congratulations and welcome!
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
> >

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Monthly Virtual Meetups and continued use of Meetup.com

[Jason Gerlowski]

Thanks for clarifying Ishan, and for using your account on the
community's behalf.

No one's chimed in to support keeping the integration alive, so let's
try running a few months without a Meetup.com membership.  If
attendance drops off in a drastic way or there are complaints, I can
consider re-upping the subscription.  But absent any direct
testimonial for it being valuable

Best,

Jason

On Sat, May 18, 2024 at 3:06 PM Ishan Chattopadhyaya
 wrote:
>
> I've been paying for our group's Meetup.com account. I no longer have the
> funds to do so anymore. I can't pay for that account any more.
>
> On Sat, 18 May 2024 at 15:01, Mark Miller  wrote:
>
> > Wow, I never knew meetup.com charged - and it looks like as much as a
> > streaming service. Wow. The only value I know of that meetup.com really
> > provides is discoverability and rsvp. Neither of which seems that valuable
> > for this.
> >
> > It would be crazy to pay that monthly fee if you didn’t already. If it did
> > have some value, I’d try and replace it with a free option. I think
> > EventBrite is free for free events, and LinkedIn has a LinkedIn Event
> > feature.
> >

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [DISCUSS] Community Virtual Meetup, May 2024

[Jason Gerlowski]

I'll vote for (2)!

On Tue, May 14, 2024 at 1:47 AM raghavan m  wrote:
>
> Hello everyone
>
> I am soliciting votes for the following days and times. Please reply to
> this thread with your preferred 1) *date and time options from below *and
> 2) *topic you want to discuss *
>
> *Voting closes May 17th 11:59 pm*
>
> *Options*
> 1. 05/22/2024 9 am pacific time
> 2. 05/23/2024 9 am pacific time
> 3. 05/24/2024 9 am pacific time
> 4. 05/27/2024 9 am pacific time
>
> thanks,
> *Raghavan*
>
>
> On Sun, May 12, 2024 at 7:03 PM raghavan m  wrote:
>
> > Thanks Jason.
> > I will start a thread and create a page, to collect topics.
> > *Raghavan*
> >
> >
> > On Sun, May 12, 2024 at 7:00 PM Jason Gerlowski 
> > wrote:
> >
> >> Raghavan - absolutely, thanks for stepping up!  (And welcome back to
> >> the country!).
> >>
> >> On Thu, May 9, 2024 at 1:51 PM raghavan m  wrote:
> >> >
> >> > Hey Jason
> >> > I am back in the country. Can I volunteer?
> >> >
> >> > Sent from iPhone
> >> >
> >> >
> >> > On Thu, May 9, 2024 at 9:35 AM Jason Gerlowski 
> >> > wrote:
> >> >
> >> > > Hey all,
> >> > >
> >> > > It's time once again to start thinking ahead to this month's virtual
> >> > > meetup!
> >> > >
> >> > > As always, two questions:
> >> > >
> >> > > 1. Does anyone have an interest in organizing?  Duties are light but
> >> > > it's an important job.  I'm happy to organize by default if there
> >> > > aren't any volunteers by mid-next-week.  (Addtl details:
> >> > > https://cwiki.apache.org/confluence/display/SOLR/Meeting+notes)
> >> > >
> >> > > 2. Does anyone have preferences on the date or time-of-day?
> >> > >
> >> > > Best,
> >> > >
> >> > > Jason
> >> > >
> >> > > -
> >> > > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> >> > > For additional commands, e-mail: dev-h...@solr.apache.org
> >> > >
> >> > >
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> >> For additional commands, e-mail: dev-h...@solr.apache.org
> >>
> >>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Package Store Implementation Questions

[Jason Gerlowski]

After some more digging I think I understand things a bit better.  The
APIs I couldn't find in the code aren't "package store" APIs at all.
Or at least not directly.  They're APIs for Solr's "file store".  I
mistook them for package APIs because the HTTP path contained
"package", but it looks like that's only by virtue of "package" being
a location within the filestore that the package-manager happens to
use.

Having found the APIs, I'm on to slightly more specific questions.  Is
there a reason that files are uploaded to "/api/cluster/files" but
fetched from "/api/node/files"?  What distinguishes the two delete
APIs that the file-store offers ("DELETE /api/cluster/files/" and
"DELETE /api/node/files")?  Are the delete APIs omitted from the
ref-guide by oversight or for a particular reason? etc.

If anyone can offer any context on these APIs, I'd greatly appreciate it!

Best,

Jason

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Package Store Implementation Questions

[Jason Gerlowski]

Hey all,

I'm looking to convert some of Solr's "Package Store" APIs over to the
JAX-RS framework, but I'm having some trouble finding the backing
code.

Some of the APIs were pretty easy to find.  For instance I see `POST
/api/cluster/package`, `GET /api/cluster/package`, and `GET
/api/clusteer/package/somePackageName` implemented in
"org.apache.solr.pkg.PackageAPI".

But I can see other APIs getting called by the PackageTool code.
`RepositoryManager` uses a SolrClient to "POST" to
`/package/somePackageName/someVersion/someFileName`. [1]  It also
makes a similar request to
`/package/somePackageName/someVersion/manifest.json`. [2]  I wondered
for awhile whether these might be requests to the external package
repository, and not to Solr itself.  But the requests are made using a
SolrClient, and it uses a Solr base URL as far as I can tell.

Does anyone know where the code for these additional APIs might live?
Or whether I'm misinterpreting them, and they really are requests made
to the repository (and not Solr)?  If the latter - is there a reason
the code uses a SolrClient instead of a Jetty or Apache HttpClient
directly?  Thanks in advance for any pointers in the right direction!

Best,

Jason

[1] 
https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/packagemanager/RepositoryManager.java#L207-L216
[2] 
https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/packagemanager/RepositoryManager.java#L198-L202

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Monthly Virtual Meetups and continued use of Meetup.com

[Jason Gerlowski]

Hey all,

Since we started them last year, we've been advertising our monthly
meetups in a few different places: here on the mailing list, in
various Slack servers and channels, and on Meetup.com.

We may have to reconsider our use of Meetup.com, or alternately: take
some action to keep that going if it's valuable.

The "group" we've been creating these Meetup.com events under
currently lacks an "organizer", and risks deletion by Meetup.com.  (It
appears they periodically delete "orphaned" events and groups.)
Anyone can sign up to be an "organizer", but doing so requires having
a paid subscription : /

So my question is: how much value do we get out of the Meetup.com
event? Typically nothing gets added to the Meetup.com event that a
reader wouldn't know by checking out our Confluence wiki page, or
reading the monthly dev@ thread for the Meetup.  So my uneducated
guess is that it doesn't provide a ton of value for us, and wouldn't
be worth signing up for a paid account just to maintain.

But if anyone can attest that the Meetup.com event is important for
some particular reason, I'm happy to look into maintaining it.
(Alternatively, if someone here already has a paid Meetup.com account
and is willing to be an organizer for us, please let me know!)

 Best,

Jason

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [DISCUSS] Community Virtual Meetup, May 2024

[Jason Gerlowski]

Raghavan - absolutely, thanks for stepping up!  (And welcome back to
the country!).

On Thu, May 9, 2024 at 1:51 PM raghavan m  wrote:
>
> Hey Jason
> I am back in the country. Can I volunteer?
>
> Sent from iPhone
>
>
> On Thu, May 9, 2024 at 9:35 AM Jason Gerlowski 
> wrote:
>
> > Hey all,
> >
> > It's time once again to start thinking ahead to this month's virtual
> > meetup!
> >
> > As always, two questions:
> >
> > 1. Does anyone have an interest in organizing?  Duties are light but
> > it's an important job.  I'm happy to organize by default if there
> > aren't any volunteers by mid-next-week.  (Addtl details:
> > https://cwiki.apache.org/confluence/display/SOLR/Meeting+notes)
> >
> > 2. Does anyone have preferences on the date or time-of-day?
> >
> > Best,
> >
> > Jason
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
> >

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

[DISCUSS] Community Virtual Meetup, May 2024

[Jason Gerlowski]

Hey all,

It's time once again to start thinking ahead to this month's virtual meetup!

As always, two questions:

1. Does anyone have an interest in organizing?  Duties are light but
it's an important job.  I'm happy to organize by default if there
aren't any volunteers by mid-next-week.  (Addtl details:
https://cwiki.apache.org/confluence/display/SOLR/Meeting+notes)

2. Does anyone have preferences on the date or time-of-day?

Best,

Jason

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: SolrJ collection creation API, replica type specificity

[Jason Gerlowski]

You didn't mention it by name, but it sounds like you're talking about
the v1 API's "replicationFactor" parameter (which has defaulted to
creating NRT replicas for awhile now)?

Personally, I'd rather see that parameter (and corresponding SolrJ
code) go away altogether.  Some things (e.g. the configset name, the
number of shards) are important enough that we force users to be
explicit about them...IMO the number and type of replicas fall into
that category.

But while the ambiguous "replicationFactor" parameter exists I think
some sort of "default replica type" concept makes sense.  (Granted
that we find a way to handle certain complications, like how "PULL"
replicas *must* be used in conjunction with replicas of other
leader-eligible replica types.)

On Wed, May 1, 2024 at 2:32 PM David Smiley  wrote:
>
> In the interests of supporting different replica types better, I'd
> like our SolrJ CollectionAdminRequest methods to not *locally* assume
> NRT when creating a request.  Calls like createCollection(collection
> name, numShards, numReplicas) are nicely ambiguous as to the type, nor
> do javadocs indicate what the type is.  This is good, I think.  Yet
> the default behavior is to create a v1 API call that specifies how
> many NRT replicas (yes of this specific type) to make.  Instead, I'd
> like to see the replica type decision made by the server (Solr).
> Today it also assumes NRT but I could imagine something as simple as
> EnvUtils (env var / sys prop) deciding what the default type should
> be.  So far this is merely changing CollectionAdminRequest and
> consequently the specificity of its v1 requests.  It'd be followed up
> by improving a number of tests to be less specific to NRT.  Any
> concerns here?
>
> *Actually* using other replica types (like TLOG or ZERO) may raise
> issues for some tests beyond this, sure.  In particular, many tests
> assume read-your-write (index, commit, query --> find it) but this
> won't hold if the query randomly routes to a non-leader.  For this I'm
> thinking an automatically applied
> shards.preference=replica.leader:true
> https://solr.apache.org/guide/solr/latest/deployment-guide/solrcloud-distributed-requests.html#shards-preference-parameter
> -- only when the default replica type isn't NRT.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: SolrJ HTTP SolrClient class hierarchy

[Jason Gerlowski]

Thanks for summarizing this all David!

We have HttpSolrClientBase AND BaseHttpSolrClient?!

A bit of the messiness here seems like it will resolve itself
"automatically" if we make good on removing the existing deprecations.
(Though given how "entrenched" the Apache client is, that will require
a good bit of work...).

I like the convention established by the JDK client, of naming our
low-level SolrClient implementations based on the HttpClient vendor in
use.  IMO doing otherwise can be confusing in a few different ways.
It's not high on my list to act on it immediately, but if there's
enough consensus on the idea of renaming (e.g.) Http2SolrClient, I
could file a ticket to document that as an ideal step and maybe it'll
hook someone eventually?

Best,

Jason

On Thu, Apr 25, 2024 at 11:34 PM Mark Miller  wrote:
>
> It's too bad HttpSolrServer setup this client philosophy. It's momentum was 
> directly opposite to what you want: a SolrClient that can optionally stream 
> or load balance and a SolrCloudClient that wraps it.
>
> [Mark Miller - Chat @ 
> Spike](https://spikenow.com/r/a/?ref=spike-organic-signature&_ts=2kigx5)  
> [2kigx5]
>
> On April 25, 2024 at 20:07 GMT, David Smiley  wrote:
>
> Our SolrJ class hierarchy is looking rather confusing right now for
> the HTTP ones especially. This message is mostly a big FYI, with some
> reflections and a recommendation or two.
>
> SolrClient
> - BaseHttpSolrClient (NOT yet deprecated but should be?)
> - - HttpSolrClient (based on Apache HttpClient; deprecated)
> - - - DelegationTokenHttpSolrClient
> - CloudSolrClient
> - - CloudHttp2SolrClient
> - - CloudLegacySolrClient (based on Apache HttpClient; deprecated)
> - ConcurrentUpdateHttp2SolrClient
> - - ...
> - ConcurrentUpdateSolrClient (based on Apache HttpClient; deprecated)
> - - ...
> - HttpSolrClientBase (this is new)
> - - Http2SolrClient
> - - HttpJdkSolrClient (this is new; based on the JDK HttpClient)
> - LBSolrClient
> - - LBHttp2SolrClient
> - - LBHttpSolrClient (based on Apache HttpClient; deprecated)
>
> In retrospect, we can see that some past names weren't so great after
> all. I think our clients should reflect the vendor/source of the
> HttpClient. "HttpJdkSolrClient" is the newest client, and it reflects
> the vendor (JDK provided HttpClient). Personally I don't care enough
> to rename all the ones with "2" in there to have "Jetty" but that's
> what they are -- if it has a "2", it's using Jetty (and it supports
> 1.1; FYI JDK also supports both 1.1 and 2 as well). The clients for
> Apache HttpClient are all deprecated so perhaps we continue to leave
> them be, mostly. Removing them will take some time; they are
> entrenched! BaseHttpSolrClient (the parent of HttpSolrClient) is at
> the moment even more confusing because HttpSolrClientBase was just
> added. BaseHttpSolrClient should be removed now; it only holds 2
> static inner classes for RemoteSolrException and
> RemoteExecutionException which should find a new home somewhere.
> Since they are referenced so much, that will happen only in main.
> HttpSolrClientBase is a tempting home but SolrClient would be fine, I
> think.
>
> Also, just because we have a nice new HttpJdkSolrClient, doesn't mean
> we can yet advise anyone to safely remove Apache & Jetty dependencies
> *yet*! We have no tests that this works, and a quick attempt I did
> recently showed there are some obscure references still! Modularizing
> SolrJ further (for Jetty & Apache) will help reveal where we have some
> references, after which we can finally free users of needing those
> dependencies.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Ref guide issue

[Jason Gerlowski]

Definitely a good thing to add to the docs.  Thanks Gus!

On Sun, Apr 28, 2024 at 1:24 PM Gus Heck  wrote:
>
> Ok it seems to be resolved now. I think I'll document this (and some other
> high level info) in the releasing doc I've been working on in SOLR-17244 so
> future release managers don't panic prematurely. Thx for the clarification.
>
> On Sun, Apr 28, 2024 at 11:00 AM Gus Heck  wrote:
>
> > Hmm if it's this:
> > https://ci-builds.apache.org/job/Solr/job/solr-reference-guide-official/
> > it's due soon.
> >
> >
> > On Sun, Apr 28, 2024 at 10:58 AM Gus Heck  wrote:
> >
> >> Is this done by a jenkins job, (or part of one)? Where can I figure out
> >> the schedule? I tried looking at http last-modified headers, but those are
> >> all over the map and I suspect (guess) those dates may relate to when it
> >> was cached by a caching layer, not when it was built (headers indicate
> >> there's a varnish cache in play (via: 1.1 varnish).
> >>
> >> -Gus
> >>
> >>
> >>
> >> On Sun, Apr 28, 2024 at 12:35 AM Houston Putman 
> >> wrote:
> >>
> >>> The ref guide gets built nightly by a jenkins job. Either the job isnt
> >>> working or we just need to wait till the next build.
> >>>
> >>> “Beta” is used when the release branch exists but the release wizard step
> >>> of updating the antora.yaml (after the release is done) has not been done
> >>> (or is waiting on the nightly build to happen)
> >>>
> >>> - Houston
> >>>
> >>> On Sat, Apr 27, 2024 at 8:40 PM Gus Heck  wrote:
> >>>
> >>> > I was about to send announcement emails for 9.6 but then I discovered
> >>> that
> >>> > somehow the ref guide didn't seem to get the right version, and isn't
> >>> quite
> >>> > working... It lists a 9.6-beta version, which I had noted this
> >>> 9.6-beta at
> >>> > one point but I assumed that since I had never typed the word "beta"
> >>> > anywhere in the process this was automatic/temporary and would get
> >>> resolved
> >>> > as the release wizard progressed.
> >>> >
> >>> > So it's unclear to me how I can fix this. If any prior release
> >>> managers can
> >>> > give me a hit as to exactly what step may have gone wrong LMK...
> >>> Otherwise
> >>> > I'll delay the announcement emails until we have this sorted out.
> >>> (unless
> >>> > the consensus is that I should still send the announcements)
> >>> >
> >>> > The site has been published already, so that's unfortunate. The staging
> >>> > site doesn't show the ref guide or the javadocs so I was unaware of
> >>> this
> >>> > issue till I checked the results of publishing the site live.
> >>> >
> >>> > I'll try to resolve it tonight, but it's getting late and I may be too
> >>> > tired soon.
> >>> >
> >>> > -Gus
> >>> >
> >>> > --
> >>> > http://www.needhamsoftware.com (work)
> >>> > https://a.co/d/b2sZLD9 (my fantasy fiction book)
> >>> >
> >>>
> >>
> >>
> >> --
> >> http://www.needhamsoftware.com (work)
> >> https://a.co/d/b2sZLD9 (my fantasy fiction book)
> >>
> >
> >
> > --
> > http://www.needhamsoftware.com (work)
> > https://a.co/d/b2sZLD9 (my fantasy fiction book)
> >
>
>
> --
> http://www.needhamsoftware.com (work)
> https://a.co/d/b2sZLD9 (my fantasy fiction book)

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [VOTE] Release Solr 9.6.0 RC1

[Jason Gerlowski]

Here's my +1 (binding)

I got the smoketester to pass without much issue.  I also tested out
local backup/restore and tried out some v2 API calls to make sure
those work OK.

Best,

Jason

On Thu, Apr 25, 2024 at 7:56 AM Gus Heck  wrote:
>
> Hi folks,
>
> This vote ends in less than 24 h and so far only 3 people have weighed in,
> two of them binding. Please take the time to build and test out the RC.
>
> -Gus
>
> On Wed, Apr 24, 2024 at 5:47 AM Arrieta, Alejandro <
> aarri...@perrinsoftware.com> wrote:
>
> > +1 not binding
> >
> > SUCCESS! [0:36:54.492872]
> >
> > docker builds successfully, too
> > [+] Building 56.6s (10/10) FINISHED
> >  => => naming to docker.io/library/solr-rc:9.6.0-1
> > [+] Building 25.8s (10/10) FINISHED
> > => => naming to docker.io/library/solr-rc:9.6.0-1-slim
> >
> > Kind Regards,
> > Alejandro Arrieta
> >
> > On Tue, Apr 23, 2024 at 4:28 PM Gus Heck  wrote:
> >
> > > As a side note the clamav process appears to have been a red herring. For
> > > me the command succeeds ~40-50% of the time (after adding --no-cache) no
> > > idea why.
> > >
> > > On Tue, Apr 23, 2024 at 9:23 AM Gus Heck  wrote:
> > >
> > > > This could be an issue with anti-virus software or gpg-agent... I got
> > the
> > > > same result with the docker build until I killed clamav
> > > >
> > > > gus@ns-l1:~$ ps aux | grep clamav
> > > > clamav  1806  0.0  0.0  59124 14080 ?Ss   09:01   0:00
> > > > /usr/bin/freshclam -d --foreground=true
> > > > gus11896  0.0  0.0   9212  2304 pts/0S+   09:09   0:00 grep
> > > > --color=auto clamav
> > > > gus@ns-l1:~$ sudo kill -9 1806
> > > > [sudo] password for gus:
> > > > gus@ns-l1:~$ ps aux | grep clamav
> > > > gus11940  0.0  0.0   9080  2304 pts/0S+   09:09   0:00 grep
> > > > --color=auto clamav
> > > > gus@ns-l1:~$ SOLR_DOWNLOAD_SERVER=
> > > >
> > >
> > https://dist.apache.org/repos/dist/dev/solr/solr-9.6.0-RC1-rev-f8e5a93c11267e13b7b43005a428bfb910ac6e57/solr
> > > > &&   docker build
> > > > $SOLR_DOWNLOAD_SERVER/9.6.0/docker/Dockerfile.official-full
> > >  --build-arg
> > > > SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER -t solr-rc:9.6.0-1 &&
> > > > docker build
> > $SOLR_DOWNLOAD_SERVER/9.6.0/docker/Dockerfile.official-slim
> > > >   --build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER -t
> > > > solr-rc:9.6.0-1-slim
> > > > [+] Building 29.1s (10/10) FINISHED
> > > >
> > > > docker:default
> > > >  => CACHED [internal] load remote build context
> > > >
> > > >   0.2s
> > > >  => [internal] load metadata for
> > > > docker.io/library/eclipse-temurin:17-jre-jammy
> > > >
> > > >   0.3s
> > > >  => CACHED [1/7] FROM
> > > >
> > >
> > docker.io/library/eclipse-temurin:17-jre-jammy@sha256:1b646daef966395c93995e73347d4c7c726c9ddba8695e984cd8dcf5d8b5b253
> > > >0.0s
> > > >  => [2/7] RUN set -ex;   apt-get update;   apt-get -y
> > > > --no-install-recommends install wget gpg gnupg dirmngr;   rm -rf
> > > > /var/lib/apt/lists/*;   export SOLR_BINARY="solr-9.6.0.tgz";   MAX_RED
> > > >  20.7s
> > > >  => [3/7] RUN set -ex;   groupadd -r --gid "8983" "solr";   useradd -r
> > > > --uid "8983" --gid "8983" "solr"
> > > >   0.4s
> > > >  => [4/7] RUN set -ex;   (cd /opt; ln -s solr-*/ solr);   rm -Rf
> > > > /opt/solr/docs /opt/solr/docker/Dockerfile;
> > > >0.4s
> > > >  => [5/7] RUN set -ex;   mkdir -p /opt/solr/server/solr/lib
> > > > /docker-entrypoint-initdb.d;   cp /opt/solr/bin/solr.in.sh
> > /etc/default/
> > > > solr.in.sh;   mv /opt/solr/bin/solr.in.sh /opt/solr/bin/so  0.5s
> > > >  => [6/7] RUN set -ex; apt-get update; apt-get -y
> > > > --no-install-recommends install acl lsof procps wget netcat gosu tini
> > > > jattach; rm -rf /var/lib/apt/lists/*;   5.3s
> > > >  => [7/7] WORKDIR /opt/solr
> > > >
> > > >   0.1s
> > > >  => exporting to image
> > > >
> > > >  0.9s
> > > >  => => exporting layers
> > > >
> > > >   0.9s
> > > >  => => writing image
> > > > sha256:e426f6991bd325a5b01a31e90f3304ce45a787004c34fc6a90a2aa86c5193afc
> > > >
> > > >0.0s
> > > >  => => naming to docker.io/library/solr-rc:9.6.0-1
> > > >
> > > >0.0s
> > > > [+] Building 19.3s (10/10) FINISHED
> > > >
> > > > docker:default
> > > >  => [internal] load remote build context
> > > >
> > > >  0.2s
> > > >  => [internal] load metadata for
> > > > docker.io/library/eclipse-temurin:17-jre-jammy
> > > >
> > > >   0.2s
> > > >  => CACHED [1/7] FROM
> > > >
> > >
> > docker.io/library/eclipse-temurin:17-jre-jammy@sha256:1b646daef966395c93995

Re: Ideas for release notes for 9.6

[Jason Gerlowski]

I made one small tweak: adding the name of the new "thin" SolrClient
to the relevant list item so readers can look it up more easily.

Otherwise the release notes look great!

On Tue, Apr 23, 2024 at 2:01 PM James Dyer  wrote:
>
> For the second item, instead of "Solrj now offers async HTTP/2
> Requests", it would be better to say "SolrJ now offers an improved
> async request API".  We previously supported async but SOLR-14763
> merely offers a better API and deprecates the existing one.  I realize
> the JIRA title is not so accurate for what we actually did!
>
> On Mon, Apr 22, 2024 at 9:48 PM Gus Heck  wrote:
> >
> > Initial release notes have been drafted here, please flesh out, refine,
> > copy edit as needed.
> >
> > https://cwiki.apache.org/confluence/display/SOLR/ReleaseNote9_6_0
> >
> > -Gus
> >
> > --
> > http://www.needhamsoftware.com (work)
> > https://a.co/d/b2sZLD9 (my fantasy fiction book)
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [DISCUSS] Community Virtual Meetup, April 2024

[Jason Gerlowski]

Here's links for everyone's meeting-related pages:

Confluence Meeting Notes:
https://cwiki.apache.org/confluence/display/SOLR/2024-04-29+Meeting+notes
TimeAndDate: 
https://www.timeanddate.com/worldclock/fixedtime.html?msg=Solr+Community+Meetup%2C+April+2024&iso=20240429T09&p1=224&ah=1
Google Meet: meet.google.com/rju-vqau-gqt
Meetup.com: 
https://www.meetup.com/apache-solr-virtual-community-meetups/events/300577073

On Mon, Apr 22, 2024 at 2:49 PM Jason Gerlowski  wrote:
>
> Alright, I'm happy to volunteer for April.
>
> Pending any more input on "when", let's shoot for Monday April 29th at
> noon ET.  The 29th gives us a full week between this decision and the
> meeting.  (I don't have strong opinions on that particular time-slot,
> but we've used it for a number of recent meetings so it seems like a
> safe choice unless someone says otherwise.)
>
> I'll create the Meeting Notes page and send out announcements in
> Slack, etc. shortly.
>
> On Fri, Apr 19, 2024 at 7:56 AM Eric Pugh
>  wrote:
> >
> > I vote for 26th, or 29 or 30 ;-)
> >
> >
> > > On Apr 17, 2024, at 12:42 PM, Jason Gerlowski  
> > > wrote:
> > >
> > > Hey all,
> > >
> > > It's time once again to start thinking ahead to our Virtual Meetup for 
> > > April!
> > >
> > > 1. Anyone have an interest in organizing?  I'm happy to organize "by
> > > default" if we don't have any volunteers by EOD Monday, but it's
> > > always great to rotate!
> > >
> > > 2. When would we like to meet?
> > >
> > > Best,
> > >
> > > Jason
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > > For additional commands, e-mail: dev-h...@solr.apache.org
> > >
> >
> > ___
> > Eric Pugh | Founder | OpenSource Connections, LLC | 434.466.1467 | 
> > http://www.opensourceconnections.com 
> > <http://www.opensourceconnections.com/> | My Free/Busy 
> > <http://tinyurl.com/eric-cal>
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 
> > <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
> > This e-mail and all contents, including attachments, is considered to be 
> > Company Confidential unless explicitly stated otherwise, regardless of 
> > whether attachments are marked as such.
> >

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [DISCUSS] Community Virtual Meetup, April 2024

[Jason Gerlowski]

Alright, I'm happy to volunteer for April.

Pending any more input on "when", let's shoot for Monday April 29th at
noon ET.  The 29th gives us a full week between this decision and the
meeting.  (I don't have strong opinions on that particular time-slot,
but we've used it for a number of recent meetings so it seems like a
safe choice unless someone says otherwise.)

I'll create the Meeting Notes page and send out announcements in
Slack, etc. shortly.

On Fri, Apr 19, 2024 at 7:56 AM Eric Pugh
 wrote:
>
> I vote for 26th, or 29 or 30 ;-)
>
>
> > On Apr 17, 2024, at 12:42 PM, Jason Gerlowski  wrote:
> >
> > Hey all,
> >
> > It's time once again to start thinking ahead to our Virtual Meetup for 
> > April!
> >
> > 1. Anyone have an interest in organizing?  I'm happy to organize "by
> > default" if we don't have any volunteers by EOD Monday, but it's
> > always great to rotate!
> >
> > 2. When would we like to meet?
> >
> > Best,
> >
> > Jason
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
>
> ___
> Eric Pugh | Founder | OpenSource Connections, LLC | 434.466.1467 | 
> http://www.opensourceconnections.com <http://www.opensourceconnections.com/> 
> | My Free/Busy <http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 
> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
> This e-mail and all contents, including attachments, is considered to be 
> Company Confidential unless explicitly stated otherwise, regardless of 
> whether attachments are marked as such.
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Welcome Jason Gerlowski as Solr's new PMC Chair

[Jason Gerlowski]

Thanks all for the kind words, and thanks especially to David for
serving this past year!

On Fri, Apr 19, 2024 at 9:00 AM Anshum Gupta  wrote:
>
> Thank you for your effort, David!
>
> Congratulations, Jason!
>
> On Thu, Apr 18, 2024 at 7:47 PM David Smiley  wrote:
>
> > The PMC has voted Jason Gerlowski as Solr's new PMC Chair.  I am the
> > outgoing chair.  The change was approved by the ASF board yesterday,
> > April 17th.
> >
> > Thanks for stepping up Jason!  I'll be happy to assist as needed.
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
> >
>
> --
> Anshum Gupta

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

[DISCUSS] Community Virtual Meetup, April 2024

[Jason Gerlowski]

Hey all,

It's time once again to start thinking ahead to our Virtual Meetup for April!

1. Anyone have an interest in organizing?  I'm happy to organize "by
default" if we don't have any volunteers by EOD Monday, but it's
always great to rotate!

2. When would we like to meet?

Best,

Jason

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

[Operator] [ANNOUNCE] Apache Solr Operator v0.8.1 released

[Jason Gerlowski]

The Apache Solr PMC is pleased to announce the release of the Apache
Solr Operator v0.8.1.

The Apache Solr Operator is a safe and easy way of managing a Solr
ecosystem in Kubernetes.

This release contains several bug fixes, some of which are highlighted
below. It also resolves CVE-2024-31391, a credential-leak
vulnerability seen in previous releases.  The release is available for
immediate download at:

  

### Solr Operator v0.8.1 Release Highlights:

* Miscellaneous bugfixes and hardening for the "managed scaling"
feature added in v0.8.0.
* Init-containers now avoid writing to "/tmp" and other root FS
locations, to better support "read-only" root filesystems
* SolrPrometheusExporter no longer fails liveness probes when the
SolrCloud is too large

A summary of important changes is published in the documentation at:

  

For the most exhaustive list, see the change log on ArtifactHub or
view the git history in the solr-operator repo.

  


  

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

CVE-2024-31391: Apache Solr Operator: Solr-Operator liveness and readiness probes may leak basic auth credentials

[Jason Gerlowski]

Severity: moderate

Affected versions:

- Apache Solr Operator 0.3.0 through 0.8.0

Description:

Insertion of Sensitive Information into Log File vulnerability in the Apache 
Solr Operator.

This issue affects all versions of the Apache Solr Operator from 0.3.0 through 
0.8.0.

When asked to bootstrap Solr security, the operator will enable basic 
authentication and create several accounts for accessing Solr: including the 
"solr" and "admin" accounts for use by end-users, and a "k8s-oper" account 
which the operator uses for its own requests to Solr.
One common source of these operator requests is healthchecks: liveness, 
readiness, and startup probes are all used to determine Solr's health and 
ability to receive traffic.
By default, the operator configures the Solr APIs used for these probes to be 
exempt from authentication, but users may specifically request that 
authentication be required on probe endpoints as well.
Whenever one of these probes would fail, if authentication was in use, the Solr 
Operator would create a Kubernetes "event" containing the username and password 
of the "k8s-oper" account.

Within the affected version range, this vulnerability affects any solrcloud 
resource which (1) bootstrapped security through use of the 
`.solrOptions.security.authenticationType=basic` option, and (2) required 
authentication be used on probes by setting 
`.solrOptions.security.probesRequireAuth=true`.

Users are recommended to upgrade to Solr Operator version 0.8.1, which fixes 
this issue by ensuring that probes no longer print the credentials used for 
Solr requests.  Users may also mitigate the vulnerability by disabling 
authentication on their healthcheck probes using the setting 
`.solrOptions.security.probesRequireAuth=false`.

This issue is being tracked as SOLR-17216 

Credit:

Flip Hess (finder)

References:

https://solr.apache.org
https://www.cve.org/CVERecord?id=CVE-2024-31391
https://issues.apache.org/jira/browse/SOLR-17216


-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [Operator] [VOTE] Release the Solr Operator v0.8.1 RC1

[Jason Gerlowski]

It's been >72h since the vote was initiated and the result is:

+1  3  (3 binding)
 0  0
-1  0

This vote has PASSED

---

Thanks to all who voted!  I'll work on finalizing the release.


On Tue, Apr 9, 2024 at 1:54 PM Anshum Gupta  wrote:
>
> +1 (binding)
>
> | Successfully smoke tested the Solr Operator v0.8.1!
>
> On Mon, Apr 8, 2024 at 10:53 PM Jason Gerlowski 
> wrote:
>
> > Please vote for release candidate 1 for the Solr Operator v0.8.1
> >
> > The artifacts can be downloaded from:
> >
> > https://dist.apache.org/repos/dist/dev/solr/solr-operator/solr-operator-v0.8.1-RC1-revcda3b06736afacfc7101e2d807f8555b96a45b0d
> >
> > You can run the full smoke tester, with instructions below.
> > However, it is also encouraged to go and use the artifacts yourself in
> > a test Kubernetes cluster.
> > The smoke tester does not require you to download or install the RC
> > artifacts before running.
> > If you plan on just running the smoke tests, then ignore all other
> > instructions.
> >
> > The artifacts are layed out in the following way:
> >   * solr-operator-v0.8.1.tgz - Contains the source release
> >   * crds/ - Contains the CRD files
> >   * helm-charts/ - Contains the Helm release packages
> >
> > The RC Docker image can be found at:
> >   apache/solr-operator:v0.8.1-rc1
> >
> > The RC Helm repo can be added with:
> >   helm repo add apache-solr-rc
> >
> > https://dist.apache.org/repos/dist/dev/solr/solr-operator/solr-operator-v0.8.1-RC1-revcda3b06736afacfc7101e2d807f8555b96a45b0d/helm-charts
> >
> > You can install the RC Solr Operator and Solr CRDs and an example Solr
> > Cloud with:
> >   curl -sL0 "https://dist.apache.org/repos/dist/release/solr/KEYS"; |
> > gpg --import --quiet
> >   # This will export your public keys into a format that helm can
> > understand.
> >   # Skip verification by removing "--verify" in the helm command below.
> >   if ! (gpg --no-default-keyring --keyring=~/.gnupg/pubring.gpg
> > --list-keys "60392455"); then gpg --export >~/.gnupg/pubring.gpg; fi
> >   kubectl create -f
> >
> > https://dist.apache.org/repos/dist/dev/solr/solr-operator/solr-operator-v0.8.1-RC1-revcda3b06736afacfc7101e2d807f8555b96a45b0d/crds/all-with-dependencies.yaml
> > || \
> > kubectl replace -f
> >
> > https://dist.apache.org/repos/dist/dev/solr/solr-operator/solr-operator-v0.8.1-RC1-revcda3b06736afacfc7101e2d807f8555b96a45b0d/crds/all-with-dependencies.yaml
> >   helm install --verify solr-operator apache-solr-rc/solr-operator
> > --set image.tag=v0.8.1-rc1
> >   helm install --verify example apache-solr-rc/solr
> >
> > You can run the full smoke tester directly with this command: (First
> > checkout the release-0.8 branch of the solr-operator)
> >
> > # First clear your go-mod cache to make sure old cache entries don't
> > cause smoke test failures
> > make mod-clean
> > ./hack/release/smoke_test/smoke_test.sh -v "v0.8.1" -s "cda3b06" -i
> > "apache/solr-operator:v0.8.1-rc1" -g "60392455" \
> > -l '
> > https://dist.apache.org/repos/dist/dev/solr/solr-operator/solr-operator-v0.8.1-RC1-revcda3b06736afacfc7101e2d807f8555b96a45b0d
> > '
> >
> > If you want to run the smoke test with a specific version of
> > kubernetes, use the -k option with a full version tag. (e.g. -k
> > v1.19.3)
> > If you want to run the smoke test with a custom version of solr, use
> > the -t option with an official Solr image version. (e.g. -t 8.10.0)
> >   However, for this smoke test, you must use a solr version that
> > supports incremental backups. (i.e. 8.9+)
> >
> > Make sure you have the following installed before running the smoke test:
> >   - Docker (Give it enough memory and CPU to run ~12 containers, 3 of
> > which are Solr nodes)
> > More information on required resources can be found here:
> > https://kind.sigs.k8s.io/docs/user/quick-start/#settings-for-docker-desktop
> >   - Go 1.20
> >   - Kubectl
> >   - GnuPG
> >   - Helm v3.4.0+
> >   - Kustomize (v4.0.0+) This will be installed for you, but NOT
> > upgraded if a lower version is already installed.
> >   - yq
> >   - jq
> >   - coreutils (if using Mac OS)
> >
> > The vote will be open for at least 72 hours i.e. until 2024-04-11 18:00
> > UTC.
> >
> > [ ] +1  approve
> > [ ] +0  no opinion
> > [ ] -1  disapprove (and reason why)
> >
> > Here is my +1
> >
> > Jason
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
> >
>
> --
> Anshum Gupta

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [EXTERNAL] - Re: PR ready fro review

[Jason Gerlowski]

> I come back to Solr only about once a year

Well welcome back!

> implementing everything needed for clustersizing v2 would be a long and 
> difficult process for me [...] I can push what I have, but then I think 
> should make the PR a draft, and hope for participation from other 
> contributors.

And of course!  Not everyone has the time, or inclination, or context
to get every PR totally finished on their own.  Totally understandable
- it comes up a lot.  And the advice we usually give is that all
contributions - even those that are incomplete, undocumented, etc. -
are worth sharing so that others can build off of them and hopefully
close whatever gaps remain.  The only thing you need to do in that
case is make sure the known gaps are mentioned somewhere - which you
already sort of did above, so you're all set!

Best,

Jason

On Wed, Apr 10, 2024 at 4:24 PM Isabelle Giguere
 wrote:
>
> Thanks for the feedback.
>
> I'm trying to implement V2.
> API interface, annotated response class: easy.
>
> Now I'm stuck.
> All the examples I can find in CollectionHandler (where clustersizing 
> resides) call CollectionsHandler.submitCollectionApiCommand(...), which 
> requires an implementation of interface CollectionApiCommand, which was never 
> implemented for clustersizing.
>
> I come back to Solr only about once a year, and usually to apply some old 
> patch on a more recent version. That means I have a limited understanding of 
> what ties into what and why.  So, implementing everything needed for 
> clustersizing v2 would be a long and difficult process for me.
>
> I can push what I have, but then I think should make the PR a draft, and hope 
> for participation from other contributors.
>
>
> Isabelle Giguère
> Computational Linguist & Java Developer
> Linguiste informaticienne & développeur java
>
>
> 
> De : Gus Heck 
> Envoyé : 9 avril 2024 11:15
> À : dev@solr.apache.org 
> Objet : [EXTERNAL] - Re: PR ready fro review
>
> CAUTION: This email originated from outside of the organization. Do not click 
> links or open attachments unless you recognize the sender and know the 
> content is safe. If you feel that the email is suspicious, please report it 
> using PhishAlarm.
>
>
> +1 for v2 in same PR
>
> On Tue, Apr 9, 2024 at 10:15 AM Jason Gerlowski 
> wrote:
>
> > > V2 could be handled in a different PR, if preferable.
> >
> > My preference at least would be that v2 is added in the same PR, so
> > there's no chance it falls through the cracks.
> >
> > I haven't found time to review yet, but I've put this on my list and
> > will try to come back through this week at some point.  If I happen to
> > forget, feel free to ping me (@gerlowskija) on the PR if I haven't
> > reviewed by Thursday or Friday.
> >
> > On Tue, Apr 9, 2024 at 8:29 AM Eric Pugh
> >  wrote:
> > >
> > > I put some comments in.On the V2 api, I think that if this is
> > specific to a collection, then /api/collections/cluster-size or maybe even
> > /api/collections/sizing If it’s overall, and not collection
> > specific, then /api/cluster/sizing would make sense.And if it supports
> > both, well then it could be both patterns!
> > >
> > > Jason G might have more insight on the best path….
> > >
> > > > On Apr 8, 2024, at 5:02 PM, Isabelle Giguere
> >  wrote:
> > > >
> > > > Hello committers;
> > > >
> > > > Please take some time to review this PR :
> > > > https://urldefense.com/v3/__https://github.com/apache/solr/pull/1638__;!!Obbck6kTJA!aD_c0LMFWboisH1fRZUyMdHu2BNU7KuwCCbeTetcPxm_kPggNqQk6TGSpDNKoCY2bcUYyNn3QQ13OIX6Tw$
> > > >
> > > > It is based on a rather old patch, so only V1 is supported, for now.
> > > >
> > > > If we wanted V2, what should be the URL path ?
> > > >
> > > >  *
> > > > /api/collections/cluster-size
> > > >  *
> > > > /api/cluster/sizing
> > > >
> > > > V2 could be handled in a different PR, if preferable.
> > > >
> > > > Thank you;
> > > >
> > > > Isabelle Giguère
> > > > Computational linguist & Java developer  |  Engineering
> > > > Linguiste informaticienne & développeur java  |  Engineering
> > > >
> > > > Phone:  (514) 908 5406 ext. 75125
> > > > Website:https://www.opentext.com/
> > > >
> > > >
> > > > [
> > https://mimage.opentext.com/

Re: PR ready fro review

[Jason Gerlowski]

> V2 could be handled in a different PR, if preferable.

My preference at least would be that v2 is added in the same PR, so
there's no chance it falls through the cracks.

I haven't found time to review yet, but I've put this on my list and
will try to come back through this week at some point.  If I happen to
forget, feel free to ping me (@gerlowskija) on the PR if I haven't
reviewed by Thursday or Friday.

On Tue, Apr 9, 2024 at 8:29 AM Eric Pugh
 wrote:
>
> I put some comments in.On the V2 api, I think that if this is specific to 
> a collection, then /api/collections/cluster-size or maybe even 
> /api/collections/sizing If it’s overall, and not collection specific, 
> then /api/cluster/sizing would make sense.And if it supports both, well 
> then it could be both patterns!
>
> Jason G might have more insight on the best path….
>
> > On Apr 8, 2024, at 5:02 PM, Isabelle Giguere 
> >  wrote:
> >
> > Hello committers;
> >
> > Please take some time to review this PR :
> > https://github.com/apache/solr/pull/1638
> >
> > It is based on a rather old patch, so only V1 is supported, for now.
> >
> > If we wanted V2, what should be the URL path ?
> >
> >  *
> > /api/collections/cluster-size
> >  *
> > /api/cluster/sizing
> >
> > V2 could be handled in a different PR, if preferable.
> >
> > Thank you;
> >
> > Isabelle Giguère
> > Computational linguist & Java developer  |  Engineering
> > Linguiste informaticienne & développeur java  |  Engineering
> >
> > Phone:  (514) 908 5406 ext. 75125
> > Website:https://www.opentext.com/
> >
> >
> > [https://mimage.opentext.com/alt_content/binary/images/email-signature/ot2023-corporate-email-signature-370x70-fy24-2-retina.png]
> >
> > This email message is confidential, may be privileged, and is intended for 
> > the exclusive use of the addressee. Any other person is strictly prohibited 
> > from disclosing or reproducing it. If the addressee cannot be reached or is 
> > unknown to you, please inform the sender by return email and delete this 
> > email message and all copies immediately.
>
> ___
> Eric Pugh | Founder | OpenSource Connections, LLC | 434.466.1467 | 
> http://www.opensourceconnections.com  
> | My Free/Busy 
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 
> 
> This e-mail and all contents, including attachments, is considered to be 
> Company Confidential unless explicitly stated otherwise, regardless of 
> whether attachments are marked as such.
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

[Operator] [VOTE] Release the Solr Operator v0.8.1 RC1

[Jason Gerlowski]

Please vote for release candidate 1 for the Solr Operator v0.8.1

The artifacts can be downloaded from:
https://dist.apache.org/repos/dist/dev/solr/solr-operator/solr-operator-v0.8.1-RC1-revcda3b06736afacfc7101e2d807f8555b96a45b0d

You can run the full smoke tester, with instructions below.
However, it is also encouraged to go and use the artifacts yourself in
a test Kubernetes cluster.
The smoke tester does not require you to download or install the RC
artifacts before running.
If you plan on just running the smoke tests, then ignore all other instructions.

The artifacts are layed out in the following way:
  * solr-operator-v0.8.1.tgz - Contains the source release
  * crds/ - Contains the CRD files
  * helm-charts/ - Contains the Helm release packages

The RC Docker image can be found at:
  apache/solr-operator:v0.8.1-rc1

The RC Helm repo can be added with:
  helm repo add apache-solr-rc
https://dist.apache.org/repos/dist/dev/solr/solr-operator/solr-operator-v0.8.1-RC1-revcda3b06736afacfc7101e2d807f8555b96a45b0d/helm-charts

You can install the RC Solr Operator and Solr CRDs and an example Solr
Cloud with:
  curl -sL0 "https://dist.apache.org/repos/dist/release/solr/KEYS"; |
gpg --import --quiet
  # This will export your public keys into a format that helm can understand.
  # Skip verification by removing "--verify" in the helm command below.
  if ! (gpg --no-default-keyring --keyring=~/.gnupg/pubring.gpg
--list-keys "60392455"); then gpg --export >~/.gnupg/pubring.gpg; fi
  kubectl create -f
https://dist.apache.org/repos/dist/dev/solr/solr-operator/solr-operator-v0.8.1-RC1-revcda3b06736afacfc7101e2d807f8555b96a45b0d/crds/all-with-dependencies.yaml
|| \
kubectl replace -f
https://dist.apache.org/repos/dist/dev/solr/solr-operator/solr-operator-v0.8.1-RC1-revcda3b06736afacfc7101e2d807f8555b96a45b0d/crds/all-with-dependencies.yaml
  helm install --verify solr-operator apache-solr-rc/solr-operator
--set image.tag=v0.8.1-rc1
  helm install --verify example apache-solr-rc/solr

You can run the full smoke tester directly with this command: (First
checkout the release-0.8 branch of the solr-operator)

# First clear your go-mod cache to make sure old cache entries don't
cause smoke test failures
make mod-clean
./hack/release/smoke_test/smoke_test.sh -v "v0.8.1" -s "cda3b06" -i
"apache/solr-operator:v0.8.1-rc1" -g "60392455" \
-l 
'https://dist.apache.org/repos/dist/dev/solr/solr-operator/solr-operator-v0.8.1-RC1-revcda3b06736afacfc7101e2d807f8555b96a45b0d'

If you want to run the smoke test with a specific version of
kubernetes, use the -k option with a full version tag. (e.g. -k
v1.19.3)
If you want to run the smoke test with a custom version of solr, use
the -t option with an official Solr image version. (e.g. -t 8.10.0)
  However, for this smoke test, you must use a solr version that
supports incremental backups. (i.e. 8.9+)

Make sure you have the following installed before running the smoke test:
  - Docker (Give it enough memory and CPU to run ~12 containers, 3 of
which are Solr nodes)
More information on required resources can be found here:
https://kind.sigs.k8s.io/docs/user/quick-start/#settings-for-docker-desktop
  - Go 1.20
  - Kubectl
  - GnuPG
  - Helm v3.4.0+
  - Kustomize (v4.0.0+) This will be installed for you, but NOT
upgraded if a lower version is already installed.
  - yq
  - jq
  - coreutils (if using Mac OS)

The vote will be open for at least 72 hours i.e. until 2024-04-11 18:00 UTC.

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Here is my +1

Jason

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Solr Operator 0.8.1 Release

[Jason Gerlowski]

Hey all,

The last few weeks have seen a handful of useful quality of life
improvements and other changes merged into the operator, so I'd like to
propose a quick 0.8.1 release.

Pending any objections I'll start on the release on Monday and hope to have
it all wrapped up (or at least out for "vote") by the end of the week!

Best,

Jason

Re: Automatic "tidy" (formatting) in PR, GitHub action

[Jason Gerlowski]

+1, if it's not too big of a "lift" to get done I think it'd be great.

I've wondered before if "tidy" could be a "commit-hook", though the idea
probably isn't workable in practice due to tidy's ~30s runtime.  I like
your PR-level check idea much better.

On Fri, Mar 22, 2024 at 2:33 PM David Smiley 
wrote:

> Sometimes we make changes and forget to run tidy.  It's rather annoying.
>
> It occurred to me that our "precommit" GitHub PR action could be modified
> to first run tidy and to commit the changes (if any) beforehand, pushing to
> the source branch (generally on someone's fork).  Here's a blog post with
> examples on how to do this:
>
> https://peterevans.dev/posts/github-actions-how-to-automate-code-formatting-in-pull-requests/
> There are some caveats listed there... like a possible permissions issue.
> It seems there may be a solution --
> https://github.com/orgs/community/discussions/26865
> Also, the post recommends a "slash command" approach instead but I think
> that would just add an extra step; we know what the solution is every
> time.  And of course a contributor can do manual follow-up editing of the
> results when it doesn't flow nicely.  Ultimately it all gets squashed
> anyway.
>
> Any thoughts on this or an alternative?
>
> ~ David
>

Re: [DISCUSS] Community Virtual Meetup, March 2024

[Jason Gerlowski]

Alright, better late than never, here are the links for this month's
Community Meetup!

Meeting Notes Page:
https://cwiki.apache.org/confluence/display/SOLR/2024-03-28+Meeting+notes
Meeting Room: https://meet.google.com/zqi-dupd-unz
TimeAndDate:
https://www.timeanddate.com/worldclock/fixedtime.html?msg=Solr+Community+Meetup%2C+March+2024&iso=20240328T09&p1=224&ah=1
Meetup.com:
https://www.meetup.com/apache-solr-virtual-community-meetups/events/38963

It's not too late to add topics for discussion on the Meeting Notes page.
Hope to see you all there!

Best,

Jason

On Fri, Mar 22, 2024 at 8:25 AM Eric Pugh 
wrote:

> Sounds good….  BTW, I’d like to make a plug for C/C NA, apparently the CfP
> is wrapping soon!
>
>
> > On Mar 21, 2024, at 10:49 PM, David Smiley  wrote:
> >
> > Sounds good; I can probably join then.  Thanks for organizing!
> >
> > On Thu, Mar 21, 2024 at 11:26 AM Jason Gerlowski 
> wrote:
> >>
> >> Hey all,
> >>
> >> Haven't heard any suggestions on scheduling, so maybe we can aim for
> noon
> >> ET on Thursday of next week?  That gives us a full week between then and
> >> now.  Pending any last minute objections I'll create the Confluence page
> >> and Google Meet link later today.
> >>
> >> Hope to see many of you there!
> >>
> >> Jason
> >>
> >> On Tue, Mar 19, 2024 at 8:34 AM Jason Gerlowski 
> >> wrote:
> >>
> >>> Hey all,
> >>>
> >>> It's time once again to start thinking ahead to our Virtual Meetup for
> >>> March!  (Apologies for starting this discussion a bit late this month,
> as
> >>> I've been afk for a few weeks.)
> >>>
> >>> Since we are pretty far into the month I'll volunteer to organize the
> >>> meeting for March, so all we need to do is pick a day and time to meet.
> >>> Does anyone have opinions on that?  Maybe one day next week would make
> for
> >>> a good target?
> >>>
> >>> Best,
> >>>
> >>> Jason
> >>>
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
>
> ___
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
>
>

Re: [DISCUSS] Community Virtual Meetup, March 2024

[Jason Gerlowski]

Hey all,

Haven't heard any suggestions on scheduling, so maybe we can aim for noon
ET on Thursday of next week?  That gives us a full week between then and
now.  Pending any last minute objections I'll create the Confluence page
and Google Meet link later today.

Hope to see many of you there!

Jason

On Tue, Mar 19, 2024 at 8:34 AM Jason Gerlowski 
wrote:

> Hey all,
>
> It's time once again to start thinking ahead to our Virtual Meetup for
> March!  (Apologies for starting this discussion a bit late this month, as
> I've been afk for a few weeks.)
>
> Since we are pretty far into the month I'll volunteer to organize the
> meeting for March, so all we need to do is pick a day and time to meet.
> Does anyone have opinions on that?  Maybe one day next week would make for
> a good target?
>
> Best,
>
> Jason
>

[DISCUSS] Community Virtual Meetup, March 2024

[Jason Gerlowski]

Hey all,

It's time once again to start thinking ahead to our Virtual Meetup for
March!  (Apologies for starting this discussion a bit late this month, as
I've been afk for a few weeks.)

Since we are pretty far into the month I'll volunteer to organize the
meeting for March, so all we need to do is pick a day and time to meet.
Does anyone have opinions on that?  Maybe one day next week would make for
a good target?

Best,

Jason

Re: MixedCase or dashed-case for long options in Solr CLI?

[Jason Gerlowski]

My guess is that "dashed-case" is slightly more common -- at least,
that's my sense from haphazardly checking a few tools I use often
("curl", "kubectl", "git", "docker").

But I don't have an opinion as long as we're internally consistent
about using one convention or the other.

Best,

Jason

On Sat, Feb 24, 2024 at 11:35 AM Eric Pugh
 wrote:
>
> Hi all,
>
> I wanted to get the communities input on formatting of long options for the 
> Solr CLI.   I noticed on https://commons.apache.org/proper/commons-cli/ that 
> their examples all are —dashed-case.
>
> However, we have —solrUrl or —zkHost as our pattern.   Though in working on 
> the PostTool, I used —solr-update-url as the parameter because I had been 
> reading the commons-cli docs...
>
> I’d like to get this sorted so that I can get 
> https://issues.apache.org/jira/browse/SOLR-16824 over the finish line.   So 
> please do speak up with preferences!   (And please let’s not support both!)
>
>
> The changes to the formatting will be a 10x thing.
>
> Eric
>
> ___
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | 
> http://www.opensourceconnections.com  
> | My Free/Busy 
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 
> 
> This e-mail and all contents, including attachments, is considered to be 
> Company Confidential unless explicitly stated otherwise, regardless of 
> whether attachments are marked as such.
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: Moving to bin/solr start defaulting to SolrCloud mode?

[Jason Gerlowski]

Thanks for the link to that 2021 discussion Jan - had forgotten about that.

I agree that SIP-14 (or some portion of it) goes a long way towards
making SolrCloud (w/ embedded ZK) a more serviceable default.

On Fri, Feb 23, 2024 at 7:14 PM Jan Høydahl  wrote:
>
> Cross referencing earlier discussion "[DISCUSS] Make Cloud mode the default 
> option in bin/solr" from May 2021:
>  https://lists.apache.org/thread/79xp2xfvqwhr9zccmsvjvj0hckgg5m6w
>
> Some valid arguments for an against in that thread.
>
> Ideally I'd prefer SIP-14 to be done before embedded zk is promoted as the 
> best way to run Solr.
>
> Jan
>
> > 23. feb. 2024 kl. 19:06 skrev Eric Pugh :
> >
> > During today’s community discussion the topic of moving to defaulting to 
> > SolrCloud mode came up.
> >
> > The idea here is that when a user run’s “bin/solr start” it fires up an 
> > embedded zookeeper.   Same behavior as “bin/solr -c” in Solr 9.5.If you 
> > have a Zookeeper Ensemble then “bin/solr start -z YOUR_ZK_SETUP” would 
> > connect to the external ensemble instead.
> >
> > If you want to continue to use the class user managed mode, then bin/solr 
> > start —user-managed maybe?   Or bin/solr start —standalone ???
> >
> > Other changes would be to go through the Ref Guide and where we have both 
> > SolrCloud and non SolrCloud content that we make sure SolrCloud content is 
> > at the top instead of at the bottom.
> >
> > To me, this feels like a change that would go on main.
> >
> > Thoughts?
> >
> > Eric
> >
> >
> >
> >
> >
> >
> >
> >
> > ___
> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | 
> > http://www.opensourceconnections.com 
> >  | My Free/Busy 
> > 
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 
> > 
> > This e-mail and all contents, including attachments, is considered to be 
> > Company Confidential unless explicitly stated otherwise, regardless of 
> > whether attachments are marked as such.
> >
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [DISCUSS] Community Virtual Meetup, February 2024

[Jason Gerlowski]

Alright, finally got the links in order!

Confluence Meeting Notes:
https://cwiki.apache.org/confluence/display/SOLR/2024-02-23+Meeting+notes

(Remember to add your discussion topics here ^^)

Meetup.com Event:
https://www.meetup.com/apache-solr-virtual-community-meetups/events/299274449/

Google Meet Link: https://meet.google.com/cbf-hsns-mxh

See you all on Friday!

Best,

Jason


On Sat, Feb 17, 2024 at 10:52 AM Eric Pugh
 wrote:
>
> Thanks!
>
>
> > On Feb 17, 2024, at 10:18 AM, Jason Gerlowski  wrote:
> >
> > Alright - looks like everyone's schedule is pretty flexible, so 2/23
> > wins with Bruno's lone vote :-p
> >
> > Will share the Confluence, Meetup.com, and Google Meet links shortly;
> > see you all next week!
> >
> > Best,
> >
> > Jason
> >
> > On Thu, Feb 15, 2024 at 5:42 PM Alessandro Benedetti
> >  wrote:
> >>
> >> Thanks for organising this! I am good with all of them!
> >> Cheers
> >> --
> >> *Alessandro Benedetti*
> >> Director @ Sease Ltd.
> >> *Apache Lucene/Solr Committer*
> >> *Apache Solr PMC Member*
> >>
> >> e-mail: a.benede...@sease.io
> >>
> >>
> >> *Sease* - Information Retrieval Applied
> >> Consulting | Training | Open Source
> >>
> >> Website: Sease.io <http://sease.io/>
> >> LinkedIn <https://linkedin.com/company/sease-ltd> | Twitter
> >> <https://twitter.com/seaseltd> | Youtube
> >> <https://www.youtube.com/channel/UCDx86ZKLYNpI3gzMercM7BQ> | Github
> >> <https://github.com/seaseltd>
> >>
> >>
> >> On Wed, 14 Feb 2024 at 00:47, Ilan Ginzburg  wrote:
> >>
> >>> Thanks for organizing this.
> >>>
> >>> I vote *not* 2/20 and *not* 2/22.
> >>>
> >>> On Tue 13 Feb 2024 at 22:53, Bruno Roustant 
> >>> wrote:
> >>>
> >>>> I vote for 2/23
> >>>>
> >>>> Thanks Jason
> >>>>
> >>>
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
>
> ___
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | 
> http://www.opensourceconnections.com <http://www.opensourceconnections.com/> 
> | My Free/Busy <http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 
> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
> This e-mail and all contents, including attachments, is considered to be 
> Company Confidential unless explicitly stated otherwise, regardless of 
> whether attachments are marked as such.
>

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [DISCUSS] Community Virtual Meetup, February 2024

[Jason Gerlowski]

Alright - looks like everyone's schedule is pretty flexible, so 2/23
wins with Bruno's lone vote :-p

Will share the Confluence, Meetup.com, and Google Meet links shortly;
see you all next week!

Best,

Jason

On Thu, Feb 15, 2024 at 5:42 PM Alessandro Benedetti
 wrote:
>
> Thanks for organising this! I am good with all of them!
> Cheers
> --
> *Alessandro Benedetti*
> Director @ Sease Ltd.
> *Apache Lucene/Solr Committer*
> *Apache Solr PMC Member*
>
> e-mail: a.benede...@sease.io
>
>
> *Sease* - Information Retrieval Applied
> Consulting | Training | Open Source
>
> Website: Sease.io 
> LinkedIn  | Twitter
>  | Youtube
>  | Github
> 
>
>
> On Wed, 14 Feb 2024 at 00:47, Ilan Ginzburg  wrote:
>
> > Thanks for organizing this.
> >
> > I vote *not* 2/20 and *not* 2/22.
> >
> > On Tue 13 Feb 2024 at 22:53, Bruno Roustant 
> > wrote:
> >
> > > I vote for 2/23
> > >
> > > Thanks Jason
> > >
> >

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

[ANNOUNCE] Apache Solr 9.5.0 released

[Jason Gerlowski]

The Solr PMC is pleased to announce the release of Apache Solr 9.5.0.

Solr is the popular, blazing fast, open source NoSQL search platform
from the Apache Solr project. Its major features include powerful
full-text search, hit highlighting, faceted search, dynamic
clustering, database integration, rich document handling, and
geospatial search. Solr is highly scalable, providing fault tolerant
distributed search and indexing, and powers the search and navigation
features of many of the world's largest internet sites.

Solr 9.5.0 is available for immediate download at:

  

### Solr 9.5.0 Release Highlights:

 * Solr now supports "node-level" Memory and CPU circuit breakers,
that serve as a default inherited by all cores on that node.
 * Collection and Replica Properties may now be used as property
substitution variables in configuration files (e.g. solrconfig.xml).
 * Solr now auto-reloads updated keystore and truststore files when
TLS is enabled.
 * Tracing support has received a number of quality-of-life
improvements, including improved tracking of distributed collection
commands and increased coverage for internal requests made with the
Apache and Jetty HTTP clients.
 * Solr now offers the ``  solr.xml tag as a means
of configuring node-level plugins in an "immutable
infrastructure"-friendly way.  This offers an alternative to using the
`/cluster/plugins`  API for managing plugins in "live" clusters.
 * Starting with 9.5.0, Solr releases now produce an OpenAPI
specification covering many of their v2 APIs. Consumers may use this
spec with an array of OpenAPI-compatible tooling to generate
documentation, clients in various programming languages, etc.  See
https://www.openapis.org/what-is-openapi for more details.

Please refer to the Upgrade Notes in the Solr Ref Guide for
information on upgrading from previous Solr versions:

  

Please read CHANGES.txt for a full list of new features, changes and bugfixes:

  

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org

Re: [VOTE] Release Solr 9.5.0 RC3

[Jason Gerlowski]

It's been >72h since the vote was initiated and the result is:

+1  6  (5 binding)
 0  0
-1  0

This vote has PASSED.

Thanks all - I'll start finalizing the release today!

On Sun, Feb 11, 2024 at 1:57 PM David Smiley  wrote:

> +1 SUCCESS! [0:51:20.054401]
>
> BTW the first run failed due to a test.  I ran the repro line and it
> did reproduce ... but just once and not again.  Didn't on main.
> Didn't see in build failures (tracked via fucit.org) but saw once in
> Gradle Enterprise albeit in that build 50 tests failed:
>
> https://ge.apache.org/s/noe3h4xdd6a5m/tests/overview?help=own-time&outcome=FAILED
> I "beasted" shortly and didn't repro.
>
> gradlew :solr:core:test --tests
>
> "org.apache.solr.cloud.LeaderVoteWaitTimeoutTest.testMostInSyncReplicasCanWinElection"
> -Ptests.jvms=8 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1
> -XX:+UseParallelGC -XX:ActiveProcessorCount=1
> -XX:ReservedCodeCacheSize=120m" -Ptests.seed=3BEF0DB1F4994AE0
> -Ptests.badapples=false -Ptests.file.encoding=UTF-8
>
> On Fri, Feb 9, 2024 at 6:48 AM Jan Høydahl  wrote:
> >
> > +1 (binding)
> >
> > SUCCESS! [0:42:27.908762]
> >
> > ALso built and started docker image.
> >
> > Jan
> >
> > > 7. feb. 2024 kl. 21:57 skrev Jason Gerlowski :
> > >
> > > Please vote for release candidate 3 for Solr 9.5.0
> > >
> > > The artifacts can be downloaded from:
> > >
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC3-rev-cdd27dd15c3a6574032e9b1b92b148ab4e383599
> > >
> > > You can run the smoke tester directly with this command:
> > >
> > > python3 -u dev-tools/scripts/smokeTestRelease.py \
> > >
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC3-rev-cdd27dd15c3a6574032e9b1b92b148ab4e383599
> > >
> > > You can build a release-candidate of the official docker images (full &
> > > slim) using the following command:
> > >
> > > SOLR_DOWNLOAD_SERVER=
> > >
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC3-rev-cdd27dd15c3a6574032e9b1b92b148ab4e383599/solr
> > > && \
> > >  docker build
> $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-full \
> > >--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
> > >-t solr-rc:9.5.0-3 && \
> > >  docker build
> $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-slim \
> > >--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
> > >-t solr-rc:9.5.0-3-slim
> > >
> > > The vote will be open for at least 72 hours i.e. until 2024-02-10
> 21:00 UTC.
> > >
> > > [ ] +1  approve
> > > [ ] +0  no opinion
> > > [ ] -1  disapprove (and reason why)
> > >
> > > Here is my +1
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: Use cases for interacting direct with ZK versus using our APIs?

[Jason Gerlowski]

> Assuming an API existed, would that then be a good way?

Yep - if there was an API, I think that'd be sufficient (for this use case
at least).  No reason a user would _want_ to read them out of ZK directly,
afaik.

On Mon, Feb 12, 2024 at 9:19 AM Eric Pugh 
wrote:

> Assuming an API existed, would that then be a good way?  Or are there
> times when an API wouldn’t do what direct ZK does?
>
> > On Feb 12, 2024, at 9:13 AM, Jason Gerlowski 
> wrote:
> >
> >> What other use cases are there for us interacting directly with ZK?
> >
> > Reading cluster/replica/collection properties for sure.  There are some
> > Solr APIs to modify these things, but afaik users that want to check the
> > value of a particular (e.g.) cluster-property don't have a way to do that
> > yet without reading it out of ZK directly.
> >
> > On Sun, Feb 11, 2024 at 1:58 PM Eric Pugh <
> ep...@opensourceconnections.com <mailto:ep...@opensourceconnections.com>>
> > wrote:
> >
> >> I like your suggestion David on the CLI supporting configset directly.
> >> I’m sort of waiting till the V2 api for putting configsets is done.
> >> According to the Solr v2 API Proposed Changes spreadsheet, while there
> is a
> >> V2 api, it hasn’t been given the “restful/jax-rs/exposed in solrj”
> >> treatment yet….
> >>
> >> I also agree that figuring out how to make ZK an internal detail is
> >> somewhat underlying my question….
> >>
> >> What other use cases are there for us interacting directly with ZK?
> >>
> >>> On Feb 11, 2024, at 12:57 PM, David Smiley  wrote:
> >>>
> >>> I suppose direct ZK has an advantage of not requiring that Solr is
> >>> running yet.  But maybe that's moot.  I do think it would make sense
> >>> to have CLI options for configset upload that use Solr's API and don't
> >>> refer to ZK.  Broadly, this is the direction we're going in; ZK is
> >>> more of an internal detail, even though it's obviously a critical
> >>> thing.
> >>>
> >>> On Sun, Feb 11, 2024 at 12:26 PM Eric Pugh
> >>>  ep...@opensourceconnections.com> <mailto:ep...@opensourceconnections.com>>
> >> wrote:
> >>>>
> >>>> Ah.. yeah, I can’t speak to Solr 6.x!   In 9x at least you could use
> >> the configset API to deploy configs and avoid the direct ZK interaction.
> >>>>
> >>>> It would be interesting to explore if the process of deploying a
> >> configset is risky, has a high chance of things failing, then how do we
> >> account for that as part of the process?So you don’t have to do
> things
> >> like upload the previous config ;-).
> >>>>
> >>>> And other common reasons to use ZK directly?
> >>>>
> >>>>> On Feb 11, 2024, at 12:14 PM, Walter Underwood <
> wun...@wunderwood.org <mailto:wun...@wunderwood.org>>
> >> wrote:
> >>>>>
> >>>>> The was deploying configs with Jenkins on Solr 6.x. Maybe the APIs
> >> were there, but I didn't know about them.
> >>>>>
> >>>>> Rebuilding the suggester did need external help, since that needs to
> >> be done separately on each node.
> >>>>>
> >>>>> I think working directly with Zookeeper is less risky. If there is
> any
> >> issue with the upload, then don’t reload the collections. You can back
> out
> >> the changes by uploading the previous config to Zookeeper.
> >>>>>
> >>>>> wunder
> >>>>> Walter Underwood
> >>>>> wun...@wunderwood.org <mailto:wun...@wunderwood.org>  wun...@wunderwood.org>  >> wun...@wunderwood.org <mailto:wun...@wunderwood.org>>
> >>>>> http://observer.wunderwood.org/  (my blog)
> >>>>>
> >>>>>> On Feb 11, 2024, at 11:07 AM, Eric Pugh <
> >> ep...@opensourceconnections.com <mailto:ep...@opensourceconnections.com>
> <mailto:ep...@opensourceconnections.com>
> >> <mailto:ep...@opensourceconnections.com>> wrote:
> >>>>>>
> >>>>>> Could you share more about “update Solr remotely” that you were
> >> doing?   Are we missing some APIs that would have made whatever you had
> to
> >> do require ZK direct access?
> >>>>>>
> >>>>>> While it’s cool that we can impact Solr via hacking around in ZK,

Re: [DISCUSS] Community Virtual Meetup, February 2024

[Jason Gerlowski]

I'll volunteer to organize this month!

For scheduling, I'll try to stick to what Raghavan has been doing the last
few months.  Feels like that's worked out well for folks?  Pending an
objection, let's meet at 9am PT some day in the week of 2/19 - 2/23.  If
you have a preference, please vote for one of the following days:

   - 2/19
   - 2/20
   - 2/21
   - 2/22
   - 2/23

Best,

Jason

On Thu, Feb 8, 2024 at 11:34 AM Jason Gerlowski 
wrote:

> Hey all,
>
> It's time once again to start thinking ahead to our Virtual Meetup for
> February!  As always, there's two main questions to answer in terms of
> planning:
>
> 1. Any volunteers to organize?  Organizer duties are pretty light and are
> summarized here:
> https://cwiki.apache.org/confluence/display/SOLR/Meeting+notes.
> Volunteers get some discretion in terms of picking a meeting time/day that
> works for them.  If there are no volunteers by the end of the week, I'm
> more than happy to set things up for this month.
>
> 2. When should we meet?  I've started the discussion early this month, so
> we've got plenty of days to choose from.  If you have any opinions, please
> discuss.
>
> Best,
>
> Jason
>

Re: Use cases for interacting direct with ZK versus using our APIs?

[Jason Gerlowski]

> What other use cases are there for us interacting directly with ZK?

Reading cluster/replica/collection properties for sure.  There are some
Solr APIs to modify these things, but afaik users that want to check the
value of a particular (e.g.) cluster-property don't have a way to do that
yet without reading it out of ZK directly.

On Sun, Feb 11, 2024 at 1:58 PM Eric Pugh 
wrote:

> I like your suggestion David on the CLI supporting configset directly.
>  I’m sort of waiting till the V2 api for putting configsets is done.
> According to the Solr v2 API Proposed Changes spreadsheet, while there is a
> V2 api, it hasn’t been given the “restful/jax-rs/exposed in solrj”
> treatment yet….
>
> I also agree that figuring out how to make ZK an internal detail is
> somewhat underlying my question….
>
> What other use cases are there for us interacting directly with ZK?
>
> > On Feb 11, 2024, at 12:57 PM, David Smiley  wrote:
> >
> > I suppose direct ZK has an advantage of not requiring that Solr is
> > running yet.  But maybe that's moot.  I do think it would make sense
> > to have CLI options for configset upload that use Solr's API and don't
> > refer to ZK.  Broadly, this is the direction we're going in; ZK is
> > more of an internal detail, even though it's obviously a critical
> > thing.
> >
> > On Sun, Feb 11, 2024 at 12:26 PM Eric Pugh
> > mailto:ep...@opensourceconnections.com>>
> wrote:
> >>
> >> Ah.. yeah, I can’t speak to Solr 6.x!   In 9x at least you could use
> the configset API to deploy configs and avoid the direct ZK interaction.
> >>
> >> It would be interesting to explore if the process of deploying a
> configset is risky, has a high chance of things failing, then how do we
> account for that as part of the process?So you don’t have to do things
> like upload the previous config ;-).
> >>
> >> And other common reasons to use ZK directly?
> >>
> >>> On Feb 11, 2024, at 12:14 PM, Walter Underwood 
> wrote:
> >>>
> >>> The was deploying configs with Jenkins on Solr 6.x. Maybe the APIs
> were there, but I didn't know about them.
> >>>
> >>> Rebuilding the suggester did need external help, since that needs to
> be done separately on each node.
> >>>
> >>> I think working directly with Zookeeper is less risky. If there is any
> issue with the upload, then don’t reload the collections. You can back out
> the changes by uploading the previous config to Zookeeper.
> >>>
> >>> wunder
> >>> Walter Underwood
> >>> wun...@wunderwood.org   wun...@wunderwood.org>
> >>> http://observer.wunderwood.org/  (my blog)
> >>>
>  On Feb 11, 2024, at 11:07 AM, Eric Pugh <
> ep...@opensourceconnections.com 
> > wrote:
> 
>  Could you share more about “update Solr remotely” that you were
> doing?   Are we missing some APIs that would have made whatever you had to
> do require ZK direct access?
> 
>  While it’s cool that we can impact Solr via hacking around in ZK, it
> also seems like an approach fraught with risk!
> 
> > On Feb 11, 2024, at 11:32 AM, Walter Underwood <
> wun...@wunderwood.org > wrote:
> >
> > I wanted something that didn’t require installing Solr locally in
> order to update Solr remotely, so I didn’t use the provided zk commands. I
> wrote some Python to dig the Zookeeper addresses out of clusterstatus (I
> think) then uploaded directly to Zookeeper with the Python kazoo package.
> >
> > The tool had a bunch of other things, like async reload checking for
> results, and rebuilding suggestion dictionaries on each node.
> >
> > wunder
> > Walter Underwood
> > wun...@wunderwood.org 
> > http://observer.wunderwood.org/  (my blog)
> >
> >> On Feb 11, 2024, at 9:04 AM, Gus Heck  gus.h...@gmail.com>> wrote:
> >>
> >> I pretty much always use zk upconfig, which also works for
> overwriting
> >> existing. I certainly tell my clients to use apis from the ref
> guide for
> >> such operations, but zk upconfig certainly counts as one. Mostly I
> tell
> >> them that they should only break out things like
> >> https://github.com/rgs1/zk_shell as a last resort (which is what I
> think of
> >> as direct modification), and if they are unsure, call me *before*
> doing
> >> anything in zk directly.
> >>
> >> By the way, I don't know if this has come up in a dev/build setting
> or not,
> >> but are you aware of https://plugins.gradle.org/search?term=solr ?
> It is
> >> presently only really suitable for local dev, with a single config
> set, but
> >> could easily grow patches and suggestions welcome of course.
> >>
> >> On Sun, Feb 11, 2024, 9:10 AM Eric Pugh <
> ep...@opensourceconnections.com >
> >> wrote:
> >>
> >>> Hi all..   I was playing around with a cluster and want

[DISCUSS] Community Virtual Meetup, February 2024

[Jason Gerlowski]

Hey all,

It's time once again to start thinking ahead to our Virtual Meetup for
February!  As always, there's two main questions to answer in terms of
planning:

1. Any volunteers to organize?  Organizer duties are pretty light and are
summarized here:
https://cwiki.apache.org/confluence/display/SOLR/Meeting+notes.  Volunteers
get some discretion in terms of picking a meeting time/day that works for
them.  If there are no volunteers by the end of the week, I'm more than
happy to set things up for this month.

2. When should we meet?  I've started the discussion early this month, so
we've got plenty of days to choose from.  If you have any opinions, please
discuss.

Best,

Jason

[VOTE] Release Solr 9.5.0 RC3

[Jason Gerlowski]

Please vote for release candidate 3 for Solr 9.5.0

The artifacts can be downloaded from:
https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC3-rev-cdd27dd15c3a6574032e9b1b92b148ab4e383599

You can run the smoke tester directly with this command:

python3 -u dev-tools/scripts/smokeTestRelease.py \
https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC3-rev-cdd27dd15c3a6574032e9b1b92b148ab4e383599

You can build a release-candidate of the official docker images (full &
slim) using the following command:

SOLR_DOWNLOAD_SERVER=
https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC3-rev-cdd27dd15c3a6574032e9b1b92b148ab4e383599/solr
&& \
  docker build $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-full \
--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
-t solr-rc:9.5.0-3 && \
  docker build $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-slim \
--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
-t solr-rc:9.5.0-3-slim

The vote will be open for at least 72 hours i.e. until 2024-02-10 21:00 UTC.

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Here is my +1

Re: [VOTE] Release Solr 9.5.0 RC2

[Jason Gerlowski]

Good catch Shawn - this is definitely worth a re-spin.  To look on the
bright side this really highlights all the changes that have been going in
lately...

"This vote has FAILED"

I'll spin RC3 as soon as I see https://github.com/apache/solr/pull/2250
merged and backported.  (Lmk if you want any help there Houston)

Best,

Jason

On Tue, Feb 6, 2024 at 7:10 PM Jan Høydahl  wrote:

> Actually, credit to Shawn for discovring the bug <
> https://the-asf.slack.com/archives/CEKUCUNE9/p1707259049474289> and to
> Houston for fixing it.
>
> :)
>
> > 7. feb. 2024 kl. 00:50 skrev Jan Høydahl :
> >
> > Thanks for catching this before the release Houston! Your PR and bats
> test looks solid. Let's re-spin..
> >
> > Jan
> >
> >> 7. feb. 2024 kl. 00:15 skrev Houston Putman :
> >>
> >> Sorry everyone, I have to -1 this. Unfortunately certain solr.in.sh
> >> functionality has broken due to a bug introduced while trying to improve
> >> the envVar handling.
> >>
> >> More details provided here: https://github.com/apache/solr/pull/2250 (
> >> SOLR-15960 <https://issues.apache.org/jira/browse/SOLR-15960>)
> >>
> >> - Houston
> >>
> >> On Tue, Feb 6, 2024 at 4:02 PM Kevin Risden  wrote:
> >>
> >>> +1 (binding)
> >>>
> >>> SUCCESS! [0:33:17.981563]
> >>>
> >>> Kevin Risden
> >>>
> >>>
> >>> On Tue, Feb 6, 2024 at 3:35 PM Arrieta, Alejandro <
> >>> aarri...@perrinsoftware.com> wrote:
> >>>
> >>>> Hello Team,
> >>>>
> >>>> SUCCESS! [0:44:46.772429]
> >>>>
> >>>> docker full and slim created fine.
> >>>>
> >>>> +1 non binding
> >>>>
> >>>> Kind Regards,
> >>>> Alejandro Arrieta
> >>>>
> >>>>
> >>>> On Tue, Feb 6, 2024 at 1:37 PM Houston Putman 
> >>> wrote:
> >>>>
> >>>>> +1 (binding)
> >>>>>
> >>>>> SUCCESS! [0:32:51.658820]
> >>>>>
> >>>>> I also built both Docker images and ran the Solr Operator integration
> >>>> tests
> >>>>> with the full image:
> >>>>>
> >>>>> •••
> >>>>>
> >>>>> Ran 23 of 23 Specs in 450.649 seconds
> >>>>> SUCCESS! -- 23 Passed | 0 Failed | 0 Pending | 0 Skipped
> >>>>>
> >>>>> - Houston
> >>>>>
> >>>>> On Tue, Feb 6, 2024 at 11:43 AM Jason Gerlowski <
> gerlowsk...@gmail.com
> >>>>
> >>>>> wrote:
> >>>>>
> >>>>>> Please vote for release candidate 2 for Solr 9.5.0
> >>>>>>
> >>>>>> The artifacts can be downloaded from:
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC2-rev-b03df01e89e64bc897a6be93de00af9ded2ee99b
> >>>>>>
> >>>>>> You can run the smoke tester directly with this command:
> >>>>>>
> >>>>>> python3 -u dev-tools/scripts/smokeTestRelease.py \
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC2-rev-b03df01e89e64bc897a6be93de00af9ded2ee99b
> >>>>>>
> >>>>>> You can build a release-candidate of the official docker images
> >>> (full &
> >>>>>> slim) using the following command:
> >>>>>>
> >>>>>> SOLR_DOWNLOAD_SERVER=
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC2-rev-b03df01e89e64bc897a6be93de00af9ded2ee99b/solr
> >>>>>> &&
> >>>>>> <
> >>>>>
> >>>>
> >>>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC2-rev-b03df01e89e64bc897a6be93de00af9ded2ee99b/solr&&;
> >>>>>>
> >>>>>> \
> >>>>>> docker build
> >>>>> $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-full
> >>>>>> \
> >>>>>>   --build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
> >>>>>>   -t solr-rc:9.5.0-2 && \
> >>>>>> docker build
> >>>>> $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-slim
> >>>>>> \
> >>>>>>   --build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
> >>>>>>   -t solr-rc:9.5.0-2-slim
> >>>>>>
> >>>>>> The vote will be open for at least 72 hours i.e. until 2024-02-09
> >>> 18:00
> >>>>>> UTC.
> >>>>>>
> >>>>>> [ ] +1  approve
> >>>>>> [ ] +0  no opinion
> >>>>>> [ ] -1  disapprove (and reason why)
> >>>>>>
> >>>>>> Here is my +1 (binding)
> >>>>>>
> >>>>>
> >>>>
> >>>
> >
>
>

[VOTE] Release Solr 9.5.0 RC2

[Jason Gerlowski]

Please vote for release candidate 2 for Solr 9.5.0

The artifacts can be downloaded from:
https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC2-rev-b03df01e89e64bc897a6be93de00af9ded2ee99b

You can run the smoke tester directly with this command:

python3 -u dev-tools/scripts/smokeTestRelease.py \
https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC2-rev-b03df01e89e64bc897a6be93de00af9ded2ee99b

You can build a release-candidate of the official docker images (full &
slim) using the following command:

SOLR_DOWNLOAD_SERVER=
https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC2-rev-b03df01e89e64bc897a6be93de00af9ded2ee99b/solr
&& \
  docker build $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-full \
--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
-t solr-rc:9.5.0-2 && \
  docker build $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-slim \
--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
-t solr-rc:9.5.0-2-slim

The vote will be open for at least 72 hours i.e. until 2024-02-09 18:00 UTC.

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Here is my +1 (binding)

Re: [VOTE] Release Solr 9.5.0 RC1

[Jason Gerlowski]

I think this is assumed from the discussion above, but because the
releaseWizard suggests a formal "FAILED" email:

This vote has FAILED.

Reason for fail is: re-spinning the release for SOLR-17149.

On Mon, Feb 5, 2024 at 1:55 PM Jason Gerlowski 
wrote:

> It's probably cleanest to create a new ticket of type="bug", and then link
> back to SOLR-16879 - that way the "Fix Version" field can be unambiguous.
>
> On Mon, Feb 5, 2024 at 1:33 PM Pierre Salagnac 
> wrote:
>
>> Thanks for your input.
>> Unfortunately there is no parameter for that. This is hardcoded at 5.
>>
>> Yes, I'm already working on a fix.
>>
>> Sorry to hijack this thread for the 9.5 release... Is a new Jira required
>> for such an issue?
>> I'm unclear with this, since the regression was introduced in a version
>> that is already released.
>>
>>
>> Le lun. 5 févr. 2024 à 19:03, Jason Gerlowski  a
>> écrit :
>>
>> > Interesting - 9.4 has been out there since October, I'm surprised no one
>> > reported this earlier.  But I guess there's always a lag for teams to
>> > upgrade to new versions...
>> >
>> > Is the number of "expensive" tasks configurable, such that there's a
>> > workaround for collections with many shards?  Assuming not, this does
>> sound
>> > serious enough to "fail" the VOTE as it'd mean that backup/restore is
>> > essentially broken for sufficiently large collections.
>> >
>> > In terms of SOLR-16879 - any chance you're willing to work on a fix
>> Pierre?
>> >
>> > Best,
>> >
>> > Jason
>> >
>> > On Mon, Feb 5, 2024 at 12:52 PM Pierre Salagnac <
>> pierre.salag...@gmail.com
>> > >
>> > wrote:
>> >
>> > > The regression was introduced in 9.4.
>> > >
>> > > Le lun. 5 févr. 2024 à 18:31, Pierre Salagnac <
>> pierre.salag...@gmail.com
>> > >
>> > > a
>> > > écrit :
>> > >
>> > > > Hi Jason,
>> > > >
>> > > > A regression was introduced in backup/restore for large collections.
>> > This
>> > > > was reported in a comment of SOLR-16879[1].
>> > > > Should this be considered as a blocker for 9.5 ?
>> > > >
>> > > > [1]
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/browse/SOLR-16879?focusedCommentId=17813066&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17813066
>> > > >
>> > > > Le lun. 5 févr. 2024 à 15:44, Jason Gerlowski <
>> gerlowsk...@apache.org>
>> > a
>> > > > écrit :
>> > > >
>> > > >> Please vote for release candidate 1 for Solr 9.5.0
>> > > >>
>> > > >> The artifacts can be downloaded from:
>> > > >>
>> > > >>
>> > >
>> >
>> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b
>> > > >>
>> > > >> You can run the smoke tester directly with this command:
>> > > >>
>> > > >> python3 -u dev-tools/scripts/smokeTestRelease.py \
>> > > >>
>> > > >>
>> > >
>> >
>> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b
>> > > >>
>> > > >> You can build a release-candidate of the official docker images
>> (full
>> > &
>> > > >> slim) using the following command:
>> > > >>
>> > > >> SOLR_DOWNLOAD_SERVER=
>> > > >>
>> > > >>
>> > >
>> >
>> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b/solr
>> > > >> &&
>> > > >> <
>> > >
>> >
>> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b/solr&&;
>> > > >
>> > > >> \
>> > > >>   docker build
>> > > >> $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-full \
>> > > >> --build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
>> > > >> -t solr-rc:9.5.0-1 && \
>> > > >>   docker build
>> > > >> $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-slim \
>> > > >> --build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
>> > > >> -t solr-rc:9.5.0-1-slim
>> > > >>
>> > > >> The vote will be open for at least 72 hours i.e. until 2024-02-08
>> > 15:00
>> > > >> UTC.
>> > > >>
>> > > >> [ ] +1  approve
>> > > >> [ ] +0  no opinion
>> > > >> [ ] -1  disapprove (and reason why)
>> > > >>
>> > > >> Here is my +1
>> > > >>
>> > > >
>> > >
>> >
>>
>

Re: [VOTE] Release Solr 9.5.0 RC1

[Jason Gerlowski]

It's probably cleanest to create a new ticket of type="bug", and then link
back to SOLR-16879 - that way the "Fix Version" field can be unambiguous.

On Mon, Feb 5, 2024 at 1:33 PM Pierre Salagnac 
wrote:

> Thanks for your input.
> Unfortunately there is no parameter for that. This is hardcoded at 5.
>
> Yes, I'm already working on a fix.
>
> Sorry to hijack this thread for the 9.5 release... Is a new Jira required
> for such an issue?
> I'm unclear with this, since the regression was introduced in a version
> that is already released.
>
>
> Le lun. 5 févr. 2024 à 19:03, Jason Gerlowski  a
> écrit :
>
> > Interesting - 9.4 has been out there since October, I'm surprised no one
> > reported this earlier.  But I guess there's always a lag for teams to
> > upgrade to new versions...
> >
> > Is the number of "expensive" tasks configurable, such that there's a
> > workaround for collections with many shards?  Assuming not, this does
> sound
> > serious enough to "fail" the VOTE as it'd mean that backup/restore is
> > essentially broken for sufficiently large collections.
> >
> > In terms of SOLR-16879 - any chance you're willing to work on a fix
> Pierre?
> >
> > Best,
> >
> > Jason
> >
> > On Mon, Feb 5, 2024 at 12:52 PM Pierre Salagnac <
> pierre.salag...@gmail.com
> > >
> > wrote:
> >
> > > The regression was introduced in 9.4.
> > >
> > > Le lun. 5 févr. 2024 à 18:31, Pierre Salagnac <
> pierre.salag...@gmail.com
> > >
> > > a
> > > écrit :
> > >
> > > > Hi Jason,
> > > >
> > > > A regression was introduced in backup/restore for large collections.
> > This
> > > > was reported in a comment of SOLR-16879[1].
> > > > Should this be considered as a blocker for 9.5 ?
> > > >
> > > > [1]
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/SOLR-16879?focusedCommentId=17813066&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17813066
> > > >
> > > > Le lun. 5 févr. 2024 à 15:44, Jason Gerlowski <
> gerlowsk...@apache.org>
> > a
> > > > écrit :
> > > >
> > > >> Please vote for release candidate 1 for Solr 9.5.0
> > > >>
> > > >> The artifacts can be downloaded from:
> > > >>
> > > >>
> > >
> >
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b
> > > >>
> > > >> You can run the smoke tester directly with this command:
> > > >>
> > > >> python3 -u dev-tools/scripts/smokeTestRelease.py \
> > > >>
> > > >>
> > >
> >
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b
> > > >>
> > > >> You can build a release-candidate of the official docker images
> (full
> > &
> > > >> slim) using the following command:
> > > >>
> > > >> SOLR_DOWNLOAD_SERVER=
> > > >>
> > > >>
> > >
> >
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b/solr
> > > >> &&
> > > >> <
> > >
> >
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b/solr&&;
> > > >
> > > >> \
> > > >>   docker build
> > > >> $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-full \
> > > >> --build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
> > > >> -t solr-rc:9.5.0-1 && \
> > > >>   docker build
> > > >> $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-slim \
> > > >> --build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
> > > >> -t solr-rc:9.5.0-1-slim
> > > >>
> > > >> The vote will be open for at least 72 hours i.e. until 2024-02-08
> > 15:00
> > > >> UTC.
> > > >>
> > > >> [ ] +1  approve
> > > >> [ ] +0  no opinion
> > > >> [ ] -1  disapprove (and reason why)
> > > >>
> > > >> Here is my +1
> > > >>
> > > >
> > >
> >
>

Re: [VOTE] Release Solr 9.5.0 RC1

[Jason Gerlowski]

Interesting - 9.4 has been out there since October, I'm surprised no one
reported this earlier.  But I guess there's always a lag for teams to
upgrade to new versions...

Is the number of "expensive" tasks configurable, such that there's a
workaround for collections with many shards?  Assuming not, this does sound
serious enough to "fail" the VOTE as it'd mean that backup/restore is
essentially broken for sufficiently large collections.

In terms of SOLR-16879 - any chance you're willing to work on a fix Pierre?

Best,

Jason

On Mon, Feb 5, 2024 at 12:52 PM Pierre Salagnac 
wrote:

> The regression was introduced in 9.4.
>
> Le lun. 5 févr. 2024 à 18:31, Pierre Salagnac 
> a
> écrit :
>
> > Hi Jason,
> >
> > A regression was introduced in backup/restore for large collections. This
> > was reported in a comment of SOLR-16879[1].
> > Should this be considered as a blocker for 9.5 ?
> >
> > [1]
> >
> https://issues.apache.org/jira/browse/SOLR-16879?focusedCommentId=17813066&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17813066
> >
> > Le lun. 5 févr. 2024 à 15:44, Jason Gerlowski  a
> > écrit :
> >
> >> Please vote for release candidate 1 for Solr 9.5.0
> >>
> >> The artifacts can be downloaded from:
> >>
> >>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b
> >>
> >> You can run the smoke tester directly with this command:
> >>
> >> python3 -u dev-tools/scripts/smokeTestRelease.py \
> >>
> >>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b
> >>
> >> You can build a release-candidate of the official docker images (full &
> >> slim) using the following command:
> >>
> >> SOLR_DOWNLOAD_SERVER=
> >>
> >>
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b/solr
> >> &&
> >> <
> https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b/solr&&;
> >
> >> \
> >>   docker build
> >> $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-full \
> >> --build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
> >> -t solr-rc:9.5.0-1 && \
> >>   docker build
> >> $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-slim \
> >> --build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
> >> -t solr-rc:9.5.0-1-slim
> >>
> >> The vote will be open for at least 72 hours i.e. until 2024-02-08 15:00
> >> UTC.
> >>
> >> [ ] +1  approve
> >> [ ] +0  no opinion
> >> [ ] -1  disapprove (and reason why)
> >>
> >> Here is my +1
> >>
> >
>

[VOTE] Release Solr 9.5.0 RC1

[Jason Gerlowski]

Please vote for release candidate 1 for Solr 9.5.0

The artifacts can be downloaded from:
https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b

You can run the smoke tester directly with this command:

python3 -u dev-tools/scripts/smokeTestRelease.py \
https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b

You can build a release-candidate of the official docker images (full &
slim) using the following command:

SOLR_DOWNLOAD_SERVER=
https://dist.apache.org/repos/dist/dev/solr/solr-9.5.0-RC1-rev-1fb7d127fc064b0bab8435a431d71a44050e654b/solr
&& \
  docker build $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-full \
--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
-t solr-rc:9.5.0-1 && \
  docker build $SOLR_DOWNLOAD_SERVER/9.5.0/docker/Dockerfile.official-slim \
--build-arg SOLR_DOWNLOAD_SERVER=$SOLR_DOWNLOAD_SERVER \
-t solr-rc:9.5.0-1-slim

The vote will be open for at least 72 hours i.e. until 2024-02-08 15:00 UTC.

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Here is my +1

Re: New branch and feature freeze for Solr 9.5.0

[Jason Gerlowski]

Alright looks like the last blocker has been merged - I'll start on the RC
this morning!

On Thu, Feb 1, 2024 at 8:57 AM Jason Gerlowski 
wrote:

> Alright - everything looks pretty good on the Lucene 9.9.2 upgrade front.
>
> We still have one "blocker" for cutting 9.5 RC1 - SOLR-17068.  It looks
> like there's an approved PR for that ticket (
> https://github.com/apache/solr/pull/2227) but I know there's been some
> discussion of the overall approach there.  (See the "CLI: bin/solr bin/post
> bin/postlogs -- what's happening" thread).  Are we still planning to target
> 9.5 for this ticket?  If so, is there a plan to get it merged?
>
> As soon as SOLR-17068 is wrapped up I'll start on 9.5 RC1.
>
> Best,
>
> Jason
>
> On Tue, Jan 30, 2024 at 1:39 PM Christine Poerschke (BLOOMBERG/ LONDON) <
> cpoersc...@bloomberg.net> wrote:
>
>> Thanks! https://issues.apache.org/jira/browse/SOLR-17142 created for it,
>> not a blocker for 9.5 I think.
>>
>> From: dev@solr.apache.org At: 01/30/24 18:28:23 UTCTo:
>> dev@solr.apache.org
>> Subject: Re: New branch and feature freeze for Solr 9.5.0
>>
>> Christine - David brought this up a few months ago -
>> https://lists.apache.org/thread/hlh1bmtgnmp55q8knhjtltf8t57pbl5q
>>
>> Kevin Risden
>>
>>
>> On Tue, Jan 30, 2024 at 1:07 PM Christine Poerschke (BLOOMBERG/ LONDON) <
>> cpoersc...@bloomberg.net> wrote:
>>
>> > Unrelated to baking times, I stumbled across a "unreferenced files under
>> > license folder" mystery:
>> > https://github.com/apache/solr/pull/2178#issuecomment-1917513343
>> >
>> > Could someone try to reproduce the "running clean once and then
>> precommit
>> > twice" sequence?
>> >
>> > Thanks,
>> > Christine
>> >
>> > From: dev@solr.apache.org At: 01/30/24 15:15:30 UTCTo:
>> > dev@solr.apache.org
>> > Subject: Re: New branch and feature freeze for Solr 9.5.0
>> >
>> > Maybe?  Obviously that was my initial plan as I mentioned above.  But I
>> was
>> > pleasantly surprised to see the number of folks that chimed in with
>> their
>> > own test results and +1's on Nazerke's Lucene-upgrade PR.  It seemed
>> > comparable to the data points we'd get out of a multi-day "bake" in
>> > Jenkins.  So I figured we might expedite the baking, or at least let it
>> > "bake" in parallel while I work on the RC.
>> >
>> > But I can also wait an additional few days if folks think it's
>> > necessary/valuable?
>> >
>> > (Also worth mentioning that this is still theoretical at this point -
>> we're
>> > still waiting on a fix for SOLR-17068 unless Eric changes his mind about
>> > getting it in 9.5.)
>> >
>> > On Tue, Jan 30, 2024 at 9:32 AM Ishan Chattopadhyaya <
>> > ichattopadhy...@gmail.com> wrote:
>> >
>> > > Should we wait at least 2-3 days for the Lucene 9.9.2 upgrade to bake
>> > > before we cut a release?
>> > >
>> > > On Tue, 30 Jan, 2024, 7:50 pm Jason Gerlowski, > >
>> > > wrote:
>> > >
>> > > > Yes - I saw that right after my last email.  Thanks for all your
>> work
>> > on
>> > > > that Nazerke!  I've merged and backported that PR; we're now on
>> Lucene
>> > > > 9.9.2!
>> > > >
>> > > > As soon as Eric's P #2227 is merged and backported I'll start on our
>> > RC!
>> > > >
>> > > > Best,
>> > > >
>> > > > Jason
>> > > >
>> > > > On Tue, Jan 30, 2024 at 7:33 AM Nazerke S 
>> > wrote:
>> > > >
>> > > > > Hi Jason,
>> > > > >
>> > > > > fyi, we have an open PR <https://github.com/apache/solr/pull/2176
>> > >for
>> > > > the
>> > > > > Lucene upgrade (v9.9.2) waiting for review.
>> > > > >
>> > > > > On Tue, Jan 30, 2024 at 6:21 PM Jason Gerlowski <
>> > gerlowsk...@gmail.com
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > > > I'm working on the Lucene upgrade now.  That will still need
>> some
>> > > time
>> > > > in
>> > > > > > review (and maybe a day to let tests bake?

Re: New branch and feature freeze for Solr 9.5.0

[Jason Gerlowski]

Alright - everything looks pretty good on the Lucene 9.9.2 upgrade front.

We still have one "blocker" for cutting 9.5 RC1 - SOLR-17068.  It looks
like there's an approved PR for that ticket (
https://github.com/apache/solr/pull/2227) but I know there's been some
discussion of the overall approach there.  (See the "CLI: bin/solr bin/post
bin/postlogs -- what's happening" thread).  Are we still planning to target
9.5 for this ticket?  If so, is there a plan to get it merged?

As soon as SOLR-17068 is wrapped up I'll start on 9.5 RC1.

Best,

Jason

On Tue, Jan 30, 2024 at 1:39 PM Christine Poerschke (BLOOMBERG/ LONDON) <
cpoersc...@bloomberg.net> wrote:

> Thanks! https://issues.apache.org/jira/browse/SOLR-17142 created for it,
> not a blocker for 9.5 I think.
>
> From: dev@solr.apache.org At: 01/30/24 18:28:23 UTCTo:
> dev@solr.apache.org
> Subject: Re: New branch and feature freeze for Solr 9.5.0
>
> Christine - David brought this up a few months ago -
> https://lists.apache.org/thread/hlh1bmtgnmp55q8knhjtltf8t57pbl5q
>
> Kevin Risden
>
>
> On Tue, Jan 30, 2024 at 1:07 PM Christine Poerschke (BLOOMBERG/ LONDON) <
> cpoersc...@bloomberg.net> wrote:
>
> > Unrelated to baking times, I stumbled across a "unreferenced files under
> > license folder" mystery:
> > https://github.com/apache/solr/pull/2178#issuecomment-1917513343
> >
> > Could someone try to reproduce the "running clean once and then precommit
> > twice" sequence?
> >
> > Thanks,
> > Christine
> >
> > From: dev@solr.apache.org At: 01/30/24 15:15:30 UTCTo:
> > dev@solr.apache.org
> > Subject: Re: New branch and feature freeze for Solr 9.5.0
> >
> > Maybe?  Obviously that was my initial plan as I mentioned above.  But I
> was
> > pleasantly surprised to see the number of folks that chimed in with their
> > own test results and +1's on Nazerke's Lucene-upgrade PR.  It seemed
> > comparable to the data points we'd get out of a multi-day "bake" in
> > Jenkins.  So I figured we might expedite the baking, or at least let it
> > "bake" in parallel while I work on the RC.
> >
> > But I can also wait an additional few days if folks think it's
> > necessary/valuable?
> >
> > (Also worth mentioning that this is still theoretical at this point -
> we're
> > still waiting on a fix for SOLR-17068 unless Eric changes his mind about
> > getting it in 9.5.)
> >
> > On Tue, Jan 30, 2024 at 9:32 AM Ishan Chattopadhyaya <
> > ichattopadhy...@gmail.com> wrote:
> >
> > > Should we wait at least 2-3 days for the Lucene 9.9.2 upgrade to bake
> > > before we cut a release?
> > >
> > > On Tue, 30 Jan, 2024, 7:50 pm Jason Gerlowski, 
> > > wrote:
> > >
> > > > Yes - I saw that right after my last email.  Thanks for all your work
> > on
> > > > that Nazerke!  I've merged and backported that PR; we're now on
> Lucene
> > > > 9.9.2!
> > > >
> > > > As soon as Eric's P #2227 is merged and backported I'll start on our
> > RC!
> > > >
> > > > Best,
> > > >
> > > > Jason
> > > >
> > > > On Tue, Jan 30, 2024 at 7:33 AM Nazerke S 
> > wrote:
> > > >
> > > > > Hi Jason,
> > > > >
> > > > > fyi, we have an open PR <https://github.com/apache/solr/pull/2176
> > >for
> > > > the
> > > > > Lucene upgrade (v9.9.2) waiting for review.
> > > > >
> > > > > On Tue, Jan 30, 2024 at 6:21 PM Jason Gerlowski <
> > gerlowsk...@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > I'm working on the Lucene upgrade now.  That will still need some
> > > time
> > > > in
> > > > > > review (and maybe a day to let tests bake?), so I'm fine if you
> > > > backport
> > > > > > SOLR-17068 in that timeframe.  But let's draw the line once the
> > > Lucene
> > > > > > 9.9.2 upgrade is in, unless something else urgent comes up.  I'll
> > > > target
> > > > > > Wednesday or Thursday for the Solr RC1.
> > > > > >
> > > > > > Jason
> > > > > >
> > > > > > On Mon, Jan 29, 2024 at 4:15 PM Eric Pugh <
> > > > > ep...@opensourceconnections.com
> > > > > > >
> > > > > > wro

Re: CLI: bin/solr bin/post bin/postlogs -- what's happening

[Jason Gerlowski]

For "post" specifically, I think option (A) is better from both a developer
and a user-interface perspective.

On the development side, "bin/solr" integration gets us a lot: it's easy to
hook new tools/subcommands into bin/solr, the logic can live in Java,
support for various features (e.g. basic-auth) comes "for free".

On the user-interface side: "bin/solr" is the only option that's available
to Windows users.  That's a huge benefit.  Even if we organized the code as
described in option (B) above, I think we'd cause a lot of confusion
requiring Linux and Windows users to execute different scripts (i.e.
"bin/post" on Linux/Mac and "bin/solr post" on Windows).

Also from a user-interface perspective - I personally don't mind having
many sub-commands accessible through one main entrypoint.  It seems
relatively common and well received nowadays - take "kubectl" and
"aws-shell", by way of example.

Best,

Jason

On Tue, Jan 30, 2024 at 5:40 PM Eric Pugh 
wrote:

> Thanks David for weighing in.   A big part of pushing the bin/post and
> bin/postlogs into bin/solr as subcommands is that then the sub command
> lives in Java land, and we just “magically” picked up Windows support.
>
> It feels like right now we are in the worst of all worlds..  We have a mix
> of bin/{somescript} that is each implemented slightly different, and then a
> lot of nested commands under bin/solr, arguably some which should be their
> own commands?   Do we then move to bin/create, bin/delete, bin/zk,
> bin/export, bin/api ?
>
> I’m also thinking of a future where we have more CLI support, for example
> a CLI for running a streaming expression.   Or a CLI for all the various
> commands that the V2 API’s are exposing, and that would be nice to have be
> scripted via a CLI.   So bin/split-shard?
>
> It may be that we have a pattern in the future like “bin/server” and
> “bin/client” or maybe you have “bin/solr api split-shard” and “bin/solr
> server start”, and then commands get grouped like that?
>
> I sometimes dream of having a separate “Solr-cli” sub project that has CLI
> that uses tooling like https://oclif.io/ to build something really
> amazing ;-).
>
> I know someone is going to say “can’t we just use Curl”, and that may work
> for some folks, but doing a lot of interactions with Solr require multiple
> steps, and that’s where the CLI shines.   Add in supporting Basic auth and
> someday oAuth, and Curl starts to break down.
>
>
> > On Jan 30, 2024, at 3:10 PM, David Smiley  wrote:
> >
> > I'm writing this to get input on where we're going in the CLI domain
> > with respect to organizational choices of our commands.  Looking on
> > 9x, I see bin/solr, bin/post, bin/postlogs scripts.  Recently, most
> > internal command plumbing has been centralized under bin/solr and thus
> > you can do "bin/solr post" or "bin/solr postlogs" instead of
> > "bin/post" and "bin/postlogs" respectively.  Disclaimer:  So I hear; I
> > haven't tried them.
> >
> > At this point, I think we have a choice:
> > (A) Remove bin/post and bin/postlogs in 10.  There's no question the
> > code duplication should be eliminated but the question is really about
> > the DX (developer user experience).  Do we want one shell command,
> > "bin/solr" to be all things Solr, not just Solr itself (starting
> > Solr), but posting data to Solr (basically a Curl alternative)?  It's
> > already a kitchen sink of some miscellaneous things, granted.  This
> > annoys my organizational sensibilities.
> > (B) Keep the scripts, but implement them as one-liners calling into
> > "bin/solr post" or similar, and possibly not document on the bin/solr
> > side that these commands are there as it's just for implementation
> > convenience.  After all, this was the actual motivation of the changes
> > that have happened lately -- it's improving code maintenance/support.
> > Good stuff; but do we need to change the DX on users too?  Is this
> > better for them?
> >
> > Separately, since there are so many commands in bin/solr that don't
> > relate to starting Solr, maybe bin/solr-start should exist?  (again,
> > could be a one-liner call into one CLI backend for our convenience).
> >
> > This decision is a matter of taste; it's not really technical.
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
>
> ___
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> This e-mail and all contents, incl

Re: BackupRepository changes

[Jason Gerlowski]

> Isn't the intent to ensure we don't waste time/space creating a
> useless backup of something that is, I suppose, already corrupted?

Space is one benefit, yep.

The other reason is to avoid giving users a false sense of security.  A
user would be very frustrated to find out at restore-time that the failsafe
backup they've been relying on is useless.  Better to surface that
information at backup time when the source collection is healthy and the
backup can be retried, etc.


On Tue, Jan 30, 2024 at 10:18 AM Bruno Roustant 
wrote:

> > Isn't the intent to ensure we don't waste time/space creating a
> useless backup of something that is, I suppose, already corrupted?
>
> That's right. And I didn't read the code enough; a clear effort has been
> put here since the last time I read the code, to make all implementations
> consistent to verify the checksum.
>
> Hum, this is annoying for directory-based encryption. The only way becomes
> to have an encryption extension for each and all different implementations.
> Less clean than a FilterBackupRepository.
>

Re: New branch and feature freeze for Solr 9.5.0

[Jason Gerlowski]

Maybe?  Obviously that was my initial plan as I mentioned above.  But I was
pleasantly surprised to see the number of folks that chimed in with their
own test results and +1's on Nazerke's Lucene-upgrade PR.  It seemed
comparable to the data points we'd get out of a multi-day "bake" in
Jenkins.  So I figured we might expedite the baking, or at least let it
"bake" in parallel while I work on the RC.

But I can also wait an additional few days if folks think it's
necessary/valuable?

(Also worth mentioning that this is still theoretical at this point - we're
still waiting on a fix for SOLR-17068 unless Eric changes his mind about
getting it in 9.5.)

On Tue, Jan 30, 2024 at 9:32 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Should we wait at least 2-3 days for the Lucene 9.9.2 upgrade to bake
> before we cut a release?
>
> On Tue, 30 Jan, 2024, 7:50 pm Jason Gerlowski, 
> wrote:
>
> > Yes - I saw that right after my last email.  Thanks for all your work on
> > that Nazerke!  I've merged and backported that PR; we're now on Lucene
> > 9.9.2!
> >
> > As soon as Eric's P #2227 is merged and backported I'll start on our RC!
> >
> > Best,
> >
> > Jason
> >
> > On Tue, Jan 30, 2024 at 7:33 AM Nazerke S  wrote:
> >
> > > Hi Jason,
> > >
> > > fyi, we have an open PR <https://github.com/apache/solr/pull/2176>for
> > the
> > > Lucene upgrade (v9.9.2) waiting for review.
> > >
> > > On Tue, Jan 30, 2024 at 6:21 PM Jason Gerlowski  >
> > > wrote:
> > >
> > > > I'm working on the Lucene upgrade now.  That will still need some
> time
> > in
> > > > review (and maybe a day to let tests bake?), so I'm fine if you
> > backport
> > > > SOLR-17068 in that timeframe.  But let's draw the line once the
> Lucene
> > > > 9.9.2 upgrade is in, unless something else urgent comes up.  I'll
> > target
> > > > Wednesday or Thursday for the Solr RC1.
> > > >
> > > > Jason
> > > >
> > > > On Mon, Jan 29, 2024 at 4:15 PM Eric Pugh <
> > > ep...@opensourceconnections.com
> > > > >
> > > > wrote:
> > > >
> > > > > Jason, I tackled SOLR-17068 (the one you reminded me of) and I’d
> love
> > > to
> > > > > get it into 9.5 since right now we have a terrible mish mash of
> > > bin/solr
> > > > > post and bin/post in the ref guide and docs.
> > > > >
> > > > > Could someone review https://github.com/apache/solr/pull/2227 and
> if
> > > it
> > > > > looks good could we sneak it into 9.5?
> > > > >
> > > > > Eric
> > > > >
> > > > >
> > > > > > On Jan 26, 2024, at 6:29 PM, Eric Pugh <
> > > > ep...@opensourceconnections.com>
> > > > > wrote:
> > > > > >
> > > > > > Backport to branch_9_5 is done.
> > > > > >
> > > > > >
> > > > > >> On Jan 26, 2024, at 1:11 PM, Jason Gerlowski <
> > gerlowsk...@gmail.com
> > > >
> > > > > wrote:
> > > > > >>
> > > > > >> Go ahead and backport on your own!  I'm still waiting on Lucene
> > > 9.9.2,
> > > > > so
> > > > > >> there shouldn't be any branch-contention on my end.
> > > > > >>
> > > > > >> Relatedly, Lucene has their RC1 out there and things look good a
> > day
> > > > or
> > > > > two
> > > > > >> into their VOTE, so with any luck we'll be able to get a Solr
> 9.5
> > RC
> > > > > >> together early next week!
> > > > > >>
> > > > > >> Best,
> > > > > >>
> > > > > >> Jason
> > > > > >>
> > > > > >> On Fri, Jan 26, 2024 at 8:49 AM Eric Pugh 
> > wrote:
> > > > > >>
> > > > > >>> I am about to merge SOLR-17112, and will backport it to
> > branch_9x.
> > > > > Jason,
> > > > > >>> do you backport it over to the branch_9_5 or do I?
> > > > > >>>
> > > > > >>> ERic
> > > > > >>>
> > > > > >>>
> > > > > >>> On 2024/01/23 19:08:10 Jas

Re: New branch and feature freeze for Solr 9.5.0

[Jason Gerlowski]

Yes - I saw that right after my last email.  Thanks for all your work on
that Nazerke!  I've merged and backported that PR; we're now on Lucene
9.9.2!

As soon as Eric's P #2227 is merged and backported I'll start on our RC!

Best,

Jason

On Tue, Jan 30, 2024 at 7:33 AM Nazerke S  wrote:

> Hi Jason,
>
> fyi, we have an open PR <https://github.com/apache/solr/pull/2176>for  the
> Lucene upgrade (v9.9.2) waiting for review.
>
> On Tue, Jan 30, 2024 at 6:21 PM Jason Gerlowski 
> wrote:
>
> > I'm working on the Lucene upgrade now.  That will still need some time in
> > review (and maybe a day to let tests bake?), so I'm fine if you backport
> > SOLR-17068 in that timeframe.  But let's draw the line once the Lucene
> > 9.9.2 upgrade is in, unless something else urgent comes up.  I'll target
> > Wednesday or Thursday for the Solr RC1.
> >
> > Jason
> >
> > On Mon, Jan 29, 2024 at 4:15 PM Eric Pugh <
> ep...@opensourceconnections.com
> > >
> > wrote:
> >
> > > Jason, I tackled SOLR-17068 (the one you reminded me of) and I’d love
> to
> > > get it into 9.5 since right now we have a terrible mish mash of
> bin/solr
> > > post and bin/post in the ref guide and docs.
> > >
> > > Could someone review https://github.com/apache/solr/pull/2227 and if
> it
> > > looks good could we sneak it into 9.5?
> > >
> > > Eric
> > >
> > >
> > > > On Jan 26, 2024, at 6:29 PM, Eric Pugh <
> > ep...@opensourceconnections.com>
> > > wrote:
> > > >
> > > > Backport to branch_9_5 is done.
> > > >
> > > >
> > > >> On Jan 26, 2024, at 1:11 PM, Jason Gerlowski  >
> > > wrote:
> > > >>
> > > >> Go ahead and backport on your own!  I'm still waiting on Lucene
> 9.9.2,
> > > so
> > > >> there shouldn't be any branch-contention on my end.
> > > >>
> > > >> Relatedly, Lucene has their RC1 out there and things look good a day
> > or
> > > two
> > > >> into their VOTE, so with any luck we'll be able to get a Solr 9.5 RC
> > > >> together early next week!
> > > >>
> > > >> Best,
> > > >>
> > > >> Jason
> > > >>
> > > >> On Fri, Jan 26, 2024 at 8:49 AM Eric Pugh  wrote:
> > > >>
> > > >>> I am about to merge SOLR-17112, and will backport it to branch_9x.
> > > Jason,
> > > >>> do you backport it over to the branch_9_5 or do I?
> > > >>>
> > > >>> ERic
> > > >>>
> > > >>>
> > > >>> On 2024/01/23 19:08:10 Jason Gerlowski wrote:
> > > >>>>> It was hoped SOLR-17112 would make 9.4.1 but it didn't as no PR
> was
> > > >>>> proposed.
> > > >>>>
> > > >>>>> SOLR-17120 [is] nominated for inclusion in the 9.5.0 release
> > > >>>>
> > > >>>> Those both sound quick and reasonable; they've got a +1 from me to
> > go
> > > >>> into
> > > >>>> 9.5 (assuming the contributor decides to continue with
> SOLR-17112).
> > > >>>>
> > > >>>>> Considering Lucene 9.9.2 is being planned, I think it would be
> > better
> > > >>> to
> > > >>>>> upgrade Solr to the to-be-released version so users have to deal
> > with
> > > >>>>> fewer upgrade cycles.
> > > >>>>
> > > >>>> Yeah, that might be best; I hadn't realized we weren't already on
> > the
> > > >>>> latest Lucene 9.x.  I've created SOLR-17128 to track our Lucene
> > > upgrade
> > > >>>> once 9.9.2 is available.
> > > >>>>
> > > >>>> Obviously this is a longer delay than some of the tickets above,
> and
> > > will
> > > >>>> mean we won't be cutting a Solr RC this week.  We can pick a new
> > date
> > > for
> > > >>>> the initial Solr 9.5 RC once Lucene 9.9.2 is available.
> > > >>>>
> > > >>>> Best,
> > > >>>>
> > > >>>> Jason
> > > >>>>
> > > >>>> On Tue, Jan 23, 2024 at 1:37 PM Anshum Gupta <
> > ans...@anshumgupta.n

Re: New branch and feature freeze for Solr 9.5.0

[Jason Gerlowski]

I'm working on the Lucene upgrade now.  That will still need some time in
review (and maybe a day to let tests bake?), so I'm fine if you backport
SOLR-17068 in that timeframe.  But let's draw the line once the Lucene
9.9.2 upgrade is in, unless something else urgent comes up.  I'll target
Wednesday or Thursday for the Solr RC1.

Jason

On Mon, Jan 29, 2024 at 4:15 PM Eric Pugh 
wrote:

> Jason, I tackled SOLR-17068 (the one you reminded me of) and I’d love to
> get it into 9.5 since right now we have a terrible mish mash of bin/solr
> post and bin/post in the ref guide and docs.
>
> Could someone review https://github.com/apache/solr/pull/2227 and if it
> looks good could we sneak it into 9.5?
>
> Eric
>
>
> > On Jan 26, 2024, at 6:29 PM, Eric Pugh 
> wrote:
> >
> > Backport to branch_9_5 is done.
> >
> >
> >> On Jan 26, 2024, at 1:11 PM, Jason Gerlowski 
> wrote:
> >>
> >> Go ahead and backport on your own!  I'm still waiting on Lucene 9.9.2,
> so
> >> there shouldn't be any branch-contention on my end.
> >>
> >> Relatedly, Lucene has their RC1 out there and things look good a day or
> two
> >> into their VOTE, so with any luck we'll be able to get a Solr 9.5 RC
> >> together early next week!
> >>
> >> Best,
> >>
> >> Jason
> >>
> >> On Fri, Jan 26, 2024 at 8:49 AM Eric Pugh  wrote:
> >>
> >>> I am about to merge SOLR-17112, and will backport it to branch_9x.
> Jason,
> >>> do you backport it over to the branch_9_5 or do I?
> >>>
> >>> ERic
> >>>
> >>>
> >>> On 2024/01/23 19:08:10 Jason Gerlowski wrote:
> >>>>> It was hoped SOLR-17112 would make 9.4.1 but it didn't as no PR was
> >>>> proposed.
> >>>>
> >>>>> SOLR-17120 [is] nominated for inclusion in the 9.5.0 release
> >>>>
> >>>> Those both sound quick and reasonable; they've got a +1 from me to go
> >>> into
> >>>> 9.5 (assuming the contributor decides to continue with SOLR-17112).
> >>>>
> >>>>> Considering Lucene 9.9.2 is being planned, I think it would be better
> >>> to
> >>>>> upgrade Solr to the to-be-released version so users have to deal with
> >>>>> fewer upgrade cycles.
> >>>>
> >>>> Yeah, that might be best; I hadn't realized we weren't already on the
> >>>> latest Lucene 9.x.  I've created SOLR-17128 to track our Lucene
> upgrade
> >>>> once 9.9.2 is available.
> >>>>
> >>>> Obviously this is a longer delay than some of the tickets above, and
> will
> >>>> mean we won't be cutting a Solr RC this week.  We can pick a new date
> for
> >>>> the initial Solr 9.5 RC once Lucene 9.9.2 is available.
> >>>>
> >>>> Best,
> >>>>
> >>>> Jason
> >>>>
> >>>> On Tue, Jan 23, 2024 at 1:37 PM Anshum Gupta 
> >>> wrote:
> >>>>
> >>>>> Considering Lucene 9.9.2 is being planned, I think it would be better
> >>> to
> >>>>> upgrade Solr to the to-be-released version so users have to deal with
> >>> fewer
> >>>>> upgrade cycles.
> >>>>>
> >>>>> To highlight, there are about 90 odd changes in the Lucene 9.9.x
> line.
> >>>>>
> >>>>> -Anshum
> >>>>>
> >>>>> On Tue, Jan 23, 2024 at 8:47 AM David Smiley 
> >>> wrote:
> >>>>>
> >>>>>> FYI It was hoped SOLR-17112
> >>>>>> https://issues.apache.org/jira/browse/SOLR-17112 "bin/solr script
> >>>>>> doesn't do ps properly on some systems" would make 9.4.1 but it
> >>> didn't
> >>>>>> as no PR was proposed.  There still isn't one but a contributor is
> >>>>>> thinking about it.
> >>>>>>
> >>>>>> On Tue, Jan 23, 2024 at 11:30 AM Christine Poerschke (BLOOMBERG/
> >>>>>> LONDON)  wrote:
> >>>>>>>
> >>>>>>> Just to cross-reference things further (Jason is already aware) --
> >>>>>> https://issues.apache.org/jira/browse/SOLR-17120 and
> >>>>>> https://github.com/apache/solr/pull/2214 are nominated for
> >>> i

Re: Collections LIST semantics

[Jason Gerlowski]

I agree with every point about the delays inherent in a distributed system,
and how any "list" call should be treated by clients as point-in-time.  And
I agree that the impact _should_ be minimal since diligent clients should
have error handling in these cases anyways.

But it still feels off to me to have a "list" op output something that's
potentially incorrect even in the point-in-time it's produced.

Not a -1 or a veto, just my 2c.  If it's an outlier opinion, please ignore
it.

Best,

Jason

On Mon, Jan 29, 2024 at 2:23 PM David Smiley  wrote:

> Yeah, I'm sympathetic to that viewpoint.  I was coming at this from
> Walter's -- clients must be tolerant always.  This mindset is
> important when working on scalable distributed systems.  But depending
> on clients being so tolerant leads to being less friendly --
> increasing the likelihood that they will have to deal with such
> errors.  Solr might even appear buggy to such a client/user.  Shrug.
>
> At work we've got this modification to add listAll to collection
> listing (thus can toggle the semantics) but for scalability reasons,
> we're finding we want this enabled everywhere, which begs the question
> if it should simply work this way to begin with.  I'm also motivated
> to contribute to Solr without adding complexity -- arguably listing
> collections shouldn't need any parameters.  But we could contribute it
> this way; okay?  And maybe make listAll's default be a system property
> so you can run Solr in this way.
>
> On Mon, Jan 29, 2024 at 1:42 PM Jason Gerlowski 
> wrote:
> >
> > Thanks for calling this out more explicitly; definitelyf worth
> discussing.
> >
> > > If a client/caller/user lists collections and then loops them to take
> > some action on them, it needs to be tolerant of the collection not
> working;
> > may seem to not exist.
> >
> > I'd go even a step further and say that users should always have
> > error-handling around their calls to Solr.
> >
> > But even so I'm leery of changing the semantics here.  I think the
> > assumption of most folks is that each entry returned by a "list" exists
> > fully, unless the response gives more granular info to augment that.  I'd
> > worry that returning partially-created or partially-deleted collections
> > would be confusing and unintuitive to most users.  (e.g. Imagine
> iterating
> > over a "list", getting a not-found error running some operation on one of
> > the entries, but still seeing the collection when you call "list" again
> to
> > double-check.)
> >
> > I understand the need for a more scalable API, or a way to detect
> orphaned
> > data in ZK.  But I'd personally rather not see us change the LIST
> semantics
> > to accomplish that.  If you need the ZK child nodes, is there maybe a
> > scalable way to invoke ZookeeperInfoHandler to get that information?
> >
> > Best,
> >
> > Jason
> >
> > On Fri, Jan 26, 2024 at 2:46 PM David Smiley  wrote:
> >
> > > https://issues.apache.org/jira/browse/SOLR-16909
> > > > Collections LIST command should fetch ZK data, not cached state
> > >
> > > I want to get further input from folks that changing the semantics is
> > > okay.  If the change is applied, LIST will be much faster but it will
> > > return collections that have not yet been fully constructed or
> > > deleted.  If a client/caller/user lists collections and then loops
> > > them to take some action on them, it needs to be tolerant of the
> > > collection not working; may seem to not exist.  I argue callers should
> > > *already* behave in this way or it may be brittle to circumstances
> > > that are hard to reason about.  On the other hand, maybe this would
> > > increase the frequency of errors to existing clients that didn't
> > > encounter this in testing?  Shrug.  I could imagine ways to solve this
> > > but it would add some complexity and it's not clear it's worthwhile.
> > >
> > > A related aside: the method ClusterStatus.getCollectionsMap is not
> > > scalable for clusters with 10K+ collections because it loops every
> > > collection to fetch the latest stake from ZK, putting a massive load
> > > on ZK.  Our implementation of collection listing calls it, as does a
> > > number of places across Solr.  Some could be changed with relative
> > > ease; some are more thorny.  I'd love to rename this thing, putting
> > > "slow" in the name so that you think twice before calling it :-)
> > >
> > > ~ David Smiley
> > > Apache Lucene/Solr Search Developer
> > > http://www.linkedin.com/in/davidwsmiley
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > > For additional commands, e-mail: dev-h...@solr.apache.org
> > >
> > >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: Collections LIST semantics

[Jason Gerlowski]

Thanks for calling this out more explicitly; definitelyf worth discussing.

> If a client/caller/user lists collections and then loops them to take
some action on them, it needs to be tolerant of the collection not working;
may seem to not exist.

I'd go even a step further and say that users should always have
error-handling around their calls to Solr.

But even so I'm leery of changing the semantics here.  I think the
assumption of most folks is that each entry returned by a "list" exists
fully, unless the response gives more granular info to augment that.  I'd
worry that returning partially-created or partially-deleted collections
would be confusing and unintuitive to most users.  (e.g. Imagine iterating
over a "list", getting a not-found error running some operation on one of
the entries, but still seeing the collection when you call "list" again to
double-check.)

I understand the need for a more scalable API, or a way to detect orphaned
data in ZK.  But I'd personally rather not see us change the LIST semantics
to accomplish that.  If you need the ZK child nodes, is there maybe a
scalable way to invoke ZookeeperInfoHandler to get that information?

Best,

Jason

On Fri, Jan 26, 2024 at 2:46 PM David Smiley  wrote:

> https://issues.apache.org/jira/browse/SOLR-16909
> > Collections LIST command should fetch ZK data, not cached state
>
> I want to get further input from folks that changing the semantics is
> okay.  If the change is applied, LIST will be much faster but it will
> return collections that have not yet been fully constructed or
> deleted.  If a client/caller/user lists collections and then loops
> them to take some action on them, it needs to be tolerant of the
> collection not working; may seem to not exist.  I argue callers should
> *already* behave in this way or it may be brittle to circumstances
> that are hard to reason about.  On the other hand, maybe this would
> increase the frequency of errors to existing clients that didn't
> encounter this in testing?  Shrug.  I could imagine ways to solve this
> but it would add some complexity and it's not clear it's worthwhile.
>
> A related aside: the method ClusterStatus.getCollectionsMap is not
> scalable for clusters with 10K+ collections because it loops every
> collection to fetch the latest stake from ZK, putting a massive load
> on ZK.  Our implementation of collection listing calls it, as does a
> number of places across Solr.  Some could be changed with relative
> ease; some are more thorny.  I'd love to rename this thing, putting
> "slow" in the name so that you think twice before calling it :-)
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: New branch and feature freeze for Solr 9.5.0

[Jason Gerlowski]

Go ahead and backport on your own!  I'm still waiting on Lucene 9.9.2, so
there shouldn't be any branch-contention on my end.

Relatedly, Lucene has their RC1 out there and things look good a day or two
into their VOTE, so with any luck we'll be able to get a Solr 9.5 RC
together early next week!

Best,

Jason

On Fri, Jan 26, 2024 at 8:49 AM Eric Pugh  wrote:

> I am about to merge SOLR-17112, and will backport it to branch_9x.  Jason,
> do you backport it over to the branch_9_5 or do I?
>
> ERic
>
>
> On 2024/01/23 19:08:10 Jason Gerlowski wrote:
> > > It was hoped SOLR-17112 would make 9.4.1 but it didn't as no PR was
> > proposed.
> >
> > > SOLR-17120 [is] nominated for inclusion in the 9.5.0 release
> >
> > Those both sound quick and reasonable; they've got a +1 from me to go
> into
> > 9.5 (assuming the contributor decides to continue with SOLR-17112).
> >
> > > Considering Lucene 9.9.2 is being planned, I think it would be better
> to
> > > upgrade Solr to the to-be-released version so users have to deal with
> > > fewer upgrade cycles.
> >
> > Yeah, that might be best; I hadn't realized we weren't already on the
> > latest Lucene 9.x.  I've created SOLR-17128 to track our Lucene upgrade
> > once 9.9.2 is available.
> >
> > Obviously this is a longer delay than some of the tickets above, and will
> > mean we won't be cutting a Solr RC this week.  We can pick a new date for
> > the initial Solr 9.5 RC once Lucene 9.9.2 is available.
> >
> > Best,
> >
> > Jason
> >
> > On Tue, Jan 23, 2024 at 1:37 PM Anshum Gupta 
> wrote:
> >
> > > Considering Lucene 9.9.2 is being planned, I think it would be better
> to
> > > upgrade Solr to the to-be-released version so users have to deal with
> fewer
> > > upgrade cycles.
> > >
> > > To highlight, there are about 90 odd changes in the Lucene 9.9.x line.
> > >
> > > -Anshum
> > >
> > > On Tue, Jan 23, 2024 at 8:47 AM David Smiley 
> wrote:
> > >
> > > > FYI It was hoped SOLR-17112
> > > > https://issues.apache.org/jira/browse/SOLR-17112 "bin/solr script
> > > > doesn't do ps properly on some systems" would make 9.4.1 but it
> didn't
> > > > as no PR was proposed.  There still isn't one but a contributor is
> > > > thinking about it.
> > > >
> > > > On Tue, Jan 23, 2024 at 11:30 AM Christine Poerschke (BLOOMBERG/
> > > > LONDON)  wrote:
> > > > >
> > > > > Just to cross-reference things further (Jason is already aware) --
> > > > https://issues.apache.org/jira/browse/SOLR-17120 and
> > > > https://github.com/apache/solr/pull/2214 are nominated for
> inclusion in
> > > > the 9.5 release, and as always additional reviews and inputs are
> welcome.
> > > > >
> > > > > Regards,
> > > > > Christine
> > > > >
> > > > > From: dev@solr.apache.org At: 01/22/24 17:30:35 UTCTo:
> > > > dev@solr.apache.org
> > > > > Subject: New branch and feature freeze for Solr 9.5.0
> > > > >
> > > > > NOTICE:
> > > > >
> > > > > Branch branch_9_5 has been cut and versions updated to 9.6 on the
> > > stable
> > > > > branch.
> > > > >
> > > > > Please observe the normal rules:
> > > > >
> > > > > * No new features may be committed to the branch.
> > > > > * Documentation patches, build patches and serious bug fixes may be
> > > > >   committed to the branch. However, you should submit all patches
> you
> > > > >   want to commit to Jira first to give others the chance to review
> > > > >   and possibly vote against the patch. Keep in mind that it is our
> > > > >   main intention to keep the branch as stable as possible.
> > > > > * All patches that are intended for the branch should first be
> > > committed
> > > > >   to the unstable branch, merged into the stable branch, and then
> into
> > > > >   the current release branch.
> > > > > * Normal unstable and stable branch development may continue as
> usual.
> > > > >   However, if you plan to commit a big change to the unstable
> branch
> > > > >   while the branch feature freeze is in effect, think twice: can't
> the
> > > > >   addition wait a couple more days? Merges of bug fixes into the
> branch
> > > > >   may become more difficult.
> > > > > * Only Jira issues with Fix version 9.5 and priority "Blocker" will
> > > delay
> > > > >   a release candidate build.
> > > > >
> > > > > The feature-freeze for the 9.5 release will go till the end of this
> > > week
> > > > -
> > > > > I'll aim to create our first RC on Thursday, January 25th.
> > > > >
> > > > > Best,
> > > > >
> > > > > Jason
> > > > >
> > > > >
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > > > For additional commands, e-mail: dev-h...@solr.apache.org
> > > >
> > > >
> > >
> > > --
> > > Anshum Gupta
> > >
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: New branch and feature freeze for Solr 9.5.0

[Jason Gerlowski]

> It was hoped SOLR-17112 would make 9.4.1 but it didn't as no PR was
proposed.

> SOLR-17120 [is] nominated for inclusion in the 9.5.0 release

Those both sound quick and reasonable; they've got a +1 from me to go into
9.5 (assuming the contributor decides to continue with SOLR-17112).

> Considering Lucene 9.9.2 is being planned, I think it would be better to
> upgrade Solr to the to-be-released version so users have to deal with
> fewer upgrade cycles.

Yeah, that might be best; I hadn't realized we weren't already on the
latest Lucene 9.x.  I've created SOLR-17128 to track our Lucene upgrade
once 9.9.2 is available.

Obviously this is a longer delay than some of the tickets above, and will
mean we won't be cutting a Solr RC this week.  We can pick a new date for
the initial Solr 9.5 RC once Lucene 9.9.2 is available.

Best,

Jason

On Tue, Jan 23, 2024 at 1:37 PM Anshum Gupta  wrote:

> Considering Lucene 9.9.2 is being planned, I think it would be better to
> upgrade Solr to the to-be-released version so users have to deal with fewer
> upgrade cycles.
>
> To highlight, there are about 90 odd changes in the Lucene 9.9.x line.
>
> -Anshum
>
> On Tue, Jan 23, 2024 at 8:47 AM David Smiley  wrote:
>
> > FYI It was hoped SOLR-17112
> > https://issues.apache.org/jira/browse/SOLR-17112 "bin/solr script
> > doesn't do ps properly on some systems" would make 9.4.1 but it didn't
> > as no PR was proposed.  There still isn't one but a contributor is
> > thinking about it.
> >
> > On Tue, Jan 23, 2024 at 11:30 AM Christine Poerschke (BLOOMBERG/
> > LONDON)  wrote:
> > >
> > > Just to cross-reference things further (Jason is already aware) --
> > https://issues.apache.org/jira/browse/SOLR-17120 and
> > https://github.com/apache/solr/pull/2214 are nominated for inclusion in
> > the 9.5 release, and as always additional reviews and inputs are welcome.
> > >
> > > Regards,
> > > Christine
> > >
> > > From: dev@solr.apache.org At: 01/22/24 17:30:35 UTCTo:
> > dev@solr.apache.org
> > > Subject: New branch and feature freeze for Solr 9.5.0
> > >
> > > NOTICE:
> > >
> > > Branch branch_9_5 has been cut and versions updated to 9.6 on the
> stable
> > > branch.
> > >
> > > Please observe the normal rules:
> > >
> > > * No new features may be committed to the branch.
> > > * Documentation patches, build patches and serious bug fixes may be
> > >   committed to the branch. However, you should submit all patches you
> > >   want to commit to Jira first to give others the chance to review
> > >   and possibly vote against the patch. Keep in mind that it is our
> > >   main intention to keep the branch as stable as possible.
> > > * All patches that are intended for the branch should first be
> committed
> > >   to the unstable branch, merged into the stable branch, and then into
> > >   the current release branch.
> > > * Normal unstable and stable branch development may continue as usual.
> > >   However, if you plan to commit a big change to the unstable branch
> > >   while the branch feature freeze is in effect, think twice: can't the
> > >   addition wait a couple more days? Merges of bug fixes into the branch
> > >   may become more difficult.
> > > * Only Jira issues with Fix version 9.5 and priority "Blocker" will
> delay
> > >   a release candidate build.
> > >
> > > The feature-freeze for the 9.5 release will go till the end of this
> week
> > -
> > > I'll aim to create our first RC on Thursday, January 25th.
> > >
> > > Best,
> > >
> > > Jason
> > >
> > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
> >
>
> --
> Anshum Gupta
>

Re: [DISCUSS] SIP-20: Separation of Compute and Storage in SolrCloud

[Jason Gerlowski]

eplica. All replicas of
> the
>shard, when they're becoming leaders, would access the same transaction
>log. Some of the logic used for implementing the shared storage for the
>index segments will be reused.
>For serving queries, the non leader replicas (also of type ZERO) update
>themselves from the shared storage directly. They behave mostly like
> PULL
>replicas (except the data doesn't come from the leader but from the
> shared
>storage), but can become leader because the shared storage is the
> "source
>of truth" and by reading all the data present there, any replica can get
>itself up to date with all acknowledged updates to become leader (there
> is
>protection so two replicas that think they're leader at the same time do
>not overwrite each other, I can describe this on another occasion).
>Currently the (basic) approach is that a replica checks if it is up to
>date as long as it's getting queries. If it is already up to date, the
> cost
>of that check is a ZooKeeper node read. If it is not up to date, it then
>fetches the updated content from the shared storage. Other strategies
>(check less often, check while not getting queries etc. are easy to
>    implement). The updates are currently done asynchronously so do not
> delay
>the queries (that serve the "previous" content, like normal SolrCloud
>replication does).
>
> I'd be happy to discuss and explain this in more detail. Quickly tomorrow
> then more once I've shared the source code branch and the design doc.
> I'd of course be happy to also follow up here with any questions, so don't
> hesitate to ask!
>
> Ilan
>
>
>
> On Wed, Jan 17, 2024 at 8:30 PM Jason Gerlowski 
> wrote:
>
> > Hey Ilan,
> >
> > Thanks for putting together this writeup.  I think I understand the goal
> > conceptually, and it sounds like a good one for Solr!  But I'm still
> having
> > trouble understanding how this all would actually work.  So a few
> > questions, inline:
> >
> > > A fourth replica type called ZERO is introduced
> >
> > Why the name "Zero"? Is it conveying something about the design that I'm
> > not picking up on?
> >
> > > At Collection creation time, it is possible to specify that the
> > collection exclusively uses replicas of type ZERO rather than being a
> > “normal” collection that uses NRT/TLOG/PULL.
> >
> > Am I correct in understanding this to mean that if "zero" is used, it
> must
> > be used for every replica in the collection?  If so, it almost sounds
> like
> > this isn't a new type of replica but a new "collection type" altogether?
> >
> > > This allows scaling compute (more queries, more indexing) independently
> > of storage
> >
> > I think the biggest question I have is: how does the "compute" side of
> this
> > actually work?
> >
> > On the indexing side: what all happens in Solr before giving a response
> > back to users?  What happens on a commit?  Are updates indexed only on
> the
> > leader (like TLOG/PULL) or on all replicas (like NRT), or some other
> > arrangement altogether?
> >
> > On the querying side: what situations cause index data to be pulled from
> > the remote store?
> >
> > (These last questions might be a bit lengthy to get into via email, but
> > they should probably be in the writeup?  Not sure what's best there...)
> >
> > Best,
> >
> > Jason
> >
> > On Sat, Jan 13, 2024 at 9:15 PM Ishan Chattopadhyaya <
> > ichattopadhy...@gmail.com> wrote:
> >
> > > +1, thanks for the contribution Ilan! Looking forward to seeing this
> > coming
> > > to fruition.
> > >
> > > On Sun, 14 Jan 2024 at 03:40, Ilan Ginzburg 
> wrote:
> > >
> > > > I have created SIP-20
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/SOLR/SIP-20%3A+Separation+of+Compute+and+Storage+in+SolrCloud
> > > >
> > > > In the next few days I will create a Jira + a branch that implements
> > > > the SIP proposal and that includes documentation on how to approach
> > > > that branch and what's in it.
> > > >
> > > > This proposed contribution is based on work done at Salesforce these
> > > > last few years and currently running at scale in multiple regions.
> > > >
> > > > Thanks,
> > > > Ilan
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > > > For additional commands, e-mail: dev-h...@solr.apache.org
> > > >
> > > >
> > >
> >
>

Re: New branch and feature freeze for Solr 9.5.0

[Jason Gerlowski]

> I'm a little surprised that this isn't caught by the
'check'/'precommit' checks - I've been relying (apparently mistakenly) on
those tasks to catch this sort of thing when reviewing and merging PRs.

Ah, I see why I didn't catch it.  It's a warning instead of an error.
Nevermind my comment.  Working on a fix now.

On Mon, Jan 22, 2024 at 2:32 PM Jason Gerlowski 
wrote:

> I've put together a first-draft for our release-notes for 9.5.  They're
> available here:
> https://cwiki.apache.org/confluence/display/SOLR/DRAFT+ReleaseNote9_5_0
>
> Would love some review on what I've put together; feel free to add, remove
> or change anything as needed!
>
> Best,
>
> Jason
>
> On Mon, Jan 22, 2024 at 2:18 PM Jason Gerlowski 
> wrote:
>
>> > I committed a bug in the build for setting up Node with a registry
>> mirror.  I assume it's ok for me to backport?
>>
>> Yep, please do!
>>
>> > One page has two of the same anchors on branch_9x and branch_9_5. We
>> should fix this before the release.
>>
>> Agreed, I'll take a look.
>>
>> Fwiw, I merged the PR that likely created that anchor conflict.  I'm a
>> little surprised that this isn't caught by the 'check'/'precommit' checks -
>> I've been relying (apparently mistakenly) on those tasks to catch this sort
>> of thing when reviewing and merging PRs.  I guess I'll have to add some
>> other ref-guide specific task to my pre-merge routine?
>>
>> On Mon, Jan 22, 2024 at 1:29 PM Houston Putman 
>> wrote:
>>
>>> Hey Jason,
>>>
>>> I committed a bug in the build for setting up Node with a registry
>>> mirror.
>>> I assume it's ok for me to backport?
>>>
>>> Also I noticed that there is an issue with the ref guide. One page has
>>> two
>>> of the same anchors on branch_9x and branch_9_5. We should fix this
>>> before
>>> the release.
>>>
>>>
>>> {"level":"warn","time":1705948125542,"name":"asciidoctor","file":{"path":"/Users/houstonputman/dev/oss/solr/solr/main/solr/solr-ref-guide/build/site-staging/modules/configuration-guide/pages/coreadmin-api.adoc","line":606},"source":{"url":"file:///Users/houstonputman/dev/oss/solr/solr/main","local":"/Users/houstonputman/dev/oss/solr/solr/main/.git","worktree":"/Users/houstonputman/dev/oss/solr/solr/main","refname":"branch_9_5","reftype":"branch","startPath":"solr/solr-ref-guide/build/site-staging"},"msg":"id
>>> > assigned to block already in use: v1coreadmin-mergeindexes"}
>>> >
>>> {"level":"warn","time":1705948125542,"name":"asciidoctor","file":{"path":"/Users/houstonputman/dev/oss/solr/solr/main/solr/solr-ref-guide/build/site-staging/modules/configuration-guide/pages/coreadmin-api.adoc","line":614},"source":{"url":"file:///Users/houstonputman/dev/oss/solr/solr/main","local":"/Users/houstonputman/dev/oss/solr/solr/main/.git","worktree":"/Users/houstonputman/dev/oss/solr/solr/main","refname":"branch_9_5","reftype":"branch","startPath":"solr/solr-ref-guide/build/site-staging"},"msg":"id
>>> > assigned to block already in use: v2coreadmin-mergeindexes"}
>>> >
>>>
>>> - Houston
>>>
>>> On Mon, Jan 22, 2024 at 12:30 PM Jason Gerlowski 
>>> wrote:
>>>
>>> > NOTICE:
>>> >
>>> > Branch branch_9_5 has been cut and versions updated to 9.6 on the
>>> stable
>>> > branch.
>>> >
>>> > Please observe the normal rules:
>>> >
>>> > * No new features may be committed to the branch.
>>> > * Documentation patches, build patches and serious bug fixes may be
>>> >   committed to the branch. However, you should submit all patches you
>>> >   want to commit to Jira first to give others the chance to review
>>> >   and possibly vote against the patch. Keep in mind that it is our
>>> >   main intention to keep the branch as stable as possible.
>>> > * All patches that are intended for the branch should first be
>>> committed
>>> >   to the unstable branch, merged into the stable branch, and then into
>>> >   the current release branch.
>>> > * Normal unstable and stable branch development may continue as usual.
>>> >   However, if you plan to commit a big change to the unstable branch
>>> >   while the branch feature freeze is in effect, think twice: can't the
>>> >   addition wait a couple more days? Merges of bug fixes into the branch
>>> >   may become more difficult.
>>> > * Only Jira issues with Fix version 9.5 and priority "Blocker" will
>>> delay
>>> >   a release candidate build.
>>> >
>>> > The feature-freeze for the 9.5 release will go till the end of this
>>> week -
>>> > I'll aim to create our first RC on Thursday, January 25th.
>>> >
>>> > Best,
>>> >
>>> > Jason
>>> >
>>>
>>

Re: New branch and feature freeze for Solr 9.5.0

[Jason Gerlowski]

I've put together a first-draft for our release-notes for 9.5.  They're
available here:
https://cwiki.apache.org/confluence/display/SOLR/DRAFT+ReleaseNote9_5_0

Would love some review on what I've put together; feel free to add, remove
or change anything as needed!

Best,

Jason

On Mon, Jan 22, 2024 at 2:18 PM Jason Gerlowski 
wrote:

> > I committed a bug in the build for setting up Node with a registry
> mirror.  I assume it's ok for me to backport?
>
> Yep, please do!
>
> > One page has two of the same anchors on branch_9x and branch_9_5. We
> should fix this before the release.
>
> Agreed, I'll take a look.
>
> Fwiw, I merged the PR that likely created that anchor conflict.  I'm a
> little surprised that this isn't caught by the 'check'/'precommit' checks -
> I've been relying (apparently mistakenly) on those tasks to catch this sort
> of thing when reviewing and merging PRs.  I guess I'll have to add some
> other ref-guide specific task to my pre-merge routine?
>
> On Mon, Jan 22, 2024 at 1:29 PM Houston Putman  wrote:
>
>> Hey Jason,
>>
>> I committed a bug in the build for setting up Node with a registry mirror.
>> I assume it's ok for me to backport?
>>
>> Also I noticed that there is an issue with the ref guide. One page has two
>> of the same anchors on branch_9x and branch_9_5. We should fix this before
>> the release.
>>
>>
>> {"level":"warn","time":1705948125542,"name":"asciidoctor","file":{"path":"/Users/houstonputman/dev/oss/solr/solr/main/solr/solr-ref-guide/build/site-staging/modules/configuration-guide/pages/coreadmin-api.adoc","line":606},"source":{"url":"file:///Users/houstonputman/dev/oss/solr/solr/main","local":"/Users/houstonputman/dev/oss/solr/solr/main/.git","worktree":"/Users/houstonputman/dev/oss/solr/solr/main","refname":"branch_9_5","reftype":"branch","startPath":"solr/solr-ref-guide/build/site-staging"},"msg":"id
>> > assigned to block already in use: v1coreadmin-mergeindexes"}
>> >
>> {"level":"warn","time":1705948125542,"name":"asciidoctor","file":{"path":"/Users/houstonputman/dev/oss/solr/solr/main/solr/solr-ref-guide/build/site-staging/modules/configuration-guide/pages/coreadmin-api.adoc","line":614},"source":{"url":"file:///Users/houstonputman/dev/oss/solr/solr/main","local":"/Users/houstonputman/dev/oss/solr/solr/main/.git","worktree":"/Users/houstonputman/dev/oss/solr/solr/main","refname":"branch_9_5","reftype":"branch","startPath":"solr/solr-ref-guide/build/site-staging"},"msg":"id
>> > assigned to block already in use: v2coreadmin-mergeindexes"}
>> >
>>
>> - Houston
>>
>> On Mon, Jan 22, 2024 at 12:30 PM Jason Gerlowski 
>> wrote:
>>
>> > NOTICE:
>> >
>> > Branch branch_9_5 has been cut and versions updated to 9.6 on the stable
>> > branch.
>> >
>> > Please observe the normal rules:
>> >
>> > * No new features may be committed to the branch.
>> > * Documentation patches, build patches and serious bug fixes may be
>> >   committed to the branch. However, you should submit all patches you
>> >   want to commit to Jira first to give others the chance to review
>> >   and possibly vote against the patch. Keep in mind that it is our
>> >   main intention to keep the branch as stable as possible.
>> > * All patches that are intended for the branch should first be committed
>> >   to the unstable branch, merged into the stable branch, and then into
>> >   the current release branch.
>> > * Normal unstable and stable branch development may continue as usual.
>> >   However, if you plan to commit a big change to the unstable branch
>> >   while the branch feature freeze is in effect, think twice: can't the
>> >   addition wait a couple more days? Merges of bug fixes into the branch
>> >   may become more difficult.
>> > * Only Jira issues with Fix version 9.5 and priority "Blocker" will
>> delay
>> >   a release candidate build.
>> >
>> > The feature-freeze for the 9.5 release will go till the end of this
>> week -
>> > I'll aim to create our first RC on Thursday, January 25th.
>> >
>> > Best,
>> >
>> > Jason
>> >
>>
>

Re: New branch and feature freeze for Solr 9.5.0

[Jason Gerlowski]

> I committed a bug in the build for setting up Node with a registry
mirror.  I assume it's ok for me to backport?

Yep, please do!

> One page has two of the same anchors on branch_9x and branch_9_5. We
should fix this before the release.

Agreed, I'll take a look.

Fwiw, I merged the PR that likely created that anchor conflict.  I'm a
little surprised that this isn't caught by the 'check'/'precommit' checks -
I've been relying (apparently mistakenly) on those tasks to catch this sort
of thing when reviewing and merging PRs.  I guess I'll have to add some
other ref-guide specific task to my pre-merge routine?

On Mon, Jan 22, 2024 at 1:29 PM Houston Putman  wrote:

> Hey Jason,
>
> I committed a bug in the build for setting up Node with a registry mirror.
> I assume it's ok for me to backport?
>
> Also I noticed that there is an issue with the ref guide. One page has two
> of the same anchors on branch_9x and branch_9_5. We should fix this before
> the release.
>
>
> {"level":"warn","time":1705948125542,"name":"asciidoctor","file":{"path":"/Users/houstonputman/dev/oss/solr/solr/main/solr/solr-ref-guide/build/site-staging/modules/configuration-guide/pages/coreadmin-api.adoc","line":606},"source":{"url":"file:///Users/houstonputman/dev/oss/solr/solr/main","local":"/Users/houstonputman/dev/oss/solr/solr/main/.git","worktree":"/Users/houstonputman/dev/oss/solr/solr/main","refname":"branch_9_5","reftype":"branch","startPath":"solr/solr-ref-guide/build/site-staging"},"msg":"id
> > assigned to block already in use: v1coreadmin-mergeindexes"}
> >
> {"level":"warn","time":1705948125542,"name":"asciidoctor","file":{"path":"/Users/houstonputman/dev/oss/solr/solr/main/solr/solr-ref-guide/build/site-staging/modules/configuration-guide/pages/coreadmin-api.adoc","line":614},"source":{"url":"file:///Users/houstonputman/dev/oss/solr/solr/main","local":"/Users/houstonputman/dev/oss/solr/solr/main/.git","worktree":"/Users/houstonputman/dev/oss/solr/solr/main","refname":"branch_9_5","reftype":"branch","startPath":"solr/solr-ref-guide/build/site-staging"},"msg":"id
> > assigned to block already in use: v2coreadmin-mergeindexes"}
> >
>
> - Houston
>
> On Mon, Jan 22, 2024 at 12:30 PM Jason Gerlowski 
> wrote:
>
> > NOTICE:
> >
> > Branch branch_9_5 has been cut and versions updated to 9.6 on the stable
> > branch.
> >
> > Please observe the normal rules:
> >
> > * No new features may be committed to the branch.
> > * Documentation patches, build patches and serious bug fixes may be
> >   committed to the branch. However, you should submit all patches you
> >   want to commit to Jira first to give others the chance to review
> >   and possibly vote against the patch. Keep in mind that it is our
> >   main intention to keep the branch as stable as possible.
> > * All patches that are intended for the branch should first be committed
> >   to the unstable branch, merged into the stable branch, and then into
> >   the current release branch.
> > * Normal unstable and stable branch development may continue as usual.
> >   However, if you plan to commit a big change to the unstable branch
> >   while the branch feature freeze is in effect, think twice: can't the
> >   addition wait a couple more days? Merges of bug fixes into the branch
> >   may become more difficult.
> > * Only Jira issues with Fix version 9.5 and priority "Blocker" will delay
> >   a release candidate build.
> >
> > The feature-freeze for the 9.5 release will go till the end of this week
> -
> > I'll aim to create our first RC on Thursday, January 25th.
> >
> > Best,
> >
> > Jason
> >
>

Re: [DISCUSS] Solr 9.5 Release

[Jason Gerlowski]

Hey all,

You've likely seen the "New branch and feature freeze for Solr 9.5.0"
email, but in case you haven't:

branch_9_5 is now created and "feature freeze" for it is currently in
effect.  There are no 9.5 blockers afaik, so my plan is to leave a few days
for any lingering bugs folks might want to squeeze in and then try for an
RC this Thursday.

Best,

Jason

On Fri, Jan 19, 2024 at 1:29 PM Jason Gerlowski 
wrote:

> Good question - I guess this is now "unblocked" now that 9.4.1 is out the
> door (thanks David!).
>
> I guess, pending any objections, that my revised timing would be to begin
> "feature freeze" and create branch_9_5 Monday or Tuesday of next week.
> With the hope that I can get the first RC out by the end of the week?
>
> Best,
>
> Jason
>
> On Fri, Jan 19, 2024 at 12:40 PM David Smiley  wrote:
>
>> When is the feature freeze?  Please create the branch_9_5 at the time of
>> that.
>>
>> On Tue, Jan 16, 2024 at 3:45 PM Jason Gerlowski 
>> wrote:
>>
>> > > Is there an urgent need for this release?
>> >
>> > Have I seemed "urgent"?  I suggested a mid-January 9.5 back on 12/6 (see
>> > the "9.4.1 release" thread).  If anything I've been slow :-p
>> >
>> > Like I said in my initial message here - my motivation around 9.5 is
>> just
>> > that I think there's some cool stuff lined up and I'd love to get that
>> out
>> > to folks.  Getting features out is a "Good Thing", so why wait on 9.5 if
>> > there's not a particular reason to?  If there is a concrete reason to
>> wait,
>> > that's fine too; I'm just curious to learn more about it.
>> >
>> > Best,
>> >
>> > Jason
>> >
>> >
>> >
>> > On Tue, Jan 16, 2024 at 1:50 PM Houston Putman 
>> wrote:
>> >
>> > > Apple isn't urging anything, and you really can assume that's true
>> across
>> > > the board for our decisions.
>> > >
>> > > There are other concerns that are urging quick releases (See other
>> > mailing
>> > > lists for more information).
>> > >
>> > > I'm happy to let you continue the 8.11.3 release, and if you want to
>> wait
>> > > for 9.4.1 that is fine with me. But we do need to get it out quickly.
>> > >
>> > > - Houston
>> > >
>> > > On Tue, Jan 16, 2024 at 12:45 PM Ishan Chattopadhyaya <
>> > > ichattopadhy...@gmail.com> wrote:
>> > >
>> > > > Is there an urgent need for this release?
>> > > > I saw that Houston took over the 8.11.3 release process from me
>> without
>> > > any
>> > > > prior intimation. Is Apple asking both of you to hurry up?
>> > > >
>> > > > On Tue, 16 Jan 2024 at 20:35, Jason Gerlowski <
>> gerlowsk...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > > but let 9.4.1 get out the door first..
>> > > > >
>> > > > > I'm happy to wait if that's the consensus, but is there a reason
>> one
>> > > > > release needs to block another?
>> > > > >
>> > > > > I could be blanking, but I don't think there's any technical
>> conflict
>> > > in
>> > > > > terms of where artifacts are staged, etc. is there?
>> > > > >
>> > > > > > getting through some one-time pains of key/GPG matters
>> > > > >
>> > > > > Definitely the worst bit in my recent experience; good luck!
>> > > > >
>> > > > > Best,
>> > > > >
>> > > > > Jason
>> > > > >
>> > > > > On Fri, Jan 12, 2024 at 11:08 PM David Smiley > >
>> > > > wrote:
>> > > > >
>> > > > > > Yeah I'm doing the 9.4.1 for real now... getting through some
>> > > one-time
>> > > > > > pains of key/GPG matters.
>> > > > > >
>> > > > > > https://issues.apache.org/jira/browse/SOLR-17096 solr.xml
>> support
>> > > for
>> > > > > >  plugins is pretty close!
>> > > > > >
>> > > > > > On Fri, Jan 12, 2024 at 3:56 PM Jason Gerlowski <
>> > > gerlowsk...@gmail.com
>> > > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hey all,
>> > > > > > >
>> > > > > > > branch_9x has accumulated 3 new features, 19 improvements, 2
>> > > > > > optimizations,
>> > > > > > > 11 bug fixes, and 7 "other changes" - I'd love to get these in
>> > > front
>> > > > of
>> > > > > > > users with a Solr 9.5.0 release.
>> > > > > > >
>> > > > > > > I'm happy to volunteer as RM, and propose that we target next
>> > > > Thursday
>> > > > > > > January 18th to cut the branch and prepare the first RC.
>> > > > > > >
>> > > > > > > Best,
>> > > > > > >
>> > > > > > > Jason
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>