Re: Solr Alpha (EA) release of Reference Branch

[Tomás Fernández Löbbe]

I was thinking (and I haven’t flushed it out completely but will throw the
idea) that an alternative approach with this timeline could be to cut 9x
branch around November/December? And then you could merge into master, it
would have the latest  changes from master plus the ref branch changes.
>From there any nightly build could be use to help test/debug.

That said I don’t know for sure what are the changes in the branch that do
not belong in 9. The problem with them being 10x only is that backports
would potentially be more difficult for all the life of 9.

On Mon, Oct 5, 2020 at 4:54 PM Noble Paul  wrote:

> >I don't think it can be said what committers do and don't do with regards
> to running Solr.  All of us would answer this differently and at different
> points in time.
>
> " I have run it in one large cluster, so it is certified to be bug
> free/stable" I don't think it's a reasonable approach. We need as much
> feedback from our users because each of them stress Solr in a
> different way. This is not to suggest that committers are not doing testing
> or their tests are not valid. When I talk to the committers out here they
> say they do not see any performance stability issues at all. But, my client
> reports issues on a day to day basis.
>
>
>
> > Definitely publish a Docker image BTW -- it's the best way to try out
> any software.
>
> Docker is not a big requirement for large scale installations. Most of
> them already have their own install scripts. Availability of docker is not
> important for them. If a user is only encouraged to install Solr because of
> a docker image , most likely they are not running a large enough cluster
>
>
>
> On Tue, Oct 6, 2020, 6:30 AM David Smiley  wrote:
>
>> Thanks so much for your responses Ishan... I'm getting much more
>> information in this thread than my attempts to get questions answered on
>> the JIRA issue months ago.  And especially,  thank you for volunteering for
>> the difficult porting efforts!
>>
>> Tomas said:
>>
>>>  I do agree with the previous comments that calling it "Solr 10" (even
>>> with the "-alpha") would confuse users, maybe use "reference"? or maybe
>>> something in reference to SOLR-14788?
>>>
>>
>> I have the opposite opinion.  This word "reference" is baffling to me
>> despite whatever Mark's explanation is.  I like the justification Ishan
>> gave for 10-alpha and I don't think I could re-phrase his justification any
>> better.  *If* the release was _not_ official (thus wouldn't show up in the
>> usual places anyone would look for a release), I think it would alleviate
>> that confusion concern even more, although I think "alpha" ought to be
>> enough of a signal not to use it without digging deeper on what's going on.
>>
>> Alex then Ishan said:
>>
>>> > Maybe we could release it to
>>> > committers community first and dogfood it "internally"?
>>>
>>> Alex: It is meaningless. Committers don't run large scale installations.
>>> We barely even have time to take care of running unit tests before
>>> destabilizing our builds. We are not the right audience. However, we all
>>> can anyway check out the branch and start playing with it, even without a
>>> release. There are orgs that don't want to install any code that wasn't
>>> officially released; this release is geared towards them (to help us test
>>> this at their scale).
>>>
>>
>> I don't think it can be said what committers do and don't do with regards
>> to running Solr.  All of us would answer this differently and at different
>> points in time.  From time to time, though not at present, I've been well
>> positioned to try out a new version of Solr in a stage/test environment to
>> see how it goes.  (Putting on my Salesforce metaphorical hat...) Even
>> though I'm not able to deploy it in a realistic way today, I'm able to run
>> a battery of tests to see if one of the features we depend on have changed
>> or is broken.  That's useful feedback to an alpha release!  And even though
>> I'm saying I'm not well positioned to try out some new Solr release in a
>> production-ish setting now, it's something I could make a good case for
>> internally since upgrades take a lot of effort where I work.  It's in our
>> interest for SolrCloud to be very stable (of course).
>>
>> Regardless, I think what you're driving at Ishan is that you want an
>> "official" release -- one that goes through the whole ceremony.  You
>> believe that people would be more likely to use it.  I think all we need to
>> do is announce (similar to a real release) that there is some unofficial
>> alpha distribution and that we want to solicit your feedback -- basically,
>> help us find bugs.  Definitely publish a Docker image BTW -- it's the best
>> way to try out any software.  I'm -0 on doing an official release for alpha
>> software because it's unnecessary to achieve the goals and somewhat
>> confusing.  I think the Solr 4 alpha/beta situation was different -- it was
>> not some fork a committer was 

terms filter in json.facet

[gopikannan]

Hi,
  In normal facet request below can be used to filter the facet terms. I am
not able to do the same using json.facet. Please let me know whether I can
raise a JIRA for this. Checked the code and I think I can work on the
changes to support this.

facet.field={!terms='alfa,betta,with\,with\',with space'}symbol

https://lucene.apache.org/solr/guide/6_6/faceting.html

Thanks
Gopi

Removing Overseer

[Ilan Ginzburg]

I'm sharing the initial drop of a proposal to remove the Overseer from
SolrCloud.

https://docs.google.com/document/d/1u4QHsIHuIxlglIW6hekYlXGNOP0HjLGVX5N6inkj6Ok/

This is a structural change that I believe requires a large consensus
to be successful or even started. Feedback is most welcome and very
much awaited!

Thanks,
Ilan

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

BlockMaxConjunctionScorer for Occur.FILTER query clauses

[Viral Gandhi]

Hello,

I am trying to utilize Block Max WAND optimization. During testing,
BooleanQuery that I passed to IndexSearcher.seach() had multiple clauses
and all of them were of Occur.FILTER type. I noticed that when queries do
not have Occur.MUST clauses, BlockMaxConjunctionScorer is not created for
those clauses.

Does anyone know why WAND optimization does not apply for Occur.FILTER
clauses?

Thanks,
Viral Gandhi

Re: Solr Alpha (EA) release of Reference Branch

[Noble Paul]

>I don't think it can be said what committers do and don't do with regards
to running Solr.  All of us would answer this differently and at different
points in time.

" I have run it in one large cluster, so it is certified to be bug
free/stable" I don't think it's a reasonable approach. We need as much
feedback from our users because each of them stress Solr in a
different way. This is not to suggest that committers are not doing testing
or their tests are not valid. When I talk to the committers out here they
say they do not see any performance stability issues at all. But, my client
reports issues on a day to day basis.



> Definitely publish a Docker image BTW -- it's the best way to try out any
software.

Docker is not a big requirement for large scale installations. Most of them
already have their own install scripts. Availability of docker is not
important for them. If a user is only encouraged to install Solr because of
a docker image , most likely they are not running a large enough cluster



On Tue, Oct 6, 2020, 6:30 AM David Smiley  wrote:

> Thanks so much for your responses Ishan... I'm getting much more
> information in this thread than my attempts to get questions answered on
> the JIRA issue months ago.  And especially,  thank you for volunteering for
> the difficult porting efforts!
>
> Tomas said:
>
>>  I do agree with the previous comments that calling it "Solr 10" (even
>> with the "-alpha") would confuse users, maybe use "reference"? or maybe
>> something in reference to SOLR-14788?
>>
>
> I have the opposite opinion.  This word "reference" is baffling to me
> despite whatever Mark's explanation is.  I like the justification Ishan
> gave for 10-alpha and I don't think I could re-phrase his justification any
> better.  *If* the release was _not_ official (thus wouldn't show up in the
> usual places anyone would look for a release), I think it would alleviate
> that confusion concern even more, although I think "alpha" ought to be
> enough of a signal not to use it without digging deeper on what's going on.
>
> Alex then Ishan said:
>
>> > Maybe we could release it to
>> > committers community first and dogfood it "internally"?
>>
>> Alex: It is meaningless. Committers don't run large scale installations.
>> We barely even have time to take care of running unit tests before
>> destabilizing our builds. We are not the right audience. However, we all
>> can anyway check out the branch and start playing with it, even without a
>> release. There are orgs that don't want to install any code that wasn't
>> officially released; this release is geared towards them (to help us test
>> this at their scale).
>>
>
> I don't think it can be said what committers do and don't do with regards
> to running Solr.  All of us would answer this differently and at different
> points in time.  From time to time, though not at present, I've been well
> positioned to try out a new version of Solr in a stage/test environment to
> see how it goes.  (Putting on my Salesforce metaphorical hat...) Even
> though I'm not able to deploy it in a realistic way today, I'm able to run
> a battery of tests to see if one of the features we depend on have changed
> or is broken.  That's useful feedback to an alpha release!  And even though
> I'm saying I'm not well positioned to try out some new Solr release in a
> production-ish setting now, it's something I could make a good case for
> internally since upgrades take a lot of effort where I work.  It's in our
> interest for SolrCloud to be very stable (of course).
>
> Regardless, I think what you're driving at Ishan is that you want an
> "official" release -- one that goes through the whole ceremony.  You
> believe that people would be more likely to use it.  I think all we need to
> do is announce (similar to a real release) that there is some unofficial
> alpha distribution and that we want to solicit your feedback -- basically,
> help us find bugs.  Definitely publish a Docker image BTW -- it's the best
> way to try out any software.  I'm -0 on doing an official release for alpha
> software because it's unnecessary to achieve the goals and somewhat
> confusing.  I think the Solr 4 alpha/beta situation was different -- it was
> not some fork a committer was maintaining; it was the master branch of its
> time, and it was destined to be the very next release, not some possible
> future release.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>

Re: Solr Alpha (EA) release of Reference Branch

[Tomás Fernández Löbbe]

> I think the Solr 4 alpha/beta situation was different -- it was not some
fork a committer was maintaining; it was the master branch of its time, and
it was destined to be the very next release, not some possible future
release.

Agree, 4’s alpha/beta was a very different situation.

> I believe it has to be an official release to have enough credibility.
People trust the Apache brand and the community. This will ensure that we
get enough people to test this out. The very objective of this release is
to get help from our users to uncover any bugs. Most big shops will not
deploy unofficial releases in their prod/staging environments. We wish to
tick all the boxes for our users

I think this is fooling ourselves/our users. They trust Apache releases
because we take them seriously, because they have the community behind,
etc. This is a release from a feature branch, so we have to be cautious and
very upfront. While I hope many go and test it and maybe deploy it to some
of their environments, I feel we need to be careful not to send the wrong
message.

> I'm -0 on doing an official release for alpha software because it's
unnecessary to achieve the goals and somewhat confusing

I’d say I’m -1 to make it an official release for those reasons.

I think we should try to merge into master (whenever you are comfortable?
Does it need to wait until 9.0 is out? What's the main reason for that? can
we split the parts that do need to wait vs the ones that don’t?) and then
people can be encouraged to run a nightly version in their test
environments to help debug possible instability. If we need to do
alpha/beta versions from master then I think that would make more sense.



On Mon, Oct 5, 2020 at 12:30 PM David Smiley  wrote:

> Thanks so much for your responses Ishan... I'm getting much more
> information in this thread than my attempts to get questions answered on
> the JIRA issue months ago.  And especially,  thank you for volunteering for
> the difficult porting efforts!
>
> Tomas said:
>
>>  I do agree with the previous comments that calling it "Solr 10" (even
>> with the "-alpha") would confuse users, maybe use "reference"? or maybe
>> something in reference to SOLR-14788?
>>
>
> I have the opposite opinion.  This word "reference" is baffling to me
> despite whatever Mark's explanation is.  I like the justification Ishan
> gave for 10-alpha and I don't think I could re-phrase his justification any
> better.  *If* the release was _not_ official (thus wouldn't show up in the
> usual places anyone would look for a release), I think it would alleviate
> that confusion concern even more, although I think "alpha" ought to be
> enough of a signal not to use it without digging deeper on what's going on.
>
> Alex then Ishan said:
>
>> > Maybe we could release it to
>> > committers community first and dogfood it "internally"?
>>
>> Alex: It is meaningless. Committers don't run large scale installations.
>> We barely even have time to take care of running unit tests before
>> destabilizing our builds. We are not the right audience. However, we all
>> can anyway check out the branch and start playing with it, even without a
>> release. There are orgs that don't want to install any code that wasn't
>> officially released; this release is geared towards them (to help us test
>> this at their scale).
>>
>
> I don't think it can be said what committers do and don't do with regards
> to running Solr.  All of us would answer this differently and at different
> points in time.  From time to time, though not at present, I've been well
> positioned to try out a new version of Solr in a stage/test environment to
> see how it goes.  (Putting on my Salesforce metaphorical hat...) Even
> though I'm not able to deploy it in a realistic way today, I'm able to run
> a battery of tests to see if one of the features we depend on have changed
> or is broken.  That's useful feedback to an alpha release!  And even though
> I'm saying I'm not well positioned to try out some new Solr release in a
> production-ish setting now, it's something I could make a good case for
> internally since upgrades take a lot of effort where I work.  It's in our
> interest for SolrCloud to be very stable (of course).
>
> Regardless, I think what you're driving at Ishan is that you want an
> "official" release -- one that goes through the whole ceremony.  You
> believe that people would be more likely to use it.  I think all we need to
> do is announce (similar to a real release) that there is some unofficial
> alpha distribution and that we want to solicit your feedback -- basically,
> help us find bugs.  Definitely publish a Docker image BTW -- it's the best
> way to try out any software.  I'm -0 on doing an official release for alpha
> software because it's unnecessary to achieve the goals and somewhat
> confusing.  I think the Solr 4 alpha/beta situation was different -- it was
> not some fork a committer was maintaining; it was the master branch of its
> time, 

Re: [VOTE] Release Lucene/Solr 8.6.3 RC1

[Anshum Gupta]

+1 (binding)

SUCCESS! [1:00:37.423566]

Tried basic indexing and search and ran the smoke tester.

On Sat, Oct 3, 2020 at 6:53 PM Jason Gerlowski 
wrote:

> Please vote for release candidate 1 for Lucene/Solr 8.6.3
>
>
> The artifacts can be downloaded from:
>
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.6.3-RC1-reve001c2221812a0ba9e9378855040ce72f93eced4
>
>
> You can run the smoke tester directly with this command:
>
>
> python3 -u dev-tools/scripts/smokeTestRelease.py \
>
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.6.3-RC1-reve001c2221812a0ba9e9378855040ce72f93eced4
>
>
> The vote will be open for at least 72 hours i.e. until 2020-10-07 02:00
> UTC.
>
>
> [ ] +1  approve
>
> [ ] +0  no opinion
>
> [ ] -1  disapprove (and reason why)
>
>
> Here is my +1
>


-- 
Anshum Gupta

Re: Solr Alpha (EA) release of Reference Branch

[David Smiley]

Thanks so much for your responses Ishan... I'm getting much more
information in this thread than my attempts to get questions answered on
the JIRA issue months ago.  And especially,  thank you for volunteering for
the difficult porting efforts!

Tomas said:

>  I do agree with the previous comments that calling it "Solr 10" (even
> with the "-alpha") would confuse users, maybe use "reference"? or maybe
> something in reference to SOLR-14788?
>

I have the opposite opinion.  This word "reference" is baffling to me
despite whatever Mark's explanation is.  I like the justification Ishan
gave for 10-alpha and I don't think I could re-phrase his justification any
better.  *If* the release was _not_ official (thus wouldn't show up in the
usual places anyone would look for a release), I think it would alleviate
that confusion concern even more, although I think "alpha" ought to be
enough of a signal not to use it without digging deeper on what's going on.

Alex then Ishan said:

> > Maybe we could release it to
> > committers community first and dogfood it "internally"?
>
> Alex: It is meaningless. Committers don't run large scale installations.
> We barely even have time to take care of running unit tests before
> destabilizing our builds. We are not the right audience. However, we all
> can anyway check out the branch and start playing with it, even without a
> release. There are orgs that don't want to install any code that wasn't
> officially released; this release is geared towards them (to help us test
> this at their scale).
>

I don't think it can be said what committers do and don't do with regards
to running Solr.  All of us would answer this differently and at different
points in time.  From time to time, though not at present, I've been well
positioned to try out a new version of Solr in a stage/test environment to
see how it goes.  (Putting on my Salesforce metaphorical hat...) Even
though I'm not able to deploy it in a realistic way today, I'm able to run
a battery of tests to see if one of the features we depend on have changed
or is broken.  That's useful feedback to an alpha release!  And even though
I'm saying I'm not well positioned to try out some new Solr release in a
production-ish setting now, it's something I could make a good case for
internally since upgrades take a lot of effort where I work.  It's in our
interest for SolrCloud to be very stable (of course).

Regardless, I think what you're driving at Ishan is that you want an
"official" release -- one that goes through the whole ceremony.  You
believe that people would be more likely to use it.  I think all we need to
do is announce (similar to a real release) that there is some unofficial
alpha distribution and that we want to solicit your feedback -- basically,
help us find bugs.  Definitely publish a Docker image BTW -- it's the best
way to try out any software.  I'm -0 on doing an official release for alpha
software because it's unnecessary to achieve the goals and somewhat
confusing.  I think the Solr 4 alpha/beta situation was different -- it was
not some fork a committer was maintaining; it was the master branch of its
time, and it was destined to be the very next release, not some possible
future release.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley

Re: [VOTE] Release Lucene/Solr 8.6.3 RC1

[Houston Putman]

+1 (non-binding)

SUCCESS! [0:59:46.680873]

On Mon, Oct 5, 2020 at 11:04 AM Adrien Grand  wrote:

> +1 SUCCESS! [1:36:10.395992]
>
> On Mon, Oct 5, 2020 at 2:15 PM Michael McCandless <
> luc...@mikemccandless.com> wrote:
>
>> +1 (binding)
>>
>> SUCCESS! [0:44:16.898412]
>>
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>
>> On Mon, Oct 5, 2020 at 3:28 AM Atri Sharma  wrote:
>>
>>> +1 (binding)
>>>
>>> SUCCESS! [1:04.32.39193]
>>>
>>> On Sun, Oct 4, 2020 at 7:24 AM Jason Gerlowski 
>>> wrote:
>>> >
>>> > Please vote for release candidate 1 for Lucene/Solr 8.6.3
>>> >
>>> >
>>> > The artifacts can be downloaded from:
>>> >
>>> >
>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.6.3-RC1-reve001c2221812a0ba9e9378855040ce72f93eced4
>>> >
>>> >
>>> > You can run the smoke tester directly with this command:
>>> >
>>> >
>>> > python3 -u dev-tools/scripts/smokeTestRelease.py \
>>> >
>>> >
>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.6.3-RC1-reve001c2221812a0ba9e9378855040ce72f93eced4
>>> >
>>> >
>>> > The vote will be open for at least 72 hours i.e. until 2020-10-07
>>> 02:00 UTC.
>>> >
>>> >
>>> > [ ] +1  approve
>>> >
>>> > [ ] +0  no opinion
>>> >
>>> > [ ] -1  disapprove (and reason why)
>>> >
>>> >
>>> > Here is my +1
>>>
>>> --
>>> Regards,
>>>
>>> Atri
>>> Apache Concerted
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>>
>
> --
> Adrien
>

Re: Solr Alpha (EA) release of Reference Branch

[Ishan Chattopadhyaya]

To conclude this thread, seems that there's general consensus for a release
around early December timeframe. If there are any outstanding concerns
against the release itself, please chime in.

The naming is something we can discuss separately, as we go along. State of
the branch can be re-evaluated at the time of the release to see if we need
to delay it, it still makes sense to go with the release, or any major
shifts in direction is needed.

Thanks everyone for your suggestions, thoughts and concerns.

On Mon, 5 Oct, 2020, 6:43 pm Ishan Chattopadhyaya, <
ichattopadhy...@gmail.com> wrote:

> IIUC, reference_impl is the branch where the changes are stable. *_dev is
> the branch where active development is going on. Once changes are baked in
> there, they are promoted to reference_impl. We want to release from
> reference_impl.
>
> On Mon, 5 Oct, 2020, 6:16 pm Erick Erickson, 
> wrote:
>
>> Uwe:
>>
>> I think it’s reference_impl_dev...
>>
>> > On Oct 4, 2020, at 6:00 PM, Uwe Schindler  wrote:
>> >
>> > Hi,
>> > I enabled the „gradlew check” runs on ASF Jenkins and Policeman Jenkins
>> (Linux, Windows, MacOS).
>> >
>> > I used branch (“reference_impl”). Is this correct, because it’s about a
>> month old?
>> > There’s also a much newer “reference_impl_dev” branch. Which one is
>> correct?
>> >
>> > I will go sleeping now, sorry for failure mails during the night.
>> Builds seem to fail, but it’s too late to do anything against it.
>> >
>> > Uwe
>> >
>> > -
>> > Uwe Schindler
>> > Achterdiek 19, D-28357 Bremen
>> > https://www.thetaphi.de
>> > eMail: u...@thetaphi.de
>> >
>> > From: Ishan Chattopadhyaya 
>> > Sent: Sunday, October 4, 2020 6:32 AM
>> > To: Uwe Schindler 
>> > Cc: Lucene Dev 
>> > Subject: Re: Solr Alpha (EA) release of Reference Branch
>> >
>> > Yes Uwe, it is ready for Jenkins testing. The Solr tests should run
>> very fast now.
>> >
>> > As for naming, our package manager in Solr will break if we don't
>> specify a major and a minor version. There's a concept of version
>> constraints for packages that need to compare against Solr version. I think
>> release process will also be much simpler if we have a major and minor
>> version. We have done a similar release in the past, 4.0.0-beta iirc.
>> >
>> > Why I favour 10.0-alpha or something like that is because users would
>> clearly know it is something that isn't coming right now (and hence very
>> early access), and it is logically a major version change (that comes after
>> 8x). With calling it 10x instead of 9x, we have the scope of abandoning the
>> effort as a whole if the early access releases have some serious problem or
>> we decide to take some other release strategy later.
>> >
>> > If the 10.0-alpha succeeds, we can always fold the changes back into a
>> 9.1 or go from 9.0 straight to 10.0, depending on what we decide later.
>> Alternatively, if the release looks good and 9.0 hasn't released yet, we
>> can fold those changes back to master, and either (a) release everything
>> normally as 9.0, or (b) call master as 10x and release from there (going
>> from 8.8 to 10.0 directly, skipping 9x altogether).
>> >
>> > Tldr, we shall have complete flexibility to go in any direction we want
>> to.
>> >
>> > WDYT?
>> >
>> > On Sun, 4 Oct, 2020, 2:31 am Uwe Schindler,  wrote:
>> >> Is the branch ready for Jenkins testing?
>> >>
>> >> If yes and "gradlew check" works, I really would like to set it up.
>> >>
>> >> Uwe
>> >>
>> >> Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com>:
>> >>> Hi Devs,
>> >>>
>> >>> As you might be aware, the reference_impl branch has a lot of
>> improvements that we want to see in Solr master. However, it is currently a
>> large deviation from master and hence the stability and reliability (though
>> improved in certain aspects) remains to be tested in real production
>> environments before we gain confidence in bringing those changes to master.
>> >>>
>> >>> I propose that we do a one off preview release from that branch, say
>> Solr 10 alpha (early access) or any other name that someone suggests, so
>> that users could try it out and report regressions or improvements etc.
>> >>>
>> >>> I volunteer to be the RM and planning to start the process around 1
>> December-15 December timeframe. Until then, we can tighten the loose ends
>> on the branch and plan for such a release.
>> >>>
>> >>> Is there any thoughts, concerns, questions?
>> >>>
>> >>> Regards,
>> >>> Ishan
>> >>
>> >> --
>> >> Uwe Schindler
>> >> Achterdiek 19, 28357 Bremen
>> >> https://www.thetaphi.de
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>

BadApple

[Erick Erickson]

We have a 12 tests failing 100% of the time over the last 24 hours. I urge 
folks to take a look at 
http://fucit.org/solr-jenkins-reports/failure-report.html and see if anything 
rings some bells.

Short form:
Raw fail count by week totals, most recent week first (corresponds to bits):
Week: 0  had  153 failures
Week: 1  had  51 failures
Week: 2  had  82 failures
Week: 3  had  94 failures


Failures in Hoss' reports in every one of the last 4 rollups.

There were 279 unannotated tests that failed in Hoss' rollups. Ordered by the 
date I downloaded the rollup file, newest->oldest. See above for the dates the 
files were collected 
These tests were NOT BadApple'd or AwaitsFix'd

Failures in the last 4 reports..
   Report   Pct runsfails   test
 0123   5.1 1821 82  HttpPartitionOnCommitTest.test
 0123   1.0 1776 12  HttpPartitionWithTlogReplicasTest.test
 0123   3.0 1792 37  MultiThreadedOCPTest.test
 0123  50.07  4  SharedFSAutoReplicaFailoverTest.test
 0123   6.8 1872125  TestExportWriter.testExpr
 0123   1.5 1464 19  TestHdfsCloudBackupRestore.test
 0123   0.4 1730  6  TestInPlaceUpdatesDistrib.test
 0123   1.0 1739 20  TestLocalFSCloudBackupRestore.test
 0123   0.6 1745 13  TestSolrConfigHandlerCloud.test


Full report:
DO NOT ENABLE LIST:
MoveReplicaHDFSTest.testFailedMove
MoveReplicaHDFSTest.testNormalFailedMove
TestControlledRealTimeReopenThread.testCRTReopen
TestICUNormalizer2CharFilter.testRandomStrings
TestICUTokenizerCJK
TestImpersonationWithHadoopAuth.testForwarding
TestLTRReRankingPipeline.testDifferentTopN
TestRandomChains


DO NOT ANNOTATE LIST
CdcrBidirectionalTest.testBiDir
IndexSizeTriggerTest.testMergeIntegration
IndexSizeTriggerTest.testMixedBounds
IndexSizeTriggerTest.testSplitIntegration
IndexSizeTriggerTest.testTrigger
InfixSuggestersTest.testShutdownDuringBuild
ShardSplitTest.test
ShardSplitTest.testSplitMixedReplicaTypes
ShardSplitTest.testSplitWithChaosMonkey
Test2BPostings.test
TestLatLonShapeQueries.testRandomBig
TestPackedInts.testPackedLongValues
TestRandomChains.testRandomChainsWithLargeStrings
TestTriggerIntegration.testSearchRate

SuppressWarnings count: last week: 4,528, this week: 4,528, delta 0


*** Files with increased @SuppressWarnings annotations:


*** Files with decreased @SuppressWarnings annotations:


Processing file (History bit 3): HOSS-2020-10-05.csv
Processing file (History bit 2): HOSS-2020-09-28.csv
Processing file (History bit 1): HOSS-2020-09-21.csv
Processing file (History bit 0): HOSS-2020-09-07.csv


Number of AwaitsFix: 31 Number of BadApples: 3


**Annotated tests that didn't fail in the last 4 weeks.

  **Tests removed from the next two lists because they were specified in 
'doNotEnable' in the properties file
 MoveReplicaHDFSTest.testNormalFailedMove

  **Annotations can be removed from the following tests because they haven't 
failed in the last 4 rollups.

  **Methods: 0


Raw fail count by week totals, most recent week first (corresponds to bits):
Week: 0  had  153 failures
Week: 1  had  51 failures
Week: 2  had  82 failures
Week: 3  had  94 failures


Failures in Hoss' reports in every one of the last 4 rollups.

There were 279 unannotated tests that failed in Hoss' rollups. Ordered by the 
date I downloaded the rollup file, newest->oldest. See above for the dates the 
files were collected 
These tests were NOT BadApple'd or AwaitsFix'd

Failures in the last 4 reports..
   Report   Pct runsfails   test
 0123   5.1 1821 82  HttpPartitionOnCommitTest.test
 0123   1.0 1776 12  HttpPartitionWithTlogReplicasTest.test
 0123   3.0 1792 37  MultiThreadedOCPTest.test
 0123  50.07  4  SharedFSAutoReplicaFailoverTest.test
 0123   6.8 1872125  TestExportWriter.testExpr
 0123   1.5 1464 19  TestHdfsCloudBackupRestore.test
 0123   0.4 1730  6  TestInPlaceUpdatesDistrib.test
 0123   1.0 1739 20  TestLocalFSCloudBackupRestore.test
 0123   0.6 1745 13  TestSolrConfigHandlerCloud.test


Failures over the last 4 weeks, but not every week. Ordered most-recent first:



   Report   Pct runsfails   test
 0120.4  674  4  
ComputePlanActionTest.testSelectedCollectionsByName
 0120.2 1335  3  MathExpressionTest.testZipFDistribution
 0120.4 1329  6  RollingRestartTest.test
 01 3   0.2 1259  4  
LBSolrClientTest.testServerIteratorTimeAllowed
 01 3   0.2 1265  3  PeerSyncReplicationTest.test
 01 3   0.4  649 14  

Re: [VOTE] Release Lucene/Solr 8.6.3 RC1

[Adrien Grand]

+1 SUCCESS! [1:36:10.395992]

On Mon, Oct 5, 2020 at 2:15 PM Michael McCandless 
wrote:

> +1 (binding)
>
> SUCCESS! [0:44:16.898412]
>
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Mon, Oct 5, 2020 at 3:28 AM Atri Sharma  wrote:
>
>> +1 (binding)
>>
>> SUCCESS! [1:04.32.39193]
>>
>> On Sun, Oct 4, 2020 at 7:24 AM Jason Gerlowski 
>> wrote:
>> >
>> > Please vote for release candidate 1 for Lucene/Solr 8.6.3
>> >
>> >
>> > The artifacts can be downloaded from:
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.6.3-RC1-reve001c2221812a0ba9e9378855040ce72f93eced4
>> >
>> >
>> > You can run the smoke tester directly with this command:
>> >
>> >
>> > python3 -u dev-tools/scripts/smokeTestRelease.py \
>> >
>> >
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.6.3-RC1-reve001c2221812a0ba9e9378855040ce72f93eced4
>> >
>> >
>> > The vote will be open for at least 72 hours i.e. until 2020-10-07 02:00
>> UTC.
>> >
>> >
>> > [ ] +1  approve
>> >
>> > [ ] +0  no opinion
>> >
>> > [ ] -1  disapprove (and reason why)
>> >
>> >
>> > Here is my +1
>>
>> --
>> Regards,
>>
>> Atri
>> Apache Concerted
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>

-- 
Adrien

Re: CWiki and IDE instructions

[David Smiley]

Moreover, with more and more helpful information showing up in the codebase
itself lately (as Dawid pointed to), I think we should remove corresponding
information in the wiki.  Less to maintain, more/different places of
information can be more confusing.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Mon, Oct 5, 2020 at 2:38 AM Dawid Weiss  wrote:

> > I'm curious if "gradlew idea" is needed.
>
> It is not needed. You should just import gradle project as-is via
> IntelliJ built-in mechanisms.
>
> https://github.com/apache/lucene-solr/blob/master/help/IDEs.txt#L1-L4
>
> D.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Solr Alpha (EA) release of Reference Branch

[Ishan Chattopadhyaya]

IIUC, reference_impl is the branch where the changes are stable. *_dev is
the branch where active development is going on. Once changes are baked in
there, they are promoted to reference_impl. We want to release from
reference_impl.

On Mon, 5 Oct, 2020, 6:16 pm Erick Erickson, 
wrote:

> Uwe:
>
> I think it’s reference_impl_dev...
>
> > On Oct 4, 2020, at 6:00 PM, Uwe Schindler  wrote:
> >
> > Hi,
> > I enabled the „gradlew check” runs on ASF Jenkins and Policeman Jenkins
> (Linux, Windows, MacOS).
> >
> > I used branch (“reference_impl”). Is this correct, because it’s about a
> month old?
> > There’s also a much newer “reference_impl_dev” branch. Which one is
> correct?
> >
> > I will go sleeping now, sorry for failure mails during the night. Builds
> seem to fail, but it’s too late to do anything against it.
> >
> > Uwe
> >
> > -
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > https://www.thetaphi.de
> > eMail: u...@thetaphi.de
> >
> > From: Ishan Chattopadhyaya 
> > Sent: Sunday, October 4, 2020 6:32 AM
> > To: Uwe Schindler 
> > Cc: Lucene Dev 
> > Subject: Re: Solr Alpha (EA) release of Reference Branch
> >
> > Yes Uwe, it is ready for Jenkins testing. The Solr tests should run very
> fast now.
> >
> > As for naming, our package manager in Solr will break if we don't
> specify a major and a minor version. There's a concept of version
> constraints for packages that need to compare against Solr version. I think
> release process will also be much simpler if we have a major and minor
> version. We have done a similar release in the past, 4.0.0-beta iirc.
> >
> > Why I favour 10.0-alpha or something like that is because users would
> clearly know it is something that isn't coming right now (and hence very
> early access), and it is logically a major version change (that comes after
> 8x). With calling it 10x instead of 9x, we have the scope of abandoning the
> effort as a whole if the early access releases have some serious problem or
> we decide to take some other release strategy later.
> >
> > If the 10.0-alpha succeeds, we can always fold the changes back into a
> 9.1 or go from 9.0 straight to 10.0, depending on what we decide later.
> Alternatively, if the release looks good and 9.0 hasn't released yet, we
> can fold those changes back to master, and either (a) release everything
> normally as 9.0, or (b) call master as 10x and release from there (going
> from 8.8 to 10.0 directly, skipping 9x altogether).
> >
> > Tldr, we shall have complete flexibility to go in any direction we want
> to.
> >
> > WDYT?
> >
> > On Sun, 4 Oct, 2020, 2:31 am Uwe Schindler,  wrote:
> >> Is the branch ready for Jenkins testing?
> >>
> >> If yes and "gradlew check" works, I really would like to set it up.
> >>
> >> Uwe
> >>
> >> Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com>:
> >>> Hi Devs,
> >>>
> >>> As you might be aware, the reference_impl branch has a lot of
> improvements that we want to see in Solr master. However, it is currently a
> large deviation from master and hence the stability and reliability (though
> improved in certain aspects) remains to be tested in real production
> environments before we gain confidence in bringing those changes to master.
> >>>
> >>> I propose that we do a one off preview release from that branch, say
> Solr 10 alpha (early access) or any other name that someone suggests, so
> that users could try it out and report regressions or improvements etc.
> >>>
> >>> I volunteer to be the RM and planning to start the process around 1
> December-15 December timeframe. Until then, we can tighten the loose ends
> on the branch and plan for such a release.
> >>>
> >>> Is there any thoughts, concerns, questions?
> >>>
> >>> Regards,
> >>> Ishan
> >>
> >> --
> >> Uwe Schindler
> >> Achterdiek 19, 28357 Bremen
> >> https://www.thetaphi.de
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Solr Alpha (EA) release of Reference Branch

[Erick Erickson]

Uwe:

I think it’s reference_impl_dev...

> On Oct 4, 2020, at 6:00 PM, Uwe Schindler  wrote:
> 
> Hi,
> I enabled the „gradlew check” runs on ASF Jenkins and Policeman Jenkins 
> (Linux, Windows, MacOS).
>  
> I used branch (“reference_impl”). Is this correct, because it’s about a month 
> old?
> There’s also a much newer “reference_impl_dev” branch. Which one is correct?
>  
> I will go sleeping now, sorry for failure mails during the night. Builds seem 
> to fail, but it’s too late to do anything against it.
>  
> Uwe
>  
> -
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: u...@thetaphi.de
>  
> From: Ishan Chattopadhyaya  
> Sent: Sunday, October 4, 2020 6:32 AM
> To: Uwe Schindler 
> Cc: Lucene Dev 
> Subject: Re: Solr Alpha (EA) release of Reference Branch
>  
> Yes Uwe, it is ready for Jenkins testing. The Solr tests should run very fast 
> now.
>  
> As for naming, our package manager in Solr will break if we don't specify a 
> major and a minor version. There's a concept of version constraints for 
> packages that need to compare against Solr version. I think release process 
> will also be much simpler if we have a major and minor version. We have done 
> a similar release in the past, 4.0.0-beta iirc.
>  
> Why I favour 10.0-alpha or something like that is because users would clearly 
> know it is something that isn't coming right now (and hence very early 
> access), and it is logically a major version change (that comes after 8x). 
> With calling it 10x instead of 9x, we have the scope of abandoning the effort 
> as a whole if the early access releases have some serious problem or we 
> decide to take some other release strategy later.
>  
> If the 10.0-alpha succeeds, we can always fold the changes back into a 9.1 or 
> go from 9.0 straight to 10.0, depending on what we decide later. 
> Alternatively, if the release looks good and 9.0 hasn't released yet, we can 
> fold those changes back to master, and either (a) release everything normally 
> as 9.0, or (b) call master as 10x and release from there (going from 8.8 to 
> 10.0 directly, skipping 9x altogether).
>  
> Tldr, we shall have complete flexibility to go in any direction we want to.
>  
> WDYT?
>  
> On Sun, 4 Oct, 2020, 2:31 am Uwe Schindler,  wrote:
>> Is the branch ready for Jenkins testing?
>> 
>> If yes and "gradlew check" works, I really would like to set it up.
>> 
>> Uwe
>> 
>> Am October 3, 2020 7:42:22 PM UTC schrieb Ishan Chattopadhyaya 
>> :
>>> Hi Devs,
>>>  
>>> As you might be aware, the reference_impl branch has a lot of improvements 
>>> that we want to see in Solr master. However, it is currently a large 
>>> deviation from master and hence the stability and reliability (though 
>>> improved in certain aspects) remains to be tested in real production 
>>> environments before we gain confidence in bringing those changes to master.
>>>  
>>> I propose that we do a one off preview release from that branch, say Solr 
>>> 10 alpha (early access) or any other name that someone suggests, so that 
>>> users could try it out and report regressions or improvements etc.
>>>  
>>> I volunteer to be the RM and planning to start the process around 1 
>>> December-15 December timeframe. Until then, we can tighten the loose ends 
>>> on the branch and plan for such a release.
>>>  
>>> Is there any thoughts, concerns, questions?
>>>  
>>> Regards,
>>> Ishan
>> 
>> --
>> Uwe Schindler
>> Achterdiek 19, 28357 Bremen
>> https://www.thetaphi.de


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [VOTE] Release Lucene/Solr 8.6.3 RC1

[Michael McCandless]

+1 (binding)

SUCCESS! [0:44:16.898412]


Mike McCandless

http://blog.mikemccandless.com


On Mon, Oct 5, 2020 at 3:28 AM Atri Sharma  wrote:

> +1 (binding)
>
> SUCCESS! [1:04.32.39193]
>
> On Sun, Oct 4, 2020 at 7:24 AM Jason Gerlowski 
> wrote:
> >
> > Please vote for release candidate 1 for Lucene/Solr 8.6.3
> >
> >
> > The artifacts can be downloaded from:
> >
> >
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.6.3-RC1-reve001c2221812a0ba9e9378855040ce72f93eced4
> >
> >
> > You can run the smoke tester directly with this command:
> >
> >
> > python3 -u dev-tools/scripts/smokeTestRelease.py \
> >
> >
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.6.3-RC1-reve001c2221812a0ba9e9378855040ce72f93eced4
> >
> >
> > The vote will be open for at least 72 hours i.e. until 2020-10-07 02:00
> UTC.
> >
> >
> > [ ] +1  approve
> >
> > [ ] +0  no opinion
> >
> > [ ] -1  disapprove (and reason why)
> >
> >
> > Here is my +1
>
> --
> Regards,
>
> Atri
> Apache Concerted
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Solr Alpha (EA) release of Reference Branch

[Shawn Heisey]

On 10/3/2020 1:42 PM, Ishan Chattopadhyaya wrote:
As you might be aware, the reference_impl branch has a lot of 
improvements that we want to see in Solr master. However, it is 
currently a large deviation from master and hence the stability and 
reliability (though improved in certain aspects) remains to be tested in 
real production environments before we gain confidence in bringing those 
changes to master.


I propose that we do a one off preview release from that branch, say 
Solr 10 alpha (early access) or any other name that someone suggests, so 
that users could try it out and report regressions or improvements etc.


How to handle this seems to come down to the answers to a couple of 
questions:


* Is this new code stable enough to work in the wild?
* Do we want to release 9.0 before we merge to master, or after?

Can someone who was involved when 4.0-ALPHA and 4.0-BETA were released 
tell me whether those releases actually did what was intended and made 
4.0.0 better than it would have been without the special releases?  If 
they did, then a special release for this new implementation before 
merging to master would probably also be helpful.  If there was no real 
benefit gained, then maybe we're better off just going ahead with the merge.


If the general feeling is that this new release is looking very solid, 
then I think we should merge to master soon, probably just before 
branch_9x is created.  If we think it needs more work, then maybe we 
should hold on merging until *after* branch_9x is created, so the new 
implementation will release with 10.0 and there will be more time to 
make it bulletproof.


My current impression, which I will admit is made with almost zero 
actual information about the code or its stability, is that we should 
plan on stabilizing the new implementation for the 9.0 release.  There's 
going to be some pain no matter how we handle this, so diving in now 
seems like a good idea.


A tangent:  How robust is our testing?  I know they take a long time to 
run, which Mark's new implementation aims to improve, but do we think 
there's enough being tested?  There has been some discussion in the past 
about benchmarks for Solr.  Benchmarks would be very useful, but I'm 
more interested in making sure that our tests will catch problems before 
they get out into the wild.  The two goals don't have to be mutually 
exclusive, though.


Thanks,
Shawn

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [VOTE] Release Lucene/Solr 8.6.3 RC1

[Atri Sharma]

+1 (binding)

SUCCESS! [1:04.32.39193]

On Sun, Oct 4, 2020 at 7:24 AM Jason Gerlowski  wrote:
>
> Please vote for release candidate 1 for Lucene/Solr 8.6.3
>
>
> The artifacts can be downloaded from:
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.6.3-RC1-reve001c2221812a0ba9e9378855040ce72f93eced4
>
>
> You can run the smoke tester directly with this command:
>
>
> python3 -u dev-tools/scripts/smokeTestRelease.py \
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.6.3-RC1-reve001c2221812a0ba9e9378855040ce72f93eced4
>
>
> The vote will be open for at least 72 hours i.e. until 2020-10-07 02:00 UTC.
>
>
> [ ] +1  approve
>
> [ ] +0  no opinion
>
> [ ] -1  disapprove (and reason why)
>
>
> Here is my +1

-- 
Regards,

Atri
Apache Concerted

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [VOTE] Release Lucene/Solr 8.6.3 RC1

[Andrzej Białecki]

+1

SUCCESS! [1:40:38.242493]


> On 4 Oct 2020, at 03:53, Jason Gerlowski  wrote:
> 
> Please vote for release candidate 1 for Lucene/Solr 8.6.3
> 
> The artifacts can be downloaded from:
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.6.3-RC1-reve001c2221812a0ba9e9378855040ce72f93eced4
>  
> 
> 
> You can run the smoke tester directly with this command:
> 
> python3 -u dev-tools/scripts/smokeTestRelease.py \
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.6.3-RC1-reve001c2221812a0ba9e9378855040ce72f93eced4
>  
> 
> 
> The vote will be open for at least 72 hours i.e. until 2020-10-07 02:00 UTC.
> 
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
> 
> Here is my +1

Re: CWiki and IDE instructions

[Dawid Weiss]

> I'm curious if "gradlew idea" is needed.

It is not needed. You should just import gradle project as-is via
IntelliJ built-in mechanisms.

https://github.com/apache/lucene-solr/blob/master/help/IDEs.txt#L1-L4

D.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org