[VOTE] Release Lucene 9.1.0 RC1

[Julie Tibshirani]

Please vote for release candidate 1 for Lucene 9.1.0

The artifacts can be downloaded from:
https://dist.apache.org/repos/dist/dev/lucene/lucene-9.1.0-RC1-rev-a6114b532a273e370528675d551d3ddfa02f4679

You can run the smoke tester directly with this command:

python3 -u dev-tools/scripts/smokeTestRelease.py \
https://dist.apache.org/repos/dist/dev/lucene/lucene-9.1.0-RC1-rev-a6114b532a273e370528675d551d3ddfa02f4679

The vote will be open for at least 72 hours i.e. until 2022-03-18 00:00 UTC.

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Here is my +1.

Julie

Re: Performance Engineering Track at ApacheCon NA?

[Michael McCandless]

Hi Sharan,

I think this is indeed a very interesting topic and would make a good
ApacheCon track!  It's a great idea.

In Lucene development we struggle with proper performance measurement
often.  We have a set of external benchmarking tools (
https://github.com/mikemccand/luceneutil ) for this purpose but they are
complex and tricky to set up and use.  Java has noisy performance from JVM
instance to instance, further complicating things.

Drawing attention to this problem and sharing ideas would be really
helpful, not just for Lucene but other complex Java projects.

I would most likely be able to give a talk about how we
approach Performance Engineering in Lucene, but probably don't have
enough bandwidth to help organize/run the full track.

Thanks,

Mike McCandless

http://blog.mikemccandless.com

On Fri, Mar 11, 2022 at 5:17 PM kujira m  wrote:

> I'd like to unsubscribe the newsletter.
>
> 2022年3月11日(金) 22:09 sharanf :
>
>> Hi All
>>
>> The call for tracks for ApacheCon NA is open. There is a suggestion to
>> try and run a Performance Engineering track at ApacheCon. At the end of
>> the message I have included some details including a definition of what
>> we mean by it and some reasoning about why it could be good to run. We
>> have a list of projects that have something to do with performance
>> engineering and if you take a look -  you will see that this project is
>> on the list!
>>
>> So what I need is some feedback as to whether the community thinks that
>> this could be an interesting track topic to run at ApacheCon..and more
>> importantly would the community be willing to submit talks for it or
>> attend ApacheCon to see it.
>>
>> Like I say - this is just an idea at this stage. If the Performance
>> Engineering track does get approval to be included at ApacheCon  - do we
>> have any volunteers willing to help with managing and promoting the
>> track on behalf of the project?
>>
>> Thanks
>> Sharan
>>
>> -
>>
>> *Performance Engineering*  is the science and practice of engineering
>> software with the required performance and scalability characteristics.
>> Many Apache projects focus on solving hard Big Data performance and
>> scalability challenges, while others provide tools for performance
>> engineering - but there are few projects that don’t care about some
>> aspect of software performance.
>>
>> This track will enable Apache projects members to share their
>> experiences of performance engineering best practices, tools,
>> techniques, and results, from their own communities, with the benefits
>> of cross-fertilization between projects. Performance Engineering in the
>> wider open source community is pervasive and includes methods and tools
>> (including automation and agile approaches) for performance:
>> architecting and design, benchmarking, monitoring, tracing, analysis,
>> prediction, modeling and simulation, testing and reporting, regression
>> testing, and source code analysis and instrumentation techniques.
>>
>> Performance Engineering also has wider applicability to DevOps, the
>> operation of cloud platforms by managed service providers (hence some
>> overlap with SRE - Site Reliability Engineering), and customer
>> application performance and tuning.  This track would therefore be
>> applicable to the wider open source community.
>>
>> *SUPPORTING DETAILS*
>>
>> *Google Searches*
>> Google “Open source performance engineering” has 4,180,000,000 results
>> Google “site:apache.org  performance” has 147,000
>> results
>>
>> *Apache Projects *which may have some interest in, or focus on,
>> performance (just the top results):
>> JMeter, Cassandra, Storm, Spark, Samza, Pulsar, Kafka, Log4J, SystemML,
>> Drill, HTTP Server, Cayenne, ActiveMQ, Impala, Geode, Flink, Ignite,
>> Impala, Lucene, TVM, Tika, YuniKorn, Solr, Iceberg, Dubbo, Hudi,
>> Accumulo, Xerces, MXNet, Zookeeper
>>
>> *Incubator projects *which may have some interest in, or focus on,
>> performance**(again just top results):
>> Crail, Eagle, Nemo, Skywalking, MXnet, HAWQ, Mnemonic, CarbonData,
>> Drill, ShenYu, Tephra, Sedona
>>
>> *References *(randomly selected to show the range of open-source
>> performance engineering topics available, rather than the quality of
>> articles):
>>
>>   1. Performance Engineering for Apache Spark and Databricks Runtime
>>  ETHZ, Big Data HS19
>>  <
>> https://archive-systems.ethz.ch/sites/default/files/courses/2019-fall/bigdata/Databricks%20ETHZ%20Big%20Data%20HS19.pdf
>> >
>>   2. Real time insights into LinkedIn's performance using Apache Samza
>>  <
>> https://engineering.linkedin.com/samza/real-time-insights-linkedins-performance-using-apache-samza
>> >
>>   3. A day in the life of an open source performance engineering team
>>  
>>   4. Locating Performance Regression Root Causes in the Field Operations
>>  

Re: [JENKINS] Lucene-9.x-MacOSX (64bit/jdk-17.0.2) - Build # 294 - Failure!

[Dawid Weiss]

Ah, thanks for clarifying, Uwe.

On Mon, Mar 14, 2022 at 10:28 AM Uwe Schindler  wrote:

> Hi Dawid,
>
>
>
> this is a problem sometimes happens on the MacOS builds (all BSD builds,
> Solaris had same problem – now deactivated). It looks like the JDK hangs in
> some deadlock. This MAY be caused by VirtualBOX, it is unknown. Previously
> the builds were hanging forever and I had to kill them manually, but since
> a few weeks I applied the maximum build time (there is a setting in jenkins
> to kill a build if it takes “significantly longer than previous builds”,
> which fits very well). When I killed the builds manually, no mail was sent
> so you havent seen this. Happens approximately once per week.
>
>
>
> Uwe
>
>
>
> -
>
> Uwe Schindler
>
> Achterdiek 19, D-28357 Bremen
>
> https://www.thetaphi.de
>
> eMail: u...@thetaphi.de
>
>
>
> *From:* Dawid Weiss 
> *Sent:* Monday, March 14, 2022 8:49 AM
> *To:* Lucene Dev 
> *Cc:* bui...@lucene.apache.org; Uwe Schindler (SD DataSolutions GmbH) <
> u...@thetaphi.de>
> *Subject:* Re: [JENKINS] Lucene-9.x-MacOSX (64bit/jdk-17.0.2) - Build #
> 294 - Failure!
>
>
>
>
>
> I'm not sure what's happening here - the gradle part finishes but the
> build is stalled after that and ultimately expires. Weird.
>
> BUILD SUCCESSFUL in 19m 4s
>
> 798 actionable tasks: 798 executed
>
> Build timed out (after 209 minutes). Marking the build as aborted.
>
> Build timed out (after 209 minutes). Marking the build as failed.
>
> Build was aborted
>
> Archiving artifacts
>
> Recording test results
>
> [Checks API] No suitable checks publisher found.
>
> Email was triggered for: Failure - Any
>
> Sending email for trigger: Failure - Any
>
> Sending email to: bui...@lucene.apache.org
>
> Finished: FAILURE
>
>
>
> On Mon, Mar 14, 2022 at 5:34 AM Policeman Jenkins Server <
> jenk...@thetaphi.de> wrote:
>
> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-MacOSX/294/
> Java: 64bit/jdk-17.0.2 -XX:-UseCompressedOops -XX:+UseParallelGC
>
> All tests passed
>
> -
> To unsubscribe, e-mail: builds-unsubscr...@lucene.apache.org
> For additional commands, e-mail: builds-h...@lucene.apache.org
>
>

RE: [JENKINS] Lucene-9.x-MacOSX (64bit/jdk-17.0.2) - Build # 294 - Failure!

[Uwe Schindler]

Hi Dawid,

 

this is a problem sometimes happens on the MacOS builds (all BSD builds, 
Solaris had same problem – now deactivated). It looks like the JDK hangs in 
some deadlock. This MAY be caused by VirtualBOX, it is unknown. Previously the 
builds were hanging forever and I had to kill them manually, but since a few 
weeks I applied the maximum build time (there is a setting in jenkins to kill a 
build if it takes “significantly longer than previous builds”, which fits very 
well). When I killed the builds manually, no mail was sent so you havent seen 
this. Happens approximately once per week.

 

Uwe

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Dawid Weiss  
Sent: Monday, March 14, 2022 8:49 AM
To: Lucene Dev 
Cc: bui...@lucene.apache.org; Uwe Schindler (SD DataSolutions GmbH) 

Subject: Re: [JENKINS] Lucene-9.x-MacOSX (64bit/jdk-17.0.2) - Build # 294 - 
Failure!

 

 

I'm not sure what's happening here - the gradle part finishes but the build is 
stalled after that and ultimately expires. Weird.

BUILD SUCCESSFUL in 19m 4s
798 actionable tasks: 798 executed
Build timed out (after 209 minutes). Marking the build as aborted.
Build timed out (after 209 minutes). Marking the build as failed.
Build was aborted
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending email to: bui...@lucene.apache.org  
Finished: FAILURE

 

On Mon, Mar 14, 2022 at 5:34 AM Policeman Jenkins Server mailto:jenk...@thetaphi.de> > wrote:

Build: https://jenkins.thetaphi.de/job/Lucene-9.x-MacOSX/294/
Java: 64bit/jdk-17.0.2 -XX:-UseCompressedOops -XX:+UseParallelGC

All tests passed

-
To unsubscribe, e-mail: builds-unsubscr...@lucene.apache.org 
 
For additional commands, e-mail: builds-h...@lucene.apache.org 
 

Re: Generate autocomplete predictions

[Michael Wechner]

Hi Adrien

Ok :-)

I think I will try to do a very rough prototype first, just to get a 
better idea and then use for discussion in JIRA.


Thanks

Michael

Am 14.03.22 um 08:19 schrieb Adrien Grand:

Hey Michael,

I like discussing ideas in JIRA first, but sometimes attaching a rough 
prototype can help show how things tie together. I guess the thing you 
want to avoid is to spend hours on the prototype but otherwise either 
is fine.


Le dim. 13 mars 2022, 23:01, Michael Wechner 
 a écrit :


Hi Adrien

Thanks for your feedback!

 From a "project management" point of view how do I best do this?

Should I just create a Pull Request with a first prototype, or
discuss
the design first in Jira tickets?

Thanks

Michael



Am 13.03.22 um 21:52 schrieb Adrien Grand:
> Hi Michael,
>
> This sounds like a good fit for Lucene to me.
>
> On Fri, Mar 11, 2022 at 11:15 PM Michael Wechner
>  wrote:
>> Hi
>>
>> I recently implemened auto-suggest based on
>>
>> https://lucene.apache.org/core/9_0_0/suggest/index.html
>>
>> whereas I am currently managing the terms / predictions (e.g.
>> "autocompletion using lucene suggesters dev") contained by the
index
>> manually.
>>
>> I would like now to generate the terms / predictions more
automatically,
>> similar to what Google does
>>
>>

https://blog.google/products/search/how-google-autocomplete-predictions-work/
>>
>> Does Lucene provide code to analyze queries in order to
generate terms /
>> predictions for an auto-suggest index?
>>
>> If not, would it make sense to contribute this kind of
functionality to
>> Lucene or should this be rather a third-party library?
>>
>> Thanks
>>
>> Michael
>>
>>
-
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-9.x-MacOSX (64bit/jdk-17.0.2) - Build # 294 - Failure!

[Dawid Weiss]

I'm not sure what's happening here - the gradle part finishes but the build
is stalled after that and ultimately expires. Weird.

BUILD SUCCESSFUL in 19m 4s
798 actionable tasks: 798 executed
Build timed out (after 209 minutes). Marking the build as aborted.
Build timed out (after 209 minutes). Marking the build as failed.
Build was aborted
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending email to: bui...@lucene.apache.org
Finished: FAILURE


On Mon, Mar 14, 2022 at 5:34 AM Policeman Jenkins Server <
jenk...@thetaphi.de> wrote:

> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-MacOSX/294/
> Java: 64bit/jdk-17.0.2 -XX:-UseCompressedOops -XX:+UseParallelGC
>
> All tests passed
>
> -
> To unsubscribe, e-mail: builds-unsubscr...@lucene.apache.org
> For additional commands, e-mail: builds-h...@lucene.apache.org

Re: Generate autocomplete predictions

[Adrien Grand]

Hey Michael,

I like discussing ideas in JIRA first, but sometimes attaching a rough
prototype can help show how things tie together. I guess the thing you want
to avoid is to spend hours on the prototype but otherwise either is fine.

Le dim. 13 mars 2022, 23:01, Michael Wechner  a
écrit :

> Hi Adrien
>
> Thanks for your feedback!
>
>  From a "project management" point of view how do I best do this?
>
> Should I just create a Pull Request with a first prototype, or discuss
> the design first in Jira tickets?
>
> Thanks
>
> Michael
>
>
>
> Am 13.03.22 um 21:52 schrieb Adrien Grand:
> > Hi Michael,
> >
> > This sounds like a good fit for Lucene to me.
> >
> > On Fri, Mar 11, 2022 at 11:15 PM Michael Wechner
> >  wrote:
> >> Hi
> >>
> >> I recently implemened auto-suggest based on
> >>
> >> https://lucene.apache.org/core/9_0_0/suggest/index.html
> >>
> >> whereas I am currently managing the terms / predictions (e.g.
> >> "autocompletion using lucene suggesters dev") contained by the index
> >> manually.
> >>
> >> I would like now to generate the terms / predictions more automatically,
> >> similar to what Google does
> >>
> >>
> https://blog.google/products/search/how-google-autocomplete-predictions-work/
> >>
> >> Does Lucene provide code to analyze queries in order to generate terms /
> >> predictions for an auto-suggest index?
> >>
> >> If not, would it make sense to contribute this kind of functionality to
> >> Lucene or should this be rather a third-party library?
> >>
> >> Thanks
> >>
> >> Michael
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >> For additional commands, e-mail: dev-h...@lucene.apache.org
> >>
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>