Re: forceMerge(1) leads to ~10% perf gains

2023-09-30 Thread Ishan Chattopadhyaya
Also, try index sorting. Often, there are performance gains to be had with the right sort key for various query workloads. On Fri, 22 Sept, 2023, 4:28 pm Adrien Grand, wrote: > > Was wondering - are there any other techniques which can be used to speed > up that work well when forceMerge works l

[ANNOUNCE] Apache Lucene 8.4.1 released

2020-01-14 Thread Ishan Chattopadhyaya
## 13 January 2020, Apache Lucene™ 8.4.1 available The Lucene PMC is pleased to announce the release of Apache Lucene 8.4.1. Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires

[ANNOUNCE] Apache Lucene 8.3.1 released

2019-12-03 Thread Ishan Chattopadhyaya
## 3 December 2019, Apache Lucene™ 8.3.1 available The Lucene PMC is pleased to announce the release of Apache Lucene 8.3.1. Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires

[ANNOUNCE] Apache Lucene 8.3.0 released

2019-11-02 Thread Ishan Chattopadhyaya
## 2 November 2019, Apache Lucene™ 8.3.0 available The Lucene PMC is pleased to announce the release of Apache Lucene 8.3.0. Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires

[ANNOUNCE] Apache Lucene 8.1.0 released

2019-05-16 Thread Ishan Chattopadhyaya
16 March 2019, Apache Lucene™ 8.1.0 available The Lucene PMC is pleased to announce the release of Apache Lucene 8.1.0. Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full

Re: CVE-2018-11802: Apache Solr authorization bug vulnerability disclosure

2019-04-24 Thread Ishan Chattopadhyaya
This fix has also been backported to Solr 6.6.6 for users who are stuck with Solr 6.x. (Sorry, I hadn't updated the issue and hence this was missed in the original mail.) On Wed, Apr 24, 2019 at 12:35 PM Noble Paul wrote: > > CVE-2018-11802: Apache Solr authorization bug disclosure > Severity: I

[ANNOUNCE] Apache Lucene 6.6.6 released

2019-04-05 Thread Ishan Chattopadhyaya
5 April 2019, Apache Lucene™ 6.6.6 available The Lucene PMC is pleased to announce the release of Apache Lucene 6.6.6. Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-

Apache Lucene 7.7.1 released

2019-02-28 Thread Ishan Chattopadhyaya
= 1 March 2019, Apache Lucene™ 7.7.1 available = The Lucene PMC is pleased to announce the release of Apache Lucene 7.7.1. Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires f

Re: Grant Ingersoll's 2009 blog article- is there a newer version?

2018-07-05 Thread Ishan Chattopadhyaya
Try these, maybe? https://lucidworks.com/2017/09/14/solr-payloads/ http://www.textsearch.io/?p=5 On Thu, Jul 5, 2018 at 8:26 PM, wrote: > Hi,- > Is there a newer version of this great article from Mr. Grant Ingersoll? > > https://lucidworks.com/2009/08/05/getting-started-with-payloads/ Thanks

[ANNOUNCE] Apache Lucene 6.6.5 released

2018-07-03 Thread Ishan Chattopadhyaya
03 July 2018, Apache Lucene™ 6.6.5 available The Lucene PMC is pleased to announce the release of Apache Lucene 6.6.5. Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-

[ANNOUNCE] Apache Lucene 6.6.4 released

2018-05-18 Thread Ishan Chattopadhyaya
18 May 2018, Apache Lucene™ 6.6.4 available The Lucene PMC is pleased to announce the release of Apache Lucene 6.6.4. Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-t

[ANNOUNCE] Apache Lucene 6.6.4 released

2018-05-18 Thread Ishan Chattopadhyaya
18 May 2018, Apache Lucene™ 6.6.4 available The Lucene PMC is pleased to announce the release of Apache Lucene 6.6.4. Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-t

[ANNOUNCE] Apache Lucene 6.6.2 released

2017-10-18 Thread Ishan Chattopadhyaya
18 October 2017, Apache Lucene™ 6.6.2 available The Lucene PMC is pleased to announce the release of Apache Lucene 6.6.2. Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires fu

Re: Re: What is the fastest way to loop over all documents in an index?

2017-09-05 Thread Ishan Chattopadhyaya
; for (int i=0; i > How can we confirm that internal Lucene IDs are subsequent numbers from 0 > to maxDoc()-1? > > I thought that they are arbitrary integers. > > Ahmet > > > > > On Tuesday, September 5, 2017, 7:54:31 AM GMT+3, Ishan Chattopadhyaya < > ichattopad

Re: What is the fastest way to loop over all documents in an index?

2017-09-04 Thread Ishan Chattopadhyaya
Maybe IndexReader#document(), looping over docids is the best here? http://lucene.apache.org/core/6_6_0/core/org/apache/lucene/index/IndexReader.html#document-int- On Tue, Sep 5, 2017 at 7:57 AM, Ahmet Arslan wrote: > Hi Jean, > > I am also interested answers to this question. I need this featur

Re: Encryption at lucene index

2017-08-07 Thread Ishan Chattopadhyaya
Harry Ochiai (Hitachi) has some index encryption solution, https://www.slideshare.net/maggon/securing-solr-search-data-in-the-cloud I think it is proprietary, but I'm not sure. Maybe more googling might help find the exact page where his solution is described. On Mon, Aug 7, 2017 at 9:59 PM, Kumar

[ANNOUNCE] Apache Solr 6.6.0 released

2017-06-06 Thread Ishan Chattopadhyaya
6 June 2017, Apache Solr 6.6.0 availableSolr is the popular, blazing fast, open source NoSQL search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted search and analytics, rich document parsing, geospatial search, extensive REST

[ANNOUNCE] Apache Lucene 6.6.0 released

2017-06-06 Thread Ishan Chattopadhyaya
6 June 2017, Apache Lucene™ 6.6.0 available The Lucene PMC is pleased to announce the release of Apache Lucene 6.6.0. Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-te

Re: [ANNOUNCE] Apache Solr 6.4.2 released

2017-03-08 Thread Ishan Chattopadhyaya
March 2017 at 01:00, Ishan Chattopadhyaya > wrote: > > > 7 March 2017, Apache Solr 6.4.2 available > > > > Solr is the popular, blazing fast, open source NoSQL search platform from > > the Apache Lucene project. Its major features include powerful full-text > > sear

RE: [ANNOUNCE] Apache Solr 6.4.2 released

2017-03-08 Thread Ishan Chattopadhyaya
l link redirects to http://www.apache.org/dyn/closer.lua/lucene/solr/6.4.1 and not http://www.apache.org/dyn/closer.lua/lucene/solr/6.4.2​ On 8 March 2017 at 01:00, Ishan Chattopadhyaya wrote: > 7 March 2017, Apache Solr 6.4.2 available > > Solr is the popular, blazing fast, open source NoSQL search

[ANNOUNCE] Apache Solr 6.4.2 released

2017-03-07 Thread Ishan Chattopadhyaya
7 March 2017, Apache Solr 6.4.2 available Solr is the popular, blazing fast, open source NoSQL search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted search and analytics, rich document parsing, geospatial search, extensive R

[ANNOUNCE] Apache Lucene 6.4.2 released

2017-03-07 Thread Ishan Chattopadhyaya
7 March 2017, Apache Lucene™ 6.4.2 available The Lucene PMC is pleased to announce the release of Apache Lucene 6.4.2 Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-t

Re: Possible to cause documents to be contiguous after forceMerge?

2016-11-16 Thread Ishan Chattopadhyaya
http://shaierera.blogspot.com/2013/04/index-sorting-with-lucene.html On Wed, Nov 16, 2016 at 11:15 AM, Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > Can IndexSort help here? > -- > From: Erick Erickson > Sent: ‎11/‎16/‎2016 9:

RE: Possible to cause documents to be contiguous after forceMerge?

2016-11-15 Thread Ishan Chattopadhyaya
Can IndexSort help here? -Original Message- From: "Erick Erickson" Sent: ‎11/‎16/‎2016 9:29 To: "java-user" Subject: Re: Possible to cause documents to be contiguous after forceMerge? Well, codecs are pluggable so if you can show that you'd get an improvement (however you measure them)

Obtain list of indexed fields from IndexWriter

2016-07-01 Thread Ishan Chattopadhyaya
Hi, If an update to a non-existent dv field is attempted, IndexWriter throws an exception: "can only update existing numeric-docvalues fields!". This exception is thrown after checking with the globalFieldNumberMap (which is obtained from the SegmentInfos). Is there a way, given an IndexWriter in

Re: Lucene indexing throughput (and Mike's lucenebench charts)

2016-04-14 Thread Ishan Chattopadhyaya
Wow, 72 cores? That sounds astounding. Are they dual Xeon E5 2699 v3 CPUs with 18 cores each, with hyperthreading = 18*2*2=72 threads? On Thu, Apr 14, 2016 at 11:33 PM, Dawid Weiss wrote: > The GC change is after this: > > BJ (2015-12-02): Upgrade to beast2 (72 cores, 256 GB RAM) > > which leads

Re: Jira issue for possibly transient resource issue, or a Lucene or JVM bug?

2016-01-21 Thread Ishan Chattopadhyaya
LUCENE-6970 On Thu, Jan 21, 2016 at 4:07 PM, Markus Jelsma wrote: > Hi - we get the above issue as well some times. I've noticed Lucene-dev > mails on this issue [1] but i couldn't find a corresponding Jira issue? Any > pointer to that one? > > Many thanks, > Markus > > [1] > http://mail-archive

Term query equivalent in Dimensional fields?

2015-12-27 Thread Ishan Chattopadhyaya
Hi, (Mike?) I am exploring Dimensional fields and excited about the potential speedups and improved efficiency. I have a DimensionalIntField (one dimensional) indexed, and wish to do something equivalent to a Term query for a particular value. Is the following the best way? I want to do: new Term