Order docIds to reduce disk seeks

*Could someone point me how to order docIds as per **http://wiki.apache.org/lucene-java/ImproveSearchingSpeed * *"Limit usage of stored fields and term vectors. Retrieving these from the index is quite costly. Typically you should only retr

Re: Order docIds to reduce disk seeks

a comparator to sort the elements > in the ScoreDoc[] by their doc field then that will put them in 'docID > order' and the reader will always be skipping forward to the next doc which > will probably reduce its seek time. > > Regards, > Stuart > > > > -Ori

Re: Order docIds to reduce disk seeks

' > the document from the index. If you write a comparator to sort the elements > in the ScoreDoc[] by their doc field then that will put them in 'docID > order' and the reader will always be skipping forward to the next doc which > will probably reduce its seek time. &g

Re: Order docIds to reduce disk seeks

your text processing step does? > > Barry > > On Nov 18, 2014 7:41 PM, "Vijay B" wrote: > > > > Hi Mike, could you provide some pointers on using inverted index. Any > > examples or what API classes to use to accomplish this. > > > >

Re: Order docIds to reduce disk seeks

tored fields. I'm not sure > how to do this with the API however. > > Barry > > On Tue, Nov 18, 2014 at 8:11 PM, Vijay B wrote: > > > Hi Barry, > > > > here is our usecase. We fetch doc text from lucene and feed it to > > http://carrotsearch.com/ libar

Index replication strategy

Hello, We index docs coming from database nightly. Current index is sitting on NFS. Due to obvious performance reasons, we are switching are planning to switch to local index. W have cluster of 4 servers and with NFS it was not a problem for us until now to share the index. but going forward, we a

Re: Index replication strategy

As indicated in my post, we use Lucene 4.2.1. On Thu, Dec 4, 2014 at 9:29 AM, Shai Erera wrote: > Do you use Lucene or Solr? Lucene also has a replication module, which will > allow you to replicate index changes. > > On Thu, Dec 4, 2014 at 4:19 PM, Vijay B wrote: > > > He

Re: Index replication strategy

rera.blogspot.com/2013/05/the-replicator.html. > > Shai > > On Thu, Dec 4, 2014 at 4:36 PM, Vijay B wrote: > > > As indicated in my post, we use Lucene 4.2.1. > > > > On Thu, Dec 4, 2014 at 9:29 AM, Shai Erera wrote: > > > > > Do you use Lucene or S

unsafe memory access operation

we are running into this issue for the first time while searching a lucene index hosted on NFS. Once, this error occurs, all subsequent requests fails. help appreciated. Our Config: 64-bit Java 1.7 (we are making use of MMapDirectory) Lucene 4.2.1 Index size: 8GB Let me know if you need any f

Re: unsafe memory access operation

/ update level are you using? Those errors occur > easily if you use an outdated JDK version. > > Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > -Original Message- > > From: Vi

Re: unsafe memory access operation

VM (build 23.25-b01, mixed mode) On Fri, Dec 12, 2014 at 12:51 PM, Vijay B wrote: > > Here is the info you requested: > > java version "1.7.0_25" > Java(TM) SE Runtime Environment (build 1.7.0_25-b15) > Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode) &g

Re: unsafe memory access operation

for the error you see here. > > I would recommend to update to 1.7.0_72 or 1.8.0_25. > > Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > > -Original Message- > > F

MMapdirectory

> > Finally we are seeing great improvement once we switch to 64-bit java and > MMapDirectory. Our Test run (multiple requests) used to take 26 minutes on > 32-bit and is now improved to 10 minutes on 64-bit java. > > We load stored documents from lucene and pass the documents to a third > party li

Benchmark testing Lucene index

We have our index located on NFS. While benchmark testing, we noticed first query would take lot of time and same query for the second time complete quickly. One of the reason for this could be fscache. To eliminate effect of caching, before start of we plan to umount and mount the NFS filesystem o

MMapdirectory Configuration

Finally we are seeing great improvement once we switch to 64-bit java and MMapDirectory. Our Test run (multiple requests) used to take 26 minutes on 32-bit and is now improved to 10 minutes on 64-bit java. We load stored documents from lucene and pass the documents to a third party libray (closed

Lucene improvements

We our production app uses Lucene 4.2.1. We are going to release a new version soon and contemplating whether or not to migrate to newer version of lucene. I see lot of new features were added in later versions but I want to get an idea of performance improvements?

re-mapping lucene index

We use MMapdirectory impl. in our search application. Occasionally we need to do a full indexing by dropping entire directory contents. How does re-mapping work with MMapDirectory as the directory contents are going to replace with new ones? is this going to be seamless or an application restart re

Re: re-mapping lucene index

t; > On Tue, Feb 10, 2015 at 1:31 PM, Vijay B wrote: > > We use MMapdirectory impl. in our search application. Occasionally we > need > > to do a full indexing by dropping entire directory contents. How does > > re-mapping work with MMapDirectory as the directory contents

Re: re-mapping lucene index

if > you still have readers open on the index. > > Regardless of which approach you use, the old mappings will remain > "alive" until you've closed all open readers agains the old index. > > Mike McCandless > > http://blog.mikemccandless.com > > > On Tue,