Thanks for the suggestion, Jian Chen's idea is very similar too. Probably optimizing that often is not necessary and not that critical for speeding up the searches.
I'll try changing the index process not to optimize at all and execute the optimization independently of the indexing on a weekly bases. Ross -----Original Message----- From: Dan Armbrust [mailto:[EMAIL PROTECTED] Sent: Thursday, June 02, 2005 11:10 AM To: java-user@lucene.apache.org Subject: Re: how long should optimizing take I would run your optimize process in a separate thread, so that your web client doesn't have to wait for it to return. You may even want to set the optimize part up to run on a weekly schedule, at a low load time. I probably wouldn't reoptimize after every 30 documents, on an index that size. Optimizing takes a while on your index, because it basically has to copy the entire index to a new index, so it will take how ever long it takes to copy 2 GB's on your hardware + a small amount of overhead... Dan Angelov, Rossen wrote: >I would like to bring that issue up again as I haven't resolved it yet and >haven't found what's causing it. > >Any help, ideas or sharing experience are welcome! > >Thanks, >Ross > >-----Original Message----- >From: Angelov, Rossen >Sent: Friday, May 27, 2005 10:42 AM >To: 'java-user@lucene.apache.org' >Subject: how long should optimizing take > > >Hi, > >I'm having problems with the Lucene optimization. Two of the indexes are >about 2BG big and every day about 30 documents are added to each of these >indexes. At the end of the indexing the IndexWriter optimize() method is >executed and it takes about 30 minutes to finish the optimization for each >index. > >The indexing happens through a web service. A servlet takes an http request >and executes methods to index the new documents and optimize the indexes. > >The problem is that the request takes too long to finish because of the >optimization and the web server doesn't return a response. The browser will >keep waiting forever. > >Has anybody else experienced similar behavior with the optimization process? > >Thanks, >Ross > >"This communication is intended solely for the addressee and is >confidential and not for third party unauthorized distribution." > > > >"This communication is intended solely for the addressee and is >confidential and not for third party unauthorized distribution." > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] "This communication is intended solely for the addressee and is confidential and not for third party unauthorized distribution."
