In addition to what Mike already said: maxMergeDocs=99999 -- do you really mean maxMergeDocs and not maxBufferedDocs?
Larg(er) maxBufferedDocs will speed up indexing. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- From: vivek sar <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Friday, January 18, 2008 4:31:26 AM Subject: Optimize for large index size Hi, We are using Lucene 2.2. We have an index of size 70G (within 3-4 days) and growing. We run optimize pretty frequently (once every hour - due to large number of index updates every min - can be up to 100K new documents every min). I have seen every now and then the optimize takes 3-4 hours to complete and up to 8 G memory (our limit). This makes the whole system slow. Few questions, 1) Is there any alternative to optimize? That is, can we do without optimize and still have our search fast? 2) What's the best way to use optimize, i.e. how can we make the optimize much faster and use lesser memory? 3) Is there a way to partition the indexes using Lucene? Let's say we partition daily, so we have to optimize only the daily indexes and not the whole thing. Our mergefactor=200 and maxMergeDocs=99999 Thanks, -vivek --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
