@Uwe Solr does support expunge deletes: http://wiki.apache.org/solr/UpdateXmlMessages#Optional_attributes_for_.22commit.22
On 16 June 2011 18:05, Erick Erickson <erickerick...@gmail.com> wrote: > OK, after more tests I'm pretty sure that my personal machine > that I'm testing on is just resource-constrained, leading to the > results I mentioned before. After all, I'm running my Solr > instance, the indexing program, etc on a Macbook > with 1 CPU and 2 cores. The indexing program is parsing the > XML. > > On a proper setup, where the indexing machine was separate > from the machine(s) feeding the index process I suspect this would > be a different story. Hmmmm, I may try that sometime too.... > > Best > Erick > > On Tue, Jun 14, 2011 at 9:25 AM, Uwe Schindler <u...@thetaphi.de> wrote: > > For simple removing deletes, there is also IW.expungeDeletes(), which is > > less intensive! Not sure if solr support this, too, but as far as I know > > there is an issue open. > > > > Also please note: As soon as one segment is selected for merging (the > merge > > policy may also do this dependent on the number of deletes in a segment), > it > > will reclaim all deleted ressources - that's what merging does. So > expunging > > deletes once per week is a good idea, if your index consists of very old > and > > large segments that are rarely merged anymore and lots of documents are > > deleted from them. > > > > ----- > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > > >> -----Original Message----- > >> From: Erick Erickson [mailto:erickerick...@gmail.com] > >> Sent: Tuesday, June 14, 2011 3:19 PM > >> To: dev@lucene.apache.org > >> Subject: Re: Indexing slower in trunk > >> > >> Optimization used to have a very noticeable impact on search speed prior > > to > >> some index format changes from quite a while ago. > >> > >> At this point the effect is much less noticeable, but the thing optimize > > does > >> do is reclaim resources from deleted documents. If you have lots of > >> deletions, it's a good idea to periodically optimize, but in that case > > it's often > >> done pretty infrequently (once a > >> day/week/month) rather than as part of any ongoing indexing process. > >> > >> Best > >> Erick > >> > >> 2011/6/14 Yury Kats <yuryk...@yahoo.com>: > >> > On 6/14/2011 4:28 AM, Uwe Schindler wrote: > >> >> indexing and optimizing was only a > >> >> good idea pre Lucene-2.9, now it's mostly obsolete) > >> > > >> > Could you please elaborate on this? Is optimizing obsolete in general > >> > or after indexing new documents? Is it obsolete after deletions? And > >> > what it "mostly"? > >> > > >> > Thanks! > >> > > >> > --------------------------------------------------------------------- > >> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For > >> > additional commands, e-mail: dev-h...@lucene.apache.org > >> > > >> > > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For > additional > >> commands, e-mail: dev-h...@lucene.apache.org > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > -- Met vriendelijke groet, Martijn van Groningen
