On Wed, Jul 20, 2011 at 2:00 PM, <v.se...@lombardodier.com> wrote: > Hi, > > I index several millions small documents per day. each day, I remove some > of the older documents to keep the index at a stable number of documents. > after each purge, I commit then I optimize the index. what I found is that > if I keep optimizing with max num segments = 2, then the index keeps > growing on the disk. but as soon as I optimize with just 1 segment, the > space gets reclaimed on the disk. so, I have currently adopted the > following strategy : every night I optimize with 2 segments, except once > per week where I optimize with just 1 segment.
what do you mean by keeps growing. you have n segments and you optimize down to 2 and the index is bigger than the one with n segments? simon > > is that an expected behavior? > I guess I am doing something special because I was not able to reproduce > this behavior in a unit test. what could it be? > > it would be nice to get some explanatory services within the product to > help get some understanding on its behavior. something that tells you some > information about your index for instance (number of docs in the different > states, how the space is being used, ...). lucene is a wonderful product, > but to me this is almost like black magic, and when there is a specific > behavior, I have got little clues to figure out something by myself. some > user oriented logging would be nice as well (the index writer info stream > is really verbose and very low level). > > thanks for your help, > > > Vince > > ************************ DISCLAIMER ************************ > This message is intended only for use by the person to > whom it is addressed. It may contain information that is > privileged and confidential. Its content does not > constitute a formal commitment by Lombard Odier > Darier Hentsch & Cie or any of its branches or affiliates. > If you are not the intended recipient of this message, > kindly notify the sender immediately and destroy this > message. Thank You. > ***************************************************************** > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
