Thanks to Shawn.
So can we say that the FileNotFoundExcep tion on DEL file was caused by merge operation of Lucene? Note that our application is running on old Lucene Core v3.6.2. Does the rule below from Core 5.3.2 apply to Core 3.6.2? Now we have 2 indexes in 68G and 76G respectively, but only 143G disk space is left. -----Original Message----- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Tuesday, September 13, 2016 8:49 PM To: dev@lucene.apache.org Subject: Re: Index partition corrupted during a regular flush due to FileNotFoundException on DEL file On 9/12/2016 7:07 PM, 郑文兴 wrote: > So you mean if there is no more than 10G free space, Lucene/Solr will > delete some files to save the disk space? Or it will cause the > misbehave of Lucene/Solr? If you do not have enough free disk space, and a segment merge takes place that requires more free space than you have, then the merge will fail. Merges are going to happen if you are adding/updating documents in your index. The Solr operation called "optimize" is known as "forceMerge" in Lucene. It forces a merge of the entire index down to (usually) one segment. Here's some more information: <https://lucene.apache.org/core/5_3_2/core/org/apache/lucene/index/IndexWriter.html#forceMerge(int)> https://lucene.apache.org/core/5_3_2/core/org/apache/lucene/index/IndexWriter.html#forceMerge(int) Lucene will not delete files to free up space. The only way that a Lucene index ever gets smaller is through segment merging, which will temporarily *increase* the space used, until the merge is complete. Any deleted documents contained in the merging segments will be removed when the merge completes. > Please note that we have several shards/partitions under the same root > directory, so which way the following is true to us. Let's assume we > have 2 partitions, and A -> 10G, B->10G > > l do we have to make sure there are at least 20G disk space available? > > l Or we just need to make sure there are at least 10G disk space > available? > The extra space required for merging is detailed in the link above. At worst your index size might temporarily increase by a factor of four. Usually a full optimize (forceMerge) on an index that's not in the compound file format will only *double* the index size while it's running, but in some situations, it might require more. My own recommendation: If you only have one large index on a server, you should have enough space available for that index to triple in size. If you have many large indexes, having enough space for *all* your indexes to double in size at the same time is probably sufficient, but if you can arrange for more free space, that would be advisable. Thanks, Shawn --------------------------------------------------------------------- To unsubscribe, e-mail: <mailto:dev-unsubscr...@lucene.apache.org> dev-unsubscr...@lucene.apache.org For additional commands, e-mail: <mailto:dev-h...@lucene.apache.org> dev-h...@lucene.apache.org
