Re: forceMerge(1) grows index and does not shrink back

2015-01-19 Thread Ian Lea
Do you need to call forceMerge(1) at all? The javadoc, certainly for recent versions of lucene, advises against it. What version of lucene are you running? It might be helpful to run lsof against the index directory before/during/after the merge to see what files are coming or going, or if there

Re: forceMerge(1) grows index and does not shrink back

2015-01-19 Thread Jürgen Albert
Hi, we use 4.8.1. We know that the javadoc advises against it. Like I wrote, the deletion of old documents (that appear during an update) would be done while closing the writer. Unfortunately we can't close the writer and we chose the force merge as alternative with less afford. Could forceMe

RE: forceMerge(1) grows index and does not shrink back

2015-01-19 Thread Uwe Schindler
Hi, > we use 4.8.1. We know that the javadoc advises against it. Like I wrote, the > deletion of old documents (that appear during an update) would be done > while closing the writer. This is not true. The merge policy continuously merges segments that contain deletions. The problem you might ha

Re: forceMerge(1) grows index and does not shrink back

2015-01-19 Thread Jürgen Albert
Hi, Am 19.01.2015 um 14:13 schrieb Uwe Schindler: Hi, we use 4.8.1. We know that the javadoc advises against it. Like I wrote, the deletion of old documents (that appear during an update) would be done while closing the writer. This is not true. The merge policy continuously merges segments t

Re: Similarity formula documentation is misleading + how to make field-agnostic queries?

2015-01-19 Thread danield
Update: I have implemented my own subclasses of QueryParser, BooleanQuery, BooleanScorer and Similarity to deal with this. I have been successful in getting the exact behaviour I want... when calling the .explain() method. However, the scores for some documents often differ when calling IndexSearc