So, basically I made the first mistake by Optimizing ? At this point, since it
seems I can't stop these optimizations from running, should I just drop all
data and start fresh ?
On Mon, Apr 23, 2018 at 01:23 PM, Erick Erickson wrote:
No, it's not "optimizing on its own". At least it better not be.
As far as your index growing after optimize, that's the little
"gotcha" with optimize, see:
https://lucidworks.com/2017/10/13/segment-merging-deleted-documents-optimize-may-bad/
(https://lucidworks.com/2017/10/13/segment-merging-deleted-documents-optimize-may-bad/)
This is being addressed in the 7.4 time frame (hopefully), see LUCENE-7976.
Best,
Erick
On Mon, Apr 23, 2018 at 10:13 AM, Scott M. wrote:
I recently installed Solr 7.1 and configured it to work with Dovecot for
full-text searching. It works great but after about 2 days of indexing, I've
pressed the 'Optimize' button. At that point it had collected about 17 million
documents and it was taking up about 60-70GB of space.
It completed once and the space dropped down to 30-45GB but since then it
appears to be doing Optimize again on its own, regularly swelling up the total
space used to double, then it shrinks again, stays a bit that way then it
starts another optimize!
Logs show:
4/22/2018, 11:04:22 PM
WARN false
DirectUpdateHandler2
Starting optimize... Reading and rewriting the entire index! Use with care.
4/23/2018, 3:18:35 AM
WARN true
DirectUpdateHandler2
Starting optimize... Reading and rewriting the entire index! Use with care.
4/23/2018, 7:33:46 AM
WARN false
DirectUpdateHandler2
Starting optimize... Reading and rewriting the entire index! Use with care.
4/23/2018, 9:48:32 AM
WARN false
DirectUpdateHandler2
Starting optimize... Reading and rewriting the entire index! Use with care.
4/23/2018, 11:25:13 AM
WARN false
DirectUpdateHandler2
Starting optimize... Reading and rewriting the entire index! Use with care.
4/23/2018, 1:00:42 PM
WARN false
DirectUpdateHandler2
Starting optimize... Reading and rewriting the entire index! Use with care.
It's absolutely killing the computer this is running on. Now it just started
another run...
In the logs all I see is entries like these, and it doesn't say anywhere
optimize=true
2018-04-23 17:12:31.995 INFO (qtp947679291-17200) [ x:dovecot]
o.a.s.u.DirectUpdateHandler2 start
commit{_version_=1598557836536709120,optimize=false,openSearcher=true,waitSearcher=false,expungeDeletes=false,softCommit=true,prepareCommit=false}
