No replies on the Solr users list, so I thought I would repost to dev. We are continuing to see inconsistent behavior with <optimize maxSegments="2"/> Out of 12 shards 1-3 of them end up with more than 2 segments at the finish of the optimize command. (and we see no errors in the logs)
The pattern seems the same in that after almost all of the segments are merged, one or two new segments are created when a startFullFlush happens after the big merge. Any suggestions on how to troubleshoot this would be appreciated. Tom ---------- Forwarded message ---------- From: Tom Burton-West <tburt...@umich.edu> Date: Mon, Feb 23, 2015 at 12:41 PM Subject: Optimize maxSegments="2" not working right with Solr 4.10.2 To: "solr-u...@lucene.apache.org" <solr-u...@lucene.apache.org> Cc: Phillip Farber <pfar...@umich.edu>, Sebastien Korner <skor...@umich.edu> Hello, We normally run an optimize with maxSegments="2" after our daily indexing. This has worked without problem on Solr 3.6. We recently moved to Solr 4.10.2 and on several shards the optimize completed with no errors in the logs, but left more than 2 segments. We send this xml to Solr <optimize maxSegments="2"/> I've attached a copy of the indexwriter log for one of the segments where there were 4 segments rather than the requested number (i.e. there should have been only 2 segments) at the end of the optimize. It looks like a merge was done down to two segments and then somehow another process flushed some postings to disk creating two more segments. Then there are messages about 2 of the remaining 4 segments being too big. (See below) What we expected is that the remainng 2 small segments (about 40MB) would get merged with the smaller of the two large segments, i.e. with the 56GB segment, since we gave the argument maxSegments=2. This didn't happen. Any suggestions about how to troubleshoot this issue would be appreciated. Tom ------- Excerpt from indexwriter log: TMP][http-8091-Processor5]: findForcedMerges maxSegmentCount=2 ... ... [IW][Lucene Merge Thread #0]: merge time 3842310 msec for 65236 docs ... [TMP][http-8091-Processor5]: findMerges: 4 segments [TMP][http-8091-Processor5]: seg=_1fzb(4.10.2):C1081559/24089:delGen=9 size=672402.066 MB [skip: too large] [TMP][http-8091-Processor5]: seg=_1gj2(4.10.2):C65236/2:delGen=1 size=56179.245 MB [skip: too large] [TMP][http-8091-Processor5]: seg=_1gj0(4.10.2):C16 size=44.280 MB [TMP][http-8091-Processor5]: seg=_1gj1(4.10.2):C8 size=40.442 MB [TMP][http-8091-Processor5]: allowedSegmentCount=3 vs count=4 (eligible count=2) tooBigCount=2
build-1.iw.2015-02-23.txt.gz
Description: GNU Zip compressed data
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
