On 5/13/2015 10:01 PM, Manohar Sripada wrote: > I have a question on segment creation on disk during indexing. > > In my solrconfig.xml, I have commented maxBufferedDocs and ramBufferSizeMB. > I am controlling the flushing of data to disk using autoCommit's maxDocs > and maxTime. > > Here, maxDocs is set to 50000 and will be hit first, so that commit of data > to disk happens every 50000 docs. So, my question here is will it create a > new segment when this commit happens? > > In the wiki > <https://wiki.apache.org/solr/SolrPerformanceFactors#mergeFactor>, it is > mentioned that a new segment creation is determined based on > maxBufferedDocs parameter. As I have commented this parameter, how a new > segment creation is determined?
In recent Solr versions, the ramBufferSizeMB setting defaults to 100 and maxBufferedDocs defaults to -1. A setting of -1 on maxBufferedDocs means that the number of docs doesn't matter, it will use ramBufferSizeMB unless a commit happens before the buffer fills up. A commit does trigger a segment flush, although if it's a soft commit, the situation might be more complicated. Unless the docs are very small, I would expect a 100MB buffer to fill up before you reach 50000 docs. It's been a while since I watched index segments get created, but if I remember correctly, the amount of space required in the RAM buffer to index documents is more than the size of the segment that eventually gets flushed to disk. Thanks, Shawn
