Re: Solr CPU spiking up on bulk indexing

2019-06-18 Thread Erick Erickson
Dynamic fields don’t make any difference, they’re just like fixed fields as far as merging is concerned. So this is almost certainly merging being kicked off by your commits. The number of documents and the more terms, the more work Lucene has to do, so I suspect this is just how things work. I’l

Re: Solr CPU spiking up on bulk indexing

2019-06-18 Thread Venu
Thanks Erick. I see the above pattern only at the time of commit. I have many fields (like around 250 fields out of which around 100 fields are dynamic fields and around 3 n-gram fields and text fields, while many of them are stored fields along with indexed fields), will a merge take a lot of t

Re: Solr CPU spiking up on bulk indexing

2019-06-16 Thread Erick Erickson
When indexing, segments periodically merged by background threads, which can be quite CPU intensive. See: http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html Segment merges can be fairly long running, so even after indexing stops it can take some time for the CPU to s

Solr CPU spiking up on bulk indexing

2019-06-15 Thread Venu
Hi While doing a batch indexing, Solr CPU is spiking regularly. I am doing the auto-commit for every 5 minutes. Please find the image below On stopping the indexing, the CPU is coming to the normal state (around 20%). In the image a

Solr CPU spiking up on bulk indexing

2019-06-15 Thread Venu
Hi While doing a batch indexing, Solr CPU is spiking regularly. I am doing the auto-commit for every 5 minutes. Please find the image below On stopping the indexing, the CPU is coming to the normal state (around 20%). In the image a