On 10/24/2019 11:50 AM, Junagade, Anup wrote:
* Attachment 1: OpenJDK 8 vs OpenJDK 8 key metrics
* Attachment 2: OpenJDK 8 vs OpenJDK 8 waiting QTP Threads
* Attachment 3: OpenJDK 11 Thread dump
There are no attachments. Apache mailing lists swallow almost all
attachments. You will need to use a file sharing website to
successfully get files to us.
Heap allocated: 32 GB
If you set your heap to 31GB, you'll actually have more memory available
to Java than a heap size of 32GB. This is because at 32GB, longer
pointers are required. Solr has a tendency to create a very large
number of small objects, so the pointer size increase ends up using a
lot of memory.
<mergePolicyFactory class="org.apache.solr.index.TieredMergePolicyFactory">
<int name="maxMergeAtOnce">100</int>
<int name="segmentsPerTier">150</int>
</mergePolicyFactory>
These numbers are huge. Without setting maxMergeAtOnceExplicit, you're
not getting the full benefit of increasing these settings beyond the
defaults of 10. Set maxMergeAtOnce and segmentsPerTier to the same
number and then use three times that number for maxMergeAtOnceExplicit.
The Explicit setting is not mentioned in Solr documentation. Numbers as
big as you have chosen will result in Solr keeping a LOT of files open,
becasue the index will end up with a large number of segments. The OS
will definitely need to have its "max open files" limit increased.
GC and ZK Settings
-DzkClientTimeout=300000
-DzkHost= XXXX,YYYY,ZZZZ
-XX:+PrintGCDetails
-XX:+UseG1GC
-XX:+UseStringDeduplication
-XX:ConcGCThreads=8
-XX:InitiatingHeapOccupancyPercent=70
-XX:MaxGCPauseMillis=200
-XX:ParallelGCThreads=32
-XX:PermSize=512m
-Xlog:gc*:file=/var/solr/logs/solr_gc.log:time,
uptime:filecount=9,
filesize=20M
-Xms32g-Xmx32g
-Xss256k-Xss256k
-verbose:gc
It looks like you have used your own GC settings instead of those that
Solr comes with. Your settings are missing one of the most important
parameters for good GC performance. You should let Solr's start script
handle GC tuning and GC logging without interference.
Thanks,
Shawn
