Thank you for your response. Now, we have no JVM monitoring. But checking GC logs, I found no major GC during load test. As you saying, heap size might be too large and I am planning to reduce that.
Thanks, Yasufumi 2019年9月30日(月) 23:19 Walter Underwood <wun...@wunderwood.org>: > 31G is still a very large heap. We use 8G for all of our different > clusters. > > Do you have JVM monitoring? Look at the heap used after a major GC. Use > that number, plus some extra, for the heap size. > > wunder > Walter Underwood > wun...@wunderwood.org > http://observer.wunderwood.org/ (my blog) > > > On Sep 30, 2019, at 1:28 AM, Yasufumi Mizoguchi <yasufumi0...@gmail.com> > wrote: > > > > Hi, Ere. > > > > Thank you for valuable feedback. > > I will try Xmx31G and Xms31G instead of current ones. > > > > Thanks and Regards, > > Yasufumi. > > > > 2019年9月30日(月) 17:19 Ere Maijala <ere.maij...@helsinki.fi>: > > > >> Just a side note: -Xmx32G is really bad for performance as it forces > >> Java to use non-compressed pointers. You'll actually get better results > >> with -Xmx31G. For more information, see e.g. > >> > >> > https://blog.codecentric.de/en/2014/02/35gb-heap-less-32gb-java-jvm-memory-oddities/ > >> > >> Regards, > >> Ere > >> > >> Yasufumi Mizoguchi kirjoitti 30.9.2019 klo 11.05: > >>> Hi, Deepak. > >>> Thank you for replying me. > >>> > >>> JVM settings from solr.in.sh file are as follows. (Sorry, I could not > >> show > >>> all due to our policy) > >>> > >>> -verbose:gc > >>> -XX:+PrintHeapAtGC > >>> -XX:+PrintGCDetails > >>> -XX:+PrintGCDateStamps > >>> -XX:+PrintGCTimeStamps > >>> -XX:+PrintTenuringDistribution > >>> -XX:+PrintGCApplicationStoppedTime > >>> -Dcom.sun.management.jmxremote.ssl=false > >>> -Dcom.sun.management.jmxremote.authenticate=false > >>> -Dcom.sun.management.jmxremote.port=18983 > >>> -XX:OnOutOfMemoryError=/home/solr/solr-6.2.1/bin/oom_solr.sh > >>> -XX:NewSize=128m > >>> -XX:MaxNewSize=128m > >>> -XX:+UseG1GC > >>> -XX:+PerfDisableSharedMem > >>> -XX:+ParallelRefProcEnabled > >>> -XX:G1HeapRegionSize=8m > >>> -XX:MaxGCPauseMillis=250 > >>> -XX:InitiatingHeapOccupancyPercent=75 > >>> -XX:+UseLargePages > >>> -XX:+AggressiveOpts > >>> -Xmx32G > >>> -Xms32G > >>> -Xss256k > >>> > >>> > >>> Thanks & Regards > >>> Yasufumi. > >>> > >>> 2019年9月30日(月) 16:12 Deepak Goel <deic...@gmail.com>: > >>> > >>>> Hello > >>>> > >>>> Can you please share the JVM heap settings in detail? > >>>> > >>>> Deepak > >>>> > >>>> On Mon, 30 Sep 2019, 11:15 Yasufumi Mizoguchi, < > yasufumi0...@gmail.com> > >>>> wrote: > >>>> > >>>>> Hi, > >>>>> > >>>>> I am trying some tests to confirm if single Solr instance can perform > >>>> over > >>>>> 1000 queries per second(!). > >>>>> > >>>>> But now, although CPU usage is 40% or so and iowait is almost 0%, > >>>>> throughput does not increase over 60 queries per second. > >>>>> > >>>>> I think there are some bottlenecks around Kernel, JVM, or Solr > >> settings. > >>>>> > >>>>> The values we already checked and configured are followings. > >>>>> > >>>>> * Kernel: > >>>>> file descriptor > >>>>> net.ipv4.tcp_max_syn_backlog > >>>>> net.ipv4.tcp_syncookies > >>>>> net.core.somaxconn > >>>>> net.core.rmem_max > >>>>> net.core.wmem_max > >>>>> net.ipv4.tcp_rmem > >>>>> net.ipv4.tcp_wmem > >>>>> > >>>>> * JVM: > >>>>> Heap [ -> 32GB] > >>>>> G1GC settings > >>>>> > >>>>> * Solr: > >>>>> (Jetty) MaxThreads [ -> 20000] > >>>>> > >>>>> > >>>>> And the other info is as follows. > >>>>> > >>>>> CPU : 16 cores > >>>>> RAM : 128 GB > >>>>> Disk : SSD 500GB > >>>>> NIC : 10Gbps(maybe) > >>>>> OS : Ubuntu 14.04 > >>>>> JVM : OpenJDK 1.8.0u191 > >>>>> Solr : 6.2.1 > >>>>> Index size : about 60GB > >>>>> > >>>>> Any insights will be appreciated. > >>>>> > >>>>> Thanks and regards, > >>>>> Yasufumi. > >>>>> > >>>> > >>> > >> > >> -- > >> Ere Maijala > >> Kansalliskirjasto / The National Library of Finland > >> > >
