Hey Shawn! Thanks for replying. Yes I meant HugePages not HugeTable, brain fart. I will give the transparent off option a go.
I have attempted to use your CMS configs as is and also the default settings and the cluster dies under our load (basically a node will get a 35-60s GC STW and then the others in the shard will take the load, and they will in turn get long STWs until the shard dies), which is why basically in a fit of desperation I tried out ParallelGC and found it to be half-way acceptable. I will run a test using your configs (and the defaults) again just to be sure (since I'm certain the machine config has changed since we used your unaltered settings). Thanks! Cas On Wed, Jun 15, 2016 at 3:41 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On 6/15/2016 3:05 PM, Cas Rusnov wrote: > > After trying many of the off the shelf configurations (including CMS > > configurations but excluding G1GC, which we're still taking the > > warnings about seriously), numerous tweaks, rumors, various instance > > sizes, and all the rest, most of which regardless of heap size and > > newspace size resulted in frequent 30+ second STW GCs, we settled on > > the following configuration which leads to occasional high GCs but > > mostly stays between 10-20 second STWs every few minutes (which is > > almost acceptable): -XX:+AggressiveOpts -XX:+UnlockDiagnosticVMOptions > > -XX:+UseAdaptiveSizePolicy -XX:+UseLargePages -XX:+UseParallelGC > > -XX:+UseParallelOldGC -XX:MaxGCPauseMillis=15000 -XX:MaxNewSize=12000m > > -XX:ParGCCardsPerStrideChunk=4096 -XX:ParallelGCThreads=16 -Xms31000m > > -Xmx31000m > > You mentioned something called "HugeTable" ... I assume you're talking > about huge pages. If that's what you're talking about, have you also > turned off transparent huge pages? If you haven't, you might want to > completely disable huge pages in your OS. There's evidence that the > transparent option can affect performance. > > I assume you've probably looked at my GC info at the following URL: > > http://wiki.apache.org/solr/ShawnHeisey#GC_Tuning_for_Solr > > The parallel collector is most definitely not a good choice. It does > not optimize for latency. It's my understanding that it actually > prefers full GCs, because it is optimized for throughput. Solr thrives > on good latency, throughput doesn't matter very much. > > If you want to continue avoiding G1, you should definitely be using > CMS. My recommendation right now would be to try the G1 settings on my > wiki page under the heading "Current experiments" or the CMS settings > just below that. > > The out-of-the-box GC tuning included with Solr 6 is probably a better > option than the parallel collector you've got configured now. > > Thanks, > Shawn > > -- Cas Rusnov, Engineer [image: Manzama Logo] <http://www.manzama.com> Visit our Resource Center <http://www.manzama.com/resource-center/>. US & Canada Office: +1 (541) 306-3271 <+15413063271> | UK Office: +44 (0)203 282 1633 <+4402032821633> | AUS Office: +61 02 9326 6264 <+610293266264> LinkedIn <http://www.linkedin.com/company/manzama>| Twitter <https://twitter.com/ManzamaInc>| Facebook <http://www.facebook.com/manzamainc>| Google + <https://plus.google.com/u/0/b/116326385357563344293/+ManzamaInc/about>| YouTube <https://www.youtube.com/channel/UCgbgt-xWBTxrbQESTVeMMHw>| Pinterest <https://www.pinterest.com/manzama1754/>
