[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[ https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282609#comment-14282609 ] Jeremy Hanna edited comment on CASSANDRA-7432 at 1/19/15 7:53 PM: -- It looks like this is in 2.0, but not 2.1 forward. was (Author: jeromatron): It looks like is in 2.0, but not 2.1 forward. > Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60 > > > Key: CASSANDRA-7432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7432 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: graham sanderson >Assignee: Brandon Williams > Fix For: 2.0.10, 2.1.1 > > Attachments: 7432.txt > > > The new flags in question are as follows: > {code} > -XX:+CMSParallelInitialMarkEnabled > -XX:+CMSEdenChunksRecordAlways > {code} > Given we already have > {code} > JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC" > JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC" > JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled" > JVM_OPTS="$JVM_OPTS -XX:+UseTLAB" > if [ "$JVM_ARCH" = "64-Bit" ] ; then > JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark" > fi > {code} > The assumption would be that people are at least running on large number CPU > cores/threads > I would therefore recommend defaulting these flags if available - the only > two possible downsides for {{+CMSEdenChunksRecordAlways}}: > 1) There is a new very short (probably un-contended) lock in the "slow" (non > TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't > detected this timing wise - this is the "slow" path after all > 2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the > default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}} > will expose you to a possible seg fault: (see > [http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[ https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282931#comment-14282931 ] Brandon Williams edited comment on CASSANDRA-7432 at 1/19/15 7:51 PM: -- Not sure what happened, I see my original commit did go into 2.1, but couldn't find where it got removed. Anyway, I put them back. was (Author: brandon.williams): Not what happened, I see my original commit did go into 2.1, but couldn't find where it got removed. Anyway, I put them back. > Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60 > > > Key: CASSANDRA-7432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7432 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: graham sanderson >Assignee: Brandon Williams > Fix For: 2.0.10, 2.1.1 > > Attachments: 7432.txt > > > The new flags in question are as follows: > {code} > -XX:+CMSParallelInitialMarkEnabled > -XX:+CMSEdenChunksRecordAlways > {code} > Given we already have > {code} > JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC" > JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC" > JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled" > JVM_OPTS="$JVM_OPTS -XX:+UseTLAB" > if [ "$JVM_ARCH" = "64-Bit" ] ; then > JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark" > fi > {code} > The assumption would be that people are at least running on large number CPU > cores/threads > I would therefore recommend defaulting these flags if available - the only > two possible downsides for {{+CMSEdenChunksRecordAlways}}: > 1) There is a new very short (probably un-contended) lock in the "slow" (non > TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't > detected this timing wise - this is the "slow" path after all > 2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the > default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}} > will expose you to a possible seg fault: (see > [http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[ https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086862#comment-14086862 ] Brandon Williams edited comment on CASSANDRA-7432 at 8/5/14 10:02 PM: -- Updated patch to check for > 1.7 and < 1.8 and patch >= 60, or just > 1.8. was (Author: brandon.williams): Update patch to check for > 1.7 and < 1.8 and patch >= 60, or just > 1.8. > Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60 > > > Key: CASSANDRA-7432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7432 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: graham sanderson >Assignee: Brandon Williams > Fix For: 1.2.19 > > Attachments: 7432.txt > > > The new flags in question are as follows: > {code} > -XX:+CMSParallelInitialMarkEnabled > -XX:+CMSEdenChunksRecordAlways > {code} > Given we already have > {code} > JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC" > JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC" > JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled" > JVM_OPTS="$JVM_OPTS -XX:+UseTLAB" > if [ "$JVM_ARCH" = "64-Bit" ] ; then > JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark" > fi > {code} > The assumption would be that people are at least running on large number CPU > cores/threads > I would therefore recommend defaulting these flags if available - the only > two possible downsides for {{+CMSEdenChunksRecordAlways}}: > 1) There is a new very short (probably un-contended) lock in the "slow" (non > TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't > detected this timing wise - this is the "slow" path after all > 2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the > default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}} > will expose you to a possible seg fault: (see > [http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809]) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[ https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14084939#comment-14084939 ] Brandon Williams edited comment on CASSANDRA-7432 at 8/4/14 5:33 PM: - -Updated patch works correctly on java 8 by inverting the check to be less than 1.8. (we're guaranteed at least 1.7 by previous checks)- nevermind, that doesn't work either. was (Author: brandon.williams): Updated patch works correctly on java 8 by inverting the check to be less than 1.8. (we're guaranteed at least 1.7 by previous checks) > Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60 > > > Key: CASSANDRA-7432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7432 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: graham sanderson >Assignee: Brandon Williams > Fix For: 1.2.19 > > Attachments: 7432.txt > > > The new flags in question are as follows: > {code} > -XX:+CMSParallelInitialMarkEnabled > -XX:+CMSEdenChunksRecordAlways > {code} > Given we already have > {code} > JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC" > JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC" > JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled" > JVM_OPTS="$JVM_OPTS -XX:+UseTLAB" > if [ "$JVM_ARCH" = "64-Bit" ] ; then > JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark" > fi > {code} > The assumption would be that people are at least running on large number CPU > cores/threads > I would therefore recommend defaulting these flags if available - the only > two possible downsides for {{+CMSEdenChunksRecordAlways}}: > 1) There is a new very short (probably un-contended) lock in the "slow" (non > TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't > detected this timing wise - this is the "slow" path after all > 2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the > default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}} > will expose you to a possible seg fault: (see > [http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809]) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[ https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059052#comment-14059052 ] graham sanderson edited comment on CASSANDRA-7432 at 7/11/14 5:34 PM: -- presumably that should be an equals for 1.7 and a greater than or equal to for patch 60 (note I'm not sure what version of 1.8 has these flags, so setting on 1.7 only for now seems reasonable was (Author: graham sanderson): presumably those should be >= rather than > > Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60 > > > Key: CASSANDRA-7432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7432 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: graham sanderson >Assignee: Brandon Williams > Fix For: 1.2.19 > > Attachments: 7432.txt > > > The new flags in question are as follows: > {code} > -XX:+CMSParallelInitialMarkEnabled > -XX:+CMSEdenChunksRecordAlways > {code} > Given we already have > {code} > JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC" > JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC" > JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled" > JVM_OPTS="$JVM_OPTS -XX:+UseTLAB" > if [ "$JVM_ARCH" = "64-Bit" ] ; then > JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark" > fi > {code} > The assumption would be that people are at least running on large number CPU > cores/threads > I would therefore recommend defaulting these flags if available - the only > two possible downsides for {{+CMSEdenChunksRecordAlways}}: > 1) There is a new very short (probably un-contended) lock in the "slow" (non > TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't > detected this timing wise - this is the "slow" path after all > 2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the > default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}} > will expose you to a possible seg fault: (see > [http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809]) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[ https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039890#comment-14039890 ] graham sanderson edited comment on CASSANDRA-7432 at 6/21/14 4:53 PM: -- Here are some performance numbers (we have an unusually large heap for C*: 8gig new, 16g old), and 16 CPU cores/32 threads h3. -XX:+CMSParallelInitialMarkEnabled ||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159| |With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019| Note this flag is just the new cousin of {{-XX:+CMSParallelRemarkEnabled}} h3. -XX:+CMSEdenChunksRecordAlways ||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394| |With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114| Note this option addresses the issue loosely described [here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the *CMSMaxAbortablePrecleanTime* section. The short version of the problem is that if you are unlucky (and bursty activity in your cluster is likely to make you less lucky), then even though you have {{-XX:+CMSParallelRemarkEnabled}}, you may get very low parallelism in your remark STW *and* you may have a very full eden when you are doing it. h3. -XX:+CMSPrintEdenSurvivorChunks Turn this on if you want even more stuff to confuse whatever tool you use to parse GC logs! was (Author: graham sanderson): Here are some performance numbers (we have an unusually large heap for C*: 8gig new, 16g old), and 16 CPU cores/32 threads h3. -XX:+CMSParallelInitialMarkEnabled ||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159| |With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019| Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}} h3. -XX:+CMSEdenChunksRecordAlways ||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394| |With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114| Note this option addresses the issue loosely described [here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the *CMSMaxAbortablePrecleanTime* section. The short version of the problem is that if you are unlucky (and bursty activity in your cluster is likely to make you less lucky), then even though you have {{-XX:+CMSParallelRemarkEnabled}}, you may get very low parallelism in your remark STW *and* you may have a very full eden when you are doing it. h3. -XX:+CMSPrintEdenSurvivorChunks Turn this on if you want even more stuff to confuse whatever tool you use to parse GC logs! > Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60 > > > Key: CASSANDRA-7432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7432 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: graham sanderson > > The new flags in question are as follows: > {code} > -XX:+CMSParallelInitialMarkEnabled > -XX:+CMSEdenChunksRecordAlways > {code} > Given we already have > {code} > JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC" > JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC" > JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled" > JVM_OPTS="$JVM_OPTS -XX:+UseTLAB" > if [ "$JVM_ARCH" = "64-Bit" ] ; then > JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark" > fi > {code} > The assumption would be that people are at least running on large number CPU > cores/threads > I would therefore recommend defaulting these flags if available - the only > two possible downsides for {{+CMSEdenChunksRecordAlways}}: > 1) There is a new very short (probably un-contended) lock in the "slow" (non > TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't > detected this timing wise - this is the "slow" path after all > 2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the > default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}} > will expose you to a possible seg fault: (see > [http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809]) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[ https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039890#comment-14039890 ] graham sanderson edited comment on CASSANDRA-7432 at 6/21/14 4:46 PM: -- Here are some performance numbers (we have an unusually large heap for C*: 8gig new, 16g old), and 16 CPU cores/32 threads h3. -XX:+CMSParallelInitialMarkEnabled ||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159| |With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019| Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}} h3. -XX:+CMSEdenChunksRecordAlways ||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394| |With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114| Note this option addresses the issue loosely described [here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the *CMSMaxAbortablePrecleanTime* section. The short version of the problem is that if you are unlucky (and bursty activity in your cluster is likely to make you less lucky), then even though you have {{-XX:+CMSParallelRemarkEnabled}}, you may get very low parallelism in your remark STW *and* you may have a very full eden when you are doing it. h3. -XX:+CMSPrintEdenSurvivorChunks Turn this on if you want even more stuff to confuse whatever tool you use to parse GC logs! was (Author: graham sanderson): Here are some performance numbers (we have an unusually large heap for C*: 8gig new, 16g old): h3. -XX:+CMSParallelInitialMarkEnabled ||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159| |With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019| Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}} h3. -XX:+CMSEdenChunksRecordAlways ||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394| |With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114| Note this option addresses the issue loosely described [here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the *CMSMaxAbortablePrecleanTime* section. The short version of the problem is that if you are unlucky (and bursty activity in your cluster is likely to make you less lucky), then even though you have {{-XX:+CMSParallelRemarkEnabled}}, you may get very low parallelism in your remark STW *and* you may have a very full eden when you are doing it. h3. -XX:+CMSPrintEdenSurvivorChunks Turn this on if you want even more stuff to confuse whatever tool you use to parse GC logs! > Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60 > > > Key: CASSANDRA-7432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7432 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: graham sanderson > > The new flags in question are as follows: > {code} > -XX:+CMSParallelInitialMarkEnabled > -XX:+CMSEdenChunksRecordAlways > {code} > Given we already have > {code} > JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC" > JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC" > JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled" > JVM_OPTS="$JVM_OPTS -XX:+UseTLAB" > if [ "$JVM_ARCH" = "64-Bit" ] ; then > JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark" > fi > {code} > The assumption would be that people are at least running on large number CPU > cores/threads > I would therefore recommend defaulting these flags if available - the only > two possible downsides for {{+CMSEdenChunksRecordAlways}}: > 1) There is a new very short (probably un-contended) lock in the "slow" (non > TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't > detected this timing wise - this is the "slow" path after all > 2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the > default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}} > will expose you to a possible seg fault: (see > [http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809]) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[ https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039890#comment-14039890 ] graham sanderson edited comment on CASSANDRA-7432 at 6/21/14 4:41 PM: -- Here are some performance numbers (we have an unusually large heap for C*: 8gig new, 16g old): h3. -XX:+CMSParallelInitialMarkEnabled ||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159| |With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019| Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}} h3. -XX:+CMSEdenChunksRecordAlways ||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394| |With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114| Note this option addresses the issue loosely described [here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the *CMSMaxAbortablePrecleanTime* section. The short version of the problem is that if you are unlucky (and bursty activity in your cluster is likely to make you less lucky), then even though you have {{-XX:+CMSParallelRemarkEnabled}}, you may get very low parallelism in your remark STW *and* you may have a very full eden when you are doing it. h3. -XX:+CMSPrintEdenSurvivorChunks Turn this on if you want even more stuff to confuse whatever tool you use to parse GC logs! was (Author: graham sanderson): Here are some performance numbers(we have an unusually large heap for C*: 8gig new, 16g old): h3. -XX:+CMSParallelInitialMarkEnabled ||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159| |With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019| Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}} h3. -XX:+CMSEdenChunksRecordAlways ||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394| |With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114| Note this option addresses the issue loosely described [here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the *CMSMaxAbortablePrecleanTime* section. The short version of the problem is that if you are unlucky (and bursty activity in your cluster is likely to make you less lucky), then even though you have {{-XX:+CMSParallelRemarkEnabled}}, you may get very low parallelism in your remark STW *and* you may have a very full eden when you are doing it. h3. -XX:+CMSPrintEdenSurvivorChunks Turn this on if you want even more stuff to confuse whatever tool you use to parse GC logs! > Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60 > > > Key: CASSANDRA-7432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7432 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: graham sanderson > > The new flags in question are as follows: > {code} > -XX:+CMSParallelInitialMarkEnabled > -XX:+CMSEdenChunksRecordAlways > {code} > Given we already have > {code} > JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC" > JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC" > JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled" > JVM_OPTS="$JVM_OPTS -XX:+UseTLAB" > if [ "$JVM_ARCH" = "64-Bit" ] ; then > JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark" > fi > {code} > The assumption would be that people are at least running on large number CPU > cores/threads > I would therefore recommend defaulting these flags if available - the only > two possible downsides for {{+CMSEdenChunksRecordAlways}}: > 1) There is a new very short (probably un-contended) lock in the "slow" (non > TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't > detected this timing wise - this is the "slow" path after all > 2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the > default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}} > will expose you to a possible seg fault: (see > [http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809]) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[ https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039890#comment-14039890 ] graham sanderson edited comment on CASSANDRA-7432 at 6/21/14 4:41 PM: -- Here are some performance numbers(we have an unusually large heap for C*: 8gig new, 16g old): h3. -XX:+CMSParallelInitialMarkEnabled ||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159| |With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019| Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}} h3. -XX:+CMSEdenChunksRecordAlways ||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394| |With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114| Note this option addresses the issue loosely described [here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the *CMSMaxAbortablePrecleanTime* section. The short version of the problem is that if you are unlucky (and bursty activity in your cluster is likely to make you less lucky), then even though you have {{-XX:+CMSParallelRemarkEnabled}}, you may get very low parallelism in your remark STW *and* you may have a very full eden when you are doing it. h3. -XX:+CMSPrintEdenSurvivorChunks Turn this on if you want even more stuff to confuse whatever tool you use to parse GC logs! was (Author: graham sanderson): Here are some performance numbers(we have an unusually large heap for C*: 8gig new, 16g old): h3. -XX:+CMSParallelInitialMarkEnabled ||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159| |With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019| Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}} h3. -XX:+CMSEdenChunksRecordAlways ||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394| |With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114| Note this option addresses the issue loosely described [here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the *CMSMaxAbortablePrecleanTime* section. The short version of the problem is that if you are unlucky (and bursty activity in your cluster is likely to make you less lucky), then even though you have {{-XX:*CMSParallelRemarkEnabled}}, you may get very low parallelism in your remark STW *and* you may have a very full eden when you are doing it. h3. -XX:+CMSPrintEdenSurvivorChunks Turn this on if you want even more stuff to confuse whatever tool you use to parse GC logs! > Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60 > > > Key: CASSANDRA-7432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7432 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: graham sanderson > > The new flags in question are as follows: > {code} > -XX:+CMSParallelInitialMarkEnabled > -XX:+CMSEdenChunksRecordAlways > {code} > Given we already have > {code} > JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC" > JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC" > JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled" > JVM_OPTS="$JVM_OPTS -XX:+UseTLAB" > if [ "$JVM_ARCH" = "64-Bit" ] ; then > JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark" > fi > {code} > The assumption would be that people are at least running on large number CPU > cores/threads > I would therefore recommend defaulting these flags if available - the only > two possible downsides for {{+CMSEdenChunksRecordAlways}}: > 1) There is a new very short (probably un-contended) lock in the "slow" (non > TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't > detected this timing wise - this is the "slow" path after all > 2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the > default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}} > will expose you to a possible seg fault: (see > [http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809]) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[ https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039890#comment-14039890 ] graham sanderson edited comment on CASSANDRA-7432 at 6/21/14 4:40 PM: -- Here are some performance numbers(we have an unusually large heap for C*: 8gig new, 16g old): h3. -XX:+CMSParallelInitialMarkEnabled ||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159| |With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019| Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}} h3. -XX:+CMSEdenChunksRecordAlways ||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394| |With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114| Note this option addresses the issue loosely described [here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the *CMSMaxAbortablePrecleanTime* section. The short version of the problem is that if you are unlucky (and bursty activity in your cluster is likely to make you less lucky), then even though you have {{-XX:*CMSParallelRemarkEnabled}}, you may get very low parallelism in your remark STW *and* you may have a very full eden when you are doing it. h3. -XX:+CMSPrintEdenSurvivorChunks Turn this on if you want even more stuff to confuse whatever tool you use to parse GC logs! was (Author: graham sanderson): Here are some performance numbers(we have an unusually large heap for C*: 8gig new, 16g old): h3. -XX:+CMSParallelInitialMarkEnabled ||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159| |With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019| Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}} h3. -XX:+CMSEdenChunksRecordAlways ||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394| |With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114| Note this option addresses the issue loosely described [here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the *CMSMaxAbortablePrecleanTime* section. The short version of the problem is that if you are unlucky (and bursty activity in your cluster will make you less lucky, then even though you have {{-XX:*CMSParallelRemarkEnabled}}, you may get very low parallelism in your remark STW *and* you may have a very full eden when you are doing it. h3. -XX:+CMSPrintEdenSurvivorChunks Turn this on if you want even more stuff to confuse whatever tool you use to parse GC logs! > Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60 > > > Key: CASSANDRA-7432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7432 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: graham sanderson > > The new flags in question are as follows: > {code} > -XX:+CMSParallelInitialMarkEnabled > -XX:+CMSEdenChunksRecordAlways > {code} > Given we already have > {code} > JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC" > JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC" > JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled" > JVM_OPTS="$JVM_OPTS -XX:+UseTLAB" > if [ "$JVM_ARCH" = "64-Bit" ] ; then > JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark" > fi > {code} > The assumption would be that people are at least running on large number CPU > cores/threads > I would therefore recommend defaulting these flags if available - the only > two possible downsides for {{+CMSEdenChunksRecordAlways}}: > 1) There is a new very short (probably un-contended) lock in the "slow" (non > TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't > detected this timing wise - this is the "slow" path after all > 2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the > default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}} > will expose you to a possible seg fault: (see > [http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809]) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[ https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039890#comment-14039890 ] graham sanderson edited comment on CASSANDRA-7432 at 6/21/14 4:39 PM: -- Here are some performance numbers(we have an unusually large heap for C*: 8gig new, 16g old): h3. -XX:+CMSParallelInitialMarkEnabled ||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159| |With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019| Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}} h3. -XX:+CMSEdenChunksRecordAlways ||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394| |With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114| Note this option addresses the issue loosely described [here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the *CMSMaxAbortablePrecleanTime* section. The short version of the problem is that if you are unlucky (and bursty activity in your cluster will make you less lucky, then even though you have {{-XX:*CMSParallelRemarkEnabled}}, you may get very low parallelism in your remark STW *and* you may have a very full eden when you are doing it. h3. -XX:+CMSPrintEdenSurvivorChunks Turn this on if you want even more stuff to confuse whatever tool you use to parse GC logs! was (Author: graham sanderson): Here are some performance numbers(we have an unusually large heap for C*: 8gig new, 16g old): h3. +CMSParallelInitialMarkEnabled ||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without +CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159| |With +CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019| Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}} h3. +CMSEdenChunksRecordAlways ||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without +CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394| |With +CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114| Note this option addresses the issue loosely described [here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the *CMSMaxAbortablePrecleanTime* section. The short version of the problem is that if you are unlucky (and bursty activity in your cluster will make you less lucky, then even though you have {{-XX:*CMSParallelRemarkEnabled}}, you may get very low parallelism in your remark STW *and* you may have a very full eden when you are doing it. > Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60 > > > Key: CASSANDRA-7432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7432 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: graham sanderson > > The new flags in question are as follows: > {code} > -XX:+CMSParallelInitialMarkEnabled > -XX:+CMSEdenChunksRecordAlways > {code} > Given we already have > {code} > JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC" > JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC" > JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled" > JVM_OPTS="$JVM_OPTS -XX:+UseTLAB" > if [ "$JVM_ARCH" = "64-Bit" ] ; then > JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark" > fi > {code} > The assumption would be that people are at least running on large number CPU > cores/threads > I would therefore recommend defaulting these flags if available - the only > two possible downsides for {{+CMSEdenChunksRecordAlways}}: > 1) There is a new very short (probably un-contended) lock in the "slow" (non > TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't > detected this timing wise - this is the "slow" path after all > 2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the > default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}} > will expose you to a possible seg fault: (see > [http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809]) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[ https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039890#comment-14039890 ] graham sanderson edited comment on CASSANDRA-7432 at 6/21/14 4:37 PM: -- Here are some performance numbers(we have an unusually large heap for C*: 8gig new, 16g old): h3. +CMSParallelInitialMarkEnabled ||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without +CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159| |With +CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019| Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}} h3. +CMSEdenChunksRecordAlways ||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without +CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394| |With +CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114| Note this option addresses the issue loosely described [here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the *CMSMaxAbortablePrecleanTime* section. The short version of the problem is that if you are unlucky (and bursty activity in your cluster will make you less lucky, then even though you have {{-XX:*CMSParallelRemarkEnabled}}, you may get very low parallelism in your remark STW *and* you may have a very full eden when you are doing it. was (Author: graham sanderson): Here are some performance numbers(we have an unusually large heap for C*: 8gig new, 16g old): h3. +CMSParallelInitialMarkEnabled ||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without +CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159| |With +CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019| h3. +CMSEdenChunksRecordAlways ||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev|| |Without +CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394| |With +CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114| Note this option addresses the issue loosely described [here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the *CMSMaxAbortablePrecleanTime* section. The short version of the problem is that if you are unlucky (and bursty activity in your cluster will make you less lucky, then even though you have {{-XX:*CMSParallelRemarkEnabled}}, you may get very low parallelism in your remark STW *and* you may have a very full eden when you are doing it. > Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60 > > > Key: CASSANDRA-7432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7432 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: graham sanderson > > The new flags in question are as follows: > {code} > -XX:+CMSParallelInitialMarkEnabled > -XX:+CMSEdenChunksRecordAlways > {code} > Given we already have > {code} > JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC" > JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC" > JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled" > JVM_OPTS="$JVM_OPTS -XX:+UseTLAB" > if [ "$JVM_ARCH" = "64-Bit" ] ; then > JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark" > fi > {code} > The assumption would be that people are at least running on large number CPU > cores/threads > I would therefore recommend defaulting these flags if available - the only > two possible downsides for {{+CMSEdenChunksRecordAlways}}: > 1) There is a new very short (probably un-contended) lock in the "slow" (non > TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't > detected this timing wise - this is the "slow" path after all > 2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the > default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}} > will expose you to a possible seg fault: (see > [http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809]) -- This message was sent by Atlassian JIRA (v6.2#6252)