[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[
https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14282931#comment-14282931
]
Brandon Williams edited comment on CASSANDRA-7432 at 1/19/15 7:51 PM:
--
Not sure what happened, I see my original commit did go into 2.1, but couldn't
find where it got removed. Anyway, I put them back.
was (Author: brandon.williams):
Not what happened, I see my original commit did go into 2.1, but couldn't find
where it got removed. Anyway, I put them back.
Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
Key: CASSANDRA-7432
URL: https://issues.apache.org/jira/browse/CASSANDRA-7432
Project: Cassandra
Issue Type: Improvement
Components: Packaging
Reporter: graham sanderson
Assignee: Brandon Williams
Fix For: 2.0.10, 2.1.1
Attachments: 7432.txt
The new flags in question are as follows:
{code}
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways
{code}
Given we already have
{code}
JVM_OPTS=$JVM_OPTS -XX:+UseParNewGC
JVM_OPTS=$JVM_OPTS -XX:+UseConcMarkSweepGC
JVM_OPTS=$JVM_OPTS -XX:+CMSParallelRemarkEnabled
JVM_OPTS=$JVM_OPTS -XX:+UseTLAB
if [ $JVM_ARCH = 64-Bit ] ; then
JVM_OPTS=$JVM_OPTS -XX:+UseCondCardMark
fi
{code}
The assumption would be that people are at least running on large number CPU
cores/threads
I would therefore recommend defaulting these flags if available - the only
two possible downsides for {{+CMSEdenChunksRecordAlways}}:
1) There is a new very short (probably un-contended) lock in the slow (non
TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't
detected this timing wise - this is the slow path after all
2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the
default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}}
will expose you to a possible seg fault: (see
[http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[
https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14282609#comment-14282609
]
Jeremy Hanna edited comment on CASSANDRA-7432 at 1/19/15 7:53 PM:
--
It looks like this is in 2.0, but not 2.1 forward.
was (Author: jeromatron):
It looks like is in 2.0, but not 2.1 forward.
Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
Key: CASSANDRA-7432
URL: https://issues.apache.org/jira/browse/CASSANDRA-7432
Project: Cassandra
Issue Type: Improvement
Components: Packaging
Reporter: graham sanderson
Assignee: Brandon Williams
Fix For: 2.0.10, 2.1.1
Attachments: 7432.txt
The new flags in question are as follows:
{code}
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways
{code}
Given we already have
{code}
JVM_OPTS=$JVM_OPTS -XX:+UseParNewGC
JVM_OPTS=$JVM_OPTS -XX:+UseConcMarkSweepGC
JVM_OPTS=$JVM_OPTS -XX:+CMSParallelRemarkEnabled
JVM_OPTS=$JVM_OPTS -XX:+UseTLAB
if [ $JVM_ARCH = 64-Bit ] ; then
JVM_OPTS=$JVM_OPTS -XX:+UseCondCardMark
fi
{code}
The assumption would be that people are at least running on large number CPU
cores/threads
I would therefore recommend defaulting these flags if available - the only
two possible downsides for {{+CMSEdenChunksRecordAlways}}:
1) There is a new very short (probably un-contended) lock in the slow (non
TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't
detected this timing wise - this is the slow path after all
2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the
default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}}
will expose you to a possible seg fault: (see
[http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[
https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086862#comment-14086862
]
Brandon Williams edited comment on CASSANDRA-7432 at 8/5/14 10:02 PM:
--
Updated patch to check for 1.7 and 1.8 and patch = 60, or just 1.8.
was (Author: brandon.williams):
Update patch to check for 1.7 and 1.8 and patch = 60, or just 1.8.
Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
Key: CASSANDRA-7432
URL: https://issues.apache.org/jira/browse/CASSANDRA-7432
Project: Cassandra
Issue Type: Improvement
Components: Packaging
Reporter: graham sanderson
Assignee: Brandon Williams
Fix For: 1.2.19
Attachments: 7432.txt
The new flags in question are as follows:
{code}
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways
{code}
Given we already have
{code}
JVM_OPTS=$JVM_OPTS -XX:+UseParNewGC
JVM_OPTS=$JVM_OPTS -XX:+UseConcMarkSweepGC
JVM_OPTS=$JVM_OPTS -XX:+CMSParallelRemarkEnabled
JVM_OPTS=$JVM_OPTS -XX:+UseTLAB
if [ $JVM_ARCH = 64-Bit ] ; then
JVM_OPTS=$JVM_OPTS -XX:+UseCondCardMark
fi
{code}
The assumption would be that people are at least running on large number CPU
cores/threads
I would therefore recommend defaulting these flags if available - the only
two possible downsides for {{+CMSEdenChunksRecordAlways}}:
1) There is a new very short (probably un-contended) lock in the slow (non
TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't
detected this timing wise - this is the slow path after all
2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the
default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}}
will expose you to a possible seg fault: (see
[http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809])
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[
https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14084939#comment-14084939
]
Brandon Williams edited comment on CASSANDRA-7432 at 8/4/14 5:33 PM:
-
-Updated patch works correctly on java 8 by inverting the check to be less than
1.8. (we're guaranteed at least 1.7 by previous checks)- nevermind, that
doesn't work either.
was (Author: brandon.williams):
Updated patch works correctly on java 8 by inverting the check to be less than
1.8. (we're guaranteed at least 1.7 by previous checks)
Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
Key: CASSANDRA-7432
URL: https://issues.apache.org/jira/browse/CASSANDRA-7432
Project: Cassandra
Issue Type: Improvement
Components: Packaging
Reporter: graham sanderson
Assignee: Brandon Williams
Fix For: 1.2.19
Attachments: 7432.txt
The new flags in question are as follows:
{code}
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways
{code}
Given we already have
{code}
JVM_OPTS=$JVM_OPTS -XX:+UseParNewGC
JVM_OPTS=$JVM_OPTS -XX:+UseConcMarkSweepGC
JVM_OPTS=$JVM_OPTS -XX:+CMSParallelRemarkEnabled
JVM_OPTS=$JVM_OPTS -XX:+UseTLAB
if [ $JVM_ARCH = 64-Bit ] ; then
JVM_OPTS=$JVM_OPTS -XX:+UseCondCardMark
fi
{code}
The assumption would be that people are at least running on large number CPU
cores/threads
I would therefore recommend defaulting these flags if available - the only
two possible downsides for {{+CMSEdenChunksRecordAlways}}:
1) There is a new very short (probably un-contended) lock in the slow (non
TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't
detected this timing wise - this is the slow path after all
2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the
default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}}
will expose you to a possible seg fault: (see
[http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809])
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[
https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059052#comment-14059052
]
graham sanderson edited comment on CASSANDRA-7432 at 7/11/14 5:34 PM:
--
presumably that should be an equals for 1.7 and a greater than or equal to for
patch 60 (note I'm not sure what version of 1.8 has these flags, so setting on
1.7 only for now seems reasonable
was (Author: graham sanderson):
presumably those should be = rather than
Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
Key: CASSANDRA-7432
URL: https://issues.apache.org/jira/browse/CASSANDRA-7432
Project: Cassandra
Issue Type: Improvement
Components: Packaging
Reporter: graham sanderson
Assignee: Brandon Williams
Fix For: 1.2.19
Attachments: 7432.txt
The new flags in question are as follows:
{code}
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways
{code}
Given we already have
{code}
JVM_OPTS=$JVM_OPTS -XX:+UseParNewGC
JVM_OPTS=$JVM_OPTS -XX:+UseConcMarkSweepGC
JVM_OPTS=$JVM_OPTS -XX:+CMSParallelRemarkEnabled
JVM_OPTS=$JVM_OPTS -XX:+UseTLAB
if [ $JVM_ARCH = 64-Bit ] ; then
JVM_OPTS=$JVM_OPTS -XX:+UseCondCardMark
fi
{code}
The assumption would be that people are at least running on large number CPU
cores/threads
I would therefore recommend defaulting these flags if available - the only
two possible downsides for {{+CMSEdenChunksRecordAlways}}:
1) There is a new very short (probably un-contended) lock in the slow (non
TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't
detected this timing wise - this is the slow path after all
2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the
default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}}
will expose you to a possible seg fault: (see
[http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809])
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[
https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039890#comment-14039890
]
graham sanderson edited comment on CASSANDRA-7432 at 6/21/14 4:37 PM:
--
Here are some performance numbers(we have an unusually large heap for C*: 8gig
new, 16g old):
h3. +CMSParallelInitialMarkEnabled
||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without +CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159|
|With +CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019|
Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}}
h3. +CMSEdenChunksRecordAlways
||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without +CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394|
|With +CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114|
Note this option addresses the issue loosely described
[here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the
*CMSMaxAbortablePrecleanTime* section. The short version of the problem is that
if you are unlucky (and bursty activity in your cluster will make you less
lucky, then even though you have {{-XX:*CMSParallelRemarkEnabled}}, you may get
very low parallelism in your remark STW *and* you may have a very full eden
when you are doing it.
was (Author: graham sanderson):
Here are some performance numbers(we have an unusually large heap for C*: 8gig
new, 16g old):
h3. +CMSParallelInitialMarkEnabled
||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without +CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159|
|With +CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019|
h3. +CMSEdenChunksRecordAlways
||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without +CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394|
|With +CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114|
Note this option addresses the issue loosely described
[here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the
*CMSMaxAbortablePrecleanTime* section. The short version of the problem is that
if you are unlucky (and bursty activity in your cluster will make you less
lucky, then even though you have {{-XX:*CMSParallelRemarkEnabled}}, you may get
very low parallelism in your remark STW *and* you may have a very full eden
when you are doing it.
Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
Key: CASSANDRA-7432
URL: https://issues.apache.org/jira/browse/CASSANDRA-7432
Project: Cassandra
Issue Type: Improvement
Components: Packaging
Reporter: graham sanderson
The new flags in question are as follows:
{code}
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways
{code}
Given we already have
{code}
JVM_OPTS=$JVM_OPTS -XX:+UseParNewGC
JVM_OPTS=$JVM_OPTS -XX:+UseConcMarkSweepGC
JVM_OPTS=$JVM_OPTS -XX:+CMSParallelRemarkEnabled
JVM_OPTS=$JVM_OPTS -XX:+UseTLAB
if [ $JVM_ARCH = 64-Bit ] ; then
JVM_OPTS=$JVM_OPTS -XX:+UseCondCardMark
fi
{code}
The assumption would be that people are at least running on large number CPU
cores/threads
I would therefore recommend defaulting these flags if available - the only
two possible downsides for {{+CMSEdenChunksRecordAlways}}:
1) There is a new very short (probably un-contended) lock in the slow (non
TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't
detected this timing wise - this is the slow path after all
2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the
default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}}
will expose you to a possible seg fault: (see
[http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809])
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[
https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039890#comment-14039890
]
graham sanderson edited comment on CASSANDRA-7432 at 6/21/14 4:39 PM:
--
Here are some performance numbers(we have an unusually large heap for C*: 8gig
new, 16g old):
h3. -XX:+CMSParallelInitialMarkEnabled
||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159|
|With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019|
Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}}
h3. -XX:+CMSEdenChunksRecordAlways
||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394|
|With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114|
Note this option addresses the issue loosely described
[here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the
*CMSMaxAbortablePrecleanTime* section. The short version of the problem is that
if you are unlucky (and bursty activity in your cluster will make you less
lucky, then even though you have {{-XX:*CMSParallelRemarkEnabled}}, you may get
very low parallelism in your remark STW *and* you may have a very full eden
when you are doing it.
h3. -XX:+CMSPrintEdenSurvivorChunks
Turn this on if you want even more stuff to confuse whatever tool you use to
parse GC logs!
was (Author: graham sanderson):
Here are some performance numbers(we have an unusually large heap for C*: 8gig
new, 16g old):
h3. +CMSParallelInitialMarkEnabled
||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without +CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159|
|With +CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019|
Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}}
h3. +CMSEdenChunksRecordAlways
||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without +CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394|
|With +CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114|
Note this option addresses the issue loosely described
[here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the
*CMSMaxAbortablePrecleanTime* section. The short version of the problem is that
if you are unlucky (and bursty activity in your cluster will make you less
lucky, then even though you have {{-XX:*CMSParallelRemarkEnabled}}, you may get
very low parallelism in your remark STW *and* you may have a very full eden
when you are doing it.
Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
Key: CASSANDRA-7432
URL: https://issues.apache.org/jira/browse/CASSANDRA-7432
Project: Cassandra
Issue Type: Improvement
Components: Packaging
Reporter: graham sanderson
The new flags in question are as follows:
{code}
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways
{code}
Given we already have
{code}
JVM_OPTS=$JVM_OPTS -XX:+UseParNewGC
JVM_OPTS=$JVM_OPTS -XX:+UseConcMarkSweepGC
JVM_OPTS=$JVM_OPTS -XX:+CMSParallelRemarkEnabled
JVM_OPTS=$JVM_OPTS -XX:+UseTLAB
if [ $JVM_ARCH = 64-Bit ] ; then
JVM_OPTS=$JVM_OPTS -XX:+UseCondCardMark
fi
{code}
The assumption would be that people are at least running on large number CPU
cores/threads
I would therefore recommend defaulting these flags if available - the only
two possible downsides for {{+CMSEdenChunksRecordAlways}}:
1) There is a new very short (probably un-contended) lock in the slow (non
TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't
detected this timing wise - this is the slow path after all
2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the
default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}}
will expose you to a possible seg fault: (see
[http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809])
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[
https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039890#comment-14039890
]
graham sanderson edited comment on CASSANDRA-7432 at 6/21/14 4:41 PM:
--
Here are some performance numbers(we have an unusually large heap for C*: 8gig
new, 16g old):
h3. -XX:+CMSParallelInitialMarkEnabled
||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159|
|With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019|
Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}}
h3. -XX:+CMSEdenChunksRecordAlways
||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394|
|With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114|
Note this option addresses the issue loosely described
[here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the
*CMSMaxAbortablePrecleanTime* section. The short version of the problem is that
if you are unlucky (and bursty activity in your cluster is likely to make you
less lucky), then even though you have {{-XX:+CMSParallelRemarkEnabled}}, you
may get very low parallelism in your remark STW *and* you may have a very full
eden when you are doing it.
h3. -XX:+CMSPrintEdenSurvivorChunks
Turn this on if you want even more stuff to confuse whatever tool you use to
parse GC logs!
was (Author: graham sanderson):
Here are some performance numbers(we have an unusually large heap for C*: 8gig
new, 16g old):
h3. -XX:+CMSParallelInitialMarkEnabled
||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159|
|With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019|
Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}}
h3. -XX:+CMSEdenChunksRecordAlways
||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394|
|With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114|
Note this option addresses the issue loosely described
[here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the
*CMSMaxAbortablePrecleanTime* section. The short version of the problem is that
if you are unlucky (and bursty activity in your cluster is likely to make you
less lucky), then even though you have {{-XX:*CMSParallelRemarkEnabled}}, you
may get very low parallelism in your remark STW *and* you may have a very full
eden when you are doing it.
h3. -XX:+CMSPrintEdenSurvivorChunks
Turn this on if you want even more stuff to confuse whatever tool you use to
parse GC logs!
Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
Key: CASSANDRA-7432
URL: https://issues.apache.org/jira/browse/CASSANDRA-7432
Project: Cassandra
Issue Type: Improvement
Components: Packaging
Reporter: graham sanderson
The new flags in question are as follows:
{code}
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways
{code}
Given we already have
{code}
JVM_OPTS=$JVM_OPTS -XX:+UseParNewGC
JVM_OPTS=$JVM_OPTS -XX:+UseConcMarkSweepGC
JVM_OPTS=$JVM_OPTS -XX:+CMSParallelRemarkEnabled
JVM_OPTS=$JVM_OPTS -XX:+UseTLAB
if [ $JVM_ARCH = 64-Bit ] ; then
JVM_OPTS=$JVM_OPTS -XX:+UseCondCardMark
fi
{code}
The assumption would be that people are at least running on large number CPU
cores/threads
I would therefore recommend defaulting these flags if available - the only
two possible downsides for {{+CMSEdenChunksRecordAlways}}:
1) There is a new very short (probably un-contended) lock in the slow (non
TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't
detected this timing wise - this is the slow path after all
2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the
default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}}
will expose you to a possible seg fault: (see
[http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809])
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[
https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039890#comment-14039890
]
graham sanderson edited comment on CASSANDRA-7432 at 6/21/14 4:40 PM:
--
Here are some performance numbers(we have an unusually large heap for C*: 8gig
new, 16g old):
h3. -XX:+CMSParallelInitialMarkEnabled
||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159|
|With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019|
Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}}
h3. -XX:+CMSEdenChunksRecordAlways
||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394|
|With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114|
Note this option addresses the issue loosely described
[here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the
*CMSMaxAbortablePrecleanTime* section. The short version of the problem is that
if you are unlucky (and bursty activity in your cluster is likely to make you
less lucky), then even though you have {{-XX:*CMSParallelRemarkEnabled}}, you
may get very low parallelism in your remark STW *and* you may have a very full
eden when you are doing it.
h3. -XX:+CMSPrintEdenSurvivorChunks
Turn this on if you want even more stuff to confuse whatever tool you use to
parse GC logs!
was (Author: graham sanderson):
Here are some performance numbers(we have an unusually large heap for C*: 8gig
new, 16g old):
h3. -XX:+CMSParallelInitialMarkEnabled
||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159|
|With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019|
Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}}
h3. -XX:+CMSEdenChunksRecordAlways
||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394|
|With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114|
Note this option addresses the issue loosely described
[here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the
*CMSMaxAbortablePrecleanTime* section. The short version of the problem is that
if you are unlucky (and bursty activity in your cluster will make you less
lucky, then even though you have {{-XX:*CMSParallelRemarkEnabled}}, you may get
very low parallelism in your remark STW *and* you may have a very full eden
when you are doing it.
h3. -XX:+CMSPrintEdenSurvivorChunks
Turn this on if you want even more stuff to confuse whatever tool you use to
parse GC logs!
Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
Key: CASSANDRA-7432
URL: https://issues.apache.org/jira/browse/CASSANDRA-7432
Project: Cassandra
Issue Type: Improvement
Components: Packaging
Reporter: graham sanderson
The new flags in question are as follows:
{code}
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways
{code}
Given we already have
{code}
JVM_OPTS=$JVM_OPTS -XX:+UseParNewGC
JVM_OPTS=$JVM_OPTS -XX:+UseConcMarkSweepGC
JVM_OPTS=$JVM_OPTS -XX:+CMSParallelRemarkEnabled
JVM_OPTS=$JVM_OPTS -XX:+UseTLAB
if [ $JVM_ARCH = 64-Bit ] ; then
JVM_OPTS=$JVM_OPTS -XX:+UseCondCardMark
fi
{code}
The assumption would be that people are at least running on large number CPU
cores/threads
I would therefore recommend defaulting these flags if available - the only
two possible downsides for {{+CMSEdenChunksRecordAlways}}:
1) There is a new very short (probably un-contended) lock in the slow (non
TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't
detected this timing wise - this is the slow path after all
2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the
default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}}
will expose you to a possible seg fault: (see
[http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809])
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[
https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039890#comment-14039890
]
graham sanderson edited comment on CASSANDRA-7432 at 6/21/14 4:41 PM:
--
Here are some performance numbers (we have an unusually large heap for C*: 8gig
new, 16g old):
h3. -XX:+CMSParallelInitialMarkEnabled
||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159|
|With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019|
Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}}
h3. -XX:+CMSEdenChunksRecordAlways
||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394|
|With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114|
Note this option addresses the issue loosely described
[here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the
*CMSMaxAbortablePrecleanTime* section. The short version of the problem is that
if you are unlucky (and bursty activity in your cluster is likely to make you
less lucky), then even though you have {{-XX:+CMSParallelRemarkEnabled}}, you
may get very low parallelism in your remark STW *and* you may have a very full
eden when you are doing it.
h3. -XX:+CMSPrintEdenSurvivorChunks
Turn this on if you want even more stuff to confuse whatever tool you use to
parse GC logs!
was (Author: graham sanderson):
Here are some performance numbers(we have an unusually large heap for C*: 8gig
new, 16g old):
h3. -XX:+CMSParallelInitialMarkEnabled
||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159|
|With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019|
Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}}
h3. -XX:+CMSEdenChunksRecordAlways
||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394|
|With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114|
Note this option addresses the issue loosely described
[here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the
*CMSMaxAbortablePrecleanTime* section. The short version of the problem is that
if you are unlucky (and bursty activity in your cluster is likely to make you
less lucky), then even though you have {{-XX:+CMSParallelRemarkEnabled}}, you
may get very low parallelism in your remark STW *and* you may have a very full
eden when you are doing it.
h3. -XX:+CMSPrintEdenSurvivorChunks
Turn this on if you want even more stuff to confuse whatever tool you use to
parse GC logs!
Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
Key: CASSANDRA-7432
URL: https://issues.apache.org/jira/browse/CASSANDRA-7432
Project: Cassandra
Issue Type: Improvement
Components: Packaging
Reporter: graham sanderson
The new flags in question are as follows:
{code}
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways
{code}
Given we already have
{code}
JVM_OPTS=$JVM_OPTS -XX:+UseParNewGC
JVM_OPTS=$JVM_OPTS -XX:+UseConcMarkSweepGC
JVM_OPTS=$JVM_OPTS -XX:+CMSParallelRemarkEnabled
JVM_OPTS=$JVM_OPTS -XX:+UseTLAB
if [ $JVM_ARCH = 64-Bit ] ; then
JVM_OPTS=$JVM_OPTS -XX:+UseCondCardMark
fi
{code}
The assumption would be that people are at least running on large number CPU
cores/threads
I would therefore recommend defaulting these flags if available - the only
two possible downsides for {{+CMSEdenChunksRecordAlways}}:
1) There is a new very short (probably un-contended) lock in the slow (non
TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't
detected this timing wise - this is the slow path after all
2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the
default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}}
will expose you to a possible seg fault: (see
[http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809])
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[
https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039890#comment-14039890
]
graham sanderson edited comment on CASSANDRA-7432 at 6/21/14 4:46 PM:
--
Here are some performance numbers (we have an unusually large heap for C*: 8gig
new, 16g old), and 16 CPU cores/32 threads
h3. -XX:+CMSParallelInitialMarkEnabled
||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159|
|With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019|
Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}}
h3. -XX:+CMSEdenChunksRecordAlways
||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394|
|With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114|
Note this option addresses the issue loosely described
[here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the
*CMSMaxAbortablePrecleanTime* section. The short version of the problem is that
if you are unlucky (and bursty activity in your cluster is likely to make you
less lucky), then even though you have {{-XX:+CMSParallelRemarkEnabled}}, you
may get very low parallelism in your remark STW *and* you may have a very full
eden when you are doing it.
h3. -XX:+CMSPrintEdenSurvivorChunks
Turn this on if you want even more stuff to confuse whatever tool you use to
parse GC logs!
was (Author: graham sanderson):
Here are some performance numbers (we have an unusually large heap for C*: 8gig
new, 16g old):
h3. -XX:+CMSParallelInitialMarkEnabled
||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159|
|With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019|
Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}}
h3. -XX:+CMSEdenChunksRecordAlways
||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394|
|With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114|
Note this option addresses the issue loosely described
[here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the
*CMSMaxAbortablePrecleanTime* section. The short version of the problem is that
if you are unlucky (and bursty activity in your cluster is likely to make you
less lucky), then even though you have {{-XX:+CMSParallelRemarkEnabled}}, you
may get very low parallelism in your remark STW *and* you may have a very full
eden when you are doing it.
h3. -XX:+CMSPrintEdenSurvivorChunks
Turn this on if you want even more stuff to confuse whatever tool you use to
parse GC logs!
Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
Key: CASSANDRA-7432
URL: https://issues.apache.org/jira/browse/CASSANDRA-7432
Project: Cassandra
Issue Type: Improvement
Components: Packaging
Reporter: graham sanderson
The new flags in question are as follows:
{code}
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways
{code}
Given we already have
{code}
JVM_OPTS=$JVM_OPTS -XX:+UseParNewGC
JVM_OPTS=$JVM_OPTS -XX:+UseConcMarkSweepGC
JVM_OPTS=$JVM_OPTS -XX:+CMSParallelRemarkEnabled
JVM_OPTS=$JVM_OPTS -XX:+UseTLAB
if [ $JVM_ARCH = 64-Bit ] ; then
JVM_OPTS=$JVM_OPTS -XX:+UseCondCardMark
fi
{code}
The assumption would be that people are at least running on large number CPU
cores/threads
I would therefore recommend defaulting these flags if available - the only
two possible downsides for {{+CMSEdenChunksRecordAlways}}:
1) There is a new very short (probably un-contended) lock in the slow (non
TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't
detected this timing wise - this is the slow path after all
2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the
default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}}
will expose you to a possible seg fault: (see
[http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809])
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7432) Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
[
https://issues.apache.org/jira/browse/CASSANDRA-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039890#comment-14039890
]
graham sanderson edited comment on CASSANDRA-7432 at 6/21/14 4:53 PM:
--
Here are some performance numbers (we have an unusually large heap for C*: 8gig
new, 16g old), and 16 CPU cores/32 threads
h3. -XX:+CMSParallelInitialMarkEnabled
||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159|
|With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019|
Note this flag is just the new cousin of {{-XX:+CMSParallelRemarkEnabled}}
h3. -XX:+CMSEdenChunksRecordAlways
||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394|
|With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114|
Note this option addresses the issue loosely described
[here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the
*CMSMaxAbortablePrecleanTime* section. The short version of the problem is that
if you are unlucky (and bursty activity in your cluster is likely to make you
less lucky), then even though you have {{-XX:+CMSParallelRemarkEnabled}}, you
may get very low parallelism in your remark STW *and* you may have a very full
eden when you are doing it.
h3. -XX:+CMSPrintEdenSurvivorChunks
Turn this on if you want even more stuff to confuse whatever tool you use to
parse GC logs!
was (Author: graham sanderson):
Here are some performance numbers (we have an unusually large heap for C*: 8gig
new, 16g old), and 16 CPU cores/32 threads
h3. -XX:+CMSParallelInitialMarkEnabled
||Initial Mark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSParallelInitialMarkEnabled|0.000|0.050|0.109|0.880|0.159|
|With -XX:+CMSParallelInitialMarkEnabled|0.000|0.010|0.019|0.090|0.019|
Note this flag is just the new cousin of {{-XX:*CMSParallelRemarkEnabled}}
h3. -XX:+CMSEdenChunksRecordAlways
||Remark STW pauses(secs)|Min|Median|Mean|Max|StdDev||
|Without -XX:+CMSEdenChunksRecordAlways|0.010|0.470|1.049|6.580|1.394|
|With -XX:+CMSEdenChunksRecordAlways|0.010|0.135|0.148|0.520|0.114|
Note this option addresses the issue loosely described
[here|https://blogs.oracle.com/jonthecollector/entry/did_you_know] in the
*CMSMaxAbortablePrecleanTime* section. The short version of the problem is that
if you are unlucky (and bursty activity in your cluster is likely to make you
less lucky), then even though you have {{-XX:+CMSParallelRemarkEnabled}}, you
may get very low parallelism in your remark STW *and* you may have a very full
eden when you are doing it.
h3. -XX:+CMSPrintEdenSurvivorChunks
Turn this on if you want even more stuff to confuse whatever tool you use to
parse GC logs!
Add new CMS GC flags to cassandra_env.sh for JVM later than 1.7.0_60
Key: CASSANDRA-7432
URL: https://issues.apache.org/jira/browse/CASSANDRA-7432
Project: Cassandra
Issue Type: Improvement
Components: Packaging
Reporter: graham sanderson
The new flags in question are as follows:
{code}
-XX:+CMSParallelInitialMarkEnabled
-XX:+CMSEdenChunksRecordAlways
{code}
Given we already have
{code}
JVM_OPTS=$JVM_OPTS -XX:+UseParNewGC
JVM_OPTS=$JVM_OPTS -XX:+UseConcMarkSweepGC
JVM_OPTS=$JVM_OPTS -XX:+CMSParallelRemarkEnabled
JVM_OPTS=$JVM_OPTS -XX:+UseTLAB
if [ $JVM_ARCH = 64-Bit ] ; then
JVM_OPTS=$JVM_OPTS -XX:+UseCondCardMark
fi
{code}
The assumption would be that people are at least running on large number CPU
cores/threads
I would therefore recommend defaulting these flags if available - the only
two possible downsides for {{+CMSEdenChunksRecordAlways}}:
1) There is a new very short (probably un-contended) lock in the slow (non
TLAB) eden allocation path with {{+CMSEdenChunksRecordAlways}}. I haven't
detected this timing wise - this is the slow path after all
2) If you are running with {{-XX:-UseCMSCompactAtFullCollection}} (not the
default) *and* you call {{System.gc()}} then {{+CMSEdenChunksRecordAlways}}
will expose you to a possible seg fault: (see
[http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8021809])
--
This message was sent by Atlassian JIRA
(v6.2#6252)
