[ https://issues.apache.org/jira/browse/CASSANDRA-13006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16118421#comment-16118421 ]
Eric Evans commented on CASSANDRA-13006: ---------------------------------------- I think this behavior (invoking {{jmap}} on OOM) is a pretty serious violation to the Element of Least-surprise. We already provide mechanisms for passing arguments to the JVM, and TTBMK, all of them provide some means for dropping a heap dump on out-of-memory. It definitely caught me be surprise. We carried over {{-XX:+HeapDumpOnOutOfMemoryError}} from our 2.2.x environment, only to have Cassandra and the JVM racing to create a dump of the same name. Additionally, something about all of this is buggy, because on more than one occasion we've had Cassandra fork-bombing {{jmap}} processes {noformat} ● cassandra-b.service - distributed storage system for structured data Loaded: loaded (/lib/systemd/system/cassandra-b.service; static) Active: active (running) since Sat 2017-08-05 22:32:07 UTC; 23h ago Main PID: 25025 (java) CGroup: /system.slice/cassandra-b.service ├─ 9213 jmap -histo 25025 ├─ 9214 jmap -histo 25025 ├─ 9284 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─ 9285 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─ 9388 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─ 9453 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─ 9519 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─ 9520 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─ 9733 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─ 9735 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─ 9736 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─14835 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─14836 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─14837 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─14839 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─14841 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─14844 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─18932 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─18933 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─18934 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─18935 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─18936 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─18937 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─18938 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─18939 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─18940 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─18942 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─18943 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─18944 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 ├─18945 jmap -dump:format=b,file=/srv/cassandra-b/cassandra-1501972327-pid24937.hprof 25025 [ ... ] {noformat} IMO, the sanest strategy here would be to leave the creation of heap dumps to the JVM. > Disable automatic heap dumps on OOM error > ----------------------------------------- > > Key: CASSANDRA-13006 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13006 > Project: Cassandra > Issue Type: Bug > Components: Configuration > Reporter: anmols > Assignee: Benjamin Lerer > Priority: Minor > Fix For: 3.0.9 > > Attachments: 13006-3.0.9.txt > > > With CASSANDRA-9861, a change was added to enable collecting heap dumps by > default if the process encountered an OOM error. These heap dumps are stored > in the Apache Cassandra home directory unless configured otherwise (see > [Cassandra Support > Document|https://support.datastax.com/hc/en-us/articles/204225959-Generating-and-Analyzing-Heap-Dumps] > for this feature). > > The creation and storage of heap dumps aides debugging and investigative > workflows, but is not be desirable for a production environment where these > heap dumps may occupy a large amount of disk space and require manual > intervention for cleanups. > > Managing heap dumps on out of memory errors and configuring the paths for > these heap dumps are available as JVM options in JVM. The current behavior > conflicts with the Boolean JVM flag HeapDumpOnOutOfMemoryError. > > A patch can be proposed here that would make the heap dump on OOM error honor > the HeapDumpOnOutOfMemoryError flag. Users who would want to still generate > heap dumps on OOM errors can set the -XX:+HeapDumpOnOutOfMemoryError JVM > option. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org